A Framework for Automated Generation of Questions Across Formal Domains
đź“ś Abstract
In this work, questions are tasks posed to secondary students to help them understand a subject, or to help educators assess their level of competency in it. Automated question generation is important today as content providers in education try to scale their efforts. In particular, MOOCs need a continuous supply of new questions in order to offer educational content to thousands of students, and to provide a fair assessment process. This paper innovates in three ways; (1) we describe question generation across domains, and, previous efforts as instances of a general framework; (2) we establish first-order logic as a suitable formal tool to describe question scenarios, questions and answers; and (3) we generalize a published question generation method based on logic programming and theorem proving to work across domains. We apply this approach to three domains in high school education—geometry, algebra and mechanics (physics)—and report initial results.
✨ Summary
The paper proposes a domain-independent framework for automatically generating educational questions and solutions from formally represented scenarios. Domains are modeled using first-order predicates and axiomatic rules, with Constraint Handling Rules implemented in SWI-Prolog providing forward-chaining deduction and pattern matching. The framework has three principal stages: generating an initial scenario, deriving new facts and candidate answers, and enriching the scenario by instantiating variables required to produce suitable questions. It represents questions as scenario–question-fact pairs and derives solutions by tracing the axioms used to obtain the questioned fact.
The authors demonstrate the approach in high-school geometry, algebra, and mechanics. Algebra is treated as a special case in which an existing question acts as a predefined scenario; numerical instantiation and equality testing then produce related questions. The reported prototype uses approximately 50 axioms for geometry and 20 for physics, generating questions in under 10 seconds in the tested cases. The main limitation identified by the authors is inadequate handling of degeneracy and validity conditions when modifying or enriching scenarios.
Subsequent evidence of influence is limited but identifiable. The work is listed in later surveys of automatic question generation and is cited as an early rule-based/formal-domain approach. (ouci.dntb.gov.ua) The authors subsequently extended the research direction with a 2016 paper on user-defined difficulty levels and interactive scenario guidance for automated question generation. (comp.nus.edu.sg) The search found no clear evidence that this specific framework was adopted in a commercial educational product or deployed at industry scale.