Current Science

Open Access Open Access  Restricted Access Subscription Access

Automatic Question Generation Approaches and Evaluation Techniques


Affiliations
1 Department of Computer Science, University of Mumbai, Santacruz, Mumbai 400 098, India
 

The objective of this article is to review several automatic question generation systems and find why automatic question generation is still an attraction for researchers. The focus is mainly on the task of question generation, analysis of the approaches and evaluation of various methods of automatic question generation. Pointers for further research are included.

Keywords

Automatic Question Generation, Evaluation Techniques, Quality Enhancers, Ranking, Sentence Simplification.
User
Notifications
Font Size

  • Rus, V. and Graesser, A. C., The question generation shared task and evaluation challenge. National Science Foundation, The Univesity of Memphis, 2009, pp. 1–48; doi:10.1016/0004-3702(73)90013-1.

  • Lindberg, D., Popowich, F., Nesbit, J. and Winne, P., Generating Natural Language Questions to Support Learning On-Line. In 14th European Workshop on Natural Language Generation, Sofia, Bulgaria, 2013, pp. 105–114.

  • Wolfe, J. H., Automatic question generation from text – an aid to independent study. In ACM SIGCUE Outlook, 1976; doi: 10.1145/953026.803459.

  • Aist, G., Towards automatic glossarization: automatically constructing and administering vocabulary assistance factoids and multiple-choice assessment. Int. J. Artif. Intell. Educ., 2001, 12, 212–231.

  • Mostow, J. et al., Using automated questions to assess reading comprehension, vocabulary, and effects of tutorial interventions. Technol. Instr. Cogn. Learn., 2004, 2, 103–140.

  • Brown, J. C., Frishkoff, G. A. and Eskenazi, M., Automatic question generation for vocabulary assessment. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, 2005, Vancouver, p. 826; doi:http://dx.doi.org/10.3115/1220575.1220678.

  • Kunichika, H., Katayama, T., Hirashima, T. and Takeuchi, A., Automated question generation methods for intelligent English learning systems and its evaluation. In International Conference on Computers in Educationa, Melbourne, Australia, 2004.

  • Chen, C. Y., Liou, H. C. and Chang, J. S., Fast: an automatic generation system for grammar tests. In Proceedings of the COLING/ACL English Instrustion and Assessment, Sydney, 2006, pp. 1–4; doi:10.3115/1225403.1225404.

  • Wang, X.-J., Tu, X., Feng, D. and Zhang, L., Ranking community answers by modeling question-answer relationships via analogical reasoning. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval – SIGIR ’09, ACM Press, 2009, pp. 179–186; doi:10.1145/1571941.1571974.

  • Liu, M., Calvo, R. A. and Rus, V., Automatic question generation for literature review writing support. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer, Berlin, Heidelbrg, 2010.

  • Liu, M., Calvo, R. and Rus, V., G-Asks: An intelligent automatic question generation system for academic writing support. Dialogue Discourse, 2012, 3, 101–124.

  • Heilman, M. and Smith, N. A., Question generation via overgenerating transformations and ranking. Framework, No. CMU/LTI-09-013, Carnegie-Mellon Univ Pittsburgh pa Language Technologies Inst, 2009.

  • Skalban, Y., Ha, L. A., Specia, L. and Mitkov, R., Automatic question generation in multimedia-based learning. COLING (Posters), 2012, 1, 1151–1160.

  • Lindberg, D., Automatic Question Generatin from Text for Self-Directed Learning, 2013.

  • Aldabe, I. and Maritxalar, M., Automatic distractor generation for domain specific texts. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6233 LNAI, 2010, pp. 27–38.

  • Kumar, G., Banchs, R. E. and Haro, L. F. D., RevUP: Automatic Gap-Fill Question Generation from Educational Texts, 2015, pp. 154–161.

  • Agarwal, M. and Mannem, P., Automatic gap-fill question generation from text books. In Proceedings of the 6th Workshop on Innovative Use of NLP for Building Educational Applications, Association for Computational Linguistics, Portland, Oregon, 2011, pp. 56–64.

  • Narendra, A., Agarwal, M. and Shah, R., Automatic cloze-questions generation. In Proceedings of Recent Advances in Natural Language Processing, Hissar, Bulgaria, 2013, pp. 511–515.

  • Mitkov, R. and Ha, L. A., Computer-aided generation of multiple-choice tests. In Proceedings of the HLT-NAACL 2003 Workshop on Building Educational Applications using Natural Language Processing, Stroudsburg, PA, USA, 2003; doi:10.3115/1118894.1118897.

  • Aldabe, I., De Lacalle, M. L., Maritxalar, M., Martinez, E. and Uria, L., ArikIturri: An automatic question generator based on corpora and NLP techniques. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Springer, Berlin, 4053 LNCS, 2006, pp. 584–594.

  • Ali, H., Chali, Y. and Hasan, S. A., Automatic question generation from sentences. In Proceedings of the Conference on Traitement Automatique de la Langue Naturelle, Montreal, Canada, 2010, pp. 19–23.

  • Chali, Y. and Hasan, S., Towards automatic topical question generation. In COLING, Mumbai, 2012, pp. 475–492.

  • Chen, W., Aist, G. and Mostow, J., Generating questions automatically from informational text. In Workshop Proceedings of the AIED 2009 14th International Conference on Artificial Intelligence in Education, Brighton, UK, 2009, pp. 17–24.

  • Ali, H., Automatic question generation: a syntactical approach to the sentence-to-question generation case. Ph D diss., Department of Mathematics and Computer Science, University fo Lethbridge, Lethbridge, Alta, 2012.

  • Corley, M. A. and Rauscher, W. C., Deeper learning through questioning, TEAL Fact Sheet No. 12, Teaching Excellence in Adult Literacy, US, 2013, pp. 1–5.

  • Yao, X., Bouma, G. and Zhang, Y., Semantics-based question generation and implementation. Dialogue Discourse, 2012, 3, 11–42.

  • Liu, M. and Calvo, R. A., An automatic question generation tool for supporting sourcing and integration in students’ essays. In Proceedings of the Fourteenth Australasian Document Computing Symposium 2009, University of Sydney, 2009, pp. 90–97.

  • Adamson, D. et al., Automatically generating discussion questions. In Artificial Intelligence Education: 16th International Conference AIED 2013, Memphis, TN, Springer, USA, 2013, vol. 7926, pp. 81–90.

  • Heilman, M. and Smith, N., Extracting simplified statements for factual question generation. In Proceedings of the 3rd Workshop on Question Generation, Pittsburg, PA, USA, 2010, pp. 11–20.

  • Heilman, M. and Smith, N. A., Ranking automatically generated questions as a shared task. In Second Workshop on Question Generation, 2009, pp. 30–37.

  • Chali, Y. and Hasan, S. A., Towards topic-to-question generation. Comput. Linguist., 2014, 70, 1–20.

  • Boyer, K. E. and Piwek, P., Proceedings of QG2010: The Third Workshop on Question Generation. In At The Tenth International Conference on Intelligent Tutoring Systems (ITS2010), Pittsburg, 2010, 91.

  • Punyakanok, V., Roth, D. and Yih, W., The importance of syntactic parsing and inference in semantic role labeling. Comput. Linguist., 2008, 34(2), 257–287.

  • Schwartz, L., Aikawa, T. and Pahud, M., Dynamic language learning tools. In Proceedings of InSTIL/ICALL Symposium 2004, Microsoft Research, 2004.

  • Liu, M., Calvo, R. A. and Rus, V., Automatic generation and ranking of questions for critical review. J. Edu. Technol. Soc., 2014, 17(2), 333–346; https://rafaelacalvo.files.wordpress.com/2013/11/14-jets_ming.pdf (accessed on 2 September 2015).

  • Mannem, P., Prasad, R. and Joshi, A., Question generation from paragraphs at UPenn: QGSTEC system description. In Proceedings of QG 2010: The Third Workshop on Question Generation, Pittsburg, PA, USA, 2010.

  • McConnell, C., Mannem, P., Prasad, R. and Joshi, A., A new approach to ranking over-generated questions. In AAAI Fall Symposium, Palo, Alto, CA, 2011, pp. 45–48.

  • Duh, K., Ranking vs. regression in machine translation evaluation. ACM Trans. Comput–Hum. Interact., 2008, 17(1), 191–194.

  • Divate, M. and Salgaonkar, A., Ranking model with a reduced feature set for an automated question generation system. In International Conference on Natural Langauge Processessing (ICON), Thiruvananthapuram, India, 2015, pp. 221–230.

  • Rus, V., Cai, Z. and Graesser, A. C., Evaluation in Natural Language Generation: The Question Generation Task. In Proceedings of the Workshop on Shared Tasks and Comparative Evaluation in Natural Language Generation, Hilton Arlington, 2007.

  • Heilman, M. and Smith, N. A., Good question! Statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational, Linguistics, Los Angeles, California, 2010, pp. 609–617.

  • Lin, C. Y., Automatic question generation from quesries. In Workshop on Question Generation Shared Task, Pittsburg, 2008, pp. 156–164.

  • Resnik, P. and Lin, J., Evaluation of NLP systems. In Handbook Computational Linguistic and Natural Language Processing, 2010, pp. 271–295; doi:10.3115/993268.993346.

  • Person, N. and Graesser, A. C., Human or computer? AutoTutor in a Bystander turing test. In Proceedings of E-Learn 2002 – World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, 2002, pp. 778–784.

  • Le, N. and Pinkwart, N., Evaluation of a question generation approach using semantic web for supporting argumentation. Res. Pract. Technol. Enhanc. Learn., 2015, 10, 3.

  • Liu, M., Calvo, R. A., Aditomo, A. and Pizzato, L. A., Using Wikipedia and conceptual graph structures to generate questions for academic writing support. IEEE Trans. Learn. Technol., 2012, 5, 251–263.

  • Zhao, S., Wang, H., Li, C., Liu, T. and Guan, Y., Automatically generating questions from queries for community-based question answering. In Proceedings of 5th International Joint Conference on Natural Language Processing, Chiang, Mai, Thailand, 2011, pp. 929–937.

  • Yao, X., Question generation with minimal recursion semantics. In Language and Communication Technologies, University of Groningen and Saarland University, 2010, pp. 1–92.

  • Varga, A. and Ha, L., Wlv: a question generation system for the qgstec 2010 task b. In Proceedings of the Third Workshp on Question Generation, Pittsburg, 2010, pp. 80–83.


Abstract Views: 374

PDF Views: 66




  • Automatic Question Generation Approaches and Evaluation Techniques

Abstract Views: 374  |  PDF Views: 66

Authors

Manisha Divate
Department of Computer Science, University of Mumbai, Santacruz, Mumbai 400 098, India
Ambuja Salgaonkar
Department of Computer Science, University of Mumbai, Santacruz, Mumbai 400 098, India

Abstract


The objective of this article is to review several automatic question generation systems and find why automatic question generation is still an attraction for researchers. The focus is mainly on the task of question generation, analysis of the approaches and evaluation of various methods of automatic question generation. Pointers for further research are included.

Keywords


Automatic Question Generation, Evaluation Techniques, Quality Enhancers, Ranking, Sentence Simplification.

References





DOI: https://doi.org/10.18520/cs%2Fv113%2Fi09%2F1683-1691