Qualitative Assessment of Software Engineering Examination Questions with Bloom’s Taxonomy

Context: Question items are intended to evaluate students for various skills and knowledge acquired during the process of learning. The knowledge and skills acquired by students are typically organized through educational taxonomies. Bloom’s taxonomy is one of the widely used taxonomies for various activities in an education field. However, there is no comprehensive understanding on how it can be employed in Software Engineering (SE) Education. Objectives: To identify the gaps in building/setting a question paper to assess the students in the context of Higher Order Cognitive Skills (HOCS) for a course on software engineering. Method/Analysis: The question papers of a course on software engineering conducted by various universities are collected and analysed with Bloom’s taxonomy. All Question items are analyzed and assessed across six cognitive levels: Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation which capture diverse cognitive skills from memory recall to problem solving. The analysis is performed for classifying the question items asked in software engineering examination conducted by various universities. Findings: Around 174 question items from various question papers were selected and analysed according to Bloom’s Taxonomy. Results of the analysis shows that most of the question items asked in the examinations for a course on software engineering test only Lower Order Cognitive Skills (LOCS) such as Knowledge and Comprehension. Very few percentages of questions are asked to test the higher order cognitive skills (HOCS) such as Analysis, Synthesis, and Evaluation. Novelty/Improvement: The course on software engineering deals with the various skills and methods for the software development. In particular, most of the software development processes such as analysis, design, construction, testing etc. required higher order cognitive skill. Thus, the percentage of questions to assess such skills must be increased in the question paper. This work can be extended to provide automated generation of a question paper with Bloom’s Taxonomy.