Challenges

Marking open-ended exam questions

EdTech

Primary challenge owner: University of Tilburg

Secondary challenge owner(s): None

 

Background

Within the School of Law of Tilburg University, open-ended questions are used in testing where the student must build an argument. This form of testing matches the learning objectives of the various law courses. A student must be able to argue in a written form. A problem thought is that lecturers have to deal with student groups of 500 students per subject. To examine the students level of knowledge on the right level, open ended questions are important. Lecturers might get overworked by this workload of marking the tests. Therefore, the School of Law is looking for an EdTech solution which makes it easier for lecturers to mark the large volumes of exams in a shorter period of time while maintaining the quality of marking and the feedback they give.

 

Some thoughts on previous research and attempts to facilitate efficient marking:

  • There are possibilities in artificial intelligence solutions, in which specific alghoritms combined with the rubric and answering model can provide a first screening of the exams. It has to be considered that an AI solutions is not standing on its own and has to be combined with marking by the assessors. The human factor in assessing should not be neglected in the process for both the teacher and the student.
  • Finally a solution could be found in using specific coding while marking. Coding makes marking more efficient. A tech solution in which this coding is integrated and feedback is standardized could be a more efficient way of marking.

The question/ the challenge to startups

Contribute to a solution with which lecturers can efficiently mark[1] open questions for exams with very large quantities of students without losing the quality of the grading and the feedback possibilities it offers.

[1] Marking is assessing the work, grading is grade the work with points.

Criteria:

  • The tool should be evidence- and research-informed. In case of the AI solution, experiments already have been done within other universities. Tilburg University also has staff that are experts in AI. If possible, involve them in the process.
  • Lecturers should be able to feed-in the system with their own defined criteria.
  • Lecturers are used to grade on paper especially with these large numbers. The tool should support fast grading on a tablet or pc that is user-friendly and body friendly.
  • Inspection of the exams by students should be possible.
  • Students should be able to see their feedback on their exam.
  • Possible integration with Canvas is a plus.
  • Calculation of the grade according to points or feedback given while marking should be easy, preferably automated.

Sign up now!