Assessment design considerations
The following features can potentially have an impact on helping to mitigate the inappropriate use of AI as part of an assessment:
- Providing a rubric that sets explicit requirements linked to delivery of the module; for example requiring students to address specific models/frameworks or use specific techniques
- Evidencing process, having development steps in the assessment where students are required to submit evidence of progress with explicit reference to how they have used developmental feedback. Hence making the process more dialogic.
- The feedback provided at each step can include a mix of tutor and peer to help manage the efficiencies of the process.
- Research by the Open University indicates that the following question types are more robust:
- Role play
- Activity Plan
- Observation of learning
- Reflections on own practice/learning environment
- The OU research also indicates that some of the features that might be used to identify potential AI use could also be indicative of poor academic skills in students. Ensuring students have good academic skills could therefore be helpful in helping to limit the possibility of false positives.
Example assessment design scenarios
Example 1 – A Project-based assessment which requires students to submit evidence of how they have undertaken the task, these steps might be; initial conceptualisation of project and plan; initial research findings; reflections on peer feedback (from a presentation on initial ideas to peers); final report. AI could be used to support some steps and how used evidenced when each step is reported.
Example 2 – Portfolio assessment – this might be in the form of a series of blog posts or multiple artefacts. A submission point for each item/artefact can be set for specific times or open during the Block. Each item/artefact weighted appropriately to contribute to final Assessment Path grade.
Example 3 – Assessment Path that includes Core Assessment and Applied Core assessment. Steps might include: core assessment with specified multiple attempts (zero weighted for final Assessment Path grade); submission of Applied Core (100% weighting)
Example 4 – Audit trail approach. This process focused approach integrates the use of AI into the assessment task. The example is based on an approach used by Haider Ali at LSE, where it was introduced to replace a 3000 word report on developing a marketing relationship. The new task involved the following steps which were evidenced as part of the audit trail:
- Specify task
- Provide prompt used to interrogate AI tools (NB while AI was built into the task students were not required to use it and so could miss this step)
- Identify AI tool used
- Result – share outcome from AI tool
- Triangulation – provide evidence of literature used to verify and triangulate outcome from AI
- Implications from the research
- Live assessment – single slide presentation combined with Q&A as final summative assessment of the task.