Author: Yang Wang
Collaborator: Colin Mack
Urban Studies & Social Policy Division, School of Social & Political Sciences
The presentation
In my presentation, “To GenAI or not to GenAI in Coding Related Assessments?”, I shared early findings from a study conducted with my colleague Colin Mack. The study began from a practical concern. While much of the discussion around generative AI has focused on written coursework, coding presents a somewhat different challenge. Because code is structured, and because there can be several acceptable ways of reaching the same solution, it may be harder for staff to judge whether a submission reflects a student’s own work and understanding.
The question behind the study was straightforward: if an assessment involves coding, how can staff judge whether the work has been generated by AI? To explore this, we used a qualitative approach based on structured reflections and focus groups with ten members of social science staff involved in coding-intensive teaching.
An early finding was a clear gap in confidence and expertise among staff. We found considerable variation in how colleagues approached possible AI use in student work, alongside shared challenges about the lack of clear institutional guidance. Many felt they were dealing with uncertainty on their own, often relying on personal judgement rather than any agreed standard or policy framework.
What the discussion raised
The discussion during the session was especially useful because it shifted the conversation away from detection alone. One colleague raised an important question about transparency: what exactly are we telling students about the use of generative AI? Are these tools being recommended, tolerated, or prohibited? Without clarity on that point, it becomes difficult to have a consistent approach to assessment and academic integrity.
Another colleague suggested that we may need to place less weight on final outputs and more on the process of coding itself. This is connected closely with a further point made during the discussion that current AI detection tools can produce high false-positive rates. If those tools are unreliable, then perhaps the more useful question is not whether we can detect AI use with certainty, but whether we can assess how far a student understands the code they submit.
Another observation that stayed with me after the session: ‘how to detect’ may be the wrong question. A more useful line of thinking may be how to design assessments that are less vulnerable to outsourcing and better able to capture understanding.
Ideas from other sessions
Discussions after my session raised a common point: how to design assessments in which understanding must be demonstrated rather than simply submitted. Several other conference sessions addressed this challenge from different but complementary perspectives.
The session closest to this reframing was Tijana Vueltic and Nicolas Labrosse’s Exploration of Viva voce assessments for meaningful knowledge attainment and improved student experience. Their use of viva-style approaches explored how oral assessments can help verify understanding by asking students to explain and discuss their knowledge directly. In the context of coding, this might involve asking students to talk through their logic or explain key decisions in a short recorded video.
Others, such as Kirsten Knox and colleagues’, Reflections on the development of confidence in students following GenAI pseudo-viva assessments, approached the issue differently. Their approach used GenAI as a kind of conversational prompt, allowing students to engage in text-based discussions about their experiments. What was also interesting and relevant to findings in the coding context is the shift away from marking a final product alone and towards marking a record of dialogue and reasoning, a form of supervised learning process.
If these two sessions addressed the design challenge from the perspective of what we assess and how, Sara Eftekhari and colleagues’ session, Enhancing Inclusive Education Through AI: Supporting Neurodivergent Students and Staff in UK Higher Education, helped me to rethink another question: for whom are we designing? This is a useful reminder that AI is not only an issue of misconduct or misuse. It also raises questions about accessibility, support, and how policies can remain academically rigorous without being overly blunt.
Closing reflections
What I took from the conference is that the answer to ‘To GenAI or not to GenAI’ is unlikely to lie in better detection software alone. The more important issue may be assessment design: how we create opportunities for students to demonstrate process, understanding, and reflection, rather than relying only on final outputs.
For coding-related assessment, this might include oral defence, process-based marking, or other ways of making students’ thinking more visible. More broadly, it points to the need for clearer staff-student dialogue about what acceptable GenAI use looks like in practice.
I am grateful to everyone who contributed their questions and ideas. The conference offered a valuable opportunity to think through these issues with colleagues, and it has already helped me sharpen the questions I want to take forward in this work.