Can Instant Feedback Make Learning to Code Harder Instead of Easier?
Since 2012, Harvard's CS50 has used an automated tool called check50 to grade student code instantly. While this framework ensures consistent, high-quality assessment and allows staff to focus on qualitative feedback, it has paradoxically led students to perceive assignments as more difficult and time-consuming than ever before.
The check50 framework was designed as an open-source, extensible solution to the challenge of assessing student programming assignments. By providing a functional API that abstracts complex tasks—such as compiling code, executing programs, supplying inputs, and verifying outputs—it enables students to receive immediate feedback on their work. This automation serves two primary purposes: it provides students with real-time guidance as they progress through their assignments, and it facilitates consistent, automatic grading for teaching staff. By offloading the mechanical aspects of evaluation, instructors are freed to dedicate more time to providing tailored, qualitative feedback that addresses the nuances of a student's approach.
However, the introduction of check50 in 2012 at Harvard University has yielded a complex psychological effect on the student body. Despite the tool's efficiency, students now report that programming assignments feel more demanding and time-consuming than they did in previous years. Researchers speculate that this shift in perception is driven by the very feedback loop the tool provides. Because students receive instant verification before final submission, they are compelled to engage in more rigorous debugging cycles than they would have in a manual grading environment. They are no longer simply submitting code; they are iterating until the automated system confirms correctness.
The result is a notable trend in academic performance: while the perceived difficulty has risen, students' actual correctness scores are higher than ever. This suggests that the framework has successfully raised the bar for technical accuracy, even if it has changed the nature of the student experience. By shifting the focus from the final submission to the iterative process of debugging, check50 has fundamentally altered how students interact with the art of programming. It highlights a critical tension in computer science education: the trade-off between the convenience of automated assessment and the increased cognitive load required to meet the high standards of a machine-verified curriculum.
Source: An Open-Source, API-Based Framework for Assessing the Correctness of Code in CS50 - ITiSCE 2020
