Leveraging Eye Tracking and Targeted Regions of Interest for Analyzing Code Comprehension

Code comprehension studies techniques for extracting information that gives insights into how software is understood. For educators, this is an important but often difficult task. This is further complicated by larger classes, limited time, and not enough grading resources for early identification of students in need of help or to provide early feedback. During a code comprehension task, analyzing where a student look can provide valuable insights into what information the student perceives as important. The instructor can then assess if the student is looking in the right areas of the code. We investigate differences in how a student's eyes traverse code during a coding comprehension exercise and propose a systematic method to distinguish between students with a good understanding of the exercise and those who need additional help. The methodology uses coding exercises seeded with errors, a graded results of completed the exercises, measured fixation counts, and average fixation durations of the students' eyes within what we refer to as the targeted region of interest (TROI) of the code. We conducted experiments using basic Java code from the Programming Principles II course, and our eye-tracking data showed that students' ability to understand the context of the code (the grade on the task) and make proper judgments (feedback on their decisions) was positively correlated with a higher ratio in the number of fixations in the TROI.