The Impact of Criterion® on Error Reduction: A Longitudinal Study

Previous studies on automated writing evaluation (AWE) were mostly based on short-term error reduction (Chodorow, Gamon, & Tetreault, 2010), and they did not fully reveal the extent to which AWEs can facilitate students error correction and progress in grammatical accuracy on a long-term basis. Therefore, this study aims to examine the potential effects of immediate feedback from a piece of AWE software, Criterion®, on students success and difficulties in error correction and changes in their correction behavior across one academic semester. As part of a larger research project on the instructional use of Criterion® in ESL academic writing, writing samples were collected at the beginning and end of a semester to investigate corrections for article errors. Articles are known as one of the most difficult elements of English for non-native speakers to master (Dalgish, 1985; Diab, 1997; Izumi et al., 2003; Bitchener et al., 2005) and thus the focus of the study. The article errors identified by Criterion® and associated error feedback were analyzed from the two aspects. First, types of Criterion® feedback were classified based on how accurately the feedback identifies an article error and how specifically it advises the students to make corrections. Second, types of students error correction on these highlighted errors were explored per feedback type, and different patterns of success, failure, or neglect in correction were identified in each feedback type. It is expected that the findings would reveal more insights into the potential of using AWEs in L2 writing classroom.