Reconsidering Counting Articles in Ranked Venues (CARV) as the Appropriate Evaluation Criteria for the Advancement of Democratic Discourse in the IS Field

For a research field to advance, scholars must be able to openly exchange ideas. For this open exchange to exist, the contexts and methods that evaluate scholarly output must encourage this interchange. We argue that the current process for evaluating scholarly output, “counting articles in ranked venues” (CARV), creates pressures that result in a distorted discourse inhibiting the growth of the field. We review the current system of evaluating scholarly output and describe its virtues and shortcomings. Then, based on works by Habermas (1984) and Mingers and Walsham (2010), we suggest that the IS field should adopt an improved method of evaluation that enhances the openness of ideas. This method should be objective, reproducible, relatively easy to compute, and standardized. The new method should also produce easy-to-compute profiles of measures that assess a construct for scholarly output founded on a well-defined theory. It should also reduce the dependence on publishing in any particular journal. The scholarly capital model (SCM) represents one such method. We believe that wide adoption of the SCM would open the discourse and help the field to develop more rapidly.