An Empirically Grounded Framework That Evaluates Argument Quality in Scientific and Social Contexts

This study was aimed to develop a general argumentation framework for evaluating the quality of causal arguments across scientific and social contexts. We designed a computer-delivered assessment that contains four scenario-based argumentation tasks. Each task asks students to identify relevant evidence from provided data sources and use the evidence to construct an argument that answers a causal question. One task is about a social issue, while the rest three tasks each requires knowledge of a scientific concept (melting/evaporation, photosynthesis, trophic cascade). The assessment was implemented with 349 students from urban middle and high schools. Based on the data and prior research, we developed an empirically grounded argumentation framework that contains four qualitatively different levels: non-causal arguments, causal arguments lacking logical connections, causal arguments with weak reasoning, and causal arguments with strong reasoning. The qualitative results provide evidence of the existence of the argumentation levels. The IRT analysis and the Wright map provide the evidence that the order of and the distinctions among the argumentation levels are meaningful. Together, the qualitative and quantitative results support the viability of the framework.