Measure for Measure: Calibrating Ten Commonly Used Calibration Scores

Gregory Schraw, Fred Kuch, Antonio P. Gutierrez

Research output: Contribution to journalArticlepeer-review

79 Scopus citations

Abstract

This study examined the dimensionality of 10 different calibration measures using confirmatory factor analysis (CFA). The 10 measures were representative of five interpretative families of measures used to assess monitoring accuracy based on a 2 (performance) × 2 (monitoring judgment) contingency table. We computed scores for each of the measures using a common data set and compared one-, two-, and five-factor CFA solutions. We predicted that the two-factor solution corresponding to measures of specificity and sensitivity used to assess diagnostic efficiency would provide the best solution. This hypothesis was confirmed, yielding two orthogonal factors that explained close to 100% of sample variance. The remaining eight measures were intercorrelated significantly with the sensitivity and specificity factors, which explained between 91 and 99 percent of variance in each measure. The two-factor solution was consistent with two different explanations, including the possibility that metacognitive monitoring may utilize two different types of processes that rely on separate judgments of correct and incorrect performance, or may be sufficiently complex that a single measurement statistic fails to capture all of the variance in the monitoring process. Our findings indicated that no single measure explains all the variance in monitoring judgments. Implications for future research are discussed.
Original languageAmerican English
JournalLearning and Instruction
Volume24
DOIs
StatePublished - Apr 2013

Keywords

  • Calibration
  • Metacognition
  • Monitoring
  • Two-factor model

DC Disciplines

  • Educational Methods
  • Curriculum and Social Inquiry
  • Curriculum and Instruction
  • Educational Assessment, Evaluation, and Research

Fingerprint

Dive into the research topics of 'Measure for Measure: Calibrating Ten Commonly Used Calibration Scores'. Together they form a unique fingerprint.

Cite this