What is the role of double-marking? Evidence from an undergraduate medical course

Authors: Smith, B., Sinclair, H., Simpson, J., Van Teijlingen, E., Bond, C. and Taylor, R.

Journal: Education for Primary Care

Volume: 13

Issue: 4

Pages: 497-503

ISSN: 1473-9879

Abstract:

Double-marking has considerable resource implications, but its value and reliability are uncertain. This paper reports a study aiming to assess the reliability and value of double-marking in an essay assessment, by measuring the level of agreement between markers and the number of grades altered by the second marker. Essays were submitted by third-year medical undergraduates on the Community Course at the University of Aberdeen, Scotland, during 1998/99 and 1999/2000, and marked by general practitioners and university academics. Marks were awarded on a Common Assessment Scale, in five bands, with two markers for each essay. Agreement between these marks was compared, and measured using the weighted kappa statistic. In 2000, 68/171 (39.8%) papers were awarded the same band from both markers, and in 1998/99, 25/68 (36.8%) papers were awarded the same band. In both years, the weighted kappa statistic was low (0.12), indicating poor agreement between markers. The reliability of double-marking is not supported by this study. However, the value in reducing marker error and subjectivity is supported. The question remains on how to reconcile disagreement between markers.

Source: Scopus