Review 2012

Quantitative content analysis procedures to analyse students' reflective essays: A methodological review of psychometric and edumetric aspects

PE
Poldner, Eric
 SP
Simons, P. Robert Jan
 WG
Wijngaards, Guus
 VD
van der Schaaf, Marieke F.(Marieke)
Educational Research Review
Journal · Vol. 7 · Issue 1 · pp. 19-37
Abstract

Reflective essays are a common way to develop higher education students' reflection ability. Researchers frequently analyse reflective essays based on quantitative content analysis procedures (QCA). However, the quality criteria that should be met in QCA are not straightforward. This article aims to: (1) develop a framework of quality requirements for QCA; and (2) explore the extent to which QCA studies of students' reflective essays meet these quality requirements. First, a methodological review of psychometric and edumetric criteria for QCA is conducted, resulting in a framework with required quality criteria. Second, 18 studies were selected in which QCA was used to analyse students' reflective essays. These studies were scrutinised using the developed framework. The results showed that none of the selected studies met all requirements. It is concluded that QCA procedures for analysing students' reflective essays need improvement and that the developed framework can be used to improve and evaluate these procedures. © 2011 Elsevier Ltd.

Keywords

Author Keywords

validity Reflection Psychometrics Content analysis procedure Edumetrics

Index Keywords

Author Affiliations
Department of Education, Universiteit Utrecht, Utrecht, Netherlands
Centre for ELearning, Hogeschool INHOLLAND, Haarlem, Netherlands
Funding & Acknowledgements
No funding information
References 10 References
1 Abou Baker El-Dib, Mervat, Levels of reflection in action research. An overview and an assessment tool, Teaching and Teacher Education, 23, 1, pp. 24-35, (2007)
2 undefined, (2008)
3 Baartman, Liesbeth K.J., The wheel of competency assessment: Presenting quality criteria for competency assessment programs, Studies in Educational Evaluation, 32, 2, pp. 153-170, (2006)
4 Baartman, Liesbeth K.J., Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks, Educational Research Review, 2, 2, pp. 114-129, (2007)
5 Bain, John D., Using journal writing to enhance student teachers' reflectivity during field experience placements, Teachers and Teaching: Theory and Practice, 5, 1, pp. 51-73, (1999)
6 undefined, (2006)
7 Teaching and Learning in Higher Education New Trends and Innovations, (2003)
8 Birenbaum, Menucha, EVALUATING THE ASSESSMENT: SOURCES OF EVIDENCE FOR QUALITY ASSURANCE, Studies in Educational Evaluation, 33, 1, pp. 29-49, (2007)
9 Borsboom, Denny, The concept of validity, Psychological Review, 111, 4, pp. 1061-1071, (2004)
10 Boud, David J., Assessment and the Promotion of Academic Values, Studies in Higher Education, 15, 1, pp. 101-111, (1990)
Quick Actions
Full Text via DOI
Citation Metrics
30
Times Cited (Scopus)
References 10
Document Identifiers