Article 2010

Facilitating evaluations of innovative, competence-based assessments: Creating understanding and involving multiple stakeholders

Evaluation and Program Planning
Journal · Vol. 33 · Issue 2 · pp. 120-127
Abstract

Schools are held more responsible for evaluating, quality assuring and improving their student assessments. Teachers' lack of understanding of new, competence-based assessments as well as the lack of key stakeholders' involvement, hamper effective and efficient self-evaluations by teachers of innovative, competence-based assessments (CBAs). While evaluating two CBAs in Agricultural Vocational Education and Training institutions, two interventions in the evaluation process aimed to tackle these problems were examined: (1) starting with explicating the CBA in the teacher team using a concrete explication format and (2) qualitatively involving key stakeholders (i.e., teachers, students and employers) in the evaluation of the CBA through mixed-group interviews. Quantitative and qualitative analysis, as well as stakeholders' perceptions are used to find indications for the added value of these interventions for evaluation and further improvement of the CBAs. Results show that external facilitation is needed to make both interventions work. However, under this condition, explicating the CBA led to more complete, concrete and shared understandings of the actual CBA among teachers and mixed-groups interviews resulted in more concrete and elaborate evaluations of the CBAs' quality and more ideas for improvement. Both interventions can facilitate building up elaborate, more valid and concrete arguments for CBA quality in self-evaluations, certainly in the case of evaluating innovative assessments. Lessons learned will provide guidelines for incorporating the interventions into other evaluations of innovative programs. © 2009 Elsevier Ltd. All rights reserved.

Keywords

Author Keywords

Evaluation Assessment Quality assurance Multiple stakeholders Teacher professionalization

Index Keywords

learning adult female human male Humans adolescent Article interview Quality control Students Vocational education curriculum student university Interviews as Topic middle aged program evaluation qualitative analysis innovation perception methodology Faculty competency-based education stakeholder standard quantitative analysis Netherlands health care quality
Author Affiliations
Wageningen University & Research, Wageningen, Gelderland, Netherlands
Department of Educational Sciences, Universiteit Utrecht, Utrecht, Netherlands
Funding & Acknowledgements
No funding information
References 10 References
1 Standards for Educational and Psychological Testing, (1999)
2 Baartman, Liesbeth K.J., Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks, Educational Research Review, 2, 2, pp. 114-129, (2007)
3 Baartman, Liesbeth K.J., Teachers' opinions on quality criteria for Competency Assessment Programs, Teaching and Teacher Education, 23, 6, pp. 857-867, (2007)
4 Baartman, Liesbeth K.J., DETERMINING THE QUALITY OF COMPETENCE ASSESSMENT PROGRAMS: A SELF-EVALUATION PROCEDURE, Studies in Educational Evaluation, 33, 3-4, pp. 258-281, (2007)
5 Educational Researcher, (2007)
6 Beers, Pieter J., Common ground, complex problems and decision making, Group Decision and Negotiation, 15, 6, pp. 529-556, (2006)
7 Biemans, Harm J.A., Competence-based vet in the netherlands: Background and pitfalls, Journal of Vocational Education and Training, 56, 4, pp. 523-538, (2004)
8 Birenbaum, Menucha, EVALUATING THE ASSESSMENT: SOURCES OF EVIDENCE FOR QUALITY ASSURANCE, Studies in Educational Evaluation, 33, 1, pp. 29-49, (2007)
9 Birenbaum, Menucha, A learning Integrated Assessment System, Educational Research Review, 1, 1, pp. 61-67, (2006)
10 Coates, Hamish B., The value of student engagement for higher education quality assurance, Quality in Higher Education, 11, 1, pp. 25-36, (2005)
Quick Actions
Full Text via DOI
Citation Metrics
19
Times Cited (Scopus)

References 10
Document Identifiers