Article Gold Open Access 2021

Practical validation framework for competence measurement in VET: a validation study of an instrument for measuring basic commercial knowledge and skills in Switzerland

Empirical Research in Vocational Education and Training
Journal · Vol. 13 · Issue 1 · Art. 18
Abstract

Competence development and measurement are of great interest to vocational education and training (VET). Although there are many instruments available for measuring competence in diverse settings, in many cases, the completed steps of validation are neither documented nor made transparent in a comprehensible manner. Understanding what an instrument actually measures is extremely important, inter alia, for evaluating test results, for conducting replication studies and for enforcing adaptation intentions. Therefore, more thorough and qualitative validation studies are required. This paper presents an approach to facilitate validation studies using the example of the simuLINCA test. The approach to validation applied in this study was developed in the field of medicine; nevertheless, it provides a promising means of assessing the validity of (computer-based) instruments in VET. We present the approach in detail along a newly developed computer-based simulation (simuLINCA) that measures basic commercial knowledge and skills of apprentices in Switzerland. The strength of the presented approach is that it provides practical guidelines that help perform the measurement process and support an increase in transparency. Still, it is flexible enough to allow different concepts to test development and validity. The approach applied proved to be practicable for VET and the measurement of occupational competence. After extending and slightly modifying the approach, a practical validation framework, including the description of each step and questions to support the application of it, is available for the VET context. The computer-based test instrument, simuLINCA, provides insights into how a computer-based test for measuring competence in various occupational fields can be developed and validated. SimuLINCA showed satisfying evidence for a valid measurement instrument. It could, however, be further developed, revised and extended. © 2021, The Author(s).

Keywords

Author Keywords

Validation Competence measurement Commercial apprenticeship Computer-based simulation Validation framework

Index Keywords

Author Affiliations
Institute of Education, Universität Zürich, Zurich, ZH, Switzerland
University of Teacher Education Lucerne, Luzern, Lucerne, Switzerland
Funding & Acknowledgements
Singapore Eye Research Institute, SERI
Funding text 1: This research has been funded by the Swiss State Secretariat for Education, Research and Innovation (SERI) through its Leading House ‘Learning and Instruction for Commercial Apprentices’ (LINCA). The funding body had no influence on the design of the study, collection, analysis, and interpretation of data and in writing the manuscript.; Funding text 2: We are grateful for the valuable feedback and support from Prof. Dr. Doreen Holtsch and Prof. Dr. Franz Eberle. We would like to thank the SERI for funding.
Staatssekretariat für Bildung, Forschung und Innovation, SBFI
Funding text 1: This research has been funded by the Swiss State Secretariat for Education, Research and Innovation (SERI) through its Leading House ‘Learning and Instruction for Commercial Apprentices’ (LINCA). The funding body had no influence on the design of the study, collection, analysis, and interpretation of data and in writing the manuscript.; Funding text 2: We are grateful for the valuable feedback and support from Prof. Dr. Doreen Holtsch and Prof. Dr. Franz Eberle. We would like to thank the SERI for funding.
References 10 References
1 Konstruktvaliditat Von Simulationsaufgaben Computergestutzte Messung Berufsfachlicher Kompetenzam Beispiel Der Ausbildung Von Industriekaufleuten Bericht an Das Bundesministerium Fur Bildung Und Forschung K350600, (2009)
2 Multidimensional Models in Conquest Tutorial, (2010)
3 Standards for Educational and Psychological Testing, (1999)
4 Standards for Educational and Psychological Testing, (1985)
5 Annual Review of Psychology, (1986)
6 Asparouhov, Tihomir, Sampling weights in latent variable modeling, Structural Equation Modeling, 12, 3, pp. 411-434, (2005)
7 the Development of Professional Skills of Plant Mechanics in the First Year of Vet, (2016)
8 Bildungsplan Kauffrau Kaufmann Efz Vom 26 September 2011 Fur Die Betrieblich Organisierte Grundbildung Stand Am 1 Mai 2017 Training Plan for Commercial Apprentices for in Company Training, (2017)
9 Blömeke, Sigrid, Beyond dichotomies: Competence viewed as a continuum, Zeitschrift fur Psychologie / Journal of Psychology, 223, 1, pp. 3-13, (2015)
10 Taxonomy of Educational Objectives, (1956)
Quick Actions
Full Text via DOI
Citation Metrics
14
Times Cited (Scopus)

References 10
Document Identifiers