Article 2013

Measuring nursing competencies in the operating theatre: Instrument development and psychometric analysis using Item Response Theory

Nurse Education Today
Journal · Vol. 33 · Issue 9 · pp. 1088-1093
Abstract

Background: Concern about the process of identifying underlying competencies that contribute to effective nursing performance has been debated with a lack of consensus surrounding an approved measurement instrument for assessing clinical performance. Although a number of methodologies are noted in the development of competency-based assessment measures, these studies are not without criticism. Research aim: The primary aim of the study was to develop and validate a Performance Based Scoring Rubric, which included both analytical and holistic scales. The aim included examining the validity and reliability of the rubric, which was designed to measure clinical competencies in the operating theatre. Research method: The fieldwork observations of 32 nurse educators and preceptors assessing the performance of 95 instrument nurses in the operating theatre were used in the calibration of the rubric. The Rasch model, a particular model among Item Response Models, was used in the calibration of each item in the rubric in an attempt at improving the measurement properties of the scale. This is done by establishing the 'fit' of the data to the conditions demanded by the Rasch model. Results: Acceptable reliability estimates, specifically a high Cronbach's alpha reliability coefficient (0.940), as well as empirical support for construct and criterion validity for the rubric were achieved. Calibration of the Performance Based Scoring Rubric using Rasch model revealed that the fit statistics for most items were acceptable. Conclusion: The use of the Rasch model offers a number of features in developing and refining healthcare competency-based assessments, improving confidence in measuring clinical performance. The Rasch model was shown to be useful in developing and validating a competency-based assessment for measuring the competence of the instrument nurse in the operating theatre with implications for use in other areas of nursing practice. © 2012.

Keywords

Author Keywords

Competence Rasch Model item response theory Clinical assessment Competency-based assessment Rubric

Index Keywords

clinical competence human procedures Humans psychometry education Educational Measurement theoretical model reproducibility validation study Reproducibility of Results Psychometrics Models, Theoretical standards operating room personnel Operating Room Nursing
Author Affiliations
Melbourne School of Health Sciences, Melbourne, VIC, Australia
Assessment Research Centre, Melbourne, VIC, Australia
Work-based Education Research Centre, Victoria University, Melbourne, VIC, Australia
Deakin University, Geelong, VIC, Australia
Funding & Acknowledgements
No funding information
References 10 References
1 Introduction to Educational Testing, (1997)
2 Australian College of Operating Room Nurses Competency Standards, (2006)
3 Acorn Standards for Perioperative Nursing, (2008)
4 Theoretical Models and the Assessment of Competency, (1993)
5 Professional Judgement in Vocational Educational and Training A Set of Resources, (2006)
6 Beck, Cheryl Tatano, Item Response Theory in Affective Instrument Development: An Illustration, Journal of Nursing Measurement, 9, 1, pp. 5-22, (2001)
7 American Journal of Pharmaceutical Education, (1995)
8 From Novice to Expert Excellence and Power in Clinical Nursing Practice, (1984)
9 Applying the Rasch Model Fundamental Measurement in the Human Sciences, (2001)
10 Bondy, Kathleen N., Criterion-referenced definitions for rating scales in clinical evaluation., The Journal of nursing education, 22, 9, pp. 376-382, (1983)
Quick Actions
Full Text via DOI
Citation Metrics
15
Times Cited (Scopus)

References 10
Document Identifiers