Augmenting Embodied Learning in Welding Training: The Co-Design of an XR- and tinyML-Enabled Welding System for Creative Arts and Manufacturing Training

Abstract

Metal welding is a craft manufacturing skill that can be unusually difficult to externalize and represent to novices. Building competency requires an apprentice to iteratively practice embodied skills and sensitize themselves to a sensorially complex practice. To explore these challenges, we organized a series of co-design workshops with a youth program in welding and fabrication. Working with eight instructors and four students, we identified opportunities for mixed reality, sensing, and tinyML processes to augment welding training and practice. This resulted in an extended reality (XR) welding helmet and torch that enhances the embodied learning of welding in three key ways: biometric sensing enhances mindfulness and stress management in sensorially challenging environments; acoustic sensing focuses learner attention on non-visual cues of weld performance; and combined motion-sensing and visual XR feedback helps improve proprioceptive and embodied learning. These features are assessed and we offer design implications for augmenting novice learning of craft-practice with XR approaches. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Keywords

Author Keywords

Augmented reality Craft 3D Interaction Embodied Interaction Prototyping Smart Objects and Environments Toolkits Ubiquitous Computing

Index Keywords

User interfaces Mixed reality Augmented reality Co-designs Arts computing welding Ubiquitous computing 3D interactions Craft Embodied interaction Prototyping Smart environment Smart objects Toolkit Welding systems Welding training

Author Affiliations

Carnegie Mellon University, Pittsburgh, PA, United States

Funding & Acknowledgements

Carnegie Mellon University, CMU

We would like to thank Tim Kaulen and our partner organization, the Industrial Arts Workshop, for their time and expertise. This work was funded by the Manufacturing Futures Institute at Carnegie Mellon University.

References 10 References

1 Contemporary Apprenticeship Reforms and Reconfigurations, (2019)

2 Vuforia Studio

3 Finke, Andrew, Lake: A digital wizard of Oz prototyping tool, Conference on Human Factors in Computing Systems - Proceedings, (2019)

4 Fisher, Robert, Still thinking: The case for meditation with children, Thinking Skills and Creativity, 1, 2, pp. 146-151, (2006)

5 Hu, Guoning, A tandem algorithm for pitch estimation and voiced speech segregation, IEEE Transactions on Audio, Speech and Language Processing, 18, 8, pp. 2067-2079, (2010)

6 Hiyama, Atsushi, Augmented reality system for measuring and learning tacit artisan skills, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8017 LNCS, PART 2, pp. 85-91, (2013)

7 Perception of the Environment Essays in Livelihood Dwelling and Skill, (2000)

8 Ipsita, Ananya, Towards Modeling of Virtual Reality Welding Simulators to Promote Accessible and Scalable Training, Conference on Human Factors in Computing Systems - Proceedings, (2022)

9 Karges, Nils, Soundscapes on edge - The real-time machine learning approach for measuring Soundscapes on resource-constrained devices, Proceedings of the International Congress on Acoustics, (2022)

10 Khng, Kiat Hui, A better state-of-mind: deep breathing reduces state anxiety and enhances test performance through regulating test cognitions in children, Cognition and Emotion, 31, 7, pp. 1502-1510, (2017)

Augmenting Embodied Learning in Welding Training: The Co-Design of an XR- and tinyML-Enabled Welding System for Creative Arts and Manufacturing Training

How to Cite