INTRODUCTION

Distal radius fractures (DRFs) are one of the most commonly occurring fractures and account for 15–20% of all fractures. The DRF is typically diagnosed on radiographs taken in the acute setting.[1] Treatment of a DRF is, in part, based on quantification of fracture displacement as measured in radiographs.[2,3] Improper patient positioning during the radiographic procedure can affect the measured values of fracture displacement; in other words, correct patient positioning is directly linked to diagnostic value.[4,5] Correct radiographic positioning is, however, a skill that requires practice and knowledge.[6]

Until recently, radiographic positioning has typically been taught in the classroom or trained using phantoms. With the advent of new technologies such as virtual reality (VR), students of radiography are given the opportunity to practice and simulate musculoskeletal radiography acquisition skills using a VR headset and controllers while in a safe and limitless environment with neither radiation nor involvement of patients.[7] The risk of radiation makes it challenging for radiographers to train radiographic positioning involving patients; this risk is eliminated using VR. Another challenge is that it may be time consuming to get real-time feedback on the radiographs from an expert. A recent study involving 1st year students of radiography showed positive correlation between VR simulation and their subsequent clinical performance across a range of parameters.[8]

The virtual three-dimensional VR universe has also been used in educational contexts within other health domains.[9-12] Studies have already explored the effect of VR learning in the areas of ultrasound, anatomy, and surgery and have found significant positive effects although further research is suggested in previous studies.[11,13]

The use of VR has also been seen within the field of X-ray training where students of radiography reported a positive attitude toward training procedures in VR. The biggest challenge they faced was lack of tactile feedback when palpating anatomical landmarks during positioning. Moreover, a general lack of interaction and communication with the patient was missing as well as feedback on positioning from the system. The students mentioned that they would like feedback on their radiographs so they could learn how to improve their positioning and improve diagnostic quality of the images.[7]

There is, to the best of our knowledge, a relatively limited amount of literature on the topic - especially on students training in a virtual simulation where the focus is positioning (rotation, deviation, and flexion/extension) of the wrist involving direct feedback and information based on their own radiographs as taken in the VR simulator. Accordingly, the primary aim of this study was to conduct a pilot study and evaluate the effect of an immersive VR simulator for training musculoskeletal X-ray of the wrist. The objectives of this study were to explore: (i) Self-perceived clinical readiness (SPCR) before and after training in the VR simulator and (ii) VR system usability.

MATERIAL AND METHODS Study design

A single group study design was used to assess feasibility of training acquisition of wrist radiographs using a VR-based simulation. The study was approved by the Local Committee for Ethics in Research (RA2008003). All students of radiography, currently attending University College, i.e., not in clinical placement were invited to participate. Ten students of radiography at varying stages of their education were recruited voluntarily. All participants were given a unique ID and neither name nor any other person-sensitive data were collected. The tests took place over a 3-day period in April 2022. The participants were informed that they could withdraw from the study at any time without having to provide a reason. They were also informed that they could take a break during the simulation if needed. An oral introduction was prepared to ensure that all participants received the same information.

Data collection

Data were collected using moderated usability testing, a real-time test with the presence of a trained facilitator with an in-depth knowledge of the VR system.[14] Each test had a duration of approximately 1 h and consisted of three questionnaires, VR introduction videos, and the acquisition of wrist radiographs in the VR simulator. All tests were performed at a classroom at University College of Radiography.

Questionnaires

Before the VR simulation, the participants answered two questionnaires - an initial questionnaire on demographics, VR- and/or gaming experience, and with wrist X-rays and next, a questionnaire on SPCR in relation to obtaining wrist radiographs was answered. The SPCR questionnaire was answered twice, a pre-SPCR before entering the VR simulator and a post-SPCR after completing the VR simulation. The SPCR questionnaire contained ten statements on the participant’s self-perceived ability when obtaining wrist X-rays using a five-point Likert scale (strongly disagree, disagree, neutral, agree, and strongly agree) with statements such as “I feel insecure on positioning the wrist of the patient when performing musculoskeletal X-ray” [Appendix 1]. The same SPCR questionnaire was answered after completion of the VR simulation. Finally, a modified questionnaire on system usability, system usability scale (SUS),[15,16] was answered by all participants containing statements such as “I felt very confident using the system.” The SUS was expanded with 2 statements (1) “I think the X-ray room was realistic” and (2) “I think the simulator can contribute with learning to the education of radiography,” totaling 2 questions [Appendix 2].

The SUS scores were calculated using the method suggested by Alathas[16] by adding the scores (from 1 = strongly disagree to 5 = strongly agree) of all the odd-numbered questions and subtracting 5 from the total to achieve a scoreodd. The total of all even-numbered questions was subtracted from 25 and this sum was multiplied by 2.5 to achieve the scoreeven. The sum of scoreodd and scoreeven was the final total SUS score. Thus, a SUS score from 0 to 100 could be achieved, with higher SUS scores indicating higher usability.[16] The SPCR score was calculated using the same method as described for the SUS scores, therefore the higher the SPCR-score the better, with 100 being the maximum score obtainable.

VR simulator

The VR simulation was developed in collaboration with a company specializing in VR software for medical education (VitaSim ApS, Odense, Denmark). In the VR simulator, a short immersive VR tutorial was given to the participants, before positioning the patient and taking posterior-anterior (PA) and lateral wrist radiographs. The tutorial consisted of an oral introduction to the VR simulator and the basic VR interactions. Hereafter, the students were led into the X-ray simulation room where an avatar introduced the tasks in the room, i.e., positioning the virtual patient, taking the wrist radiographs and how they could get feedback on radiographic positioning. In Figure 1, the test flow is presented.

Figure 1:: Flowchart of the virtual reality simulation test.

Export to PPT

The virtual patient had the following anatomical movement of the shoulder (abduction, adduction, and rotation), elbow (flexion and extension), and wrist (supination, pronation, flexion, extension, radial, and ulnar deviation). Moreover, the participants had the opportunity to remove the skin to expose the bones. Figures 2 and 3, respectively, depict the X-ray room as seen in the VR simulator and the VR patient with the bones exposed.

Figure 2:: The virtual X-ray room with the patient seated and positioned for a wrist radiograph.

Export to PPT

Figure 3:: The patient is positioned for a posterior-anterior wrist radiograph and the bones have been exposed to help with correct positioning for a retake.

Export to PPT

After positioning the patient and collimating the image, PA and lateral wrist radiographs were obtained of the VR patient, the participants were shown the corresponding radiographs and given immediate feedback on positioning, i.e., deviation, flexion, extension, and rotation. The feedback was visually depicted on the wall behind the patient. The user could choose between feedback following each single radiograph or after acquisition of both PA and lateral radiographs. An avatar appeared in the X-ray room and gave further information on how to interpret the feedback as well as theoretical background knowledge on positioning criteria of PA and lateral wrist radiographs. Following the feedback, the participants could reposition the patient and obtain as many retakes as needed. Updated feedback was available following each new wrist radiograph.

Technical VR setup

An Acer Predator Helios 300 with RTX 3070 GPU and 16 GB RAM (Compal Electronics Inc., Taiwan) was used for developing and running the VR simulator tethered. An Oculus Quest headset (Meta Technologies, Meta Inc., Menlo Park, CA, USA) was used for immersive VR.

Data analysis

Demographics of participants were presented descriptively with mean and range. The SUS scores were presented by mean and standard deviation (SD). The modified questions on simulator realism and educational usability were reported in percentage.

The SPCR score was presented by mean and SD with corresponding 95% confidence intervals (CIs). Assuming normality of data, a student’s paired t-test was applied to assess differences between pre- and post-SPCR scores. Results were visually depicted in a box plot. Stata version 16 (StataCorp. 2019, TX) was used for statistical analyses.

RESULTS

Ten participants (6 females and 4 males) with a mean age of 23.5 years (range: 20–27) completed the VR simulation. The students were at different stages of their education (1st semester n = 1; 3rd semester n = 5; 5th semester n = 4). The 3rd and 5th semester students had educational and practical experience with musculoskeletal radiographs of the wrist. Demographics on experience with video games, VR, and wrist X-rays are shown in Figure 4.

Figure 4:: Demographics. n = 10 (6 female; 4 male).

Export to PPT

Based on subjective assessment of how confident the participants felt obtaining a radiograph of the wrist, the mean pre-SPCR score was 75 (95% CI: 54–96) and SD: 29.8. Average post-SPCR score was 77 (95% CI: 59–95), SD (25) [Figure 5]. Thus, no statistically significant differences were shown between the pre- and post-SPCR score, P = 0.4574 [Figure 6].

Figure 5:: Depicting individual pre- and post-self-perceived clinical readiness (SPCR) scores (n = 10). Cases with only one red dot signify equal pre- and post-SPCR scores, as seen in participants 1, 5, and 10.

Export to PPT

Figure 6:: Box plot with medians, quartiles, and ranges comparing the pre- and post-self-perceived clinical readiness scores.

Export to PPT

The average SUS score was 77.5 (SD: 19.1) out of 100. Most of the participants, 90% (n = 9), strongly agreed that the X-ray room in the VR simulator was realistic whereas one participant was neutral on this question. Regarding learning potential, 90% (n = 9) of the participants strongly agreed that the VR simulator could contribute to learning at the radiographer education.

DISCUSSION

This study piloted feasibility of using a VR simulator as an educational tool when training correct positioning of wrist radiographs with specific objectives of exploring: (i) SPCR before and after training in the VR simulator and (ii) system usability. In the simulator, feedback was offered based on patient positioning along with educational pointers on wrist radiographs and positioning regarding anatomical landmarks and how to assess anatomical positioning of the wrist and hand in a radiograph.

Despite positive feedback on the use of the simulator, the SPCR score was improved only by an average of 2 points after the completion of the VR sessions. This finding could indicate what psychologists Kruger and Dunning described in 1999 – the Dunning–Kruger effect. They described after four experiments that students who objectively performed well often subjectively underestimated their own performances.[17] On the other hand, students that performed objectively poor overestimated their own performances.[17] Considering that all participants in the current study were students, they had a rather high SPCR regarding obtaining wrist radiographs. This may in part be because of the relatively low esteem of musculoskeletal radiography. However, while the radiography skills and competencies have not altered overtime, the importance of accuracy and recognition of the skill may have become downgraded by some.[6] Aforementioned Dunning–Kruger[17] effect may also explain the student’s trust in own competences, or perhaps, more likely, a combination of the two.

As technologies and imaging modalities have developed, musculoskeletal radiography has been increasingly seen as less complex compared to magnetic resonance imaging and computed tomography.[6] A hypothesis could be that the students overestimated their own ability to obtain wrist radiographs assuming that it is a common and therefore simple procedure. In a recent rejection analysis study, it was, however, uncovered that improper positioning was one of the most common reasons for retake of digital musculoskeletal radiographs.[18] Hence, in a profession such as radiography, it is critical that clinical competency assessments are part of the curriculum both as formative assessments for identifying the strengths and weaknesses and for highlighting deficiencies, or as summative pass/fail tests, e.g., for high-risk or high-stake procedures. By implementing tests, trainees, in this case, radiography students can train the procedure repeatedly until sufficient proficiency. Some students might need more training, guidance, and supervision than others due to, e.g., different previous experiences and learning paces. This principle is called mastery learning and is a frequent and well-described method for health-care education.[19] VR supports the principles of mastery learning, because it is possible to incorporate assessment tools within the virtual simulation. It is possible to make the training and learning period more flexible to the students and decrease the number of instructors needed compared to, e.g., classroom-based education and training. Mastery learning, however, requires clear learning objectives, a competence level to meet or exceed, and the opportunity for continuous feedback. To our knowledge, no studies have been published presenting VR tests for assessing radiography positioning skills but solid validity evidence have been published for VR tests in ultrasound and for preparing students of radiography for clinical practice.[8,13]

Particularly, regarding positioning and musculoskeletal radiography, even small degrees of malpositioning can influence the diagnostic value derived from the radiograph. In the case of wrist radiographs, forearm rotation away from the mid-prone position can depict the apparent angulation of the articular surface of the radius differently as seen in the lateral image. Studies have shown that forearm supination can decrease the radiographically measured value of dorsal angulation and conversely, pronation may increase the apparent dorsal angulation.[5,20,21] Patient positioning is also important in other anatomical regions, such as the pelvis, where pelvic rotation during the radiographic procedure will impact the appearance of radiographic signs of acetabular retroversion such as the crossover sign.[22] Hence, the value of VR training and positioning in musculoskeletal radiography is important in, but not limited to, wrist radiographs.

Limitations

An inherent limitation in the current study is the low number of participants (n = 10) combined with the fact that the participants were at various stages of their education. A larger and more homogenous cohort of participants might have changed the results. The path from educational research to clinical outcomes is long with multiple factors and variables affecting the pathway. The first step could be to investigate the clinical effect of the VR training, e.g., exploring the transfer to a clinical setting by comparing the image quality (positioning) of radiographers or students who have trained using VR compared to those who have trained using normal methods. An alternative approach could be to gather validity evidence for a VR-based simulation test in radiography competences such as the wrist X-ray. Moreover, this VR simulator allows for training in patient positioning only, and thus, factors such as radiation protection, radiation dose, and subsequent technical image quality are not assessed in this study.

CONCLUSION

This pilot study showed no difference in SPCR before and after the use of VR. The feasibility of using VR simulation for training correct positioning of wrist radiographs was identified though with positive feedback from the participants. Additional studies are recommended to further elaborate on this initial research.