Remember me
Major depressive disorder (MDD) is the leading cause of disability worldwide []. Current treatment regimens, while effective in improving core symptoms of MDD, fall short in addressing MDD-related disability [-]. Cognitive dysfunction—encompassing deficits in memory, processing speed, attention, and executive functioning—has emerged as an important determinant of functional outcomes in MDD [-]. Advancing cognitive-targeting therapies such as cognitive remediation has accordingly become a critical treatment priority. Cognitive remediation is a behavioral-based therapy aimed at enhancing neurocognitive skills, typically through a combination of “drill-and-practice” cognitive games and implementation of compensatory strategies []. Traditional cognitive remediation programs delivered in paper-and-pencil or computerized formats have demonstrated robust improvements in both cognition and real-world functioning in schizophrenia [], attention-deficit/hyperactivity disorder, and autism spectrum disorder []. In MDD, results to date have been equivocal. Although cognitive remediation in patients with MDD appears to improve performance in select cognitive domains, this has not translated to appreciable improvements in real-world functioning [,].
Immersive virtual reality (VR) has emerged as a promising modality to enhance the real-world effectiveness of cognitive remediation. Such technologies allow for almost complete sensory immersion with extensive design possibilities. By creating simulations that engage users via multiple senses (sight, hearing, and touch) while permitting natural movement, immersive VR creates a sense of presence that can elicit a genuine response in the individual. The use of VR environments also permits tight experimental control, which makes it feasible to measure and study everyday functioning that can be otherwise prohibitive in real-world settings. The ability of VR to deliver and control stimuli while capturing responses with high fidelity during an exercise provides a controlled and repeatable tool that is unavailable in traditional testing or remediation methods and makes it ideal for assessing and training cognitive skills in the performance of simulated real-life tasks. There has been growing interest in the use of VR for both assessment and rehabilitation from the research and clinical neuropsychology communities in recent years [,]. The feasibility of using VR for cognitive assessment and care has been demonstrated across various cognitive domains, such as attention [-], executive functions [-], memory [-], and spatial abilities [-].
The National Research Council Canada (NRC) has developed bWell, a VR-based immersive platform for cognitive assessment and remediation []. bWell is a foundational platform featuring configurable exercises and designed as a broadly applicable toolkit targeting general aspects of cognition that are commonly impacted across many disorders. A video sample of bWell’s interface and laboratory exercise are presented in . Preliminary acceptability studies in healthy individuals have indicated that the platform is enjoyable, engaging, and well tolerated [,]. These studies have also shown that using immersive VR for clinical applications not only is technically feasible but also has advantages over traditional 2D equivalents. The first study showed that subjective reports of engagement when performing a task in bWell (an immersive environment) were greater than those when performing the same task on a tablet (a nonimmersive environment). The second study showed that the use of more complex types of immersion (eg, the user was not static) did not increase subjective reports of cybersickness during a visual attention task and provided the required support for testing in clinical populations.
bWell is currently being incorporated into clinical settings using an interdisciplinary approach successfully developed by the NRC [-]. This method, which has proven effective in creating surgical simulation platforms, most notably for neurosurgery (eg, NeuroTouch, distributed worldwide as NeuroVR by CAE Healthcare), ensures a comprehensive and collaborative framework for implementation. Specifically, the team established long-term collaborative research agreements with 5 world-renowned clinical sites across Canada and 1 in Japan—the NRC’s cognitive care network (CCN). The CCN consists of an early adopter group that provided domain-specific perspectives as well as feedback on iterative deliveries of the platform as bWell development progressed. The CCN is currently launching several studies investigating the content of bWell exercises, as well as its use, specific to target populations.
In one of these collaborations, with the University of British Columbia, we customized bWell to deficits observed in depression as bWell-D, a prototype of immersive VR cognitive remediation for MDD. bWell-D targets cognitive and real-world challenges germane to MDD depression []. It uses previously identified common components of successful cognitive remediation in psychiatric populations []. These include (1) errorless learning, wherein participants are provided support in learning all task components, with supports gradually removed as skills and confidence increase; and (2) adaptivity, where tasks become increasingly complex to match the participants’ growing competence.
In the development of a new digital technology, there is a need to determine the effective components and clinical relevance using systematic methodologies. A working group of international experts has published the first consensus guidelines for best practices in the design and testing of VR clinical applications []. In these guidelines, incorporation of qualitative feedback from multiple end users is identified as a critical component of VR program content development. In addition, the Medical Research Council (United Kingdom), in a guidance document on the development of complex interventions to improve health and health care [], recommends including prevailing theories and frameworks in a given domain to anchor the feedback obtained.
From an implementation perspective, the effectiveness of digital intervention delivery is challenged by low uptake and high noncompliance rates, which gamification may play a role in addressing. A systematic study of technology use in the treatment of mental health disorders found that, specific to depression, gamification was being used as an approach aiming to improve adherence and engagement []. The application of gamification to mental health domains is still a new research field with limited studies [] that tend to be heterogeneous []. The results are inconsistent; for example, it was observed that its application to web-based interventions did not lead to a significant increase in adherence []. There is also a lack of comparison between gamified and nongamified interventions, which makes it difficult to evaluate the impact of such features on adherence and engagement. Finally, it has been suggested that gamification might be used without fully understanding the behavioral theories or mechanisms behind it [,]. For instance, adding too many gamification elements that rely on external rewards can overwhelm the user with unnecessary cognitive demands, potentially invalidating the task [].
ObjectivesUsing behavioral theory as a framework for understanding likely processes for change induced by an intervention is hypothesized to enhance its effectiveness [,]. The capability, opportunity, and motivation–behavior (COM-B) model [] is one such theory that provides valuable insights. It is widely used and identifies capability, motivation, and opportunity as the 3 essential elements for all behavior. Effecting behavior change, such as improving compliance with a digital intervention, requires addressing at least one of these drivers of behavior. Another relevant behavioral theory is self-determination theory, which delves deeper into the aspects of motivation. This theory posits that people are intrinsically motivated to grow and change when their needs for autonomy, competence, and connection are met [].
In this work, we aimed to address these challenges through an iterative process for the development of bWell-D that involves co-design for the development of content and in the application of gamification using behavior change theories as a framework.
Following best practice guidelines, qualitative data were collected from patients and care providers to understand end-user perceptions on the use of VR technologies for cognitive remediation and obtain suggestions for changes specific to the bWell-D program. This qualitative study phase was reported in a previous paper from the authors []. Briefly, participants comprising individuals with lived experience of depression (n=15) and clinicians (n=12) were interviewed, and the results were transcribed and coded followed by thematic analysis. The demographic and clinical characteristics of these end users are described in our previous paper. Interviewees were asked questions to probe drivers of behavior change related to the use of VR (eg, perceptions and experiences with VR, desired outcomes from a workplace cognitive and functional remediation program, and perceived barriers). Participants were also asked for feedback specific to the bWell-D program itself, for which they were shown previously recorded video samples of the tasks and probed for potential relevance to daily life and how to improve the tasks to enhance engagement and real-world applicability. Emerging themes from patients and clinicians were identified and ranked based on prevalence.
The focus of this paper was to structure these previously collected qualitative data into a theoretically grounded model to guide software and functionality development for the next iterations of bWell-D. Specifically, we aimed to use these findings to implement features for the next iterations of bWell-D specific to its use in cognitive remediation while promoting the required motivation, engagement, and ease of use or access to obtain favorable outcomes. Following best practice guidelines for human-centered VR content development [], the themes suggested by end users to improve the bWell-D program were discussed among the research team and translated into representative software functionalities during team brainstorming sessions. It was found that the functionalities that could be implemented could also be described in many ways. In an effort to standardize the categorization, all functionalities were encoded against gamification elements based on the taxonomy by Cheng et al [], who describe 18 elements used in digital technologies for 17 mental health and well-being domains, including depressive disorders.
The gamification elements were then evaluated against behavioral theories to characterize each element in terms of (1) factors that drive behavior change and (2) potential to intrinsically promote user engagement. The COM-B factors and psychological needs of self-determination are presented in [,] to provide a clearer definition.
Taking into consideration the factors identified from behavioral theories (eg, opportunity), the ideation included both updating the concepts for each bWell-D exercise as well as the overall delivery and use of the bWell-D program itself. The modifications were assigned priority for implementation based on (1) how frequently these changes were suggested by end users and (2) whether these changes were feasible for development in a VR environment.
The steps of the methodology used are summarized in . End-user feedback was thematically grouped and developed into software functionalities. These functionalities were categorized using a taxonomy for digital technologies used in mental health and well-being applications. Furthermore, the functionalities were assessed using selected behavior change theories to view the distribution of needs identified by end users for platform improvement. The last step involves validating the implemented software functionalities through an iterative development process that gathers new feedback from end users for further refinement. Multiple rounds of iteration will help create a comprehensive overview of the functionalities that end users want and refine these implementations to ensure that they effectively meet users’ needs. This overview will highlight the range of engagement methods preferred by users organized according to a taxonomy of gamification elements for digital health applications. In addition, selected behavior change theories will provide insights into what motivates users to engage with and adhere to treatments.
Finally, in this study, new participants were recruited for a pilot feasibility trial with a healthy population. With the aim of evaluating the impact of the gamification elements, participants trialed the original version for 40 minutes, after which they were given the option to either try out the expanded features of the bWell-D platform for up to 15 minutes or embark on a training program for 8 sessions (occurring 2 times a week for 4 weeks). Feedback regarding tolerability and cybersickness was obtained using the Simulator Sickness Questionnaire (SSQ) [] before and after each session. User experience feedback was obtained using the User Engagement Scale (UES) [] and Game User Experience Satisfaction Scale (GUESS) [] after the initial assessment session and after the first, fourth, and last training sessions. Ad hoc comments made by the participants during the trials were also noted. For participants who chose to try out the expanded feature without attending the training sessions, feedback was obtained via a semistructured interview while performing the exercises.
Textbox 1. Categorization in terms of factors that drive behavior change and potential to intrinsically promote engagement.Categorization for effective behavior change
The capability, opportunity, and motivation–behavior model posits that there are 3 drivers for behavior change. (1) Capability is the ability to engage in an activity; (2) opportunity is factors that lie outside the individual that can make behavior possible, prompt it, or present a barrier to it; and (3) motivation is the brain processes that energize and direct behavior.Categorization for intrinsic engagement
According to self-determination theory, 3 needs must be met to achieve psychological (inner) growth. (1) Autonomy is when a person feels in control of their own behaviors and that their action will result in real change, it plays a major part in helping them feel self-determined; (2) competence is when a person feels equipped with the skills needed, they are more likely to take action; (3) connection or relatedness is a sense of belonging and attachment to others promotes the actualization of inherent potential. Social environments supporting these needs enhance one’s internal motivation.
Figure 1. Iterative software development process using end-user feedback, thematic segmentation of feedback, software functionality development, taxonomy assignment, assessment using selected behavior change theories, distribution review, and functionality validation for improvement. Ethical ConsiderationsThe research and studies were reviewed and approved by the institutional research ethics board at the University of British Columbia (H21-00028 and H20-00746) and the NRC (2020-122). Informed consent was obtained before the start of the study. Data were deidentified before any analysis. Participants were compensated for taking part with CAD $100 (US $70).
Results from the previously published qualitative study [] showed that, overall, patients and clinicians considered bWell-D to be interesting, acceptable, and potentially feasible and provided suggestions to enhance its applicability. Five main themes were identified from the patient interviews: (1) patients would prefer to undergo VR treatment at home, (2) the VR treatment should emulate real-life settings, (3) the VR intervention needs to be multisensory, (4) the VR treatment should be challenging, and (5) the VR treatment should be engaging. Three main themes were identified from the clinician interviews: (1) the VR treatment should be multi-domain, (2) the VR tasks should become more challenging over time, and (3) remote treatment would be more convenient for patients. In addition, end users suggested having a user-friendly interface and tutorials to have an opportunity to practice the tasks, which they considered especially important for at-home treatment. Participants also recommended including an additional VR activity that specifically targets their mood.
In this study, these themes were evaluated against the identified behavioral theories, and it was found that the themes encompassed all 3 COM-B drivers for behavior change. Regarding motivation, VR is seen as a potential, interesting, but underexplored treatment option for remediating cognitive deficits in depression. Many suggestions for bWell were related to better harnessing the affordances of VR to deliver multisensorial and multimodal exercises. Regarding capability, VR was seen as “the second-best thing to real life” as it allows for immersion in highly realistic settings. Some skepticism about effectiveness in the short and long term was also expressed, especially when the purpose and real-world applications of practiced skills were not made explicit. Regarding motivation, the gamified experience of VR adds fun, engaging, and immersive elements to the treatment. There is a need for a good balance in challenge (motivating but not too hard) for continued engagement. Regarding opportunity, there is a need for inclusivity and customizability in treatment delivery. Participants believed that the option for home-based treatment provided a sense of comfort and control. However, it was not the preference for everyone. Specifically, patients with limited space or suboptimal conditions for treatment at home would prefer having the option to receive treatment at a clinic if necessary. Concerns about equipment accessibility were also highlighted.
In terms of feedback specific to bWell-D, the main areas of improvement suggested by both clinicians and patients centered on (1) motivation—taking greater advantage of VR multisensorial affordances, (2) capability—the inclusion of more realistic VR environments, and (3) motivation—providing sufficient challenge for fun and engaging tasks.
It was observed that patient and clinician themes overlapped on several common topics, and thus, they were combined. Similarly, the feedback specific to bWell was related to that for the general use of VR. However, while the suggestions specific to bWell addressed 2 of the drivers—motivation and capability—opportunity was not captured. In an aim to capture all drivers for behavior change as well as all the major suggestions from end users, the following overall themes emerged as priorities for the development of bWell-D: (1) provide sufficient challenge over time for fun and engaging tasks (patient themes 4 and 5 and clinician theme 2), (2) take greater advantage of the potential of VR to create tasks that are more multi-domain and multisensorial (visual, auditory, and tactile stimuli; patient theme 3 and clinician theme 1), (3) include more representative VR elements (to reflect more realistic VR environments; patient theme 2), (4) increase autonomy, opportunity, or enablement with remote or in-home use (patient theme 1 and clinician theme 3), and (5) provide mood-focused features and tasks (participant recommendation).
The gamification elements for each exercise, including relevant original elements and the newly implemented modifications based on qualitative feedback, are summarized in the upcoming sections. Each element is classified according to gamification taxonomy and characterized according to the drivers of behavior change from the COM-B framework it relates to (motivation, capability, or opportunity), as well as the need from self-determination theory that it most addresses (autonomy, competence, or connection). Each element is also associated with overall themes summarized from the qualitative analysis.
General Interface, Tutorial, and bWell-D AdministrationThe general modifications to bWell-D are described in this section, and provides a summary of the categorization.
bWell-D has 3 distinct modes to optimize user engagement and outcomes. (1) Tutorial mode is for initiation with the virtual environment, providing a comprehensive walkthrough of the tasks to be undertaken. It ensures that users are comfortable and well prepared to navigate the tasks effectively. (2) Assessment mode serves as a diagnostic tool to evaluate the users’ abilities by monitoring their performance at a consistent level of difficulty. The primary objective is to pinpoint the users’ unique requirements and areas where they may need additional support or intervention. (3) Training mode is tailored to foster user growth. The training mode dynamically adjusts the difficulty level in response to individual performance. This personalized approach ensures that the tasks remain challenging yet achievable, promoting continuous skill development. Users receive instantaneous feedback on progress via level or score change notifications.
The modes can be configured and customized via a clinical interface as bWell-D is intended for administration by a facilitator.
On the basis of the emerging theme of increased opportunity with at-home use, a new user interface was implemented to support autonomous use. As a result, bWell-D’s hardware-agnostic core has now been leveraged to integrate stand-alone headsets such as the PICO Neo3 Eye and the Oculus Quest 2. Such headsets have come a long way, from clunky, tethered devices to performant and comfortable stand-alone units aimed at the mass market that enable seamless onboarding of nontechnical users new to VR.
Existing features of bWell did not include the emerging theme of representative real-world elements (). With the aim to make the purpose and real-world applications of practiced skills more explicit, bWell-D administration now includes “bridge exercises” as a supplement to the VR program. These exercises consist of asking participants to reflect on how they think the VR tasks will apply to their personal contexts to help them understand how practiced skills could be related to their real life.
Figure 2. General interface, tutorial, and program administration, including modifications made after end-user feedback. Checkmarks (✓) denote existing elements and plus signs (+) denote new elements that were added in the updated version of bWell-D. Gray shading indicates that the theme does not apply. VR: virtual reality. Egg SceneThe egg scene takes place in an office work environment. The user is tasked with scanning the environment for eggs, drawing on visual processing and sustained attention. They are then required to direct and hold their gaze on a located egg long enough to make it hatch. Modifications to the scene and a summary of their categorization are presented in , where it can be observed that only the theme of representative real-world elements has been addressed in the existing features (ie, creation of a simulated real-world environment).
In the feedback obtained, end users recommended including more real-world VR elements. To address this, the egg target was changed to new, more context-appropriate ones for the office-themed virtual environment. These included staplers, legal pads, and coffee mugs.
An in-game user interface was added to the player’s wrist during the egg scene to display important information, including the current score, target object, and target location. This type of interface takes advantage of the user’s natural arm movements, allowing them to easily access information or controls by looking at or interacting with their wrist, much like checking a smartwatch. It is designed to be intuitive and accessible without interrupting the immersive experience. This artificial assistance element addressed the need for autonomy.
Ambient, multisensorial distractors were added as options to the scene. Visual distractors include videos playing on computer screens, coworkers walking around the office, and fake eggs that resemble the real target eggs. Audio distractors include phone rings, coworkers talking, and a buzzing sound that emanates from randomized locations within the office. These distractors add artificialchallenge as well as addressing the mood theme by introducing an element of hot cognition.
A customization option was added in which users follow auditory rather than the usual visual instructions. This addresses the user’s motivation, providing an alternative sensorial interaction pathway to engage with the user.
An adaptive difficulty progression including multisensorial stimuli was developed for the scene. To address the user’s capability, difficulty adjusts based on user performance, making this a personalization element. Difficulty adjustment includes adjusting the number of targets, the reaction delay allowed to obtain bonus points, the time it takes to hatch an egg, and the number and frequency of audio distractors. The user’s performance is assessed based on how they respond to multiple stimuli: a visual (egg turns green), tactile (controller vibrates), or audio (clock ticking sound clip) stimulus.
The last modification to the scene was the addition of a preset difficulty progression. The user is given tasks, starting off simple at lower levels and becoming increasingly complex at higher levels. These tasks include finding a new target, going to a target location, and pressing a button sequence to obtain bonus points. Because these levels are chosen by the user or clinician rather than being adaptive, they are considered a customization element.
Figure 3. Egg exercise (targeting states of attention), including modifications made after end-user feedback. Checkmarks (✓) denote existing elements and plus signs (+) denote new elements that were added in the updated version of bWell-D. Laboratory SceneIn the laboratory scene, the user must complete 2 recipes simultaneously by pouring the correct ingredients at the correct times into 2 different tubs. The scene challenges the user’s executive function and attention across several dimensions, including divided, selective, and sustained attention. A supervisor avatar is present in the scene and adds artificialassistance by giving auditory feedback on the user’s performance and advice on how to improve.
Modifications to the scene and a summary of their categorization are presented in . On the basis of the identified theme of added challenge and the inclusion of more representative elements, a customization option was added in which the supervisor avatar gives the user auditory instructions rather than having the recipes visually presented on tablets. An aspect of hot cognition was also added by modifying the avatar behavior so that it can make distracting or irrelevant comments to the user.
In total, 2 modifications were made to the scene explicitly to add artificialchallenge. The first one simply added 3 new nonrelevant ingredient bottles as distractions while the user attempts to complete the recipes. The second modification was the introduction of new secondary ingredients to recipes. The user must first create the secondary ingredient by combining 2 primary ingredients in a beaker before pouring the beaker in the tub, adding complexity to the task. A chart showing possible ingredient combinations was added to the scene.
Figure 4. Laboratory exercise (targeting multitasking), including modifications made after end-user feedback. Checkmarks (✓) denote existing elements and plus signs (+) denote new elements that were added in the updated version of bWell-D. Mole SceneThe mole scene is a Whac-A-Mole variation in which the user has a colored hammer in each hand with which they are to hit matching colored cylinders (moles) that pop up in front of them. Hammer colors change unexpectedly from time to time. Hitting valid moles exercises the user’s impulse control with selective attention, visual processing, and executive functioning.
A score streak animation (visual reward) is used to provide level and progressfeedback, serving to motivate the user to maintain a fast pace as the difficulty increases. This occurs when multiple moles are correctly hit in sequence. In addition, to enable investigations of potential bias in MDD, the moles were modified to have a simple facial expression, either a smile or a frown. The program checks whether the particular facial expression influences the accuracy of the hits. In one mode, valid moles (ones with a color matching a hammer) are assigned a smiling expression, and invalid moles are frowning. In another mode, the expressions are randomized.
Modifications to the scene and a summary of their categorization are presented in . From , it can be observed that the existing features did not include the emerging theme of more multi-domain and multisensorial elements. The changes focused on exercise variants to address other domains. The multiple-hit mole customization variant was added to target mood. The multiple-hit mole is a variant taken from the bWell core battery of exercises designed to frustrate the user by requiring repeated hits for the mole to go down. In total, 2 other bWell core customization variants were added to the mole scene. The side moles and directional hit variants require additional physical effort or dexterity. Side moles appear to the side of the main table, requiring the user to turn their head and extend their arms further to hit them. In the directional hit variant, moles must be hit on the top of their head to score points. This is more challenging compared to the usual case in which moles can be hit from any angle and still score points. It was hoped that the additional physical engagement required by these variations would increase motivation for users to continue using the program.
Figure 5. Mole exercise (targeting response inhibition and cognitive control, including modifications made after end-user feedback). Checkmarks (✓) denote existing elements and plus signs (+) denote new elements that were added in the updated version of bWell-D. Theater SceneThe theater scene is a matching task for visual and short-term memory. The user is presented with a set of target shapes for a short period and asked to remember it. The target shapes are then hidden from view. After a set time, random shapes start falling into view, and the user must reproduce the initial series of shapes. The scene was modeled after a university classroom, with the teacher’s desk in front and the students’ desks laid out in a semicircle in front of it. This setting provides an ecological environment for the sample population, which were recruited on a university campus. The user is placed in front of the teacher, who is seated at their desk. A few student avatars who may be enabled or disabled are seated in the classroom behind the user. This arrangement was intended to be somewhat daunting. For additional distraction, the teacher in front can create noise by taking down notes, and the student avatars can make loud whispering noises. The arrangement and the distractions add a level of artificialchallenge for the user as well as hot cognition.
Further artificialchallenge was added by including distracting shapes that are not part of the correct answer. The pool size of distracting shapes was increased to further enhance the challenge.
Modifications to the scene and a summary of their categorization are presented in , where it can be observed that the existing features did not include the emerging theme of multisensorial and multi-domain elements. A customization option was added to the scene in which a virtual voice gives instructions about which shape to place in which order instead of presenting the target shapes visually. Anticipating that this audio mode would be too difficult when the presented shapes were complex or unfamiliar, a new collection of number-shaped objects was created for use in the auditory mode.
Figure 6. Theater exercise (targeting visual memory), including modifications made after end-user feedback. Checkmarks (✓) denote existing elements and plus signs (+) denote new elements that were added in the updated version of bWell-D. Tent SceneThe tent scene is intended as a space for relaxation and sensory immersion. The user is placed in a relaxing nature environment and allowed to freely explore. A ball that grows and shrinks to guide the user’s breathing, as well as additional user customization promoting relaxation, is included. Specifically, the user was given control of the selection of the environment and background music that plays. A virtual book placed in front of the user includes controls that allow them to navigate through a variety of virtual environments. Giving users control over their environment is expected to add motivation for continued use and address their need for autonomy.
While this scene was part of the bWell platform, it was not initially included in bWell-D. Feedback from end users about the need for activities that target mood prompted its inclusion. The scene was designed to follow an exploratoryopen-world approach, allowing users to freely look around the environment for as much or as little time as they wished. A summary of the features and their categorization is presented in .
Figure 7. Tent exercise (targeting relaxation). A task to target mood was added to bWell-D based on end-user recommendations. This task was taken as is from the bWell platform core battery of exercises. Checkmarks (✓) denote existing elements. Iterative FeedbackIn the previously published qualitative study [], a second round of interviews was conducted with 4 patients, during which they were shown the original bWell-D videos and updated videos that highlighted the added functionalities and features. All the participants reacted positively to these changes, felt that their input was considered, and thought that the initial version of bWell-D was improved.
In this study, the updated version was formally evaluated in participant trials with the original version in a single standardized session for initial assessment and as multi-session cognitive training for the improvements and gamification elements. In both the assessment and training sessions, the participants carried out the exercises for a total of 40 minutes. Of the 20 participants, 15 (75%) completed the assessment session, 4 (20%) opted to do the training sessions, and 1 (5%) tried the expanded features. The demographics and clinical characteristics of the participants of this study are summarized in for reference. It was planned for each participant to interact with bWell for 1 assessment session, and those who opted in for training would have 8 more sessions. However, 25% (1/4) of the participants who elected to do the training sessions withdrew from the study after the second training session. User experience scales (GUESS and UES) were administered on training sessions 1, 4, and 8; therefore, 75% (3/4) of the participants rated the bWell-D expanded features 3 times, and 25% (1/4) of the participants rated bWell-D only once, for a total of 10 measures taken.
The results indicated that bWell-D was well tolerated by the tested participants in the initial and training sessions, with scores within the acceptable range in the literature (total SSQ score of <5 after the session) []. The SSQ scores, as summarized in , showed no significant increases (after vs before the session) in cybersickness symptoms in the initial assessment session or in the training sessions for the 20% (4/20) of the participants who opted in (the scores for the training sessions were measured as average SSQ score changes across all sessions). The GUESS, a 7-point Likert scale and whose results are summarized in , demonstrated high scores for user experience. In addition, for the improved version of bWell-D, an increase in score in 4 subscales (usability, enjoyment, audio aesthetics, and visual aesthetics) was observed. The effect size was large for usability and audio aesthetics, medium for visual aesthetics, and small for enjoyment []. While all subscales were higher for the improved bWell-D version, the small sample size and limited effect size do warrant further studies with larger populations. Although not statistically significant, engagement scores measured using the UES, a 5-point Likert scale with 4 subscales, remained high, with an average score of 15.07 (SD 1.53) for the initial session and 15.48 (SD 0.42) for the training sessions out of a possible maximum score of 20, as shown in .
Regarding the unstructured feedback during the training sessions, one participant mentioned that they did not feel the passage of time while playing the mole game and was surprised when their 10-minute game session was over. Another participant compared the mole game to the original arcade games they used to play years before, and their intense engagement with the game led to a noticeable perspiration and higher heartbeat levels. The theater game was mentioned as the most liked game by a participant, whereas a different person stated that they did not like the game in the initial session as it was hard to obtain a sense of achievement. One person specifically enjoyed the relaxation scene (tent), in particular the ability to freely explore the environment.
The participant that tried 1 session of bWell-D with expanded features stated that moving within the office space was interesting and led to a better experience compared to the original version of bWell-D. Regarding the laboratory scene, they reacted positively to the addition of ingredient mixing, believed that it would also make it more difficult, and made additional suggestions to further increase the complexity. In the mole game, they recognized the addition of faces to the moles and the multiple-hit moles as variants. Furthermore, they approved of the option to have moles coming from the sides more than an expanding table as it involves peripheral vision. However, they also mentioned that these elements should be introduced after practicing the initial task to avoid possible frustration. They also noted that the tent scene was indeed a relaxing environment.
Table 1. Characteristics of the nonclinical participants who took part in this study (N=20).Participant characteristicsValuesAge (y), mean (SD)41.8 (12.58)Age (y), median (IQR)43.5 (34.25-53.25)Sex (female), n (%)12 (60)BDIa (score range 0-63), mean (SD)3.7 (5.55)BDI (score range 0-63), median (IQR; SEM)2 (0-4.25; 1.24)STAIb, mean (SD)59.85 (20.3)STAI, median (IQR; SEM)50.5 (46.50-68.25; 4.54)aBDI: Beck Depression Inventory.
bSTAI: State-Trait Anxiety Inventory.
Table 2. Scores for before and after the virtual reality session for the initial assessment and training sessions (Simulator Sickness Questionnaire [SSQ] scores averaged over all sessions).aaTotal SSQ scores were computed based on the study by Bouchard et al []. Comparing post- versus presession scores, it was observed that there were no significant increases in SSQ scores in the initial session (15 participants) and the training sessions (4 participants).
Table 3. Comparison of Game User Experience Satisfaction Scale results across 15 participants for the assessment session (original version) and across 4 participants who opted for the training sessions (improved version), averaging 3 recordings per participant.aaAll scores were, on average, higher in the training sessions than in the assessment session. This trend of high training means relative to assessment means is captured in the effect size, with the usability/playability and audio aesthetics subscales having large effect sizes (<0.80) [].
Table 4. Comparison of User Engagement Scale results across 15 participants for the assessment session (original version) and across 4 participants who opted for the training sessions (improved version), averaging 3 recordings per participant.aaHigh engagement was observed for all sessions (overall engagement score of >15 out of 20).
In this paper, we propose a 2-pronged approach using end-user feedback for the co-design of content as well as in the application of gamification. This process considers behavior change theories to structure feedback obtained and provide insights in terms of drivers that potentially promote engagement and enhance intervention efficacy. Ultimately, the aim was to improve the compliance rate of a VR cognitive remediation intervention for depression.
Several interesting findings emerged with implications for technology designers involved in developing interventions for people with depression, as well as for the implementation of such interventions. Optimizing engagement is critical for the success of digital interventions for any patient cohort. However, it is an especially vital consideration for interventions targeting depression, which is defined by decreased ability to experience enjoyment or interest, decreased motivation, and a lack of self-efficacy []. Grounded in feedback obtained from persons with lived experience, this analysis provides useful guidance for developing intrinsically engaging, motivating digital interventions that can overcome these hallmark features of depression.
Identification of Gamification ElementsThe classification of the gamification elements added to bWell-D indicated that feedback from end users centered on using gamification to add personalization and customization options, artificialchallenges, and artificialassistance. In defining customization versus personalization elements, the former consist of settings controlled by the clinician or user, whereas the latter are settings automatically imposed by bWell-D. In defining pers
Comments (0)