This project took place in a hospital-based pediatric rheumatology clinic at a moderate-sized, urban, pediatric tertiary care center in the southeastern United States. Patient care in our clinic is provided by a multidisciplinary team including five attending pediatric rheumatologists, three fellows, three nurse practitioners, three nurse case managers, a physical therapist, an occupational therapist, and a social worker. Outreach clinics were excluded because of a lack of required support staff at these sites (i.e. social workers, emergency rooms, and dedicated rheumatology nurses). We serve a diverse population of patients with cSLE, ranging in age from children to young adults. According to self-reported data within the EHR, most of our patient population is female (86%) and of African American (47%), Caucasian (39%), Asian (9%), or Hispanic (0.8%) race. Our patients with cSLE have follow-up visits as frequently as once per week to once every 6 months, depending on disease activity and severity. In a prior study performed in a cohort of patients with cSLE seen at our center, Systemic Lupus Erythematosus Disease Activity Index (SLEDAI) scores ranged from 2–8 with an average of 5.1 [2]. Our site utilizes Epic (EpicCare Ambulatory, Epic Systems Corporation, Verona, WI, USA) as the EHR.

We used the Institute for Healthcare Improvement’s (IHI) Model for Improvement [16] to structure and execute our QI project. The IHI Model for Improvement is an accepted framework used to accelerate process improvement within healthcare settings [17]. Through application of this model, we identified key drivers that contribute to achieving a reliable mental health screening system. Our key drivers included knowledgeable clinic staff and providers, a standardized process for administering screens and reviewing screen results, a defined process for addressing positive screens and suicidal ideation, bidirectional communication between clinic staff and providers, EHR automation of the screening process, and patient and family awareness (Fig. 1). We then developed interventions that directly related to these key drivers. Per the Model for Improvement, these interventions were iteratively planned, tested, and adapted using Plan-Do-Study-Act (PDSA) cycles to ensure changes to the process resulted in improvement of our desired metric – the rate of standardized depression screening for patients with cSLE presenting for routine follow-up appointments at our pediatric rheumatology clinic.

Our QI team included an attending pediatric rheumatologist, a clinic nurse, nurse case managers, a pediatric resident, a pediatric rheumatology fellow, an Epic physician builder, a social worker, a member of our front desk staff, and a discharge coordinator. The QI team met biweekly to review the results of each intervention via run charts, control charts, and qualitative feedback from members of the QI team. The team used concepts of highly reliable systems [17] to develop and refine workflow processes for each PDSA cycle. Qualitative feedback was also elicited from providers and patients to gauge satisfaction and plan future process improvements. The Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) guidelines were used in the preparation of this manuscript [18]. The Vanderbilt University Medical Center institutional review board (IRB) reviewed the proposed project and deemed it exempt from IRB oversight as a QI activity (IRB #190,126).

We used the Patient Health Questionnaire-9 modified for adolescents (PHQ-A) to screen for depressive symptoms. This is a standardized screening tool for depressive symptoms validated for ages 12–18 years. In validation studies, the PHQ-A was found to be self-administered in less than 5 min [19]. The sensitivity and specificity of the PHQ-A for major depressive disorder is 75% and 94% respectively. We chose this screening tool due to its specific age validation for adolescents as compared to the PHQ-9, its fast self-administration, and its wide use in clinical and research settings in pediatric rheumatology [2, 4, 7, 19,20,21,22,23]. The screen was offered to patients identified with cSLE using specified International Classification of Diseases 10th Revision (ICD-10) codes (Supplemental Table 1), who presented to our clinic for a follow-up visit. Patients who were greater than or equal to 12 years of age and English-speaking were eligible, in accordance with the validation parameters of the PHQ-A and the limited availability of translated screening materials at our institution upon initiation of the project.

The PHQ-A scoring algorithm and suggested treatment plan are illustrated in Supplemental Fig. 1. For patients with moderate depressive symptoms or higher, a referral to a mental health provider was advised. Patients with severe depressive symptoms were seen by a social worker for suicide risk assessment and safety planning. Patients who were deemed actively suicidal were referred to the emergency department for further evaluation and urgent intervention. We performed frequent reviews of our data to ensure that these appropriate interventions were taking place in response to positive screens.

The primary outcome measure was the percentage of eligible patient encounters for which depression screening was offered using the PHQ-A screening tool. Eligible patient encounters were identified by an EHR report of patient encounters within the pediatric rheumatology clinic during two-week intervals with a documented clinic visit billing code related to SLE as defined by the ICD-10 codes previously described. The screening results were manually extracted and reviewed by members of the QI team (V.N., A.D., E.D., N.B., S.P.) to determine if the PHQ-A was completed at each visit and if appropriate clinical actions were taken.

The primary outcome measure was plotted on an SPC chart (P-chart) using a mean centerline and upper and lower control limits to assess for special cause variation. Special cause variation is reflective of specific circumstances that led to improvements in our system compared with common cause variation, which reflects causes of variation inherent in the system itself [24]. Statistical rules are used to identify nonrandom patterns on a control chart that determine special cause variation [24]. These rules include (1) a single point outside the control limits, (2) a run of eight or more points in a row above (or below) the centerline (shift), (3) six consecutive points increasing (trend up) or decreasing (trend down), (4) two out of three consecutive points near (outer one third) a control limit, and (5) fifteen consecutive points close (inner one third of the chart) to the centerline [24].

We also collected and analyzed secondary data using descriptive statistical methods to help inform ongoing efforts aimed at improving mental health care for this population. We measured the total number of completed PHQ-A screens, the percentage of positive PHQ-A screens, defined as those indicating moderate to severe depressive symptoms, and the percentage of screens with positive suicidal ideation for which clinically appropriate actions were taken. We also measured the number of individual patients with suicidal ideation.

Prior to the initiation of this project, we were not administering mental health screening questionnaires in our clinic. Our project was completed in three phases over a four-year period as illustrated in Fig. 2.

In this phase, we focused on creating a standardized process to administer the PHQ-A with defined safety protocols to address positive screens. Nurse case managers used an EHR generated report to perform pre-visit planning to identify eligible patient visits on a weekly basis. These nurses and front desk staff provided the PHQ-A to patients on an electronic tablet at the time of in-person check-in for their appointment. Patients completed the screen prior to starting the provider visit. Thus, no additional time was taken during the visit for completion of the screen. The PHQ-A form was provided via Research Electronic Data Capture (REDCap), a secure-HIPPA-compliant web-based software platform [25, 26]. We chose to provide our screening tool via an electronic tablet rather than on paper to maintain patient confidentiality and facilitate immediate manual electronic upload of the result to the EHR for provider review. Nurse case managers, who collected the tablets upon completion of the screen and uploaded the results to the EHR, also reviewed the screen results and helped verbally notify the provider of the score. During the visit, the visit provider and/or social worker were expected to address a positive screen. The project was paused at the start of the SARS-CoV-2 pandemic due to safety concerns amid an increase in telehealth visits and a change in clinic staffing protocols.

The second phase of our project focused on refining our screening process to increase the reliability of our system. We focused on interventions that would reduce the dependence of our process on specific individuals, improve bidirectional communication among clinic staff, and bolster safety mechanisms.

The third phase of our project focused on the automation of PHQ-A screening and safety procedures through the EHR to further improve reliability and promote sustainability by decreasing the dependence of our process on clinical staff. The EHR was programmed to automatically assign and display the PHQ-A as a pre-visit screening questionnaire at eligible patient visits. The screen was still completed by the patient on a tablet after in-person check-in, before starting the provider visit. The EHR then scored the screen and displayed the results in an easily visible location within the visit encounter. Similar to the prior phases, the visit provider and/or social worker were expected to address a positive screen during the visit. The PHQ-A was only visible to patients at the time of their in-person check-in rather than via the patient portal a few days before their appointment. This was intentionally done to ensure that the results were immediately reviewed upon completion of the screen to protect the safety of our patients. Another safety mechanism was the best practice advisory alert (BPA), which was created to alert staff to patients with suicidal ideation upon opening the chart. A rheumatology physician in our group who is an Epic physician builder (L.B.) helped build this system.