Introduction

Homelessness is a growing problem in North America. According to a 2022 annual report by the US Department of Housing and Urban Development, 582 462 people were experiencing homelessness across America.1 In Canada, over 235 000 individuals experience homelessness each year, with many more becoming precariously housed.2 In addition to the lack of permanent housing, a significant body of evidence has demonstrated that individuals who are precariously housed often experience diverse and complex medical and social issues, including income and food insecurity, poverty, as well as mental health and addiction needs.3 4 A Canadian study illustrated that 76.6% of individuals experiencing homelessness used the emergency department (ED), with an average of two ED visits annually.5 Systemic challenges and structural barriers within and beyond the ED often result in discontinuity of care and missed follow-up appointments, clients returning to the ED for unmet social needs, as well as poor patient care experience and navigation within the ED and after discharge to the community. Further support and resources are needed for PEH who frequently use the ED to facilitate smooth care transitions and coordination of essential health and social services.

Downtown East Toronto Ontario Health Team (DET-OHT)

OHTs are a novel approach to organising and delivering integrated care to a defined geographic location in the province of Ontario, Canada, to improve patient and provider experience, enhance population health and reduce the costs of healthcare. The DET-OHT is a group of partnered health service provider organisations in the DET community. Its goal is to provide seamless, inclusive and responsive care through an integrated, connected system of diverse providers and partners with lived experience. In its first year of implementation, the focus of the DET-OHT was to improve coordination of care and services and healthcare outcomes for its two priority populations: people experiencing homelessness (PEH) and adults with mental health and addiction needs. One of the local initiatives within the DET-OHT, involving the PEH priority population, is the ED Outreach Programme at Unity Health Toronto- St. Michael’s Hospital (UHT-SMH) in Toronto, Ontario, Canada.

UHT-SMH ED Outreach Programme

The ED within UHT-SMH provides health and social care to at least 4500 PEH annually; 20% of all ED visits are by PEH. Overall, 45% of PEH visits to any ED were to UHT-SMH. To facilitate continuity of care and a safe transition for PEH from the ED to the community, a service navigation and critical time intervention programme was developed. A critical time intervention is one of the most common case management approaches for PEH and involves bridging the gap at a ‘critical’ transition point for clients. The critical time intervention, referred to as the ‘ED Outreach Programme’, aims to improve ED service navigation and connection to community resources for PEH. Individuals experiencing homelessness presenting in the ED at UHT-SMH are supported by an ED Outreach Worker who determines their immediate needs (eg, food security, income support and clothing) and facilitates necessary referrals and support for a safe discharge to the community (eg, housing, primary and/or specialty care follow-up). The ED Outreach Worker establishes a therapeutic relationship with the client grounded in trust and respect and helps clients navigate the healthcare system while empowering and supporting their personal agency. Following a client’s discharge to the community, the ED Outreach Worker maintains contact with clients to ensure continuity of care and assists with the coordination, navigation and follow-up of healthcare and community services. The ED interventions provided through the programme have been widely supported by the literature and shown to improve social determinants of health.6 To our knowledge, the ED Outreach Programme at UHT-SMH is the first for PEH in Canada. Figure 1 provides an overview of the ED Outreach Programme.

Figure 1

Overview of Emergency Department (ED) Outreach Programme.

Patient-reported experience measures (PREMs)

PREMs are surveys that evaluate patient perceptions and experiences with care processes, interventions, and programmes and provide data that can be used to assess the performance of health systems and enhance patient-centred care. Beyond understanding patient experiences to evaluate interventions and programmes, PREMs play a crucial role in providing the necessary evidence to key stakeholders such as government bodies to facilitate continued programme funding and support. While the development of PREMs in collaboration with patients has been previously cited in the extant literature, none have specifically involved PEH and/or precariously housed communities.7–10 Engaging these populations in participatory research and healthcare decision-making through co-design and co-development of PREMs is one method to promote health equity and strengthen marginalised, socially disadvantaged communities.11 12 To our knowledge, no other study has engaged PEH in co-designing PREMs to evaluate patient experience in an ED-based outreach programme. Co-developing PREMs with PEH from the ED Outreach Programme will facilitate rigorous, meaningful evaluation of the programme and its services and inform invaluable process and programme improvements for the programme and other ED-based interventions. More importantly, findings from this study will facilitate new knowledge on engaging marginalised populations in co-design research, specifically, the development of PREMs in evaluating care processes and programmes, which will facilitate a more equitable healthcare system that is inclusive and representative of disadvantaged and marginalised populations. Using the ED Outreach Programme as an exemplar, this paper aims to present a community-based participatory approach guided by principles of shared ownership and empowerment to engage disadvantaged populations in the development of PREMs that can rigorously and meaningfully evaluate healthcare programmes and services.

Objectives

This study aims to (1) examine experiences of the ED Outreach Programme from the perspective of programme clients and staff/providers, (2) co-design a set of PREMs with clients, staff/providers and partners with lived experience to evaluate the patient experience in the ED Outreach Programme and (3) pilot test the PREMs to determine their implementation feasibility, utility and appropriateness in the clinical setting, as well as gather preliminary information regarding patient experience.

Methods and analysis

This three-phase study involves using a community-based participatory research (CBPR) approach to engage people with lived experience of homelessness and staff/providers to co-design PREMs that will measure and evaluate patient experience, including patients’ ability to navigate barriers and/or access services, within an ED-based outreach programme. This project is important given the dearth of evidence on co-designing evaluation measures with PEH, as well as the lack of existing literature concerning patient experience among PEH and specifically within ED-based programmes and services. Each study objective is sequentially examined through each study phase. This protocol outlines an ongoing study that commenced in September 2022, with data collection and analysis currently in progress.

CBPR and co-design

The CBPR method applies a power-focused lens to researching groups and communities that are often vulnerable, marginalised and/or seldom heard.13 14 One participatory research approach is ‘co-design’, a dynamic process of collaborating and partnering with individuals with lived experience in an authentic and equitable manner to solve a health service problem or challenge.15 Greater emphasis has been placed on co-design because of the increasing focus on ensuring health services research is meaningful and impactful to end users. The principles of ‘distribution of power in research’, ‘amelioration of the human experience’ and ‘positive societal impact’ are particularly important when engaging individuals who are considered vulnerable due to social and systemic barriers, such as PEH15 (p.1). While healthcare should aim to meet the needs of vulnerable groups, especially those who are disadvantaged because of their health, economic, cultural and/or social circumstances, the literature has continued to highlight that these marginalised populations are often under-represented in the research and evaluation of health services.12 15 16 CBPR encompasses a variety of research approaches and involves empowering and collaborating with key stakeholders (ie, clients with lived experience, staff/providers and other community partners) in the research process, from defining the problem to devising solutions, while acknowledging the unique perspective of each participant in the process of constructing knowledge.

The 10-step Ottawa Citizen Engagement and Action Model (OCEAM)17 provides a useful framework to meaningfully engage and involve individuals with lived experience and ensure they remain at the centre of the research process, from study inception to dissemination. Notably, the OCEAM has demonstrated positive outcomes by emphasising community participation to support health behaviour interventions among hard-to-reach, inner-city populations experiencing homelessness.17 18 Adhering to the principles of the CBPR, OCEAM and other related literature,17–20 a working group and community expert group (CEG), composed of representatives from local stakeholders and community members with lived experience of homelessness, have been actively involved in shaping the research questions and study design of the present study and will continue to contribute and provide feedback on subsequent steps of the research process to ensure the study findings reflect the priorities and perspectives of the community.

While previous studies serve as a strong foundation for CBPR in health interventions and patient-reported outcome measures, the present study aims to contribute by developing measures of patient experience to evaluate the impact of an integrated care programme aimed at improving service navigation and access to supports for PEH. Engaging community participants in the co-design process empowers them to actively contribute to developing and evaluating health services rather than being solely partners who ‘have a say’ in their care.21 22 The co-design process in the present study will facilitate shared leadership and collective ownership among clients and professionals in developing an evaluation tool (eg, PREMs) for the ED Outreach Programme that is patient-centred and driven. Specifically, this study will use a CBPR study design to co-design PREMs to evaluate patient experience in the ED Outreach Programme. To complement the CBPR approach, grounded theory principles will be applied to develop a rich, contextual understanding of the underlying psychological and social processes underpinning community participants’ experience with the outreach programme.23 24 The study will comprise three major phases. Phase 1 involves conducting in-depth interviews with clients and staff/providers of the ED Outreach Programme to gain contextual awareness of the priority population (ie, PEH) and programme. In Phase 2, a co-design group comprising programme clients, partners with lived experience and staff/providers of the ED Outreach Programme will be assembled to develop PREMs to evaluate patient experience. In Phase 3, the PREMs will be pilot-tested with active clients in the ED Outreach Programme.

Phase 1: Conceptualisation of population and ED Outreach Programme

The first phase involves conducting in-depth interviews with clients and staff/providers of the ED Outreach Programme to gain contextual awareness of the priority population (ie, PEH) and understand their experiences with the programme. Findings from these interviews will provide insight into the care and services provided to clients through the programme (eg, resources, supports) and inform appropriate and pertinent patient experience constructs, domains and measures that should be evaluated as PREMs. Furthermore, the interviews will provide a comprehensive understanding of care processes and service delivery involved in the programme and important objective and subjective data regarding patient and provider experiences. The interviews will also facilitate the development of trust and rapport with programme clients and a deeper understanding of how to engage PEH in research in an inclusive, equitable manner, particularly in co-design research, programme evaluation and quality improvement activities of healthcare programmes and services. Contextual awareness of the priority population (ie, PEH) and the programme is imperative for rigorous programme evaluation and ethical and sensitive engagement of partners with lived experience in research.

Study population and sampling strategy

The first study phase will examine the perspectives of clients and staff/providers involved with the programme. Clients are eligible to participate if they (a) are an adult (aged ≥18 years) and (b) were or have been participating in the ED Outreach Programme for at least a period of 1 month. Clients unable to provide informed consent will be excluded. Staff/providers are eligible to participate if they have provided care and/or services to one or more clients as part of the programme in their professional role at a healthcare or community partner organisation within the DET-OHT. Participants will be purposively selected by referrals from the ED Outreach Team. A total of 4–5 clients and 4–5 staff/providers are anticipated; however, additional participants will be recruited, if necessary, and interviews will be conducted in an iterative process until data saturation is achieved.

Data collection and analysis

In-depth, one-on-one interviews will be conducted using a semistructured interview guide to examine client and staff/provider perceptions and experiences with the programme. The interviews will facilitate the opportunity to develop trust and rapport with the participants and identify programme components that are perceived to be most important for clients. Themes emerging from these interviews will form the foundation and basis for the patient experience constructs, domains and measures of the programme. Findings from this study phase will also facilitate meaningful insights and knowledge about how to engage clients and staff/providers in the subsequent study phase, which involves collaboration and engagement of clients, partners with lived experience and staff/providers in the co-design and co-development of PREMs. Online supplemental appendix A illustrates a sample of interview questions and general themes that will be discussed and explored with client participants, while online supplemental appendix B outlines questions for staff/provider participants. Demographic information will also be collected from all interview participants using a short questionnaire to facilitate contextualisation of the qualitative findings (see online supplemental appendix C for client demographic questions and online supplemental appendix D for staff/providers). Quantitative data (eg, demographic information) will be entered into IBM Statistical Package for Social Sciences (SPSS) V.28 for descriptive analyses. Guided by the grounded theory approach, interview transcripts will be transcribed verbatim and analysed using thematic analysis by a member of the research team. The analysis process will follow initial, focused and theoretical levels of coding, in which qualitative comments are sorted and grouped into similar categories and coded according to themes.25 26

Phase 2: co-design of PREMs

In Phase 2, a Co-design Research and Evaluation Advisory Committee composed of programme clients, staff/providers and partners with lived experience will be assembled to engage in focus group discussions. Using a shared decision-making model, the lived experience and ‘insider knowledge’ of clients and lived experience partners regarding the ED Outreach Programme and the topic of homelessness will be invaluable to develop PREMs that are relevant, applicable and feasible in the local context. To meaningfully engage community participants, the principles of CBPR will be paramount.

Study population and sampling strategy

Like Phase 1, this study phase involves engaging programme clients and staff/providers and drawing on their experiences with the programme. Additionally, individuals who have experienced homelessness or are at risk of homelessness are invited and eligible to participate, providing more diverse perceptions and perspectives regarding the PREMs to evaluate the ED Outreach Programme.

Purposive and convenience sampling will be used to select approximately 5–8 clients, partners with lived experience and staff/providers to form the Co-design Research and Evaluation Advisory Committee. Potential client participants from the programme will be purposefully identified by ED Outreach Workers and/or community partners. other community participants will be identified through purposive and convenience sampling from a local CEG composed of persons with experience of homelessness or being precariously housed. Broadening the eligibility criteria in this study phase to include individuals with lived experience of homelessness will facilitate more diverse and inclusive perspectives in the co-design process. Staff/providers will also be identified through convenience sampling. To recruit staff/providers, the study description, including participant eligibility criteria and investigator information, will be emailed to community partner agencies involved in the programme.

Data collection and analysis

A series of focus group discussions with the Co-design Research and Evaluation Advisory Committee will be held to develop PREMs that will evaluate patient experience in the ED Outreach Programme. Two to three virtual or in-person meetings, ranging from 60 to 90 min, are anticipated; however, the final number of meetings will be determined by the iterative analysis process and data saturation of themes.

The facilitator will moderate all co-design meetings using an unstructured discussion guide. Based on the findings from Phase 1, this study phase involves developing a core set of PREMs to evaluate patient experience in the ED Outreach Programme. To this end, a literature review will first be conducted to identify existing PREMs relevant to the programme, ED setting and marginalised populations. The patient experience constructs, domains and measures identified in Phase 1 will then be mapped onto existing measures to develop a preliminary draft set of PREMs for review by the Co-design Research and Evaluation Advisory Committee. The committee members will be asked to (a) broadly comment on their knowledge and experiences related to the ED Outreach Programme and/or experiences with ED-based programmes and interventions, (b) review and examine the preliminary set of PREMs adapted from existing patient experience surveys, instruments and measures identified from the literature and (c) identify and develop additional PREMs as needed. The committee will also be consulted to determine if the PREMs are in an accessible language and how often patient experience will be evaluated (eg, baseline, 3 months and/or 6 months). Demographic information will also be collected from all participants.

All focus group discussions will be recorded and transcribed verbatim. Qualitative data and narrative information collected will be analysed using the constant comparative method, where qualitative comments are sorted and grouped into similar categories and coded according to themes.25 26

Phase 3: Pilot test of patient experience evaluation tool

In Phase 3, the co-designed survey with PREMs will be pilot-tested with active clients in the ED Outreach Programme. Pilot testing is an essential component of survey research; the PREMs will be pilot-tested in a feasibility study, that will be systematically administered to a diverse cross-sectional sample. The pilot test will support the appropriateness of the PREMs and their feasibility and adequacy in the clinical setting and identify potential methodological issues, such as data collection limitations.27 28 Data from the pilot testing phase will also improve the standardisation of measurement procedures, identify potential implementation challenges (eg, language and accessibility barriers) and facilitate the evaluation of the PREM survey’s validity and reliability. Furthermore, the pilot test will also provide preliminary data regarding patient experience in the ED Outreach Programme and identify process and quality improvement opportunities. To increase accessibility, both paper-based and electronic iterations of the PREM survey will be developed. The electronic iteration of the PREMs will be integrated into REDCap (Research Electronic Data Capture), a secure, web-based application designed for data collection and management in research studies.

Study population and sampling strategy

Clients will be approached in person (in the ED or community as part of programme follow-up) or by phone/email. The PREM survey will be administered to approximately 200 clients in the ED Outreach Programme through convenience sampling29 ; however, this sample size will vary according to the number of items or questions within the developed survey and accessible clients in the programme who can be contactedor reached to participate in the study. In the pilot testing of the survey instrument, the recommended number of participants is 5–10 per item or question.30

Data collection and analysis

Clients will be provided with the PREM survey in the form of their choice (ie, online/electronic through REDCap or in-person/paper-copy) and given the option to complete the survey on their own or with the guidance of a research assistant. Descriptive statistics will be generated to determine and summarise informant characteristics and quantitative responses. Qualitative data and narrative information collected will be analysed using a thematic analysis framework.25 26 Findings and feedback from this pilot testing phase will be reviewed by the Co-design Research and Evaluation Advisory Committee and may result in modifications and/or refinements to the PREM survey if limitations or challenges to the instructions, content and/or data collection procedures are identified by clients or staff; further revisions and refinements may be made to the PREMs prior to full-scale administration and implementation in the ED Outreach Programme.

Patient and public involvement

Partners with lived experience and patient advocacy groups were and will continue to be involved throughout this research project, from the programme’s inception to its evaluation, to ensure a truly participatory approach and that study findings are relevant and responsive to their needs. The ED Outreach Programme was developed with community partner organisations and partners with lived experience in the DET-OHT and established in response to the need for improved access to social and primary care for PEH in the ED. The present study is guided and overseen by two key groups: the DET-OHT PEH Working Group and the SMH MAP CEG on Homelessness, Housing and Health. These groups, composed of local healthcare and social care organisations and individuals with lived experience of homelessness, were actively involved and consulted throughout the development/implementation of the programme, the design of the research project and will continue to guide the data collection, analysis and interpretation of findings and dissemination of results. To illustrate, the proposed study design and methods, including areas of inquiry and evaluation, were reviewed and discussed with the CEG to ensure that the research project was well-aligned with various community-based research pillars, including community relevance, capacity building, equity, collaboration and shared ownership. The CEG provided their support and positive feedback regarding the study’s aims and design and highlighted the need for more research efforts and studies that focus on ‘doing with’ clients with lived experience of homelessness rather than ‘doing for’, as exemplified in our study.

Ethics and disseminationEthical and safety considerations

The study will be carried out in accordance with sound ethical principles, ensuring beneficence, respect for self-determination and human dignity and the right to privacy. Written informed consent will be obtained, and a verbal consent checklist will be reviewed with participants. Clients will be informed about the study aims, data collection processes and use of the collected data, as well as procedures to ensure data protection, privacy and confidentiality. All clients will be informed that their feedback and experiences will not be shared with their ED Outreach Workers or staff/providers involved in their care as part of the programme. Clients will also be informed that their participation is voluntary; they can withdraw from the project at any point, and it will not impact the careor services they receive from their service providers or the programme.

Dissemination plan

Dissemination materials such as presentations, briefing notes and reports with knowledge gained and lessons learned will be developed in collaboration with clients and staff/providers participating in the project and shared through affiliated hospitals, community organisations and government agencies. Clients and partners with lived experience will be acknowledged as co-authors with their consent. Findings from this project will be submitted and published in open-access, peer-reviewed scientific journals to ensure high accessibility and availability to study participants and the wider public and patient community. Furthermore, results from this project will be presented at national and international conferences, particularly those organised by health system leaders, to (a) facilitate further development of programmes that will improve access, coordination and navigation of care and services for individuals who are at risk of experiencing homelessness and (b) more engagement of marginalised or socially disadvantaged groups in research, evaluation and quality improvement activities of health services and programmes.