WHAT IS ALREADY KNOWN ON THIS TOPIC

Rapid review (RR) conduct stems from the systematic review process but has unique considerations. Known to be poorly reported, it is essential that readers have access to the fulsome information, transparently reported to understand scope, methods, findings, limitations, and implications.

WHAT THIS STUDY ADDS

Provides interim guidance for the reporting of RRs, including a preliminary list of items specific to RRs, in advance of the development of a consolidated checklist, Preferred Reporting Items for Systematic Reviews and Meta-Analyses for RRs (PRISMA-RR).

HOW THIS STUDY MIGHT AFFECT RESEARCH, POLICY OR PRACTICEIntroduction

This paper provides interim reporting guidance for rapid reviews (RRs) as part of a series from the Cochrane Rapid Reviews Methods Group.1–4 RRs have emerged to support urgent decision-making; producers use abbreviated SR methods to generate synthesised evidence in a resource-efficient manner.5 Although RRs have been in use for more than two decades, their prominence has increased over time, and they were an important vehicle to support health decisions during the COVID-19 pandemic.6

With the motivation to support decision-making comes the responsibility to transparently report research. Producers need to communicate essential information so that interested readers can understand the review’s scope, how it was undertaken, the relevant evidence base and synthesised research findings, and any additional considerations or limitations. Reporting should be such that others could, in theory, replicate methods and findings. Although intuitive that all essential information should be provided, studies on SRs show a need for improvement.7–9 Several articles have signalled reporting issues with RRs,10–13 including two empirical studies.14 15 With RRs, there is the added consideration of ensuring differences to full SR methods are communicated, particularly as reports tend to be shorter and produced more quickly, and methods are not standardised.

This paper provides considerations and recommendations informed by empirical studies on the reporting of RRs of primary studies,14 15 survey input, and the authors’ collective experience. The collation of empirical studies and survey deployment reflected the initial development phase of an extension of the Preferred Reporting Items for SR and Meta-Analyses (PRISMA) checklist for RRs of primary studies, including PRISMA for Abstract items.16 Soon following, the PRISMA 2020 team started updating PRISMA 2009; there was desire by all to integrate PRISMA 2020 into the extension for RRs. However, the timing was such that further development was halted by the COVID-19 pandemic through shifts in research activity to support COVID-19 decision-making. Therefore, the preliminary list of reporting items outlined in this paper will be considered in the development of PRISMA for RRs (PRISMA-RR), supported by funding from the Canadian Institutes of Health Research (CIHR).17 In addition to integrating more recently developed reporting guidance, timing is opportune to not only leverage learnings from the production of RRs in the context of the COVID-19 pandemic, but to incorporate newer developments in RR methods, such as automation.18

Making the preliminary reporting items available now allows RR producers to implement as an interim measure and to provide feedback on their use as we look to develop PRISMA-RR. We intend for flexibility in the use of these items; for example, RR producers using PRISMA 2020 alongside, rather than PRISMA 2009, is sensible. As with PRISMA, this guidance is geared to reviews addressing intervention questions; RR producers would need to adapt reporting for other types of research questions, accordingly.

General considerations

General considerations for the reporting of RRs are detailed below, from which general recommendations for reporting are provided in box 1.

Box 1 Recommendations for reporting

Use existing, robustly developed reporting guidelines as the foundation for writing rapid reviews (RRs): notably Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 (PRISMA 2020), but consider preferred reporting items for overviews of reviews (PRIOR) items where systematic reviews (SRs) are included in the RR.

In addition, consider the items in table 1 as a minimum set of items for RRs.

Explicitly report any deviation from standard SR methods, including omitted steps.

As RRs can take various formats and packaging to facilitate decision-making, it is practical to consider complete reporting as across the documents that comprise the information package, and explicit linking among documents would be required to accomplish this. Additional, minimum essential information is provided as an appendix or in an open-access journal or repository that is barrier-free. We discourage information made available by request or posting on non-permanent websites.

Table 1

Preliminary reporting items for rapid reviews in addition to Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 (PRISMA 2020) and preferred reporting items for overviews of reviews (PRIOR)

Face validity of PRISMA items for RRs

As RRs are typically understood to be products that stem from SR methods, starting first with a consideration of the relevant PRISMA guidelines is logical. However, RRs cannot simply be thought of as modified SRs, where, for example, the unit of inclusion is the primary study and the report structure typically reflects the Introduction-Methods-Results-and-Discussion (IMRaD) format. Depending on what is initially scoped or uncovered during RR conduct, RRs may include a summary of existing SRs (sometimes referred to as secondary evidence), with or without a summary of more recently published primary studies, or a synthesis of primary studies alone; indeed, initial characterisation of a sample of 76 journal-published RRs showed that 40% included secondary evidence.16 When considering PRISMA 2009, for example, we deemed that an estimated one-third of items would not have sufficient face validity when attempting to apply them to RRs that include secondary evidence. When developing the survey (described in more detail below), we focused on the reporting of RRs of primary studies as the first step in developing guidance for RRs. Although RRs that include secondary evidence would not be considered akin to an expedited version of an overview of reviews, their future reporting guidance would require the consideration of the preferred reporting items for overviews of reviews (PRIOR) checklist.19 Until this is further developed in context of PRISMA-RR, we recommend that RR developers consider items within PRIOR if including secondary evidence. For example, PRIOR addresses not only specifying the definition of SR for including in the report, but the reporting of an assessment of SRs themselves (ie, A MeaSurement Tool to Assess systematic Reviews 2 [AMSTAR 2] or ROBIS)20 21 in addition to the primary studies within them. Those items are relevant to RRs with secondary evidence, even if a brief statement of the risk of bias of primary studies from the SRs is provided, for example.

Reporting in relation to RR format

A second consideration in terms of using reporting guidance is RR format. To date, related checklists19 22–24 are structured around a typical IMRaD format, the predominant format for reporting research in the biomedical community and other areas of science. Not surprisingly, an empirical study showed that 92% of RRs published in journals were formatted in that manner.25 However, non-journal published RR reports, which greatly outnumber those published in journals, were shown to primarily take other forms, such as graded entry formats or packages (eg, 1:3:25 report graded-entry report structure).25 These alternative formats emphasise presenting key information upfront to support decision-making, followed by more in-depth information such as methods, findings, and risk of bias or quality appraisal and not necessarily in that order.

RRs can, therefore, comprise information in one document or a series of documents of increasing detail. Given that various formats are available, it is practical to think of complete reporting as across a set of accompanying documents and not necessarily that all details need to be made available in one document, as would be expected for reports of SRs. For example, if an RR commissioner wishes to receive a document of no more than 10 pages, then the RR producer can provide access to additional documents that would facilitate complete reporting for items not in the main report. Of key importance is offering flexibility for different packaging or presentation needs while providing easy (eg, open) access to all information to uphold complete and transparent reporting. RR producers should ensure these documents are explicitly linked. Supplemental information could be included as an appendix to the main report or in open-access journal websites or repositories, such as Open Science Framework (osf.io/). We discourage making information available by request or posting on websites that may not have permanence.

Transparent reporting of omitted methods in RRs

Explicitly declaring where methods items or steps were omitted is a third consideration that bears noting. Although this would be sensible guidance for the reporting of any health research report, there is particular consideration for RRs in understanding their methods relative to SRs. Some survey respondents had suggested modifying the wording of items in relation to relevance (eg, ‘if done, ‘if applicable’), such as for risk-of-bias assessments. We instead recommend reporting methods explicitly, such as when there is a deviation from or modification to SR methods, including the omission of steps, as this makes the process transparent for readers.

Preliminary reporting items for RRs

As a summary of the survey process, all items within the PRISMA 2009 and PRISMA for Abstract checklists were endorsed by 100 respondents. Nine new items achieved consensus, and four items were modified, of which some were subsequently reflected in PRISMA 2020. No additional items were proposed on the survey regarding the writing of an abstract. As informed by our survey, a handful of reporting items can be considered relevant to RRs. We provide the rationale for those items below, with a summary provided in table 1 and example for each of the methods-related items. Details of the methods, participant characteristics, survey results, and disposition to comments are comprehensively provided in data (online supplemental supplement 1).

A priori iterative methods

RR producers may need to build into their protocol the points during conduct at which decisions may need to be made in light of the emerging nature (eg, types of study designs) and volume of evidence (eg, number of studies) to meet the decision-making timeline. This is unique to RR conduct and typically reflective of a short period of time to scope and refine topics prior to conduct. For example, when developing the RR protocol on the effectiveness of personal protective equipment in the context of filovirus disease, the authors indicated that if studies on filovirus disease were limited, the scope could be broadened to include indirect evidence from other infectious diseases with a similar route of transmission and infectivity.26 This option was instituted during conduct of the RR, indicated as an expansion of scope but not included in the protocol modification section. Similarly, outcomes of interest were listed in descending order of the importance to decision-making; in the protocol, the authors indicated that the evaluation could be limited to a subset, according to that priority list, if the volume of evidence was too large to complete the RR for the decision-making timeline. Placing emphasis on including high-quality study designs relevant to the review question is another example provided by the Cochrane Rapid Reviews Methods Group.27 Naturally, the key concern in this process is making decisions in relation to when the findings were known. Therefore, we recommend stating at what point during conduct those decisions were made (eg, prior to data extraction). Post hoc changes made during conduct of the RR that were not outlined in the protocol would be declared as an amendment to the protocol (eg, PRISMA 2020 item 24 c).

Distinguishing the RR from a systematic review

With the diversity of RR methods comes potentially differing impacts on conclusions. As such, it is important for producers to signal why they do not consider their report to be an SR; Cochrane provides an SR definition that readers could refer to.28 The use of one person to review titles and abstracts of citation records, not including a search for grey literature, and foregoing risk-of-bias assessments (although we would discourage this) would be examples. We acknowledge that a continuum exists as to how producers may relate particular methods approaches to an SR or RR,29 which underscores the need to make this explicit; for example, whether limiting inclusion to English language literature is viewed as SR or RR methods. We recommend authors frame this declaration as to why they deem the product to be an RR. In addition to providing transparency, those distinctions may also help inform the growing empirical base of the impacts of RR methods. Although a substantive proportion of RRs also include secondary evidence, we have kept the comparison here in relation to SRs for two reasons. First, the process for rigorously conducted SRs and overviews of SRs largely overlap in terms of steps of production. Second, RRs including SRs would not have the level of sophistication of an overview, serving more as a knowledge translation product of existing SRs.30 31

Knowledge user involvement

Integrated knowledge translation (iKT) involves knowledge users as co-producers of research, with the intent of increasing relevance and use in decision-making.32 Examples of knowledge users are policymakers, guideline developers, healthcare providers and patients. Given the typically accelerated nature of producing RRs, a closely collaborative relationship between the producer and knowledge users provides important context to shaping the scope of the RR to realise a fit-for-purpose product.3 We direct readers to another article in this series that provides a thorough discussion and considerations of knowledge user involvement in RRs.3 The article provides evidence of inadequate reporting of knowledge user involvement, which we hope to improve through this reporting item. Other relevant reporting guidance should be considered in this context, such as the use of the second version of the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) for the inclusion of patient partners.33 At a minimum, we recommend RR producers to report who was involved, at what stages, and providing input for what items.

Authorship and corresponding author

Listing an authorship byline in addition to identifying a corresponding author and their contact information are standard attributes of journal article publications. However, RRs that are not published in journals do not report this as frequently.15 As there are important publication ethics principles to uphold, namely, giving appropriate attribution to intellectual content and providing accountability to the research undertaken and reported, we recommend reporting an authorship list and contact information for a corresponding author; the International Committee of Medical Journal Editors (ICMJE) information on the roles of authors and contributors is the most widely recognised framework to support reporting in this regard.34 Further consideration could be given to listing contributors, whether authors or others, and their respective roles during conduct. The Contributor Roles Taxonomy (CRediT) is one such framework to structure contributorship; however, it is not intended to define what constitutes authorship.35

Acknowledgements

Providing attribution to those who were involved in the work but did not meet the criteria for authorship would reflect ethical publishing practice. To distinguish from the item ‘Knowledge user involvement’, the latter is intended to assist in understanding the iKT process undertaken. However, individuals providing input from an iKT perspective should be listed here if not meeting the authorship criteria; an example would be knowledge user involvement in research question development but not reviewing and approving the final report.

Peer review

The main consideration for this item is providing an opportunity, in an urgent environment, to have one or more individuals external to the RR producer team critically review the report. This can help provide validity from a content and/or methodological perspective and correct inadvertent errors prior to submitting to the commissioner, to optimise the quality of the product. This can be attractive to RR-producing teams to obtain a particular knowledge user’s input if unable to involve in an iKT process. How this compares to a journal editorial peer-review process is beyond the scope and intention of discussion here, but is worthy of consideration as many RRs are not journal-published.15

Other reporting items

Several other reporting items that were either endorsed through survey feedback but did not achieve consensus or were modifications to PRISMA 2009 are now reflected in the PRISMA 2020 checklist. Those include reporting methods on assessing the certainty of evidence, outlining protocol modifications and providing a statement on data sharing and supplemental information. Modifications made in the survey to PRISMA 2009 items were largely reflected in PRISMA 2020. This provides support that PRISMA 2020 can be readily integrated into developing PRISMA-RR.

Additional survey items not included in PRISMA 2020 and not achieving consensus will be further explored in the development of PRISMA-RR (box 2). Readers may be interested in exploring the feedback in the supplement to consider other reporting items until PRISMA-RR is available. For example, RR producers could consider providing a rationale as to why an RR rather than an SR was undertaken as part of the ‘Rationale’ PRISMA 2020 reporting item; we direct readers to another paper in this methods series that outlines the appropriateness of conducting an RR.36 No items proposed for the main checklist nor the abstract achieved consensus for exclusion.

Box 2 Main checklist items proposed but not achieving consensus for RR reporting.

Timeframe of conduct. This item would specify time parameters, such as the number of weeks from finalisation of protocol to draft report.

Intended users. This item was envisioned to specify the audience of interest, from which readers could understand the lens to which a discussion of the applicability and implications of the evidence were applied.

Comprehensive assessment. Producers could indicate whether a systematic review is warranted given the results of the rapid review.

Reviewing methodological advances with respect to RRs will need to occur with the development of PRISMA-RR. To this regard, we encourage RR producers to become familiar with other articles within this series. For example, producers considering team characteristics and organisation guidance could elect to report on the SR methodological expertise within the RR team and the number of team members participating at various conduct steps.2

Conclusions

Reporting has shown to be poor in RRs based on tools developed for SRs. As interim guidance pending the development of PRISMA-RR, we encourage RR producers to use PRISMA 2020 as the foundation for reporting and to consider PRIOR items when including secondary evidence. We further present additional items that can be considered, endorsed through an expert survey. We encourage the RR community to provide feedback to the corresponding author on the use of those items as we look to develop a consolidated list for PRISMA-RR. To strike a balance between practicality of presenting information for decision-makers and ensuring complete reporting, consider reporting clearly linked and easily accessible materials made available in open-access journals or repositories that are barrier-free.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statementsPatient consent for publication

Not applicable.

Ethics approval

This study involved human participants and was approved via delegated review by the Ottawa Health Science Network Research Ethics Board (Protocol 20180588-01H). Participants gave informed consent to participate in the study before taking part.

Acknowledgments

We thank David Moher (DM) for receipt of initial funding and important input on the project protocol and development of the survey.