In a single-centre study of preoperative frailty assessments performed by nurses and anesthesiologists, we found good agreement between CFS ratings between the two groups, consistent with our hypothesis. Agreement was stronger when the individual CFS score was used rather than a dichotomized score. Histograms showing the distribution of ratings by each group of assessors were similar. Sensitivity analysis confirmed good agreement within different age groups, although the lower confidence limit dropped to or below 0.5 in three of four age groups.

Our study is consistent with perioperative studies from the UK comparing agreement between anesthesiologists and perioperative nurses (good agreement; kappa, 0.61) and from Norway comparing agreement between an anesthesiologist and two medical students (good to very good agreement; kappa, 0.74 to 0.85).12,13 In critically ill patients, interrater reliability of the CFS was found to be good in three separate studies in the UK, Australia, and Wales.23,24,25 In the emergency department setting, a USA study reported very good interrater agreement between nurses and doctors (weighted kappa, 0.90) over the entire spectrum of CFS scores and also when dichotomized as vulnerable/frail vs not frail (weighted kappa, 0.80).26 A 2019 Canadian study in the emergency department found patient-registered nurse and patient-doctor interrater agreement was moderate (kappa, 0.51 and kappa, 0.42, respectively) and physician-registered nurse agreement was good (kappa, 0.72).27 In the cardiology setting, a 2011 Swedish study showed extremely high interrater reliability (intraclass correlation coefficient, 0.97).28 In the community setting, a 2018 Canadian study was conducted by a geriatric outreach service, showing good interrater reliability (kappa, 0.64).29

Most reports of interrater reliability in various populations have shown moderate or good interrater agreement, with the exception of the 2011 Swedish cardiology study observing extremely high agreement and the Norwegian perioperative study showing good to very good agreement.13,28 Studies of interrater reliability in distinguishing frail (CFS ≥ 5) from nonfrail (comprising both nonfrail [CFS 1–2] and prefrail [CFS 3–4]) have generally found “good” or “substantial” agreement (Cohen’s kappa, 0.61 to 0.80). This study improves on these prior investigations by examining agreement across each increment of the scale, highlighting the improved agreement that occurs when we use the full granularity of the scale. Conversely, the Bland–Altman plot shows that the 95% limits of agreement are quite wide (± 3), suggesting opportunities exist to improve routine frailty assessment in clinical practice, to apply scores across the full range of the scale, and to support application of the CFS by training.

Our study has several limitations. First, no formal training was provided to nursing or medical staff in CFS assessment, potentially leading to variability in assessment. Rater training should be considered in future.30 Second, we cannot exclude informal consultation between nursing and medical staff when assigning CFS rating, as part of good communication in routine clinical care. Third, we conducted a single centre study and our results in this setting require external validation. Finally, results from interrater reliability studies apply to a population of patients; the accuracy of interrater assessment for individuals can vary from patient to patient.

Strengths of our study include the large sample size achieved by collection of routine clinical data during a period of transition in the PAC, making this one of the larger CFS inter-rater reliability studies to date. Blinding of assessors was achieved through the workflow structure of nursing and medical assessors at PAC. This was a real-world study in a large regional university teaching hospital with a high PAC patient load, with no formal training provided to nursing or medical staff in CFS assessment, no exclusion criteria, and two thirds of patients being high risk (ASA–PS III–IV).

Our single-centre study suggests the CFS can be applied in routine clinical practice and is reliable across a population of patients when applied by perioperative nursing staff or anesthesiologists with no formal training in its use. It shows that CFS assessment can be included in the normal PAC workflow with a high degree of data capture, and that when part of routine workflow, it was recorded with almost 100% completeness by nursing staff but less frequently by anesthesiologists. These findings should ideally be confirmed in a multicentre study across a range of public and private hospital facilities in different geographical settings.

At John Hunter Hospital, CFS assessment will be formally moved to the nursing workflow for elective surgery patients seen in PAC. This model for CFS assessment could easily be adopted by other hospitals to enable its completion. A mechanism is similarly required for routine capture of CFS in the emergency surgery setting, and this will likely require it to be systematically recorded by the treating anesthesiologist as standard practice.

Frailty is an important predictor of perioperative outcomes and should be included in future risk-adjustment models in clinical trials, cohort studies, and registries. Further, the addition of frailty as a covariable improves the accuracy of surgical mortality risk prediction models.31,32,33,34 To undertake this modelling, a large data set containing frailty assessment will be required. Implementation of international recommendations for routine assessment and recording of frailty status in the perioperative setting should be an immediate priority for Australian state and territory health systems, led by specialist colleges of surgery and anesthesia. At the health system level, this can be achieved by including frailty status as a variable in Australia’s Admitted Patient Care National Minimum Data Set, regularly reported by public and private hospitals to state and territory health departments.35