Treatment with antiresorptive agents for osteoporosis is associated with an elevated risk of developing hypocalcemia. Other medications have also been associated with hypocalcemia.1 Postauthorisation monitoring of drug safety using health and administrative registries and databases is essential. Hypocalcemia was a safety outcome of interest in a long-term postauthorization safety study (PASS) of denosumab among women with postmenopausal osteoporosis (PMO).2 This international study used claims data in the United States and administrative data from population-based registries in Denmark, Norway and Sweden. Hypocalcemia was operationalized as the presence of a diagnosis code in the primary position recorded at a hospitalization or an emergency visit. In the included Scandinavian registries diagnoses are coded using the International Classification of Diseases, 10th Revision (ICD-10). Denmark uses a local adaptation of ICD-10, whereby some codes are more granular than the codes of the World Health Ortanization (WHO) version of ICD-10. Norway and Sweden use the WHO version. Specifically, the Danish adaptation of ICD-10 contains distinct codes for disorders of calcium metabolism, hypocalcemia and hypercalcemia. By contrast, the WHO ICD-10 provides only the general code ICD-10 code E83.5 “Disorders of the calcium metabolism”, which can be used both for hypo- and hypercalcemia. In Denmark, where registry data on laboratory values are available in addition to the diagnosis codes, a comparison of the general and the specific codes against laboratory values showed that among patients with the general code E83.5, only 20% had lowered calcium levels.3 Thus, if use of the general code to operationalize hypocalcemia may substantially overestimate the risk of this adverse event.

The objective of this study was to estimate the positive predictive value (PPV) of a case ascertainment algorithm for hypocalcemia leading to hospitalization or emergency room (ER) visit in the Swedish National Patient Register among women with PMO treated with bisphosphonates or denosumab. As there are no formal full-length studies validating ICD-10-based algorithms for hypocalcemia, the aim of the study was to validate this safety outcome, which is included in an ongoing regulator-mandated multi-database PASS.2

This validation study was conducted in Sweden,4 as it was one of the participating sites in the PASS. Sweden is a welfare state, with tax-funded health care contributing routinely collected data to multiple individually linkable population registries, including Swedish Patient Register, Population Register, and Prescribed Drug Register.4–8 Participants were a a convenience sample representing patients contributing to an interim report of the PASS, with the same eligibility criteria: women with PMO treated with bisphosphonates or denosumab between 26 May 2010 to 31 December 2016. PMO was defined based on age (≥55 years), combined with subsequent diagnoses indicative of osteoporosis, fragility fracture, or with a record of osteoporosis-specific treatment proxy. Patients with a history of Paget’s disease or cancer were excluded. The specific additional inclusion criterion for the validation study was a record of hypocalcemia identified based on being hospitalized or having an ER visit with the primary diagnosis ICD-10 code E83.5 between the date of on-study initiation of denosumab or bisphosphonate through 31 December of 2018. The cohort of the potential cases was defined using data from the Swedish Patient Register.9 Applying these eligibility criteria identified 164 women treated at 87 different hospitals across Sweden (Figure 1).

Figure 1 Flowchart of Patient Identification.

Medical chart review was used as the reference standard to establish the presence of hypocalcemia, and information from the medical charts was obtained using a standardized abstraction form. After obtaining ethical approval, we requested each department/hospital to provide the medical charts for the visits associated with the ICD-10 code. The hypocalcemia case definition was based on a combination of serum calcium levels, intravenous calcium administration, and clinical assessment. Chart abstractors were blinded with respect to the antirresorptive treatment type (bisphosphonates or denosumab), and case status was adjudicated by a physician as definite case, definite non-case, or insufficient information. An independent second physician adjudicated a random 20% sample of the potential cases, and the inter-rater variability was measured using the kappa coefficient.

The number and proportion of the potential cases with and without retrieved charts for validation were reported. Patients’ characteristics, as collected based on the registry data in the PASS were tabulated descriptively overall and by chart retrieval status (medical records requested, records not obtained, records obtained, records with insufficient information). Among the patients with retrieved charts and with sufficient information to determine the case status, the number, and proportion of confirmed cases and PPV was reported with exact 95% confidence intervals (CIs).

The study was approved by the Swedish Ethical Board (Dnr 2021-02304). The data accessed complied with all relevant local data protection and privacy regulations. No informed consent was required per local regulations for retrospective studies of medical charts. Each head of staff of the different clinics where the patients had received their diagnosis had to comply to providing data.

Of the 164 eligible patients with potential hypocalcemia, hospital charts were available for 125 (76.2%) patients. Out of the 39 patients who lacked medical charts, the records were unavailable for 4 patients, access to the records was declined for 2 patients, and the medical department did not respond for 33 patients (Table 1). A total of 121 of 125 patients (96.8%) with available medical charts had sufficient information for adjudication.

Table 1 Characteristics of Treated Women with PMO with Potential Hypocalcemia Leading to Hospitalization or Emergency Room Visit by Chart Retrieval Status

The median age at the recorded hospital visit for hypocalcemia was 76 years (interquartile range (IQR): 69–82 years). The number of potential cases increased with the calendar year. Most patients (n=90) were treated at a department of internal medicine. Additionally, the majority of patients (n=135) were inpatients, and 29 patients had an ER visit.

Among the 121 patients with sufficient information for adjudication, 19 patients fulfilled the criteria for definite case of hypocalcemia. The remaining patients either had hypercalcemia (n=97) or had no calcium level abnormalities (n=5). The PPV for the cases with sufficient information was 15.7% (95% CI: 10.0% to 23.0%) and ranged from 5.3% to 40.0% in different departments (Table 2). Twenty-six cases were subjected to adjudication by a second physician, and for one of these cases, the opinion did not correspond with the original adjudication, kappa coefficient 0.96.

Table 2 Positive Predictive Value of ICD-10 Code for Hypocalcemia in the Swedish National Patient Register Among Women with Postmenopausal Osteoporosis

In this nationwide validation study of hypocalcemia diagnoses among women with PMO, the PPV for definite hypocalcemia associated with the ICD-10 code E83.5 at hospitalization or ER visit recorded in the Swedish Patient Registry was 15.7% (95% CI: 10.0% to 23.0%).

Adjudication was possible for 76.2% of the eligible patients with potential hypocalcemia, limiting the precision of the estimated PPV. The case adjudication, initially performed by one physician, showed high correlation with the validation process conducted by a second physician. Although access was not granted to some patients, the prevalence of nonresponse was not associated with geographic regions or medical departments, suggesting that nonresponse was unlikely to have affected the PPV. As the study was not designed to estimate the sensitivity of the case-finding algorithm, the proportion of cases not captured remains unknown. In addition to non-response, low precision of some estimates is a limitation.

The result was in line with the findings from Denmark, where it was observed that only a minority (20%) of the patients with the nonspecific code for disturbances of calcium metabolism had hypocalcemia.3 Similarly, the same study, even when using a specific code, yielded a PPV of only around 40% during case adjudication.3 One reason for this finding could be the low prevalence of hypocalcemia in the target population, as the PPV is directly proportional to prevalence.

The records in the Swedish National Patient Register have demonstrated high validity for many, but not for all diagnoses.9 The validity of a specific algorithm depends on various factors, including proportion of cases leading to an encounter that triggers a record in the data source. In this study, the primary diagnosis at hospitalization or ER visit was part of the case-finding algorithm, with the aim of identifying hypocalcemia as the main reason for the encounter. Furthermore, the available nonspecific diagnosis code could be used for both hypo- and hypercalcemia. The results of this study underscore the importance of validating case-defining algorithms in routinely collected data.10 In the absence of information about whether the validity of diagnosis varies by type of treatment received and about sensitivity of the code in capturing true cases, the direction of the resulting bias cannot be predicted. The same principles are likely to generalize to other data sources using non-specific diagnosis codes.

The ICD-10 code assessed here is not appropriate for operationalizing hypocalcemia in epidemiologic studies based on the Swedish Patient Register.

The authors acknowledge Leslie Spangler for contributions to the conception and design of the study and for input into early versions of the manuscript.

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

This study received institutional funding from Amgen Inc. and administered by Aarhus University, with further subcontract to the Karolinska Institutet.

AK, POL, TT, AT, HT, and VE are salaried employees of their respective institutions. DH is employed at the Centre for Pharmacoepidemiology, which carries out research projects funded by several entities (pharmaceutical companies, regulatory authorities and contract research organizations), all regulator-mandated Phase IV-studies, all with funds paid to the institution where he is employed (no personal fees) and with no relation to the work reported in this study. MK is an employee and owns equity in Amgen. The authors report no other conflicts of interest in this work.

1. Fong J, Khan A. Hypocalcemia: updates in diagnosis and management for primary care. Can Fam Physician. 2012;58(2):158–162. doi:10.1002/jbmr.483

2. Xue F, Ma H, Stehman-Breen C, et al. Design and methods of a postmarketing pharmacoepidemiology study assessing long-term safety of Prolia(R) (denosumab) for the treatment of postmenopausal osteoporosis. Pharmacoepidemiol Drug Saf. 2013;22(10):1107–1114. doi:10.1002/pds.3477

3. Antonsen S, Christiansen CF, Nørgaard M, et al. The validity of hypercalcemia and hypocalcemia diagnoses in Danish registries (Abstract at the 27th International Conference on Pharmacoepidemiology and Therapeutic Risk Management). Pharmacoepidemiol Drug Saf. 2011;20(S1):S161.

4. Ludvigsson JF, Almqvist C, Bonamy AK, et al. Registers of the Swedish total population and their use in medical research. Eur J Epidemiol. 2016;31(2):125–136. doi:10.1007/s10654-016-0117-y

5. Ludvigsson JF, Otterblad-Olausson P, Pettersson BU, et al. The Swedish personal identity number: possibilities and pitfalls in healthcare and medical research. Eur J Epidemiol. 2009;24(11):659–667. doi:10.1007/s10654-009-9350-y

6. Johansson LA, Westerling R. Comparing Swedish hospital discharge records with death certificates: implications for mortality statistics. Int J Epidemiol. 2000;29(3):495–502. doi:10.1093/ije/29.3.495

7. Swedish Registers at the National Board of Health and welfare. Available from: http://www.socialstyrelsen.se/register. Accessed August15, 2025.

8. National Prescribed Drug Register. Available from: https://www.socialstyrelsen.se/en/statistics-and-data/registers/national-prescribed-drug-register/. Accessed August19, 2025.

9. Ludvigsson JF, Andersson E, Ekbom A, et al. External review and validation of the Swedish national inpatient register. BMC Public Health. 2011;11(1):450. doi:10.1186/1471-2458-11-450

10. Ehrenstein V, Hellfritzsch M, Kahlert J, et al. Validation of algorithms in studies based on routinely collected health data: general principles. Am J Epidemiol. 2024;193(11):1612–1624. doi:10.1093/aje/kwae071