Study design

Validation study comparing two ultrasound techniques, conducted during routine clinical practice.

It was carried out in the general ultrasound unit of the San Francisco de Asís University Hospital in Madrid, between November 2022 and September 2023. All measures were obtained by a family doctor with more than 15 years of experience in the central imaging department of this hospital.

All the patients recruited underwent two ultrasound examinations by the same technician: (1) first, a POCUS (point-of-care ultrasound) with a General Electric® Vscan Air, a test which is performed in a short period of time and must answer a specific clinical question; and (2) standard ultrasound in the general ultrasound unit, with no time limit, using a high-end Samsung RS 80® ultrasound scanner. This second examination was considered the reference standard or gold standard, and was the test used to establish the final diagnosis. Both tests were done at the same time, first the POCUS and second the standard ultrasound. This sequence prevents the physician from having any reliable information during the POCUS examination, but not in the standard ultrasound. Our design avoids bias related to inter-observer variability and also avoids any advantage for the HUD, which is the device that we were evaluating.

This study was approved by the Ethics Committee for Investigation with Medicinal products of the Princesa University Hospital, Madrid, with the registration code SEM_ECO_1_22, on 23-06-22.

Patients

This research included all the patients from the emergency department and inpatient unit who underwent abdominal ultrasound during the study period, and who were referred to the Principal Investigator’s office.

Patients who did not sign the informed consent form and those who had the two scans performed by different technicians were excluded.

The sample size required to assess agreement between two observer-dependent diagnostic imaging tests was calculated, ensuring that same professional performed all the examinations. This significantly minimizes observation bias. A sample size of more than 100 patients was considered sufficient to ensure adequate statistical power to confirm agreement between the two techniques. The sample size was estimated based on the prevalence of each group of diseases considering the prevalence of each disease [6, 7], as explained in the following section.

Based on this estimate, the recruitment period was limited to six months. According to the previous year’s records (with an average of 300 requests for abdominal ultrasound per year), this interval was considered sufficient to reach the required sample and also to avoid temporal biases associated with longer recruitment periods.

Diagnostic criteria

The diagnosis of the different abdominal pathologies was made according to the following criteria:

Hepatomegaly: a longitudinal axis of the right liver lobe greater than 15.5 cm at a midclavicular longitudinal section [8].

Hepatic steatosis: diffusely increased hepatic echogenicity with posterior attenuation and loss of definition of the intra-hepatic vessels [8].

Cirrhosis was suspected if there was coarse-grained structure, nodular surface, hypertrophy of the caudate lobe, spleno-portal axis abnormalities, portal vein greater than 13 mm, and presence of indirect signs such as ascites or collateral circulation [8].

Splenomegaly: greater than 12 cm at the longitudinal axis [9].

Gallbladder pathology: enlarged gallbladder when the transverse axis was greater than 4 cm and the wall was thickened more than 3 mm. The bile duct should be < 8 mm [6].

Pancreas: normal size between 12.5 and 20 cm [7].

Data analysis

Categorical data were presented as absolute frequency and percentage (%). Age was summarized as mean, standard deviation, minimum and maximum.

The diagnoses were grouped for the analysis by anatomical regions, considering the following classification:

HEPATIC PATHOLOGY: hepatomegaly, space-occupying lesions, signs of diffuse liver disease.

INTESTINAL PATHOLOGY: signs of appendicitis, signs of diverticulitis, signs of intestinal obstruction.

PANCREATIC PATHOLOGY: enlargement, space-occupying lesions, altered echogenicity.

RENAL PATHOLOGY: hydronephrosis, renal lithiasis, solid lesions.

COMPLICATED RENAL PATHOLOGY: urinoma, pyonephrosis, signs of pyelonephritis, abscess.

BLADDER PATHOLOGY: polyp, lithiasis, trabeculated bladder, clot.

GALLBLADDER AND BILIARY PATHOLOGY: enlargement, wall thickening, lithiasis, biliary sludge, intra- or extrahepatic bile duct dilatation.

OTHERS: aortic pathology, spleen pathology, ascites, pleural effusion, prostate pathology.

Normal result

IBM SPSS Statistics version 23 was used for statistical analyses. The Forest Plot was made with Review Manager 5.4.1.

Outcomes

The positive and negative predictive values, and the sensitivity and specificity of POCUS were calculated in comparison with standard ultrasound, which was the gold standard. A Forest Plot was designed with the sensitivities and specificities of the different pathologies and their 95% confidence intervals (95% CI).

Cohen’s kappa was used to assess the agreement between the two ultrasound techniques. Values 0.00 to 0.20 were interpreted as no or slight agreement, 0.21 to 0.40 as fair agreement, 0.41 to 0.60 as moderate agreement, 0.61 to 0.80 as substantial agreement, and 0.81 to 1.00 as almost perfect agreement.

In the hypothesis test, the null hypothesis was rejected in comparisons with p values < 0.05.

Results

A total of 93 patients were recruited, of which 52.7% were women. The mean age was 65.6 (23.6) years, with 38.7% over 80 and 36.6% under 60. As regards body mass index (BMI), 11.8% had a BMI greater than 30 kg/m2 and 14.0% had a BMI < 20 kg/m2.

Of the examinations performed, 69.9% were abdominal and the rest were urological. By anatomical systems, renal pathology was the most prevalent (23.6%, n = 22), followed by gallbladder and biliary pathology (14%, n = 13), bladder pathology (9.7%, n = 9) and gastrointestinal pathology (9.7%, n = 9). Liver pathology accounted for 6.5% (n = 6) of diagnoses, other pathological findings were 8.6% (n = 8) of cases and 20.4% (n = 19) were normality cases. The other pathological findings included ascites (n = 2), prostatic hypertrophy (n = 2), abdominal aortic aneurysm (n = 1), pelvic mucocele (n = 1), splenic cyst (n = 1) and pleural effusion (n = 1). Figure 1 shows agreement for diagnosis of the different pathologies by system.

Fig. 1

Kappa indices for each group of pathologies

A diagnostic agreement of 89% was found between HUDs and high-end ultrasound devices. Figure 2 represents sensitivity (Sn) and specificity (Sp) for abdominal pathologies.

Fig. 2

Forest Plot of sensitivity and specificity for each of the pathologies. TP: true positive; FP: false positive; TN: true negative; FN: false negative. 95% CI: 95% confidence interval

The 11 cases in which agreement between the two devices was not detected corresponded to the following pathologies: 3 (27.3%) to gastrointestinal tract disorders (diverticulitis, inguinal hernia), 3 (27.3%) to pancreatic pathology (mild pancreatitis), 3 (27.3%) to complicated renal pathology (pyonephrosis, pyelonephritis and complex cyst), 1 (9%) to gallbladder pathology (polyp) and 1 (9%) to ascites.