The navigation workflow previously developed in [17,18,19] for open liver surgery was adapted for laparoscopic use. Instrument tracking was performed using the Aurora® V2 electromagnetic (EM) planar field generator (Northern Digital Inc., Waterloo, Canada). Three instruments were equipped with EM sensors: a laparoscopic pointer, a laparoscopic ultrasound (LUS) probe (I12C4f (9066), BK Medical, Denmark), and a 30° laparoscope (Endoeye 30° HD, Olympus, Japan). EM sensors used in this study were commercial products (Northern Digital Inc., Waterloo, Canada).

High-resolution 3D scans of the standard laparoscopic instruments (ultrasound probe and laparoscope) were acquired to map their exact geometry (Artec 3D, Luxembourg, Luxembourg). Using these scans, we designed and 3D-printed form-fitting adapters that house the EM sensors. This ensures a unique, stable fit for each instrument, which is critical for accurate calibration. The design of the adapters was performed using SolidWorks 2022 (Dassault Systèmes, Vélizy-Villacoublay, France) and 3D printed using a Formlabs Form 3B + stereolithography printer (Formlabs Inc., Somerville, MA, USA). The 3D-printed adapters are non-sterile components; they are attached to the instruments by the technician and covered with a sterile surgical drape prior to the procedure. Conversely, the liver sensor and the pointer are sterile devices that can be handled directly by the surgeon. This workflow ensures that standard hospital equipment can be tracked without permanent modification or compromised sterility.

The laparoscopic pointer was constructed from stainless-steel (316L) tubing with an EM sensor embedded at its distal tip. The LUS adapter was designed to create a unique alignment between the sensor and imaging plane while fitting through a standard 12-mm trocar. The laparoscope adapter positioned the EM sensor externally around the trocar sleeve to maintain proximity to the operative field and to avoid EM interference with the internal electronics of the laparoscope, as described by Liu et al. [20]. All tracked components were designed to remain within the effective 50 × 50 × 50 cm operational tracking volume of the field generator when used clinically (Fig. 1).

Schematic overview of the tracked instruments, including the laparoscope, laparoscopic ultrasound probe, pointer, and liver sensor with sensor positions denoted by red circles

Preoperative calibration of the LUS probe was performed with an EM-tracked pointer, and the laparoscope was calibrated using Zhang’s method [21, 22]. The instrument 3D models, together with their calibrations, were integrated into the CustusX version 18.04 (SINTEF, Trondheim, Norway) navigation software. To enable AR navigation, the laparoscopic video feed was incorporated into the platform by routing the Olympus video output through a frame grabber (Epiphan DVI2USB 3.0) and importing it into the navigation system. A virtual environment was developed in Unity 3D (Unity Technologies, San Francisco, USA), allowing real-time integration of 3D instrument models, tracking data, and laparoscopic video.

The workflow for liver-sensor placement, landmark acquisition, and registration is described in the Clinical feasibility study section.

Four AR display modes were designed in Unity 3D to optimize anatomical visualization: (a) complete 3D model showing internal structures and liver surface; (b) the planned resection plane and vascular branches supplying the resected segment; (c) 3D model limited to a circular region centered on the laparoscopic image to reduce visual clutter; and (d) increased transparency of structures further away from the laparoscope to improve depth perception.

Ex vivo experiments of the four AR displays were performed using the IOUSFAN phantom (Kyoto Kagaku Co., Ltd, Kyoto, Japan). Video recordings of these experiments were presented to seven hepatobiliary surgeons from two hospitals. The surgeons were asked to indicate which hepatic structures they considered essential for AR display and which visualization modes they preferred for clinical use. Multiple preferences could be selected. The phantom study provided a general impression of overall preferred visualization modes and familiarized surgeons with the available options. Visualization preferences varied between surgeons, and were also influenced by the specific clinical task. Given the feasibility nature of the study, no standardized visualization mode was enforced in the clinical study. Instead, for each clinical procedure, the operating surgeon selected the preferred visualization mode based on prior experience with the phantom study and the anticipated intraoperative requirements.

A prospective, single center feasibility study was conducted at the Netherlands Cancer Institute. Consecutive patients scheduled for laparoscopic resection of liver tumors between September 2023 and March 2025 were eligible for inclusion, irrespective of tumor type, size, or location. Exclusion criteria included patients with vanished lesions on intraoperative ultrasound. While this patient group represents a primary indication for image guidance, tumor visibility was strictly required in this study to serve as a ground truth for calculating the target registration error (TRE) and validating system accuracy. Patients with a pacemaker were also excluded because of potential interference with the EM field generator.

This study was conducted in accordance with the Declaration of Helsinki and was approved by the institutional medical ethics committee in August 2023 (NL65724.031.18) and written informed consent was obtained from all participants prior to surgery. Demographic characteristics and perioperative variables were recorded, including operative time, conversion to open surgery, postoperative complications, length of hospital stay, length of hospital stay and resection margin status (R0/R1).

3D models of the hepatic anatomy were generated preoperatively from diagnostic scans according to [18, 19]. Prior to each procedure, the preferred AR visualizations were discussed with the operating surgeon and adjusted accordingly for the specific case. The LUS probe, laparoscope, and field generator were covered with sterile drapes. To compensate for local movement due to respiration and surgical manipulation, an EM sensor was secured intraoperatively to the parenchyma adjacent to the target tumor using a laparoscopic glue applicator (Glubran®, GEM, Italy). The sensor was positioned within the planned resection planes, ensuring that it was removed together with the tumor. Registration of the preoperative three-dimensional (3D) liver model to the intraoperative anatomy was performed in CustusX. Corresponding anatomical landmarks near the target lesion were identified on the preoperative model and digitized intraoperatively using tracked instruments. Subsurface landmarks, such as vascular bifurcations and cysts, were sampled with the EM-tracked LUS probe, whereas surface landmarks, including ligament insertions, were digitized using the tracked laparoscopic pointer. Registration accuracy was verified by superimposing the 3D model onto the live ultrasound image and assessing correspondence of structures such as vessels and tumor (Fig. 2). To enable AR, the registration matrix was transferred to Unity.

Navigation visualization with a 3D model of the liver with the parenchyma (brown), hepatic vein (blue), portal vein (purple) and tumor (yellow) on the right side and a cross-sectional overlay of the 3D model over the live ultrasound image on the top left. This allows for direct assessment of registration accuracy

The primary outcome was feasibility of the complete navigation workflow, including sensor attachment, registration and AR visualization without technical errors. Secondary outcomes included registration accuracy, time required for individual study steps, and the intraoperative performance of augmented reality overlays. Registration accuracy was assessed using the target registration error (TRE), defined as the positional difference between the tumor location indicated by the navigation system and its intraoperative location. To obtain an intraoperative reference, the tracked LUS probe was used to acquire a sweep encompassing the entire tumor, from which a 3D ultrasound volume was reconstructed. The tumor was segmented from this volume and its center was used as the reference position. TRE was calculated as the Euclidean distance between this ultrasound-derived tumor center and the corresponding tumor center in the navigation system after registration. In addition, the fiducial registration error (FRE) was computed as the root-mean-square of the residual distances between all digitized anatomical landmarks (ranging from 3 to 6 landmarks per case) and their corresponding points on the preoperative model after registration. In addition, time was recorded for key steps of the workflow, including attachment of the liver sensor, registration and initializing of AR. In this feasibility study, AR visualization was restricted to observational use and not applied for surgical guidance.