In medical documentation, vast amounts of unstructured text are generated that are still underutilized in current prognostic models. We investigate the potential of self-hosted large language models (LLM) to extract clinically meaningful, patient-specific information from routine clinical notes for personalized risk stratification in cancer care.

We collected real-world medical notes from 2,708 non-small cell lung cancer (NSCLC) patients and 814 colon cancer patients documented before treatment at a large comprehensive cancer center. LLMs extracted key prognostic indicators, including comorbidities, metastatic sites, and qualitative descriptors of patient condition, in a zero-shot manner without prior task-specific training. Integrating these LLM-derived features into machine learning models significantly improved the prediction of overall survival compared to TNM staging alone (C-Index: NSCLC, 0.72 vs 0.64; colon cancer, 0.70 vs 0.59), and surpassed models using text embeddings. Based on the LLM-informed risk scores, patients were stratified into four distinct risk groups, enabling reclassification of 61.4% of NSCLC and 68.3% of colon cancer patients. Analysis of model drivers revealed that LLM-derived factors, such as the physical condition, substantially modulated the prognostic impact of TNM stage.

These findings highlight the potential of self-hosted LLM to extract clinically meaningful information from unstructured clinical documentation and support clinical decision-making.

J.T.S. receives honoraria as a consultant or for continuing medical education presentations from AstraZeneca, Bayer, Boehringer Ingelheim, Bristol Myers Squibb, Immunocore, MSD, Novartis, Roche/Genentech, and Servier. His institution receives research funding from Abalos Therapeutics, Boehringer Ingelheim, Bristol Myers Squibb, Celgene, Eisbach Bio, and Roche/Genentech; he holds ownership in FAPI Holding (<3%); all are outside the submitted work. All other authors declare no conflicts of interest related to this study.

J.Keyl is supported by a German Research Foundation (DFG)-funded clinician scientist program (FU 356/12-2). The work of Markus Pauly was funded by the Deutsche Forschungsgemeinschaft (grant 352692197).

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The study was approved by the Ethics Committee of the Medical Faculty of the University Duisburg-Essen (No. 22-10881-BO). The requirement for written informed consent was waived due to the retrospective design of the study.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Anonymized data are available from the corresponding author upon reasonable request. Data cannot be shared with investigators outside the institution without consent.