Human-AI teaming to improve accuracy and efficiency of eligibility criteria prescreening for oncology trials: a randomized evaluation trial using retrospective electronic health records.

This study evaluated whether a large language model can safely and effectively assist research staff in prescreening cancer patients for clinical trial eligibility.

Design:

• Randomized noninferiority trial using retrospectively collected charts from 355 adults with non-small cell lung or colorectal cancer.
• Compared two arms:
• Human-alone: prescreening by trained research staff.
• Human+AI: same staff augmented with a pretrained neurosymbolic AI language model that abstracted trial eligibility criteria from unstructured clinical notes.
• Primary endpoint: chart-level prescreening accuracy.
• Secondary endpoint: efficiency (average time per chart).

Key findings:

• Accuracy: Human+AI prescreening was both noninferior and statistically superior to Human-alone (76.5% vs. 71.1%).
• Efficiency: No meaningful change in time per chart (37.4 vs. 37.8 minutes).
• Domains of greatest benefit: biomarker status, disease staging, and treatment response criteria, where AI support improved correct identification of trial-eligible patients.
• Limitations: Performance gains were constrained in some areas due to automation bias, where staff appeared to over-rely on AI outputs.

Implications:

• AI-augmented prescreening can modestly improve the accuracy of identifying trial-eligible oncology patients without increasing staff time.
• By better capturing eligible patients—especially in complex criteria domains—this human-AI framework has potential to increase clinical trial enrollment, though workflow and cognitive biases need further optimization.