Accessing AI mammography reports impacts patient follow-up behaviors: the unintended consequences of including AI in patient portals.

Study Overview

This randomized online experiment evaluated how including AI-generated mammography results in patient portals affects women’s reactions to a negative radiologist report.

Population and Design

• Participants: 1,623 English-speaking women in the US with at least one prior mammogram.
• Radiologist report: All participants saw a report indicating no breast cancer.
• Randomization: Participants were assigned to 13 conditions:
• Control: Radiologist report only.
• Intervention (12 conditions): Radiologist report plus an AI report with:
  • AI scores: One of four scores:
    • 0 or 29: No suspicion of cancer.
    • 31 or 50: Suspicion of cancer.
  • Presentation of AI score:
    • Alone
    • With an abnormality cutoff threshold
    • With the threshold and either the AI’s False Discovery Rate (FDR) or False Omission Rate (FOR)

Outcomes

• Primary outcome: Stated likelihood of consulting an attorney for litigation if advanced cancer were found one year later.
• Secondary outcomes:
  • Intended follow-up actions
  • Concern about cancer
  • Trust

Key Findings

• Litigation intent: Adding AI information increased participants’ stated likelihood of pursuing litigation compared with the radiologist report alone.
• Discordant results: The increase in litigation intent was strongest when the AI output suggested suspicion of cancer despite the negative radiologist report.
• Performance metrics: Presenting AI performance metrics (FDR or FOR) along with thresholds reduced the inclination toward litigation and similarly tempered changes in intended follow-up behavior.
• Psychological impact: These communication strategies also influenced concern about cancer and trust, generally moderating the disruptive impact of concerning AI scores.

Implication

How AI mammography results are presented in patient portals meaningfully shapes patients’ perceived medicolegal response, follow-up intentions, and psychological reactions. Including clear performance metrics (FDR/FOR) can mitigate adverse responses when AI findings conflict with a negative radiologist report.