Large language models (LLMs) with script concordance testing (SCT)

The benchmarking study that evaluated the clinical reasoning performance of large language models (LLMs) using Script Concordance Testing (SCT) provided several insights into their capabilities and limitations in medical applications. Below is a detailed summary of the findings and their implications:

---

### What is Script Concordance Testing (SCT)?

SCT is a validated method for assessing clinical reasoning under uncertainty. Unlike multiple-choice questions (MCQs) that focus on knowledge recall, SCT evaluates how new information influences diagnostic or management decisions. It is particularly suited for scenarios where there is no single "correct" answer, but instead a range of plausible responses that depend on context.

---

### Key Findings from the Study

#### 1. **Clinical Context and Need for SCT**

• Current benchmarks for LLMs in medicine, like licensing exams, primarily assess knowledge recall rather than real-time diagnostic reasoning under uncertainty.
• SCT is better aligned with real-world clinical reasoning, making it a valuable tool for evaluating LLMs in medical contexts.

#### 2. **Benchmark Scope**

• The study created a 750-question SCT benchmark using datasets from 10 international sources spanning specialties like pediatrics, neurology, internal medicine, emergency medicine, surgery, and physiotherapy.
• This broad scope ensured that the test covered diverse clinical scenarios and minimized the risk of data leakage.

#### 3. **Human vs. Model Comparison**

• The SCT performance of LLMs was compared to 1,070 medical students, 193 residents, and 300 attending physicians.
• This direct comparison provided a clear understanding of how LLMs measure up to human clinicians at different levels of expertise.

#### 4. **Performance of LLMs**

• Among the 10 LLMs tested, OpenAI’s o3 model performed best, achieving a score of 67.8%, followed by GPT-4o at 63.9%.
• These scores approached, but did not match, the performance of senior residents or attending physicians.
• Other models, such as Google’s Gemini 2.5, scored lower (52.1%), and reasoning-optimized models like DeepSeek R1 (55.5%) underperformed expectations.

#### 5. **Comparison with Medical Students**

• LLMs performed on par with or better than medical students in many cases.
• However, they consistently underperformed compared to senior residents and attending physicians, indicating that LLMs still have limitations in advanced clinical reasoning.

#### 6. **Overconfidence Bias**

• LLMs demonstrated a tendency to choose extreme Likert-scale ratings (+2 or −2) more often than neutral responses (0).
• This overconfidence in probability adjustments is a significant deviation from the balanced approach used by expert clinicians.

#### 7. **Performance of Reasoning-Optimized Models**

• Surprisingly, models fine-tuned for explicit reasoning (e.g., chain-of-thought models) performed worse on SCT.
• This suggests that such tuning may impair the models' ability to exercise flexible judgment under uncertainty.

#### 8. **Differences in Reasoning Patterns**

• Human clinicians tended to use more balanced probability shifts, while LLMs exaggerated belief changes, often missing cases where new information should not alter the decision.
• This highlights a key limitation in the probabilistic reasoning of LLMs.

#### 9. **Benchmark Validity**

• SCT demonstrated a performance gradient consistent with human expertise levels (students < residents < attendings), reinforcing its validity as a measure of clinical reasoning.

#### 10. **Advantages of SCT Over MCQs**

• SCT emphasizes probabilistic, context-sensitive reasoning, which is critical in clinical practice but often not captured in MCQ-style tests.
• This makes SCT a more rigorous and realistic benchmark for evaluating LLMs in medicine.

#### 11. **Challenges and Limitations**

• SCT is abstracted and does not fully replicate the complexities of real-world clinical reasoning, such as interpersonal dynamics or system-level decision-making.
• The scoring system may inflate results for models that mimic conservative responses and penalize models that reason well but deviate from consensus.

---

### Implications for the Use of LLMs in Medicine

1. **Clinical Decision Support**

• LLMs show promise as tools for supporting medical decision-making, particularly for medical students and junior clinicians.
• However, their overconfidence and inflexibility highlight the need for human oversight to ensure safe deployment.

2. **Training and Refinement**

• The study underscores the importance of refining LLM training to improve probabilistic reasoning and reduce overconfidence.
• Future efforts should focus on aligning model reasoning more closely with expert clinician patterns.

3. **Benchmarking and Validation**

• The public release of the SCT benchmark (https://www.concor.dance) will enable researchers to develop better prompts, improve model training, and integrate LLMs more effectively into clinical workflows.

4. **Limitations in Current Models**

• The study highlights that LLMs are not yet ready to independently handle complex clinical reasoning tasks, particularly under uncertainty.
• Their performance on SCT suggests that they still lack the nuanced judgment required for advanced medical practice.

5. **Educational Applications**

• The weak correlation between SCT and MCQ scores in both humans and models supports the idea that SCT measures reasoning skills beyond factual knowledge.
• LLMs could potentially be used as educational tools to help medical students and trainees develop clinical reasoning skills.

---

### Future Directions

• **Model Improvement:** Researchers can use the insights from this study to refine LLMs, focusing on probabilistic reasoning and reducing overconfidence.
• **Expanded Testing:** Additional benchmarks that incorporate real-world complexities, such as interpersonal and systemic factors, could further evaluate LLM capabilities.
• **Integration with Clinicians:** LLMs may be better suited for augmenting, rather than replacing, human decision-making in clinical settings.

---

In summary, while LLMs demonstrate promising capabilities in clinical reasoning as evaluated by SCT, they are not yet on par with senior clinicians. Their overconfidence, exaggerated belief changes, and difficulty with nuanced judgment under uncertainty limit their current utility in independent clinical practice. However, the SCT benchmark has provided valuable insights that can guide the development of more robust and reliable medical AI systems.