ORCHESTRATOR_TEST_CASE.md

# 🎼 Comprehensive Orchestrator Test Case

## Test Scenario: Complex Multi-Drug Resistant Infection

This test case demonstrates the orchestrator's ability to coordinate multiple specialist agents to solve a complex infectious disease case requiring expertise in multiple domains.

---

## 🎯 Setup: Create Your Agent Team

### Step 1: Create Specialist Subagents

Create these **4 specialist agents** first (they will be the orchestrator's team):

#### Agent 1: Stewardship Specialist
```
Agent Type: 🎯 Specialist
Agent Name: Stewardship Expert
Mission: Expert in antimicrobial stewardship, antibiotic selection, de-escalation strategies, and optimizing duration of therapy
Skills:
  ✓ recommend_deescalation
  ✓ alert_prolonged_antibiotic_use
  ✓ search_pubmed (if you want literature support)
```

#### Agent 2: ID Diagnostician
```
Agent Type: 🎯 Specialist
Agent Name: ID Diagnostician
Mission: Expert in infectious disease diagnosis, differential diagnosis, culture interpretation, and diagnostic workup
Skills:
  ✓ differential_diagnosis
  ✓ generate_ddx_mermaid
  ✓ search_pubmed
```

#### Agent 3: ICU Consultant
```
Agent Type: 🎯 Specialist  
Agent Name: ICU Sepsis Consultant
Mission: Expert in critical care infectious diseases, sepsis management, hemodynamic support, and ICU-specific considerations
Skills:
  ✓ Any relevant tools you want
  ✓ search_pubmed
```

#### Agent 4: Infection Prevention Specialist
```
Agent Type: 🎯 Specialist
Agent Name: IPC Specialist
Mission: Expert in infection prevention and control, isolation precautions, outbreak management, and transmission-based precautions
Skills:
  ✓ ipc_reporting (if available)
  ✓ Any other relevant tools
```

### Step 2: Create the Orchestrator
```
Agent Type: 🎼 Orchestrator
Agent Name: ID Maestro
Mission: Coordinate multiple ID specialists to provide comprehensive infectious disease consultation for complex cases requiring multidisciplinary expertise
Skills:
  (Orchestrators have access to all your other agents automatically)
API Key: Your OpenAI API key
```

---

## 📋 Test Case: Complex Septic Patient

### Patient Presentation:

Copy and paste this detailed case to the **ID Maestro** orchestrator:

```
I need help with a complex case:

Patient: 68-year-old male
PMH: Type 2 diabetes (A1C 9.2%), ESRD on hemodialysis (MWF), recent hospitalization 3 weeks ago for MRSA bacteremia from infected dialysis catheter

Current Presentation:
- Day 3 ICU admission for septic shock
- Fever 39.8°C, HR 118, BP 85/50 on norepinephrine 0.15 mcg/kg/min
- WBC 24,000 with 18% bands, procalcitonin 8.5
- Lactate 4.2 → 2.8 after 4L crystalloid

Cultures:
- Blood cultures (×2 sets): Gram-positive cocci in clusters at 14 hours
- Preliminary: MRSA (same strain as 3 weeks ago)
- Sensitivities pending (results in 24 hours)
- Urine culture: Negative
- CXR: Right lower lobe infiltrate

Current Antibiotics (started 72 hours ago):
- Vancomycin 1g IV q12h (trough pending)
- Piperacillin-tazobactam 3.375g IV q6h

Additional Info:
- New tunneled dialysis catheter placed 2 weeks ago
- Patient on contact precautions
- Last vancomycin trough (from previous admission): 18 mcg/mL
- CrCl: Not applicable (on dialysis)
- Patient improving clinically: BP improving, lactate trending down

Questions:
1. Is current antibiotic coverage appropriate?
2. Should we de-escalate or change therapy?
3. What's the optimal duration?
4. Any diagnostic workup needed?
5. Are isolation precautions correct?
6. What are the key stewardship considerations?
```

---

## ✅ Expected Orchestrator Behavior

### Phase 1: Planning
The orchestrator should:
1. **Analyze the request** and identify multiple distinct tasks
2. **Create an execution plan** listing which agents to invoke:
   - ID Diagnostician (for differential diagnosis and culture interpretation)
   - Stewardship Expert (for antibiotic optimization and de-escalation)
   - ICU Sepsis Consultant (for critical care considerations)
   - IPC Specialist (for isolation precautions)
3. **Display the plan** with numbered steps
4. **Wait for your confirmation** ("proceed")

### Phase 2: Execution (after you say "proceed")
The orchestrator should:
1. **Invoke each specialist agent** sequentially or in parallel
2. **Show progress**: "🚀 Invoking [Agent Name]..."
3. **Collect responses** from each specialist
4. **Display intermediate results** as they come in

### Phase 3: Synthesis
The orchestrator should:
1. **Synthesize all specialist inputs** into a comprehensive recommendation
2. **Address all 6 questions** from the original query
3. **Highlight agreements** between specialists
4. **Resolve conflicts** if specialists disagree
5. **Provide prioritized recommendations**
6. **Include specific details** from each specialty area

---

## 📊 What to Look For

### ✅ Good Orchestrator Performance:

**Planning Phase:**
- [ ] Identifies 4-6 distinct tasks in the case
- [ ] Plans to invoke 3-4 specialist agents
- [ ] Explains rationale for each agent selection
- [ ] Asks for confirmation before proceeding

**Execution Phase:**
- [ ] Shows "Invoking [Agent Name]" messages
- [ ] Displays each agent's response as it arrives
- [ ] Maintains context between agent calls
- [ ] Handles any agent errors gracefully

**Synthesis Phase:**
- [ ] Comprehensive final answer addressing all questions
- [ ] Specific recommendations from each specialist:
  - **Diagnostician**: Culture interpretation, workup recommendations
  - **Stewardship**: De-escalation plan, duration recommendations
  - **ICU Consultant**: Hemodynamic considerations, monitoring
  - **IPC**: Isolation precautions, transmission prevention
- [ ] Prioritized action items (urgent vs routine)
- [ ] Clinical reasoning and evidence-based rationale

### ❌ Signs of Issues:

- Orchestrator doesn't invoke any subagents (just gives generic answer)
- Shows execution plan but doesn't actually call the agents
- Can't find subagents (error: "agent not found")
- Synthesis doesn't incorporate subagent responses
- Generic response without specialty-specific details

---

## 🔍 Detailed Expected Outputs

### From ID Diagnostician:
```
Expected content:
- MRSA bacteremia recurrence vs new infection
- Differential diagnosis for persistent bacteremia
- Recommendations for:
  * Echo to rule out endocarditis
  * Consider removing/replacing dialysis catheter
  * Imaging for metastatic foci
- Discussion of complicated vs uncomplicated bacteremia
```

### From Stewardship Expert:
```
Expected content:
- Vancomycin optimization (check trough, AUC/MIC)
- Piperacillin-tazobactam: likely unnecessary (can de-escalate)
- Alternative options: daptomycin, ceftaroline, linezolid
- Duration: 14 days for uncomplicated, longer if endocarditis
- Monitoring: Weekly vancomycin troughs, renal function
- De-escalation timeline: After sensitivities available
```

### From ICU Sepsis Consultant:
```
Expected content:
- Hemodynamic status: Improving (decreasing vasopressor needs)
- Source control: Consider catheter removal
- Fluid resuscitation: Adequate (lactate improving)
- Monitoring: Daily blood cultures until clearance
- ICU-specific considerations: Dialysis timing with antibiotics
- Prognosis: Good if source controlled
```

### From IPC Specialist:
```
Expected content:
- Contact precautions: Appropriate for MRSA
- Isolation duration: Until cultures negative
- Staff education: Hand hygiene, PPE compliance
- Cohorting considerations
- Decolonization protocols (if recurrent MRSA)
- Environmental cleaning protocols
```

### Orchestrator's Synthesis:
```
Expected structure:
1. Summary of case (MRSA bacteremia, improving septic shock)
2. Answers to each question with specialist input:
   Q1: Coverage - vanc appropriate, pip-tazo can stop
   Q2: De-escalation - stop pip-tazo, optimize vanc dosing
   Q3: Duration - 14+ days depending on source control
   Q4: Workup - echo, consider catheter removal
   Q5: Isolation - contact precautions correct
   Q6: Stewardship - multiple opportunities to optimize
3. Prioritized action plan:
   - Urgent (today): Echo, check vanc trough, stop pip-tazo
   - Within 24h: Review sensitivities, daily blood cultures
   - Ongoing: Monitor clinical response, consider catheter removal
4. Key takeaways and follow-up plan
```

---

## 🧪 Alternative Test Cases

### Quick Test (Simpler):
```
"Patient with pneumonia needing antibiotic choice and stewardship guidance. 
How should I treat and what's the optimal duration?"
```

### Complex Multi-System Test:
```
"78F with UTI, pneumonia, and C. diff. Multiple antibiotics on board. 
Need help with antibiotic optimization, infection prevention, and diagnostic workup."
```

### Outbreak Scenario:
```
"3 patients in ICU with carbapenem-resistant Enterobacterales. 
Need infection control measures, treatment options, and stewardship guidance."
```

---

## 📝 Testing Checklist

### Before Testing:
- [ ] All 4 specialist agents created
- [ ] Orchestrator agent created
- [ ] NCBI_EMAIL and API keys set (for PubMed searches)
- [ ] Browser ready in builder panel

### During Test:
- [ ] Paste complete case to orchestrator
- [ ] Wait for execution plan
- [ ] Type "proceed" to start execution
- [ ] Watch for agent invocation messages
- [ ] Note any errors or missing agents

### After Test:
- [ ] Review final synthesis
- [ ] Verify all questions answered
- [ ] Check if recommendations are actionable
- [ ] Confirm specialty-specific details included
- [ ] Test passed: ✅ or needs debugging: ❌

---

## 🐛 Troubleshooting

### Problem: Orchestrator doesn't invoke subagents
**Solution**: Verify you created the subagents first and they're visible in your "Active Agents" list

### Problem: "Agent not found" errors
**Solution**: Check agent names match exactly (case-sensitive). Recreate agents if needed.

### Problem: Generic response without specialist details
**Solution**: Orchestrator might not have access to subagents. Verify per-user isolation working correctly.

### Problem: Execution plan created but agents not invoked
**Solution**: Make sure to type "proceed" after the plan is shown

### Problem: Only one agent invoked instead of multiple
**Solution**: Case might not be complex enough. Use the detailed test case above.

---

## 🎓 Learning Objectives

This test demonstrates:
1. ✅ **Multi-agent coordination** - Orchestrator managing 4+ specialists
2. ✅ **Complex reasoning** - Breaking down multifaceted clinical case
3. ✅ **Information synthesis** - Combining multiple expert opinions
4. ✅ **Conflict resolution** - Handling differing recommendations
5. ✅ **Prioritization** - Urgent vs routine actions
6. ✅ **Comprehensive coverage** - Addressing all aspects of care
7. ✅ **Clinical applicability** - Actionable recommendations

---

## 📚 Documentation Reference

- **Orchestrator Setup**: See `FIX_ORCHESTRATOR_SUBAGENTS.md`
- **Agent Isolation**: See `AGENT_ISOLATION_COMPLETE.md`
- **Testing Guide**: See `TEST_AGENT_ISOLATION.md`

---

**Ready to test?** Start by creating your 4 specialist agents, then the orchestrator, then paste the complex case! 🎼🔬