# 🎼 Comprehensive Orchestrator Test Case ## Test Scenario: Complex Multi-Drug Resistant Infection This test case demonstrates the orchestrator's ability to coordinate multiple specialist agents to solve a complex infectious disease case requiring expertise in multiple domains. --- ## 🎯 Setup: Create Your Agent Team ### Step 1: Create Specialist Subagents Create these **4 specialist agents** first (they will be the orchestrator's team): #### Agent 1: Stewardship Specialist ``` Agent Type: 🎯 Specialist Agent Name: Stewardship Expert Mission: Expert in antimicrobial stewardship, antibiotic selection, de-escalation strategies, and optimizing duration of therapy Skills: βœ“ recommend_deescalation βœ“ alert_prolonged_antibiotic_use βœ“ search_pubmed (if you want literature support) ``` #### Agent 2: ID Diagnostician ``` Agent Type: 🎯 Specialist Agent Name: ID Diagnostician Mission: Expert in infectious disease diagnosis, differential diagnosis, culture interpretation, and diagnostic workup Skills: βœ“ differential_diagnosis βœ“ generate_ddx_mermaid βœ“ search_pubmed ``` #### Agent 3: ICU Consultant ``` Agent Type: 🎯 Specialist Agent Name: ICU Sepsis Consultant Mission: Expert in critical care infectious diseases, sepsis management, hemodynamic support, and ICU-specific considerations Skills: βœ“ Any relevant tools you want βœ“ search_pubmed ``` #### Agent 4: Infection Prevention Specialist ``` Agent Type: 🎯 Specialist Agent Name: IPC Specialist Mission: Expert in infection prevention and control, isolation precautions, outbreak management, and transmission-based precautions Skills: βœ“ ipc_reporting (if available) βœ“ Any other relevant tools ``` ### Step 2: Create the Orchestrator ``` Agent Type: 🎼 Orchestrator Agent Name: ID Maestro Mission: Coordinate multiple ID specialists to provide comprehensive infectious disease consultation for complex cases requiring multidisciplinary expertise Skills: (Orchestrators have access to all your other agents automatically) API Key: Your OpenAI API key ``` --- ## πŸ“‹ Test Case: Complex Septic Patient ### Patient Presentation: Copy and paste this detailed case to the **ID Maestro** orchestrator: ``` I need help with a complex case: Patient: 68-year-old male PMH: Type 2 diabetes (A1C 9.2%), ESRD on hemodialysis (MWF), recent hospitalization 3 weeks ago for MRSA bacteremia from infected dialysis catheter Current Presentation: - Day 3 ICU admission for septic shock - Fever 39.8Β°C, HR 118, BP 85/50 on norepinephrine 0.15 mcg/kg/min - WBC 24,000 with 18% bands, procalcitonin 8.5 - Lactate 4.2 β†’ 2.8 after 4L crystalloid Cultures: - Blood cultures (Γ—2 sets): Gram-positive cocci in clusters at 14 hours - Preliminary: MRSA (same strain as 3 weeks ago) - Sensitivities pending (results in 24 hours) - Urine culture: Negative - CXR: Right lower lobe infiltrate Current Antibiotics (started 72 hours ago): - Vancomycin 1g IV q12h (trough pending) - Piperacillin-tazobactam 3.375g IV q6h Additional Info: - New tunneled dialysis catheter placed 2 weeks ago - Patient on contact precautions - Last vancomycin trough (from previous admission): 18 mcg/mL - CrCl: Not applicable (on dialysis) - Patient improving clinically: BP improving, lactate trending down Questions: 1. Is current antibiotic coverage appropriate? 2. Should we de-escalate or change therapy? 3. What's the optimal duration? 4. Any diagnostic workup needed? 5. Are isolation precautions correct? 6. What are the key stewardship considerations? ``` --- ## βœ… Expected Orchestrator Behavior ### Phase 1: Planning The orchestrator should: 1. **Analyze the request** and identify multiple distinct tasks 2. **Create an execution plan** listing which agents to invoke: - ID Diagnostician (for differential diagnosis and culture interpretation) - Stewardship Expert (for antibiotic optimization and de-escalation) - ICU Sepsis Consultant (for critical care considerations) - IPC Specialist (for isolation precautions) 3. **Display the plan** with numbered steps 4. **Wait for your confirmation** ("proceed") ### Phase 2: Execution (after you say "proceed") The orchestrator should: 1. **Invoke each specialist agent** sequentially or in parallel 2. **Show progress**: "πŸš€ Invoking [Agent Name]..." 3. **Collect responses** from each specialist 4. **Display intermediate results** as they come in ### Phase 3: Synthesis The orchestrator should: 1. **Synthesize all specialist inputs** into a comprehensive recommendation 2. **Address all 6 questions** from the original query 3. **Highlight agreements** between specialists 4. **Resolve conflicts** if specialists disagree 5. **Provide prioritized recommendations** 6. **Include specific details** from each specialty area --- ## πŸ“Š What to Look For ### βœ… Good Orchestrator Performance: **Planning Phase:** - [ ] Identifies 4-6 distinct tasks in the case - [ ] Plans to invoke 3-4 specialist agents - [ ] Explains rationale for each agent selection - [ ] Asks for confirmation before proceeding **Execution Phase:** - [ ] Shows "Invoking [Agent Name]" messages - [ ] Displays each agent's response as it arrives - [ ] Maintains context between agent calls - [ ] Handles any agent errors gracefully **Synthesis Phase:** - [ ] Comprehensive final answer addressing all questions - [ ] Specific recommendations from each specialist: - **Diagnostician**: Culture interpretation, workup recommendations - **Stewardship**: De-escalation plan, duration recommendations - **ICU Consultant**: Hemodynamic considerations, monitoring - **IPC**: Isolation precautions, transmission prevention - [ ] Prioritized action items (urgent vs routine) - [ ] Clinical reasoning and evidence-based rationale ### ❌ Signs of Issues: - Orchestrator doesn't invoke any subagents (just gives generic answer) - Shows execution plan but doesn't actually call the agents - Can't find subagents (error: "agent not found") - Synthesis doesn't incorporate subagent responses - Generic response without specialty-specific details --- ## πŸ” Detailed Expected Outputs ### From ID Diagnostician: ``` Expected content: - MRSA bacteremia recurrence vs new infection - Differential diagnosis for persistent bacteremia - Recommendations for: * Echo to rule out endocarditis * Consider removing/replacing dialysis catheter * Imaging for metastatic foci - Discussion of complicated vs uncomplicated bacteremia ``` ### From Stewardship Expert: ``` Expected content: - Vancomycin optimization (check trough, AUC/MIC) - Piperacillin-tazobactam: likely unnecessary (can de-escalate) - Alternative options: daptomycin, ceftaroline, linezolid - Duration: 14 days for uncomplicated, longer if endocarditis - Monitoring: Weekly vancomycin troughs, renal function - De-escalation timeline: After sensitivities available ``` ### From ICU Sepsis Consultant: ``` Expected content: - Hemodynamic status: Improving (decreasing vasopressor needs) - Source control: Consider catheter removal - Fluid resuscitation: Adequate (lactate improving) - Monitoring: Daily blood cultures until clearance - ICU-specific considerations: Dialysis timing with antibiotics - Prognosis: Good if source controlled ``` ### From IPC Specialist: ``` Expected content: - Contact precautions: Appropriate for MRSA - Isolation duration: Until cultures negative - Staff education: Hand hygiene, PPE compliance - Cohorting considerations - Decolonization protocols (if recurrent MRSA) - Environmental cleaning protocols ``` ### Orchestrator's Synthesis: ``` Expected structure: 1. Summary of case (MRSA bacteremia, improving septic shock) 2. Answers to each question with specialist input: Q1: Coverage - vanc appropriate, pip-tazo can stop Q2: De-escalation - stop pip-tazo, optimize vanc dosing Q3: Duration - 14+ days depending on source control Q4: Workup - echo, consider catheter removal Q5: Isolation - contact precautions correct Q6: Stewardship - multiple opportunities to optimize 3. Prioritized action plan: - Urgent (today): Echo, check vanc trough, stop pip-tazo - Within 24h: Review sensitivities, daily blood cultures - Ongoing: Monitor clinical response, consider catheter removal 4. Key takeaways and follow-up plan ``` --- ## πŸ§ͺ Alternative Test Cases ### Quick Test (Simpler): ``` "Patient with pneumonia needing antibiotic choice and stewardship guidance. How should I treat and what's the optimal duration?" ``` ### Complex Multi-System Test: ``` "78F with UTI, pneumonia, and C. diff. Multiple antibiotics on board. Need help with antibiotic optimization, infection prevention, and diagnostic workup." ``` ### Outbreak Scenario: ``` "3 patients in ICU with carbapenem-resistant Enterobacterales. Need infection control measures, treatment options, and stewardship guidance." ``` --- ## πŸ“ Testing Checklist ### Before Testing: - [ ] All 4 specialist agents created - [ ] Orchestrator agent created - [ ] NCBI_EMAIL and API keys set (for PubMed searches) - [ ] Browser ready in builder panel ### During Test: - [ ] Paste complete case to orchestrator - [ ] Wait for execution plan - [ ] Type "proceed" to start execution - [ ] Watch for agent invocation messages - [ ] Note any errors or missing agents ### After Test: - [ ] Review final synthesis - [ ] Verify all questions answered - [ ] Check if recommendations are actionable - [ ] Confirm specialty-specific details included - [ ] Test passed: βœ… or needs debugging: ❌ --- ## πŸ› Troubleshooting ### Problem: Orchestrator doesn't invoke subagents **Solution**: Verify you created the subagents first and they're visible in your "Active Agents" list ### Problem: "Agent not found" errors **Solution**: Check agent names match exactly (case-sensitive). Recreate agents if needed. ### Problem: Generic response without specialist details **Solution**: Orchestrator might not have access to subagents. Verify per-user isolation working correctly. ### Problem: Execution plan created but agents not invoked **Solution**: Make sure to type "proceed" after the plan is shown ### Problem: Only one agent invoked instead of multiple **Solution**: Case might not be complex enough. Use the detailed test case above. --- ## πŸŽ“ Learning Objectives This test demonstrates: 1. βœ… **Multi-agent coordination** - Orchestrator managing 4+ specialists 2. βœ… **Complex reasoning** - Breaking down multifaceted clinical case 3. βœ… **Information synthesis** - Combining multiple expert opinions 4. βœ… **Conflict resolution** - Handling differing recommendations 5. βœ… **Prioritization** - Urgent vs routine actions 6. βœ… **Comprehensive coverage** - Addressing all aspects of care 7. βœ… **Clinical applicability** - Actionable recommendations --- ## πŸ“š Documentation Reference - **Orchestrator Setup**: See `FIX_ORCHESTRATOR_SUBAGENTS.md` - **Agent Isolation**: See `AGENT_ISOLATION_COMPLETE.md` - **Testing Guide**: See `TEST_AGENT_ISOLATION.md` --- **Ready to test?** Start by creating your 4 specialist agents, then the orchestrator, then paste the complex case! πŸŽΌπŸ”¬