How agasthya736/multi-agent-system works
Purpose and Entry Points
Radiology MAS generates structured chest X-ray reports through a staged multi-agent pipeline: Vision → Knowledge → Writing → Verifier. The interactive entry point is the Streamlit application at app.py; batch/pipeline execution is represented by pipeline_runner.py. A standalone baseline path exists in baseline.py.
Orchestration and Agent Stages
The core architecture is a sequential, file-oriented agent workflow:
1. Vision analysis interprets chest X-ray inputs: vision_agent/agent.py, vision_agent/runner.py.
2. Knowledge augmentation adds medical/radiology context: knowledge_agent/agent.py.
3. Report writing converts upstream findings into radiology-report text: writing_agent/agent.py.
4. Verification reviews generated content for hallucinations or unsupported claims: verifier_agent/agent.py.
The verifier is an explicit quality boundary between generated draft content and final report outputs.
Data Boundaries
Local dataset assets are central dependencies. Image data is stored under data/images/NLMCXR_png, while source radiology reports are a large XML corpus under data/report/NLMCXR_reports/ecgen-radiology. data/dataset.csv appears to provide the dataset-level index or sample metadata connecting these artifacts.
Dataset parsing is separated into parse_dataset.py, creating a boundary between raw XML/image resources and pipeline-ready records.
Generated Artifacts and Comparative Runs
Outputs are persisted by pipeline stage beneath results/generated_reports, including results/generated_reports/vision, results/generated_reports/knowledge, results/generated_reports/writing, and results/generated_reports/verifier. This preserves intermediate agent outputs for inspection and supports comparison against results/generated_reports/baseline.
The result layout indicates runs are keyed by sample ID and materialized as text files, rather than stored in a database or served through an API.
Evaluation Layer
Evaluation is decoupled from generation. Clinical scoring is represented by clinical_metric.py, while general and multi-agent evaluation scripts include evaluate.py, evaluation_mas.py, and evaluate_mas.py. Persisted aggregates include results/clinical_metrics.csv, results/scores.csv, and results/summary.csv.
This separation enables comparison of baseline and staged-agent reports using the same sample-level artifacts.
Runtime and Infrastructure
The repository is Python-based, with dependencies declared in requirements.txt and evaluation-specific dependencies in requirements_eval.txt. It runs as a Streamlit service locally or in Docker via Dockerfile; Streamlit settings reside in .streamlit/config.toml.
The system supports external LLM providers—Gemini, OpenAI, and Ollama—per the project description. Docker deployment specifically expects GOOGLE_API_KEY, making Gemini an operationally documented integration. No external persistence, message queue, or backend service is evidenced in the repository tree.
Main entry points: generated_reports Stage report outputs · parse_dataset.py Dataset parser · vision_agent/agent.py Vision analysis · knowledge_agent/agent.py Knowledge augmentation
Components 19
Entry points
- Streamlit app
app.py - Pipeline runner
pipeline_runner.py - Baseline generator
baseline.py
MAS pipeline
- Vision analysis · Interprets chest X-ray inputs.
vision_agent/agent.py - Vision runner
vision_agent/runner.py - Knowledge augmentation · Adds medical and radiology context.
knowledge_agent/agent.py - Report writing · Produces radiology-report text from upstream findings.
writing_agent/agent.py - Report verifier · Rejects hallucinated or unsupported claims before final output.
verifier_agent/agent.py
Dataset
- Dataset parser · Transforms raw dataset resources into pipeline-ready records.
parse_dataset.py - Dataset index
data/dataset.csv - Chest X-ray images
data/images/NLMCXR_png - Source XML reports
data/report/NLMCXR_reports/ecgen-radiology
Generated artifacts
- Stage report outputs · Intermediate vision, knowledge, writing, and verifier outputs keyed by sample ID.
results/generated_reports - Baseline outputs
results/generated_reports/baseline
Evaluation
- Clinical metric
clinical_metric.py - MAS evaluation · Evaluates baseline and staged-agent reports.
evaluation_mas.py - Score aggregates · Clinical metrics, scores, and summaries are persisted as CSV files.
results/scores.csv
Other components
- LLM providers · Gemini, OpenAI, or Ollama supply model inference.
- Docker runtime · Documented deployment expects GOOGLE_API_KEY for Gemini.
Dockerfile
Connections 25
- Streamlit app → Pipeline runner: starts pipeline
- Pipeline runner → Dataset parser: loads records
- Dataset index → Dataset parser: indexes samples
- Chest X-ray images → Dataset parser: image assets
- Source XML reports → Dataset parser: reference reports
- Dataset parser → Vision runner: prepared samples
- Vision runner → Vision analysis: executes
- Vision analysis → Knowledge augmentation: findings
- Knowledge augmentation → Report writing: augmented context
- Report writing → Report verifier: draft report
- Vision analysis → Stage report outputs: persists output
- Knowledge augmentation → Stage report outputs: persists output
- Report writing → Stage report outputs: persists draft
- Report verifier → Stage report outputs: persists final report
- Baseline generator → Baseline outputs: persists reports
- Stage report outputs → Clinical metric: generated reports
- Baseline outputs → MAS evaluation: comparison input
- Stage report outputs → MAS evaluation: comparison input
- Clinical metric → Score aggregates: clinical metrics
- MAS evaluation → Score aggregates: scores and summary
- Vision analysis → LLM providers: model inference
- Knowledge augmentation → LLM providers: model inference
- Report writing → LLM providers: model inference
- Report verifier → LLM providers: model inference
- Docker runtime → Streamlit app: runs locally