How geliang0114/nvagent works
Project Type & Purpose
• nvAgent is a full-stack AI workflow for Natural-Language-to-Visualization (NL2Vis).
• It comprises three collaborative agents (processor, composer, validator), a web UI for live demos, and an evaluation framework (VisEval) for benchmarking.
File Structure
• Top level:
• core/ – orchestration of agents, LLM config, prompts, utilities
• web_vis/ – Flask‐based web interface (app.py, templates)
• viseval/ – evaluation code and checks (evaluate.py, check/*)
• run_evaluate.py – entry point for automated benchmarking
• Observations:
• MVC pattern in web_vis (app.py controllers + templates views)
• Agent workflow in core (chat_manager orchestrates multiple LLM calls)
• visEval as a separate evaluation micro-service
Extract Architectural Insights from README
• Three Agents:
a. ProcessorAgent – filters/queries tabular data
b. ComposerAgent – plans visualization (chart type, axes, encodings)
c. ValidatorAgent – translates plan into code + verifies output
• Interactive demo: Flask UI where users upload CSVs and type requests
• Evaluation: uses Selenium + GPT-4o-mini (vision model) as a judge
Main Components & Their Interactions
A. Frontend (Web UI)
• Flask app (web_vis/app.py)
• Jinja2 templates (web_vis/templates/index.html)
• Static/demo assets for user upload & text input
B. Backend Core (Agent Workflow)
• ChatManager (core/chat_manager.py) orchestrates:
• ProcessorAgent (core/agents.py)
• ComposerAgent (core/agents.py)
• ValidatorAgent (core/agents.py)
• LLM Interface (core/llm.py) via Azure OpenAI / OpenAI API
• Prompt Templates (core/const.py), Config (core/api_config.py), Utilities
C. Data Layer
• User‐uploaded CSV files (runtime)
• VisEval Dataset (visEval_dataset.zip) for evaluation
D. Visualization Engine
• Matplotlib / Seaborn code generated by agents
• Output images / chart objects returned to UI
E. Evaluation Framework (visEval)
• Dataset loader (viseval/dataset.py)
• Evaluation runner (viseval/evaluate.py + run_evaluate.py)
• Check modules (viseval/check/*.py) for layout, readability, etc.
• Vision-language judge (GPT-4o-mini via Langchain) + Selenium
Relationships & Data Flow
• User → Frontend UI → Backend API endpoint (Flask)
• Backend → ChatManager → ProcessorAgent → (reads CSV) → calls LLM
• ProcessorAgent → ComposerAgent → LLM → returns visualization plan
• ComposerAgent → ValidatorAgent → LLM → returns executable Python code
• ValidatorAgent → executes code → generates chart → returns image URL/data
• Backend → responds to UI with rendered chart
• For batch evaluation: run_evaluate.py → loads visEval dataset → ChatManager → generates & saves charts → Selenium captures screenshots → Vision LLM judges → evaluation scores
Architectural Patterns / Principles
• Agent-based micro-workflow: clear separation of responsibilities among agents
• MVC for web interface: Controllers (app.py) ↔ Views (templates)
• Layered design: Presentation (web_vis) → Application (core) → Data (files, datasets) → External (LLM APIs)
• Modular evaluation service decoupled from main inference pipeline
Technologies & Frameworks to Call Out
• Python 3.9+, Flask, Jinja2
• Langchain for LLM orchestration
• Azure OpenAI / OpenAI API for code generation & vision judging
• Matplotlib / Seaborn for charting
• Selenium (chromedriver) for screenshot‐based evaluation
• GPT-4o-mini as MLLM-as-Judge