How hoangsonww/agentic-ai-pipeline works
Project Type and Purpose
• This is a full-stack, agentic AI platform with a primary Research & Outreach Agent.
• It combines a FastAPI+SSE web UI, a CLI, a LangGraph-driven multi-stage agent, memory stores (SQLite & ChromaDB), atomic tools, client SDKs, and companion pipelines (Agentic-RAG, Agentic-Coding).
• It includes complete DevOps/infrastructure code (Docker, Ansible, Terraform, Kubernetes, AWS CD/CFN, GitHub Actions).
File Structure
• Top-level “src/agentic_ai/” contains the core 7-layer agent logic:
• Experience (app.py + web/)
• Discovery, Composition, Reasoning, Tools, Memory, Infrastructure.
• “web/” holds the static chat UI (Vue3 via CDN).
• “clients/” contains TypeScript and Python SDKs.
• Two side-car pipelines: “Agentic-RAG-Pipeline/” and “Agentic-Coding-Pipeline/” with their own agents, graph runners, and tools.
• An “mcp/” micro-service exposes a shared control plane (HTTP/SSE) for all pipelines.
• “ansible/”, “aws/”, “k8s/”, “hashicorp/terraform/” orchestrate deployment.
• “Makefile” and “scripts/” automate setup, ingestion, testing, and deployment.
README Insights on Architecture
• 7-Layer Architecture: Experience → Discovery → Composition → Reasoning → Tools/API → Memory/Feedback → Infrastructure.
• LangGraph Agent Graph: A state machine with PLAN→DECIDE→ACT→TOOLS→REFLECT→FINALIZE.
• Atomic Tools: web_search, web_fetch, kb_search, calculator, file_write, emailer.
• Memory: SQLite for messages, ChromaDB for vector KB.
• External Services: OpenAI/Anthropic LLMs, DuckDuckGo Search.
• Frontend:
• Web UI (browser)
• CLI & SDKs (Python, TS)
• API Server: FastAPI/Uvicorn
• Agent Core: LangGraph runner (graph.py + layers/*)
• Tool Layer: modules under tools/ (webtools, ops, knowledge)
• Memory Layer: SQLite store, ChromaDB vector store
• Companion Pipelines: Agentic-RAG, Agentic-Coding, each with its own agent graph & tools
• MCP Server: centralized HTTP/SSE control plane (mcp/server.py)
• External Systems: OpenAI, Anthropic, DuckDuckGo, SMTP (mock), file system for artifacts
• Infrastructure: Rate limiter, logging, config (infra/), plus Docker, Ansible, Terraform, k8s
• User → Web UI/CLI → FastAPI (Experience)
• FastAPI → Rate limiter → Agent graph entry
• Agent graph steps → Tool calls → External APIs or Memory → results back into graph
• Agent final output → SSE stream → UI/CLI → persists to SQLite & file system
• Ingestion CLI → Memory vector store (ChromaDB)
• Companion pipelines & MCP → reuse the same tools, memory, and control plane
c. Architectural Patterns & Principles
• Layered Architecture: clear separation of concerns in 7 layers
• Single-action per step: each DECIDE/ACT binds one tool call
• Event-driven/SSE streaming: incremental updates to the UI/CLI
• Micro-service hooks: mcp server and side-pipelines treated as peer services
• Infrastructure as Code: Docker/Ansible/Terraform/K8s manifests
d. Key Technologies & Frameworks
• FastAPI, Uvicorn, SSE
• LangGraph (state machine orchestrator)
• LangChain (tool abstractions for RAG)
• OpenAI & Anthropic LLMs
• SQLite + SQLAlchemy, ChromaDB
• Docker, Ansible, Terraform, Kubernetes
• Vue.js (UI), TypeScript & Python (SDKs)
• GitHub Actions for CI/CD