# ra189zor/mindflow architecture

> Architecture diagram of the GitHub repository ra189zor/mindflow, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ra189zor/mindflow
- Repository: https://github.com/ra189zor/mindflow

## Overview

##### Project Type and Purpose
- MindFlow is a Python-based, AI-powered content generation pipeline with a Streamlit UI front-end.
- It orchestrates multiple specialized “agents” (Idea Generator, Filter, Researcher, Writer, Validator) via CrewAI/LangChain, persists research in a vector store, and exposes a live dashboard.

##### File Structure
- Top‐level files:
- app.py – Streamlit application (UI + orchestration)
- requirements.txt / .env.example – dependencies and API key configuration
- agents/ directory:
- Five agent modules (boss_agent.py, filter_agent.py, idea_agent.py, research_agent.py, writer_agent.py) – encapsulated AI tasks
- vectorstore/ directory:
- chroma_setup.py – ChromaDB initialization for caching research embeddings

##### Key README Insights
- Pipeline stages are linear with an iterative feedback loop for validation.
- Uses CrewAI + LangChain for agent definitions.
- External services: OpenAI LLM, DuckDuckGo scraper (Medium trends), ChromaDB, optional SQLite/InMemory cache.
- UI: Streamlit panels, status badges, live progress bar.

##### Main Components
a. Streamlit Frontend (app.py)
– Sidebar inputs (niche, keywords, parameters)
– Status pane & progress bar
– Orchestrator invoking each agent in sequence
b. Agent Modules
– Idea Agent (scrape Medium via DuckDuckGo → brainstorm)
– Filter Agent (rank & JSON‐output)
– Research Agent (gather facts, use vectorstore cache)
– Writer Agent (draft content)
– Boss/Validator Agent (JSON feedback, revision loop)
c. Vector Store
– ChromaDB instance (via chroma_setup.py) for embedding storage
– Optional in-memory or SQLite fallback
d. External Services
– OpenAI API (LLM calls with tenacity retries & concurrency throttling)
– DuckDuckGo search / Medium scraping

##### Relationships & Data Flows
- User → Streamlit UI
- UI → Agent Orchestrator → sequential AI calls
- Research Agent ↔ Vectorstore (read/write cache)
- All Agents → OpenAI API (model_name, max_concurrency)
- Idea Agent → DuckDuckGo/Medium scraper
- Validator Agent → Writer Agent (feedback loop, up to max_revisions)
- Final content → UI + export

##### Architectural Patterns & Design Principles
- Pipeline Pattern: discrete, single‐responsibility agents chained in order
- Feedback Loop: Boss Agent triggers Writer Agent revisions until approval
- Caching Layer: vectorstore for idempotency and performance
- Concurrency Control & Retry Logic: tenacity + max_concurrency parameters
- Separation of Concerns: UI, orchestration, agent logic, and persistence are modular

##### Technologies & Frameworks
- Streamlit (UI)
- CrewAI & LangChain (agent definitions & orchestration)
- ChromaDB (vectorstore)
- DuckDuckGo search API or web scraping
- tenacity (retry logic)
- InMemoryCache / SQLiteCache (LLM caching)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Layers and Components
    UI["Streamlit App"]:::ui
    IdeaAgent["Idea Agent"]:::agent
    FilterAgent["Filter Agent"]:::agent
    ResearchAgent["Research Agent"]:::agent
    WriterAgent["Writer Agent"]:::agent
    BossAgent["Boss/Validator Agent"]:::agent
    UICompletion["Final Content Display"]:::ui
    ChromaDB["ChromaDB Vector Store"]:::persistence
    OpenAI["OpenAI API"]:::external
    DuckDuckGoCloud["DuckDuckGo/Medium Scraper"]:::external
    ConfigReq["requirements.txt"]:::config
    ConfigEnv[".env.example"]:::config
    AgentsInit["agents/__init__.py"]:::config
    VectorInit["vectorstore/__init__.py"]:::config

    %% Pipeline Flow
    UI -->|start pipeline| IdeaAgent
    IdeaAgent -->|ideas| FilterAgent
    FilterAgent -->|filtered ideas| ResearchAgent
    ResearchAgent -->|research data| WriterAgent
    WriterAgent -->|draft content| BossAgent
    BossAgent -->|feedback| WriterAgent
    BossAgent -->|approved content| UICompletion

    %% Storage Interaction
    ResearchAgent <-->|"read/write cache"| ChromaDB

    %% External Services
    IdeaAgent -->|"DuckDuckGo search"| DuckDuckGoCloud
    IdeaAgent -->|"LLM calls"| OpenAI
    FilterAgent -->|"LLM calls"| OpenAI
    ResearchAgent -->|"LLM calls"| OpenAI
    WriterAgent -->|"LLM calls"| OpenAI
    BossAgent -->|"LLM calls"| OpenAI

    %% Config and Initializers
    UI --> ConfigReq
    UI --> ConfigEnv
    IdeaAgent --> AgentsInit
    ResearchAgent --> VectorInit

    %% Click Events
    click UI "https://github.com/ra189zor/mindflow/blob/main/app.py"
    click IdeaAgent "https://github.com/ra189zor/mindflow/blob/main/agents/idea_agent.py"
    click FilterAgent "https://github.com/ra189zor/mindflow/blob/main/agents/filter_agent.py"
    click ResearchAgent "https://github.com/ra189zor/mindflow/blob/main/agents/research_agent.py"
    click WriterAgent "https://github.com/ra189zor/mindflow/blob/main/agents/writer_agent.py"
    click BossAgent "https://github.com/ra189zor/mindflow/blob/main/agents/boss_agent.py"
    click ChromaDB "https://github.com/ra189zor/mindflow/blob/main/vectorstore/chroma_setup.py"
    click ConfigReq "https://github.com/ra189zor/mindflow/blob/main/requirements.txt"
    click ConfigEnv "https://github.com/ra189zor/mindflow/blob/main/.env.example"
    click AgentsInit "https://github.com/ra189zor/mindflow/blob/main/agents/__init__.py"
    click VectorInit "https://github.com/ra189zor/mindflow/blob/main/vectorstore/__init__.py"

    %% Styles
    classDef ui fill:#bbdefb,stroke:#0d47a1,color:#0d47a1;
    classDef agent fill:#c8e6c9,stroke:#1b5e20,color:#1b5e20;
    classDef persistence fill:#ffe0b2,stroke:#e65100,color:#e65100;
    classDef external fill:#eeeeee,stroke:#616161,color:#424242;
    classDef config fill:#f3e5f5,stroke:#6a1b9a,color:#4a148c;
```
