# amritkochar/gtmind architecture

> Architecture diagram of the GitHub repository amritkochar/gtmind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amritkochar/gtmind
- Repository: https://github.com/amritkochar/gtmind

## Overview

##### Project Type and Purpose
- GTMind is a Python-based “autonomous research agent.”
- It implements a linear pipeline (Search → Parse → Extract → Aggregate → Persist/Serve) and exposes three user interfaces: CLI, HTTP API, and a Streamlit UI.
- It integrates with external services (Serper.dev for search, OpenAI for content extraction) and persists history to SQLite.

##### File Structure
- src/gtmind/core: pipeline modules (search.py, parse.py, extract.py, aggregate.py, settings.py)
- src/gtmind/api: Typer (CLI) + FastAPI (HTTP endpoint)
- src/gtmind/ui: Streamlit front-end
- src/gtmind/persistence.py: SQLite persistence logic
- sample_outputs.json: example report artifacts
- tests/: unit + integration tests validating each core component and end-to-end pipeline

##### README Insights
- The tech stack per layer is enumerated (httpx, trafilatura, OpenAI, RapidFuzz, sqlmodel, FastAPI, Typer, Streamlit).

##### Main Components & Their Relationships
A. External Dependencies
1. Serper.dev search API
2. The Web (HTML endpoints)
3. OpenAI GPT-4o API
B. Core Pipeline Container
1. Search Service (core/search.py)
2. Downloader + Boilerplate Stripper (core/parse.py)
3. Extractor (core/extract.py)
4. Aggregator (core/aggregate.py)
C. Persistence Layer
1. SQLite Database via SQLModel (persistence.py)
D. Serving Interfaces
1. CLI (Typer)
2. FastAPI HTTP Server
3. Streamlit UI

Data Flow (sequential):
[User → Interface] → Pipeline.Search → Pipeline.Parse → Pipeline.Extract → Pipeline.Aggregate → ResearchReport JSON → {CLI output, Database write, HTTP response, Streamlit render}

##### Architectural Patterns & Principles
- Pipeline pattern: each core module performs one transformation step.
- Separation of concerns: core logic vs. persistence vs. serving.
- Dependency injection via settings (core/settings.py) for API keys.
- Asynchronous I/O for HTTP calls (httpx in parse, search, and FastAPI).

##### Essential Elements
- Clear component labels and module paths (e.g., src/gtmind/core/extract.py).
- Versioned externals (OpenAI GPT-4o, Serper.dev).
- Data formats (JSON, SQLite).
– Rectangles for micro-services or Python modules
– Clouds for external APIs

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    subgraph "External Services"
        direction TB
        Serper["Serper.dev API\n(HTTP/JSON)"]:::external
        Web["The Web\n(HTML)"]:::external
        OpenAI["OpenAI GPT-4o API\n(HTTP/JSON)"]:::external
    end

    %% Core Pipeline
    subgraph "Core Pipeline"
        direction TB
        Settings["Settings\n(src/gtmind/core/settings.py)"]:::core
        Models["Data Models & Schemas\n(src/gtmind/core/models.py)"]:::core
        Prompts["Prompt Templates\n(src/gtmind/core/prompts.py)"]:::core
        Search["Search Service\n(src/gtmind/core/search.py)"]:::core
        Parse["Downloader & Boilerplate Stripper\n(src/gtmind/core/parse.py)"]:::core
        Extract["Extractor\n(src/gtmind/core/extract.py)"]:::core
        Aggregate["Aggregator\n(src/gtmind/core/aggregate.py)"]:::core
        Report["ResearchReport JSON\n(Function Output)"]:::core
    end

    %% Persistence Layer
    subgraph "Persistence"
        direction TB
        DB["SQLite Database\n(src/gtmind/persistence.py)"]:::persistence
    end

    %% Serving Layer
    subgraph "Serve Layer"
        direction TB
        CLI["CLI Interface\n(src/gtmind/api/cli.py)"]:::serve
        HTTP["HTTP API Server\n(src/gtmind/api/routes.py\n& run.py)"]:::serve
        Streamlit["Streamlit UI\n(src/gtmind/ui/app.py)"]:::serve
    end

    %% External to Pipeline
    Serper -->|HTTP/JSON| Search
    Web -->|HTML| Parse
    OpenAI -->|HTTP/JSON| Extract

    %% Pipeline Flow
    Settings --> Search
    Models --> Search
    Prompts --> Extract
    Search -->|Function call| Parse
    Parse -->|Function call| Extract
    Extract -->|Function call| Aggregate
    Aggregate -->|Function call| Report

    %% Pipeline to Persistence & Serve
    Report -->|Write Result| DB
    Report -->|Return JSON| CLI
    Report -->|HTTP Response| HTTP
    Report -->|Render Report| Streamlit

    %% Serve triggers pipeline
    CLI -->|Invoke Pipeline| Search
    HTTP -->|Invoke Pipeline| Search
    Streamlit -->|Invoke Pipeline| Search
    Streamlit -->|Read Recent Reports| DB

    %% Click Events
    click Search "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/search.py"
    click Parse "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/parse.py"
    click Extract "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/extract.py"
    click Aggregate "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/aggregate.py"
    click Settings "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/settings.py"
    click Models "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/models.py"
    click Prompts "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/core/prompts.py"
    click DB "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/persistence.py"
    click CLI "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/api/cli.py"
    click HTTP "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/api/routes.py"
    click Streamlit "https://github.com/amritkochar/gtmind/blob/main/src/gtmind/ui/app.py"

    %% Styles
    classDef external fill:#e0e0e0,stroke:#888,stroke-width:1px
    classDef core fill:#cfe2f3,stroke:#609,stroke-width:1px
    classDef persistence fill:#f4cccc,stroke:#d00,stroke-width:1px
    classDef serve fill:#d9ead3,stroke:#290,stroke-width:1px
```
