# ansn07/book-gen architecture

> Architecture diagram of the GitHub repository ansn07/book-gen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ansn07/book-gen
- Repository: https://github.com/ansn07/book-gen

## Overview

##### File structure
- Top-level files:
- `main.py` (the orchestration script)
- `requirements.txt` (Python dependencies)
- `README.md` (project description & setup)
- `generated-books/…` (output directories & `book.md`)
- External integrations: GitHub API (via `--repo` URL or local path), Ollama HTTP inference server.

##### README for additional insights
- Key steps: clone/read repo → parse `.py` files → generate prompts → call local LLM → post-process LLM output → merge Markdown.
- Dependencies: Python 3, pip packages, Ollama server hosting Mistral (~4–6 GB).

##### Components & relationships
a. Main Components
1. CLI Interface (user runs `python main.py --repo …`)
2. Repo Fetcher
- GitHub client (HTTPS clone or local FS reader)
3. File Discovery & Parser
- Walks directory, finds `.py`, optionally builds AST
4. Prompt Generator
- Takes each `.py` file’s content → constructs a prompt template
5. LLM Inference Engine
- Ollama server (HTTP endpoint) hosting Mistral model
6. Post-Processor & Markdown Generator
7. Chapter Assembler
- Collates individual chapter files into a single `book.md` + TOC
8. File System I/O
- Writes to `generated-books/<project>/book.md` and `/explained/` subfolder
9. External Services
- GitHub API (rate-limited; optionally authenticated via `.env`), Ollama local server

b. Relationships & data flows
- CLI → Repo Fetcher (HTTP clone or local FS)
- Repo Fetcher → File Discovery → Parser
- Parser → Prompt Generator → LLM Inference (HTTP request to Ollama)
- LLM Inference → Post-Processor → Markdown files
- Markdown files → Chapter Assembler → Final `book.md`
- All steps read/write to local file system; errors bubble back to CLI user

##### Architectural patterns & principles
- Pipelines & staged processing (each file goes through the same sequence)
- Modular separation of concerns (fetching, parsing, prompting, inference, formatting)
- Externalizing heavy computation to a service (Ollama)
- Idempotent file outputs for caching / re-runs

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% CLI and Config
    subgraph "CLI & Config"
        CLI["CLI Interface"]:::internal
        Env[".env / GITHUB_TOKEN"]:::config
    end

    %% Fetching components
    subgraph "Fetching"
        Fetcher["Repo Fetcher"]:::internal
        GitHubAPI["GitHub API"]:::external
    end

    %% Processing pipeline
    subgraph "Processing Pipeline"
        Parser["File Discovery & Parser"]:::internal
        Prompt["Prompt Generator"]:::internal
        LLMServer["LLM Server\n(Ollama)"]:::external
        PostProc["Post-Processor & Markdown Generator"]:::internal
        Assembler["Chapter Assembler"]:::internal
    end

    %% I/O components
    subgraph "Output & Storage"
        FSIO["Output FS\n(generated-books/)"]:::data
    end

    %% Project metadata
    subgraph "Project Files"
        README["README.md"]:::data
        Requirements["requirements.txt"]:::data
    end

    %% Generated outputs
    subgraph "Generated Books"
        Output1["tcm/book.md"]:::data
        Output2["triton-util/book.md"]:::data
        Output3["triton/book.md"]:::data
    end

    %% Data flows
    CLI --> Fetcher
    Env -.-> Fetcher
    Fetcher -->|"clone/read"| GitHubAPI
    Fetcher --> Parser
    Parser -->|".py code"| Prompt
    Prompt -->|".json HTTP"| LLMServer
    LLMServer --> PostProc
    PostProc --> Assembler
    Assembler --> FSIO
    FSIO --> Output1
    FSIO --> Output2
    FSIO --> Output3

    %% Project files to CLI
    README --> CLI
    Requirements --> CLI

    %% Click events
    click CLI "https://github.com/ansn07/book-gen/blob/main/main.py"
    click Fetcher "https://github.com/ansn07/book-gen/blob/main/main.py"
    click Parser "https://github.com/ansn07/book-gen/blob/main/main.py"
    click Prompt "https://github.com/ansn07/book-gen/blob/main/main.py"
    click LLMServer "https://github.com/ansn07/book-gen/blob/main/main.py"
    click PostProc "https://github.com/ansn07/book-gen/blob/main/main.py"
    click Assembler "https://github.com/ansn07/book-gen/blob/main/main.py"
    click FSIO "https://github.com/ansn07/book-gen/tree/main/generated-books/"
    click README "https://github.com/ansn07/book-gen/blob/main/README.md"
    click Requirements "https://github.com/ansn07/book-gen/blob/main/requirements.txt"
    click Output1 "https://github.com/ansn07/book-gen/blob/main/generated-books/tcm/book.md"
    click Output2 "https://github.com/ansn07/book-gen/blob/main/generated-books/triton-util/book.md"
    click Output3 "https://github.com/ansn07/book-gen/blob/main/generated-books/triton/book.md"

    %% Styles
    classDef internal fill:#ADD8E6,stroke:#000;
    classDef external fill:#90EE90,stroke:#000;
    classDef data fill:#FFFFE0,stroke:#000;
    classDef config fill:#FFDAB9,stroke:#000;
```
