# dicklesworthstone/mindmap-generator architecture

> Architecture diagram of the GitHub repository dicklesworthstone/mindmap-generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dicklesworthstone/mindmap-generator
- Repository: https://github.com/dicklesworthstone/mindmap-generator

## Overview

##### Project Type and Purpose
- It implements advanced document‐type detection, intelligent chunking, non-linear exploration, semantic deduplication, reality checking, emoji selection, asynchronous orchestration, and cost-optimization strategies.

##### Key Structural Elements (from the file tree)
- Root files:
- mindmap_generator.py (entry point / orchestrator)
- .env, requirements.txt, .python-version (configuration)
- sample_input_document…md (input examples)
- mindmap_outputs/ (all generated outputs)
- screenshots/ (illustrative assets)
- No separate “frontend” or “backend” directories—this is a self-contained service.

##### Main Components
A. CLI Interface
- mindmap_generator.py
- Reads config (.env) and input document path
- Kicks off the Processing Pipeline

B. Configuration & Environment
- .env loader (API keys, PROVIDER)
- Python venv / dependencies (requirements.txt)

C. Processing Pipeline (Orchestrator)
- Task Orchestrator / Async Scheduler
- Rate Limiter / Semaphore Manager (for LLM API calls)
- Caching Layer (persistent memoization of prompts/responses)
- Feedback loops & dynamic priority adjustment

D. Document Analysis Subsystems
1. Document Type Detector
– Samples document, classifies technical/scientific/narrative/legal/etc.
2. Intelligent Chunking Engine
– Overlapping chunk creation
– Boundary optimization & context preservation
3. Topic Extraction Engine
– Parallel initial extraction
– Frequency analysis & consolidation
– Semantic deduplication
– Importance ranking
4. Adaptive Exploration Controller
– Breadth-depth balancing
– Diminishing returns detection
5. Reality Check System
– Factual verification against source
– Confidence scoring
6. Semantic Redundancy Detector
– Textual, fuzzy, token-based, LLM-based checks
7. Emoji Selector
– Context-aware, hierarchical emoji assignment

E. Output Renderer
- Markdown Outline Generator
- Writes to mindmap_outputs/ folder

F. External Services
- LLM Providers: OpenAI, Anthropic (Claude), DeepSeek, Google Gemini
- Each accessed via provider-specific client modules

G. Logging & Monitoring
- Token usage tracker & cost estimates
- Error reporting & retry/backoff dashboards

##### Relationships & Data Flows
- Pipeline feeds document → Document Type Detector → chunking → Topic Extraction → Adaptive Controller → (loops back as needed)
- Responses pass through Redundancy Detector & Reality Checker, then to Emoji Selector
- Final hierarchical structure sent to Output Renderer → writes files
- Logging & Monitoring taps into each stage, writing logs to console and optional dashboards
- Caching layer interposes between pipeline and LLM Providers

##### Architectural Patterns & Principles
- Modular pipeline with clear separation of concerns
- Asynchronous, concurrent task orchestration
- Graph-like exploration (non-linear) with feedback loops
- Heuristic-guided processing (adaptive breadth/depth)
- Resilience: rate limiting, exponential backoff, graceful degradation
- Cost-optimization via provider selection, caching, batching

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration & Environment
    subgraph "Configuration & Environment"
        direction TB
        EnvLoader[".env"]:::config
        ReqTxt["requirements.txt"]:::config
        PyVersion[".python-version"]:::config
    end

    %% CLI Interface
    CLI["mindmap_generator.py"]:::core

    %% Processing Pipeline
    subgraph "Processing Pipeline" 
        direction TB
        Orchestrator["Task Orchestrator / Async Scheduler"]:::core
        RateLimiter["Rate Limiter / Semaphore Manager"]:::core
        CacheLayer["Caching Layer"]:::cache
        Detector["Document Type Detector"]:::core
        Chunker["Intelligent Chunking Engine"]:::core
        TopicExtractor["Topic Extraction Engine"]:::core
        AdaptiveController["Adaptive Exploration Controller"]:::core
        RealityChecker["Reality Check System"]:::core
        RedundancyDetector["Semantic Redundancy Detector"]:::core
        EmojiSelector["Emoji Selector"]:::core
    end

    %% Output Renderer
    subgraph "Output Renderer"
        direction TB
        MermaidGen["Mermaid Syntax Generator"]:::core
        HTMLRenderer["HTML Renderer"]:::core
        MarkdownGen["Markdown Outline Generator"]:::core
        Outputs["mindmap_outputs/"]:::output
    end

    %% External Services
    subgraph "LLM Providers"
        direction TB
        OpenAI[(OpenAI)]:::external
        Anthropic[(Claude)]:::external
        DeepSeek[(DeepSeek)]:::external
        Gemini[(Google Gemini)]:::external
    end

    %% Logging & Monitoring
    Logging["Logging & Monitoring"]:::logging

    %% Sample Inputs
    subgraph "Sample Input Documents"
        direction TB
        Sample1["sample_input_document_as_markdown__durnovo_memo.md"]:::input
        Sample2["sample_input_document_as_markdown__small.md"]:::input
    end

    %% Illustrative Assets
    Screenshots["screenshots/"]:::asset

    %% Flows
    CLI -->|"reads config & input"| EnvLoader
    CLI -->|"starts pipeline"| Orchestrator
    EnvLoader --> CLI
    ReqTxt --> CLI
    PyVersion --> CLI

    Orchestrator --> RateLimiter
    Orchestrator --> CacheLayer
    Orchestrator --> Detector
    Detector --> Chunker
    Chunker --> TopicExtractor
    TopicExtractor --> AdaptiveController
    AdaptiveController --> TopicExtractor
    TopicExtractor --> RealityChecker
    RealityChecker --> RedundancyDetector
    RedundancyDetector --> EmojiSelector
    EmojiSelector --> MermaidGen

    MermaidGen --> HTMLRenderer
    MermaidGen --> MarkdownGen
    HTMLRenderer --> Outputs
    MarkdownGen --> Outputs

    RateLimiter -->|controls calls| OpenAI
    RateLimiter --> Anthropic
    RateLimiter --> DeepSeek
    RateLimiter --> Gemini

    CacheLayer -->|caches prompts/responses| OpenAI
    CacheLayer --> Anthropic
    CacheLayer --> DeepSeek
    CacheLayer --> Gemini

    OpenAI -->|LLM JSON response| Detector
    Anthropic --> Detector
    DeepSeek --> Detector
    Gemini --> Detector

    Logging --> Orchestrator
    Logging --> RateLimiter
    Logging --> CacheLayer
    Logging --> Detector
    Logging --> Chunker
    Logging --> TopicExtractor
    Logging --> AdaptiveController
    Logging --> RealityChecker
    Logging --> RedundancyDetector
    Logging --> EmojiSelector
    Logging --> MermaidGen
    Logging --> HTMLRenderer
    Logging --> MarkdownGen

    Sample1 --> CLI
    Sample2 --> CLI
    Screenshots -.-> Outputs

    %% Click Events
    click CLI "https://github.com/dicklesworthstone/mindmap-generator/blob/main/mindmap_generator.py"
    click EnvLoader "https://github.com/dicklesworthstone/mindmap-generator/blob/main/.env"
    click ReqTxt "https://github.com/dicklesworthstone/mindmap-generator/blob/main/requirements.txt"
    click PyVersion "https://github.com/dicklesworthstone/mindmap-generator/blob/main/.python-version"
    click Sample1 "https://github.com/dicklesworthstone/mindmap-generator/blob/main/sample_input_document_as_markdown__durnovo_memo.md"
    click Sample2 "https://github.com/dicklesworthstone/mindmap-generator/blob/main/sample_input_document_as_markdown__small.md"
    click Outputs "https://github.com/dicklesworthstone/mindmap-generator/tree/main/mindmap_outputs/"
    click Screenshots "https://github.com/dicklesworthstone/mindmap-generator/tree/main/screenshots/"

    %% Styles
    classDef core fill:#cce5ff,stroke:#004085,color:#004085,stroke-width:1px;
    classDef config fill:#e2e3e5,stroke:#6c757d,color:#6c757d,stroke-width:1px,stroke-dasharray: 5 5;
    classDef cache fill:#d6d8d9,stroke:#6c757d,color:#383d41,stroke-width:1px;
    classDef external fill:#ffeeba,stroke:#856404,color:#856404,stroke-width:1px,stroke-dasharray: 2 2;
    classDef logging fill:#d4edda,stroke:#155724,color:#155724,stroke-width:1px;
    classDef output fill:#f8d7da,stroke:#721c24,color:#721c24,stroke-width:1px;
    classDef input fill:#fff3cd,stroke:#856404,color:#856404,stroke-width:1px;
    classDef asset fill:#d1ecf1,stroke:#0c5460,color:#0c5460,stroke-width:1px;
```
