How dicklesworthstone/mindmap-generator works
Project Type and Purpose
• It implements advanced document‐type detection, intelligent chunking, non-linear exploration, semantic deduplication, reality checking, emoji selection, asynchronous orchestration, and cost-optimization strategies.
Key Structural Elements (from the file tree)
• Root files:
• mindmap_generator.py (entry point / orchestrator)
• .env, requirements.txt, .python-version (configuration)
• sample_input_document…md (input examples)
• mindmap_outputs/ (all generated outputs)
• screenshots/ (illustrative assets)
• No separate “frontend” or “backend” directories—this is a self-contained service.
Main Components
A. CLI Interface
• mindmap_generator.py
• Reads config (.env) and input document path
• Kicks off the Processing Pipeline
B. Configuration & Environment
• .env loader (API keys, PROVIDER)
• Python venv / dependencies (requirements.txt)
C. Processing Pipeline (Orchestrator)
• Task Orchestrator / Async Scheduler
• Rate Limiter / Semaphore Manager (for LLM API calls)
• Caching Layer (persistent memoization of prompts/responses)
• Feedback loops & dynamic priority adjustment
D. Document Analysis Subsystems
1. Document Type Detector
– Samples document, classifies technical/scientific/narrative/legal/etc.
2. Intelligent Chunking Engine
– Overlapping chunk creation
– Boundary optimization & context preservation
3. Topic Extraction Engine
– Parallel initial extraction
– Frequency analysis & consolidation
– Semantic deduplication
– Importance ranking
4. Adaptive Exploration Controller
– Breadth-depth balancing
– Diminishing returns detection
5. Reality Check System
– Factual verification against source
– Confidence scoring
6. Semantic Redundancy Detector
– Textual, fuzzy, token-based, LLM-based checks
7. Emoji Selector
– Context-aware, hierarchical emoji assignment
E. Output Renderer
• Markdown Outline Generator
• Writes to mindmap_outputs/ folder
F. External Services
• LLM Providers: OpenAI, Anthropic (Claude), DeepSeek, Google Gemini
• Each accessed via provider-specific client modules
G. Logging & Monitoring
• Token usage tracker & cost estimates
• Error reporting & retry/backoff dashboards
Relationships & Data Flows
• Pipeline feeds document → Document Type Detector → chunking → Topic Extraction → Adaptive Controller → (loops back as needed)
• Responses pass through Redundancy Detector & Reality Checker, then to Emoji Selector
• Final hierarchical structure sent to Output Renderer → writes files
• Logging & Monitoring taps into each stage, writing logs to console and optional dashboards
• Caching layer interposes between pipeline and LLM Providers
Architectural Patterns & Principles
• Modular pipeline with clear separation of concerns
• Asynchronous, concurrent task orchestration
• Graph-like exploration (non-linear) with feedback loops
• Heuristic-guided processing (adaptive breadth/depth)
• Resilience: rate limiting, exponential backoff, graceful degradation
• Cost-optimization via provider selection, caching, batching