# thiagofinch/mega-brain architecture

> Architecture diagram of the GitHub repository thiagofinch/mega-brain, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thiagofinch/mega-brain
- Repository: https://github.com/thiagofinch/mega-brain

## Overview

Mega Brain turns user-provided material into organized, searchable knowledge, then supports evidence-grounded strategic decisions. The documented flow is ingest, process and index, query, and deliberate; separate capabilities include video analysis and JARVIS voice interaction. The graph preserves major README features and indexed core modules. Some implementation details, especially Conclave and unsampled pipeline stages, are documented but not shown in source excerpts, so those links are identified as documentation-based or left unverified.

Main entry points: [`engine/intelligence/pipeline/batch_auto_creator.py`](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/batch_auto_creator.py) (Batch Processing), [`engine/jarvis/voice/orchestrator.py`](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/orchestrator.py) (Voice Orchestrator), [`engine/intelligence/rag/pipeline.py`](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/pipeline.py) (RAG Pipeline), [`scripts/ingest-with-entity-discovery.py`](https://github.com/thiagofinch/mega-brain/blob/main/scripts/ingest-with-entity-discovery.py) (Ingestion Entry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Ingestion and Processing"]
  node_ingest_cmd["Ingest Command"]
  node_ingest_entry["Ingestion Entry"]
  node_video["Video Analysis<br/>[pipeline.py]"]
  node_batch["Batch Processing"]
  node_bucket["Bucket Routing<br/>[pipeline_router.py]"]
  node_llm["LLM Routing<br/>[llm_router.py]"]
end

subgraph group_search["Knowledge and RAG"]
  node_knowledge[("Local Knowledge<br/>[memory_manager.py]")]
  node_rag_pipeline["RAG Pipeline<br/>[pipeline.py]"]
  node_adaptive["Adaptive Retrieval<br/>[adaptive_router.py]"]
  node_hybrid_query["Hybrid Search<br/>[hybrid_query.py]"]
  node_hybrid_index[("Hybrid Index<br/>[hybrid_index.py]")]
  node_bucket_query["Bucket Queries"]
end

subgraph group_decision["Decision Support"]
  node_conclave["Conclave Council"]
end

subgraph group_voice["JARVIS Voice"]
  node_voice_main["Voice Application<br/>[main.py]"]
  node_voice_orchestrator["Voice Orchestrator<br/>[orchestrator.py]"]
  node_stt["Speech Recognition<br/>[stt_handler.py]"]
  node_connector["Brain Connector"]
  node_tts["Speech Output<br/>[tts_handler.py]"]
end

node_user(("User"))

node_user -->|"submits material"| node_ingest_cmd
node_ingest_cmd -.->|"starts ingestion"| node_ingest_entry
node_ingest_entry -->|"uses video pipeline"| node_video
node_ingest_entry -->|"creates batches"| node_batch
node_batch -.->|"routes content"| node_bucket
node_batch -.->|"extracts insights"| node_llm
node_batch -->|"enriches memory"| node_knowledge
node_user -->|"asks questions"| node_rag_pipeline
node_rag_pipeline -->|"routes retrieval"| node_adaptive
node_adaptive -->|"runs hybrid search"| node_hybrid_query
node_adaptive -->|"uses index"| node_hybrid_index
node_bucket_query -->|"searches buckets"| node_hybrid_query
node_bucket_query -->|"selects bucket index"| node_hybrid_index
node_rag_pipeline -->|"builds memory context"| node_knowledge
node_user -.->|"requests deliberation"| node_conclave
node_conclave -.->|"grounds decisions"| node_rag_pipeline
node_voice_main -->|"starts"| node_voice_orchestrator
node_voice_orchestrator -->|"receives transcripts"| node_stt
node_voice_orchestrator -->|"uses brain state"| node_connector
node_voice_orchestrator -->|"speaks responses"| node_tts

click node_ingest_entry "https://github.com/thiagofinch/mega-brain/blob/main/scripts/ingest-with-entity-discovery.py"
click node_video "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/video/pipeline.py"
click node_batch "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/batch_auto_creator.py"
click node_bucket "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/pipeline_router.py"
click node_llm "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/mce/llm_router.py"
click node_knowledge "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/memory_manager.py"
click node_rag_pipeline "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/pipeline.py"
click node_adaptive "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/adaptive_router.py"
click node_hybrid_query "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/hybrid_query.py"
click node_hybrid_index "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/hybrid_index.py"
click node_bucket_query "https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/bucket_query_router.py"
click node_conclave "https://github.com/thiagofinch/mega-brain/tree/main/engine/intelligence/pipeline/mce"
click node_voice_main "https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/main.py"
click node_voice_orchestrator "https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/orchestrator.py"
click node_stt "https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/stt_handler.py"
click node_connector "https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/mega_brain_connector.py"
click node_tts "https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/tts_handler.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_cmd,node_ingest_entry,node_video,node_batch,node_bucket,node_llm,node_user toneBlue
class node_knowledge,node_rag_pipeline,node_adaptive,node_hybrid_query,node_hybrid_index,node_bucket_query toneAmber
class node_conclave toneMint
class node_voice_main,node_voice_orchestrator,node_stt,node_connector,node_tts toneRose
```

## Components

### Ingestion and Processing

- **Ingest Command**
- [Ingestion Entry](https://github.com/thiagofinch/mega-brain/blob/main/scripts/ingest-with-entity-discovery.py): `scripts/ingest-with-entity-discovery.py`
- [Video Analysis](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/video/pipeline.py): `engine/intelligence/pipeline/video/pipeline.py`
- [Batch Processing](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/batch_auto_creator.py): `engine/intelligence/pipeline/batch_auto_creator.py`
- [Bucket Routing](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/pipeline_router.py): `engine/intelligence/pipeline/pipeline_router.py`
- [LLM Routing](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/pipeline/mce/llm_router.py): `engine/intelligence/pipeline/mce/llm_router.py`

### Knowledge and RAG

- [Local Knowledge](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/memory_manager.py): `engine/intelligence/memory_manager.py`
- [RAG Pipeline](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/pipeline.py): `engine/intelligence/rag/pipeline.py`
- [Adaptive Retrieval](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/adaptive_router.py): `engine/intelligence/rag/adaptive_router.py`
- [Hybrid Search](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/hybrid_query.py): `engine/intelligence/rag/hybrid_query.py`
- [Hybrid Index](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/hybrid_index.py): `engine/intelligence/rag/hybrid_index.py`
- [Bucket Queries](https://github.com/thiagofinch/mega-brain/blob/main/engine/intelligence/rag/bucket_query_router.py): `engine/intelligence/rag/bucket_query_router.py`

### Decision Support

- [Conclave Council](https://github.com/thiagofinch/mega-brain/tree/main/engine/intelligence/pipeline/mce): `engine/intelligence/pipeline/mce`

### JARVIS Voice

- [Voice Application](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/main.py): `engine/jarvis/voice/main.py`
- [Voice Orchestrator](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/orchestrator.py): `engine/jarvis/voice/orchestrator.py`
- [Speech Recognition](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/stt_handler.py): `engine/jarvis/voice/stt_handler.py`
- [Brain Connector](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/mega_brain_connector.py): `engine/jarvis/voice/mega_brain_connector.py`
- [Speech Output](https://github.com/thiagofinch/mega-brain/blob/main/engine/jarvis/voice/tts_handler.py): `engine/jarvis/voice/tts_handler.py`

### Other components

- **User**

## Connections

- User → Ingest Command: submits material
- Ingest Command → Ingestion Entry: starts ingestion
- Ingestion Entry → Video Analysis: uses video pipeline (evidence: `scripts/ingest-with-entity-discovery.py`)
- Ingestion Entry → Batch Processing: creates batches (evidence: `scripts/ingest-with-entity-discovery.py`)
- Batch Processing → Bucket Routing: routes content
- Batch Processing → LLM Routing: extracts insights
- Batch Processing → Local Knowledge: enriches memory (evidence: `engine/intelligence/pipeline/batch_auto_creator.py`)
- User → RAG Pipeline: asks questions
- RAG Pipeline → Adaptive Retrieval: routes retrieval (evidence: `engine/intelligence/rag/pipeline.py`)
- Adaptive Retrieval → Hybrid Search: runs hybrid search (evidence: `engine/intelligence/rag/adaptive_router.py`)
- Adaptive Retrieval → Hybrid Index: uses index (evidence: `engine/intelligence/rag/adaptive_router.py`)
- Bucket Queries → Hybrid Search: searches buckets (evidence: `engine/intelligence/rag/bucket_query_router.py`)
- Bucket Queries → Hybrid Index: selects bucket index (evidence: `engine/intelligence/rag/bucket_query_router.py`)
- RAG Pipeline → Local Knowledge: builds memory context (evidence: `engine/intelligence/rag/pipeline.py`)
- User → Conclave Council: requests deliberation
- Conclave Council → RAG Pipeline: grounds decisions
- Voice Application → Voice Orchestrator: starts (evidence: `engine/jarvis/voice/main.py`)
- Voice Orchestrator → Speech Recognition: receives transcripts (evidence: `engine/jarvis/voice/orchestrator.py`)
- Voice Orchestrator → Brain Connector: uses brain state (evidence: `engine/jarvis/voice/orchestrator.py`)
- Voice Orchestrator → Speech Output: speaks responses (evidence: `engine/jarvis/voice/orchestrator.py`)
