# vinayakpnandi/insightai architecture

> Architecture diagram of the GitHub repository vinayakpnandi/insightai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vinayakpnandi/insightai
- Repository: https://github.com/vinayakpnandi/insightai

## Overview

InsightAI is a Streamlit analytics and decision-support application. Its main workflow takes an uploaded dataset through quality checks and profiling, then supports SQL analysis, visualization, automated model training, predictions, explainability, and business reporting. A separate document workflow ingests PDFs into a vector store for question answering. The graph follows documented capabilities and source-index references; unsampled implementation details are represented only where README capabilities support them, and their internal links are omitted when unverified.

Main entry points: [`app.py`](https://github.com/vinayakpnandi/insightai/blob/main/app.py) (Streamlit App), [`src/ml/service.py`](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/service.py) (ML Analysis Service), [`src/database/sql_agent.py`](https://github.com/vinayakpnandi/insightai/blob/main/src/database/sql_agent.py) (Natural-Language SQL), [`src/database/loader.py`](https://github.com/vinayakpnandi/insightai/blob/main/src/database/loader.py) (Dataset Loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Application"]
  node_ui["Streamlit App<br/>[app.py]"]
end

subgraph group_data["Dataset Analytics"]
  node_upload["Dataset Loader<br/>[loader.py]"]
  node_quality["Data Quality<br/>[data_quality.py]"]
  node_profile["Dataset Profiler<br/>[profiler.py]"]
  node_sqlite[("SQLite Database<br/>[database.py]")]
  node_sqlagent["Natural-Language SQL<br/>[sql_agent.py]"]
  node_schema["Schema Formatter<br/>[schema.py]"]
  node_visual["Visualization<br/>[visualizer.py]"]
end

subgraph group_ml["Machine Learning"]
  node_mlservice["ML Analysis Service<br/>[service.py]"]
  node_predictor["Model Training<br/>[predictor.py]"]
  node_preprocess["Preprocessing<br/>[preprocessor.py]"]
  node_models["Candidate Models<br/>[models.py]"]
  node_leakage["Leakage Analysis<br/>[leakage.py]"]
  node_explain["Feature Importance<br/>[explainability.py]"]
  node_prediction["Interactive Prediction<br/>[prediction.py]"]
  node_insights["Business Insights"]
end

subgraph group_docs["Document Q&amp;A"]
  node_pdf["PDF Ingestion<br/>[ingest.py]"]
  node_docloader["Document Loading<br/>[document_loader.py]"]
  node_vectordb[("Vector Store<br/>[vector_store.py]")]
  node_embeddings["Embeddings<br/>[embeddings.py]"]
  node_retriever["Document Retriever<br/>[retriever.py]"]
  node_qa["Document Q&amp;A<br/>[qa.py]"]
end

node_user(("Analyst"))
node_llm["Local Ollama LLM"]

node_user -->|"uses"| node_ui
node_ui -->|"loads datasets"| node_upload
node_upload -->|"checks quality"| node_quality
node_upload -->|"writes dataset"| node_sqlite
node_ui -->|"submits questions"| node_sqlagent
node_sqlagent -->|"formats schema"| node_schema
node_sqlagent -->|"generates SQL"| node_llm
node_sqlagent -->|"executes query"| node_sqlite
node_ui -->|"requests charts"| node_visual
node_ui -->|"runs analysis"| node_mlservice
node_mlservice -->|"profiles data"| node_profile
node_mlservice -->|"trains models"| node_predictor
node_mlservice -->|"checks leakage"| node_leakage
node_mlservice -->|"computes importance"| node_explain
node_predictor -->|"builds preprocessing"| node_preprocess
node_predictor -->|"selects candidates"| node_models
node_ui -->|"collects inputs"| node_prediction
node_ui -->|"requests report"| node_insights
node_insights -->|"generates report"| node_llm
node_ui -->|"ingests PDFs"| node_pdf
node_pdf -->|"loads and splits"| node_docloader
node_pdf -->|"stores chunks"| node_vectordb
node_vectordb -->|"uses embeddings"| node_embeddings
node_ui -->|"asks document questions"| node_qa
node_qa -->|"retrieves context"| node_retriever
node_qa -->|"answers question"| node_llm
node_retriever -->|"searches chunks"| node_vectordb

click node_ui "https://github.com/vinayakpnandi/insightai/blob/main/app.py"
click node_upload "https://github.com/vinayakpnandi/insightai/blob/main/src/database/loader.py"
click node_quality "https://github.com/vinayakpnandi/insightai/blob/main/src/tools/data_quality.py"
click node_profile "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/profiler.py"
click node_sqlite "https://github.com/vinayakpnandi/insightai/blob/main/src/database/database.py"
click node_sqlagent "https://github.com/vinayakpnandi/insightai/blob/main/src/database/sql_agent.py"
click node_schema "https://github.com/vinayakpnandi/insightai/blob/main/src/database/schema.py"
click node_visual "https://github.com/vinayakpnandi/insightai/blob/main/src/tools/visualizer.py"
click node_mlservice "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/service.py"
click node_predictor "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/predictor.py"
click node_preprocess "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/preprocessor.py"
click node_models "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/models.py"
click node_leakage "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/leakage.py"
click node_explain "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/explainability.py"
click node_prediction "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/prediction.py"
click node_insights "https://github.com/vinayakpnandi/insightai/blob/main/src/ml/business_insights.py"
click node_pdf "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/ingest.py"
click node_docloader "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/document_loader.py"
click node_vectordb "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/vector_store.py"
click node_embeddings "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/embeddings.py"
click node_retriever "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/retriever.py"
click node_qa "https://github.com/vinayakpnandi/insightai/blob/main/src/rag/qa.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui toneBlue
class node_upload,node_quality,node_profile,node_sqlite,node_sqlagent,node_schema,node_visual toneAmber
class node_mlservice,node_predictor,node_preprocess,node_models,node_leakage,node_explain,node_prediction,node_insights toneMint
class node_pdf,node_docloader,node_vectordb,node_embeddings,node_retriever,node_qa toneRose
class node_user,node_llm toneIndigo
```

## Components

### Application

- [Streamlit App](https://github.com/vinayakpnandi/insightai/blob/main/app.py): `app.py`

### Dataset Analytics

- [Dataset Loader](https://github.com/vinayakpnandi/insightai/blob/main/src/database/loader.py): `src/database/loader.py`
- [Data Quality](https://github.com/vinayakpnandi/insightai/blob/main/src/tools/data_quality.py): `src/tools/data_quality.py`
- [Dataset Profiler](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/profiler.py): `src/ml/profiler.py`
- [SQLite Database](https://github.com/vinayakpnandi/insightai/blob/main/src/database/database.py): `src/database/database.py`
- [Natural-Language SQL](https://github.com/vinayakpnandi/insightai/blob/main/src/database/sql_agent.py): `src/database/sql_agent.py`
- [Schema Formatter](https://github.com/vinayakpnandi/insightai/blob/main/src/database/schema.py): `src/database/schema.py`
- [Visualization](https://github.com/vinayakpnandi/insightai/blob/main/src/tools/visualizer.py): `src/tools/visualizer.py`

### Machine Learning

- [ML Analysis Service](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/service.py): `src/ml/service.py`
- [Model Training](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/predictor.py): `src/ml/predictor.py`
- [Preprocessing](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/preprocessor.py): `src/ml/preprocessor.py`
- [Candidate Models](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/models.py): `src/ml/models.py`
- [Leakage Analysis](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/leakage.py): `src/ml/leakage.py`
- [Feature Importance](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/explainability.py): `src/ml/explainability.py`
- [Interactive Prediction](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/prediction.py): `src/ml/prediction.py`
- [Business Insights](https://github.com/vinayakpnandi/insightai/blob/main/src/ml/business_insights.py): `src/ml/business_insights.py`

### Document Q&A

- [PDF Ingestion](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/ingest.py): `src/rag/ingest.py`
- [Document Loading](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/document_loader.py): `src/rag/document_loader.py`
- [Vector Store](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/vector_store.py): `src/rag/vector_store.py`
- [Embeddings](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/embeddings.py): `src/rag/embeddings.py`
- [Document Retriever](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/retriever.py): `src/rag/retriever.py`
- [Document Q&A](https://github.com/vinayakpnandi/insightai/blob/main/src/rag/qa.py): `src/rag/qa.py`

### Other components

- **Analyst**
- **Local Ollama LLM**

## Connections

- Analyst → Streamlit App: uses (evidence: `README.md`)
- Streamlit App → Dataset Loader: loads datasets (evidence: `app.py`)
- Dataset Loader → Data Quality: checks quality (evidence: `src/database/loader.py`)
- Dataset Loader → SQLite Database: writes dataset (evidence: `README.md`)
- Streamlit App → Natural-Language SQL: submits questions (evidence: `app.py`)
- Natural-Language SQL → Schema Formatter: formats schema (evidence: `src/database/sql_agent.py`)
- Natural-Language SQL → Local Ollama LLM: generates SQL (evidence: `src/database/sql_agent.py`)
- Natural-Language SQL → SQLite Database: executes query (evidence: `src/database/sql_agent.py`)
- Streamlit App → Visualization: requests charts (evidence: `app.py`)
- Streamlit App → ML Analysis Service: runs analysis (evidence: `app.py`)
- ML Analysis Service → Dataset Profiler: profiles data (evidence: `src/ml/service.py`)
- ML Analysis Service → Model Training: trains models (evidence: `src/ml/service.py`)
- ML Analysis Service → Leakage Analysis: checks leakage (evidence: `src/ml/service.py`)
- ML Analysis Service → Feature Importance: computes importance (evidence: `src/ml/service.py`)
- Model Training → Preprocessing: builds preprocessing (evidence: `src/ml/predictor.py`)
- Model Training → Candidate Models: selects candidates (evidence: `src/ml/predictor.py`)
- Streamlit App → Interactive Prediction: collects inputs (evidence: `app.py`)
- Streamlit App → Business Insights: requests report (evidence: `app.py`)
- Business Insights → Local Ollama LLM: generates report (evidence: `src/ml/business_insights.py`)
- Streamlit App → PDF Ingestion: ingests PDFs (evidence: `app.py`)
- PDF Ingestion → Document Loading: loads and splits (evidence: `src/rag/ingest.py`)
- PDF Ingestion → Vector Store: stores chunks (evidence: `src/rag/ingest.py`)
- Vector Store → Embeddings: uses embeddings (evidence: `src/rag/vector_store.py`)
- Streamlit App → Document Q&A: asks document questions (evidence: `app.py`)
- Document Q&A → Document Retriever: retrieves context (evidence: `src/rag/qa.py`)
- Document Q&A → Local Ollama LLM: answers question (evidence: `src/rag/qa.py`)
- Document Retriever → Vector Store: searches chunks (evidence: `src/rag/retriever.py`)
