# hss3107/ai_app architecture

> Architecture diagram of the GitHub repository hss3107/ai_app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hss3107/ai_app
- Repository: https://github.com/hss3107/ai_app

## Overview

##### Main Components
- Web Frontend – The Django templates (in the “app/templates/app” directory) provide user-facing pages (e.g., home, login, register, transcript_query).
- Django Application (Backend) – The “ai_project/ai_project” directory contains settings, URLs, WSGI, and ASGI configuration; “ai_project/app” contains models, views, middleware, and processors that implement business logic.
- AI / Document Processing Pipeline – This is represented by several directories:
– “processed_documents”, “transcripts”, “summaries”, and “content_maps” are used to store raw and processed document data.
– “ai_formatted_transcripts” and “Chain” (with JSON execution logs and a chain_logger module) capture AI chain execution details.
– “processors” (like smart_document_processor.py and nlp_resources.py) implement the AI logic and document handling.
- Vector Stores – The “vector_stores” folder holds many subdirectories that contain FAISS index files (index.faiss and index.pkl) for semantic search and similarity matching.
- Data Storage – The project uses SQLite (found as “db.sqlite3”) locally; however, other production databases might be used (as indicated by the presence of a “postgresql.crt”).
- Deployment and Infrastructure – The “docker” directory (with Prod Dockerfile, entrypoint.sh, and uwsgi.ini) and GitHub workflows outline the CI/CD/deployment pipeline.

##### Relationships and Data Flow
- The Web Frontend interacts with the Django backend using standard HTTP flows. User requests hit URL endpoints (defined in “urls.py”) and are routed to views.
- The Django backend (models, views, serializers, middleware) handles business logic, including user sessions, document uploads, and data processing.
- Once documents are uploaded (in the “media/temp_documents” or similar), they are processed into “processed_documents”, with detailed processing information logged.
- Transcripts and summaries are generated, and content maps are produced from the document contents.
- The AI processors (in “app/processors”) read formatted transcripts and automatically create embeddings. These embeddings are then stored and indexed using FAISS in the “vector_stores” directories. Data flows from document processing components to the vector store for retrieval during search queries.
- External services (e.g., video analysis indicated by “Video Analysis API.docx”) or additional AI modules might be invoked via chain execution (as logged in “Chain”).

- Use the Model-View-Controller (or Model-Template-View) pattern to represent Django’s separation of concerns.
- A clear separation between presentation (templates/front-end), business logic (views/controllers & middleware), and data persistence (database, file storage).
- A separate processing pipeline for AI tasks that includes document ingestion, processing (text extraction, summarization, embedding generation), and indexing (vector stores).
- Docker containerization and deployment components as part of an infrastructure layer.

##### Technologies, Frameworks, and Libraries
- Django (web framework)
- FAISS (vector similarity indexing)
- Python libraries for NLP/document processing (as evidenced by modules such as smart_document_processor.py and nlp_resources.py)
- Docker for deployment
- Likely additional libraries for logging, JSON processing, and possibly machine learning frameworks in the processors

– At the top, show the Client/Browser layer (with UI pages).
- URL Routing and Controllers (views)
- Template Rendering (static and dynamic content)
– Side-by-side or below, include two clusters:
- Data Layer: database (SQLite, potential PostgreSQL), media storage for PDFs and images.
- AI Processing Pipeline:
– Document Ingestion Module (from media/temp_documents)
– Processing & Formatting Module (processed_documents, transcripts, summaries, content_maps)
– NLP & Embedding Generation (processors, chain execution logs)
– Finally, a cluster for the Vector Stores (all FAISS indexes) that connect back to the AI pipeline.
– At the bottom or side, add an Infrastructure layer:
- Docker (Prod build, entrypoint) and CI/CD (GitHub workflows).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global Entities
    Client["Client/Browser"]:::frontend

    %% Web Frontend Layer
    subgraph "Web Frontend"
        FE["Django Templates (UI)"]:::frontend
    end

    %% Django Backend Layer
    subgraph "Django Backend"
        DCR["Core Config & Routing"]:::backend
        DBM["Business Logic & Models"]:::backend
    end

    %% Data Storage Layer
    subgraph "Data Storage"
        SQL["SQLite Database"]:::data
        PG["PostgreSQL Certificate"]:::data
        Media["Media File Storage (temp_documents)"]:::data
    end

    %% AI Processing Pipeline
    subgraph "AI Processing Pipeline"
        DP["Document Processing (Processors)"]:::ai
        subgraph "Processed Content Storage"
            PD["processed_documents"]:::ai
            TR["transcripts"]:::ai
            SM["summaries"]:::ai
            CM["content_maps"]:::ai
            AFT["AI Formatted Transcripts"]:::ai
        end
        VS["Vector Search (FAISS Index)"]:::ai
    end

    %% Deployment & Infrastructure
    subgraph "Deployment & Infrastructure"
        Docker["Docker Deployment"]:::deployment
        CICD["CI/CD Pipeline"]:::deployment
    end

    %% Connections / Data Flow
    Client -->|"HTTP_request"| FE
    FE -->|"renders_request"| DCR
    DCR -->|"routes to"| DBM

    DBM -->|"reads/writes"| SQL
    DBM -->|"reads/writes"| PG
    DBM -->|"uploads"| Media

    Media -->|"document ingestion"| DP
    DP -->|"processes to"| PD
    PD -->|"indexes content"| VS
    VS -->|"retrieval for search"| DBM

    Docker -->|"deploys"| DCR
    CICD -->|"triggers"| Docker

    %% Click Events
    click FE "https://github.com/hss3107/ai_app/tree/main/ai_project/app/templates/app/"
    click DCR "https://github.com/hss3107/ai_app/tree/main/ai_project/ai_project/"
    click DBM "https://github.com/hss3107/ai_app/tree/main/ai_project/app/"
    click DP "https://github.com/hss3107/ai_app/tree/main/ai_project/app/processors/"
    click PD "https://github.com/hss3107/ai_app/tree/main/ai_project/processed_documents"
    click TR "https://github.com/hss3107/ai_app/tree/main/ai_project/transcripts"
    click SM "https://github.com/hss3107/ai_app/tree/main/ai_project/summaries"
    click CM "https://github.com/hss3107/ai_app/tree/main/ai_project/content_maps"
    click AFT "https://github.com/hss3107/ai_app/tree/main/ai_project/ai_formatted_transcripts"
    click VS "https://github.com/hss3107/ai_app/tree/main/ai_project/vector_stores/"
    click SQL "https://github.com/hss3107/ai_app/blob/main/ai_project/db.sqlite3"
    click PG "https://github.com/hss3107/ai_app/blob/main/ai_project/postgresql.crt"
    click Media "https://github.com/hss3107/ai_app/tree/main/ai_project/media/temp_documents"
    click Docker "https://github.com/hss3107/ai_app/tree/main/ai_project/docker/"
    click CICD "https://github.com/hss3107/ai_app/blob/main/.github/workflows/ci_cd.yml"

    %% Styles
    classDef frontend fill:#a2d9ce,stroke:#117864,stroke-width:2px;
    classDef backend fill:#f9e79f,stroke:#b7950b,stroke-width:2px;
    classDef data fill:#d2b4de,stroke:#6c3483,stroke-width:2px;
    classDef ai fill:#fad7a0,stroke:#cb6e17,stroke-width:2px;
    classDef deployment fill:#f5b7b1,stroke:#922b21,stroke-width:2px;
```
