# oxapavan/loganalysis-llm architecture

> Architecture diagram of the GitHub repository oxapavan/loganalysis-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/oxapavan/loganalysis-llm
- Repository: https://github.com/oxapavan/loganalysis-llm

## Overview

##### Project type and purpose
- GenAI_LogAnalyzer is a full-stack AI-powered log analysis application built in Python.
- Purpose: Ingest raw IT log files, perform root cause analysis via large language models (Cohere, AWS Bedrock), generate summaries, and integrate with ITSM tools (e.g., ServiceNow).

##### File structure
- Top-level scripts:
- app.py, main.py – orchestration/entry points
- streamlitUI.py / streamlitUI_showcase.py – Streamlit-based frontend
- OneTimeExecution.py – AWS setup (credentials, infrastructure)
- LLM modules: LLM.py, “LLM Chatbot…” .py/.ipynb – exploratory notebooks / CLI demos
- Summary modules: SmallSummary.py, LargeSummary.py, Chunking.py
- VectorDB.py – wrapper around vector database for embedding storage & retrieval
- utils/bedrock.py – AWS Bedrock integration
- utils/tools_agents.py – connector to external services (e.g., ServiceNow API)
- input/ & output/ – sample log files and report outputs
- requirements.txt / requirement_whole.txt – dependencies

##### README for additional insights
- Describes AWS prerequisites (OneTimeExecution.py).
- Launch via `streamlit run streamlitUI.py`.
- Mentions integration plugin for ServiceNow.

##### Main components and their interactions
a. Streamlit Frontend
– Components: streamlitUI.py (primary), optional showcase.
– Presents upload widget for log files, progress indicators, and displays analysis reports.
b. Orchestration Layer (API & Controllers)
– app.py / main.py handle request routing from UI to backend logic.
– OneTimeExecution.py provisions AWS resources before runtime.
c. Preprocessing & Chunking
– Chunking.py breaks raw logs into token-limited segments.
d. Embedding & Vector Database
– VectorDB.py creates embeddings (via LLM module) and performs similarity search.
e. LLM Processing
– LLM.py abstracts calls to Cohere or AWS Bedrock (via utils/bedrock.py).
– “LLM Chatbot” scripts demonstrate conversational analysis mode.
f. Summarization & Analysis
– SmallSummary.py for quick summaries, LargeSummary.py for in-depth reports, LargeSummary writes final_report.txt.
g. External Integrations
– utils/tools_agents.py handles ServiceNow API calls to push tickets or retrieve context.
h. Data Stores & Outputs
– Local file system (output/*.txt) for interim and final reports.
– (Optionally) vector DB can be hosted in cloud (e.g., AWS memory store, not in file tree but configurable).

##### Architectural patterns and design principles
- Layered architecture: UI → API/Orchestrator → Domain Services (preprocessing, vector store, LLM) → External Integrations.
- Modular design: each functional area in its own module; easy to extend or swap LLM providers.
- Separation of concerns: chunking, embedding, summarization separated into discrete scripts/classes.
- Unidirectional data flow: user uploads → preprocessing → embedding → analysis → summarization → output/integration.

##### Actionable next steps
- Lay out the layers horizontally: UI on the left, processing pipeline in the center, external services on the right.
- Every major `.py` or module in your file tree is represented or grouped logically.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% UI Layer
    subgraph "UI Layer"
        StreamlitUI["StreamlitUI"]:::ui
        StreamlitShowcase["StreamlitUI_Showcase"]:::ui
    end

    %% Orchestrator/API Layer
    subgraph "Orchestrator/API Layer"
        App["app.py"]:::backend
        Main["main.py"]:::backend
        OneTime["OneTimeExecution.py"]:::backend
    end

    %% Preprocessing Service
    subgraph "Preprocessing & Chunking"
        Chunk["Chunking.py"]:::backend
    end

    %% Embedding & VectorDB
    subgraph "Embedding & VectorDB"
        DB[(VectorDB)]:::storage
    end

    %% LLM Processing Service
    subgraph "LLM Processing Service"
        LLMService["LLM.py"]:::backend
        Bedrock["AWS Bedrock"]:::external
    end

    %% Summarization & Analysis
    subgraph "Summarization & Analysis"
        SmallSum["SmallSummary.py"]:::backend
        LargeSum["LargeSummary.py"]:::backend
    end

    %% External Integrations & Data Stores
    subgraph "External Integrations & Data Stores"
        Tools["ServiceNow Connector\n(utils/tools_agents.py)"]:::external
        FSin["input/"]:::storage
        FSout["output/"]:::storage
        Cohere["Cohere API"]:::external
    end

    %% Exploratory Demos
    subgraph "Exploratory LLM Demos"
        Demo1["LLM Chatbot using Streamlit\n(Cohere).py"]:::backend
        Demo2["LLM Notebook Chatbot\n(Cohere).ipynb"]:::backend
    end

    %% Connections
    StreamlitUI -->|"upload log file"| App
    StreamlitShowcase -->|"upload log file"| App
    App -->|"invoke setup"| OneTime
    App -->|"route requests"| Main
    Main -->|"tokenize & chunk"| Chunk
    Chunk -->|"create embeddings"| DB
    Main -->|"invoke analysis"| LLMService
    LLMService -->|"call Cohere"| Cohere
    LLMService -->|"call Bedrock"| Bedrock
    DB -->|"retrieve vectors"| LLMService
    LLMService -->|"generate summary"| SmallSum
    LLMService -->|"generate in-depth summary"| LargeSum
    SmallSum -->|"write report"| FSout
    LargeSum -->|"write final_report.txt"| FSout
    LargeSum -->|"push ticket"| Tools
    FSin --> Main
    Demo1 -.-> Cohere
    Demo2 -.-> Cohere

    %% Click Events
    click StreamlitUI "https://github.com/oxapavan/loganalysis-llm/blob/main/streamlitUI.py"
    click StreamlitShowcase "https://github.com/oxapavan/loganalysis-llm/blob/main/streamlitUI_showcase.py"
    click App "https://github.com/oxapavan/loganalysis-llm/blob/main/app.py"
    click Main "https://github.com/oxapavan/loganalysis-llm/blob/main/main.py"
    click OneTime "https://github.com/oxapavan/loganalysis-llm/blob/main/OneTimeExecution.py"
    click Chunk "https://github.com/oxapavan/loganalysis-llm/blob/main/Chunking.py"
    click DB "https://github.com/oxapavan/loganalysis-llm/blob/main/VectorDB.py"
    click LLMService "https://github.com/oxapavan/loganalysis-llm/blob/main/LLM.py"
    click Bedrock "https://github.com/oxapavan/loganalysis-llm/blob/main/utils/bedrock.py"
    click SmallSum "https://github.com/oxapavan/loganalysis-llm/blob/main/SmallSummary.py"
    click LargeSum "https://github.com/oxapavan/loganalysis-llm/blob/main/LargeSummary.py"
    click Tools "https://github.com/oxapavan/loganalysis-llm/blob/main/utils/tools_agents.py"
    click Demo1 "https://github.com/oxapavan/loganalysis-llm/blob/main/LLM Chatbot using Streamlit (Cohere).py"
    click Demo2 "https://github.com/oxapavan/loganalysis-llm/blob/main/LLM Notebook Chatbot (Cohere).ipynb"
    click FSin "https://github.com/oxapavan/loganalysis-llm/tree/main/input/"
    click FSout "https://github.com/oxapavan/loganalysis-llm/tree/main/output/"

    %% Styles
    classDef ui fill:#ADD8E6,stroke:#000
    classDef backend fill:#B0E57C,stroke:#000
    classDef external fill:#FFA500,stroke:#000
    classDef storage fill:#D3D3D3,stroke:#000
```
