# mohanasrujana/message-analyser architecture

> Architecture diagram of the GitHub repository mohanasrujana/message-analyser, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mohanasrujana/message-analyser
- Repository: https://github.com/mohanasrujana/message-analyser

## Overview

##### Project Type and Purpose
- The Message Analyzer is a data-driven application focused on analyzing conversations to identify crime-related elements, specifically murder-related. It leverages a modified Gemma:2B model, exported to ONNX format, for efficient inference using ONNXRuntime.
- The README clarifies that the system supports both a Command Line Interface (CLI) and a Flask-ML-based web API for deploying the model.

##### Analyzing the File Structure
- Major directories:
- data: Contains raw and processed conversation datasets along with supplementary files (e.g., CSVs, JSON, TXT files). This indicates an important data ingestion and preprocessing layer.
- models: Houses the ONNX model(s) (gemma-2b.onnx) that are used for inference.
- scripts: Contains various scripts including:
- API related files (cli.py, server.py, and documentation server_info.md) that serve as the entry points for CLI usage and Flask-ML API deployment.
- Model export and inference scripts (export_to_onnx.py, run_onnx_inference.py) which handle conversion from PyTorch to ONNX and perform inference.
- Jupyter notebooks for experimentation and further analysis.
- questions: Storage for conversation-related question files that likely balance or guide the analysis.
- test: Includes unit and integration tests ensuring the reliability of the model’s functionalities.
- Files like requirements.txt and .gitignore help manage dependencies and version control, while the README provides comprehensive usage and export instructions.

##### Key Architectural Components
a. Data Layer:
- Data Ingestion & Preprocessing: Represent the storage and processing of both raw and processed conversation datasets from the “data” directory. This layer feeds into the analysis module.

b. Model and Inference Layer:
- Model Export: Highlight the export_to_onnx.py script that converts the Gemma-2B model to ONNX format (placed in the “models” directory).
- ONNX Model Inference: Illustrate the run_onnx_inference.py script and the underlying ONNXRuntime used for efficient inference.

c. Interface Layer:
- Command Line Interface (CLI): Connect the CLI (scripts/api/cli.py) for users to interact with the inference capabilities directly.
- Flask-ML API: Represent the Flask-based server (scripts/api/server.py) which offers a RESTful API for handling requests (such as image uploads or CSVs containing conversation data) and returning predictions.

d. Infrastructure & Testing:
- Testing: Use the “test” directory to denote modules that validate model performance and API functionalities.
- External Dependencies: Emphasize that technologies such as ONNXRuntime, Python (>=3.8), and Flask-ML are part of the stack.

##### Relationships and Interactions
- Data flow beginning with input data (raw CSVs in the “data” folder) that is either preprocessed or directly passed into the analysis modules.
- Flow from the preprocessing/import scripts to the model export script which generates the ONNX model.
- The ONNX model is then used by the inference scripts (either via CLI or API) to analyze conversation data.
- Finally, output (results or predicted categories) is stored in the “results” folder and may be further reviewed or served back to the user.

##### Relevant Architectural Patterns & Design Principles
- Modular and Layered Design: Segregation between data processing, model inference, and client interfacing (CLI/API) ensures clarity and maintainability.
- Separation of Concerns: Different scripts and modules for preprocessing, model export, inference, and API deployment.
- Extensibility: The use of configuration files and modular Jupyter notebooks indicates areas designed for experimentation or future enhancement (e.g., improving prediction accuracy or implementing ONNX quantization).

##### Specific Elements
- Clear labels on each component such as “Raw Data (CSV)”, “Preprocessed Data”, “Model Export (export_to_onnx.py)”, “ONNX Model (gemma-2b.onnx)”, “ONNX Inference (run_onnx_inference.py)”, “CLI (cli.py)”, “Flask-ML API (server.py)”, and “Results.”
- Data ingestion → Preprocessing → Model Export → ONNX Model → Inference Scripts.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        RawData["Raw Data (CSV)"] 
        Preprocess["Data Preprocessing (scripts/preprocessing.py)"]
        Questions["Questions (Analysis)"]
        RawData -->|"preprocess"| Preprocess
    end

    %% Model & Inference Layer
    subgraph "Model and Inference Layer"
        ModelExport["Model Export (export_to_onnx.py)"]
        ONNXModel["ONNX Model (gemma-2b.onnx)"]
        Inference["ONNX Inference (run_onnx_inference.py)"]
        Results["Results"]
        Preprocess -->|"prepare_export"| ModelExport
        ModelExport -->|"exports_model"| ONNXModel
        ONNXModel -->|"runs_inference"| Inference
        Inference -->|"generates"| Results
        Questions ---|"supplements"| Inference
    end

    %% Interface Layer
    subgraph "Interface Layer"
        CLI["Command Line Interface (cli.py)"]
        FlaskAPI["Flask-ML API (server.py)"]
        CLI ---|"CLI_command"| Inference
        FlaskAPI ---|"HTTP_POST"| Inference
    end

    %% Testing
    Testing["Testing Framework (test/)"]
    Testing ---|"unit/integration_tests"| CLI
    Testing ---|"unit/integration_tests"| FlaskAPI
    Testing ---|"validates"| Inference

    %% Click Events
    click RawData "https://github.com/mohanasrujana/message-analyser/tree/main/data/"
    click Preprocess "https://github.com/mohanasrujana/message-analyser/blob/main/scripts/preprocessing.py"
    click Questions "https://github.com/mohanasrujana/message-analyser/tree/main/questions/"
    click ModelExport "https://github.com/mohanasrujana/message-analyser/blob/main/scripts/export_to_onnx.py"
    click Inference "https://github.com/mohanasrujana/message-analyser/blob/main/scripts/run_onnx_inference.py"
    click CLI "https://github.com/mohanasrujana/message-analyser/blob/main/scripts/api/cli.py"
    click FlaskAPI "https://github.com/mohanasrujana/message-analyser/blob/main/scripts/api/server.py"
    click Testing "https://github.com/mohanasrujana/message-analyser/tree/main/test/"

    %% Styles
    classDef data fill:#E0F7FA,stroke:#006064,stroke-width:2px;
    classDef model fill:#E8F5E9,stroke:#1B5E20,stroke-width:2px;
    classDef interface fill:#FFF8E1,stroke:#FF6F00,stroke-width:2px;
    classDef testing fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px;

    class RawData,Preprocess,Questions data;
    class ModelExport,ONNXModel,Inference,Results model;
    class CLI,FlaskAPI interface;
    class Testing testing;
```
