# sathvick-sudarsan/natural-language-to-code architecture

> Architecture diagram of the GitHub repository sathvick-sudarsan/natural-language-to-code, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sathvick-sudarsan/natural-language-to-code
- Repository: https://github.com/sathvick-sudarsan/natural-language-to-code

## Overview

##### Overall Project Type and Purpose
- This is a machine learning application with a clear focus on natural language processing for code generation.
- The project incorporates both model training/evaluation components and a Streamlit-based UI for end-user interaction.

##### Main Components and Their Roles
A. User Interface:
- Components: Streamlit apps (app.py, app_seq2seq.py, app_rag.py).
- Responsibility: Accept user inputs (English text queries) and display the generated Python code.

B. Model Layer:
- Components:
- Baseline Model: baseline_seq2seq.py (Seq2Seq model with attention).
- Transformer Model: (While the README indicates “transformer_model.py”, the file tree shows transformer_viz.py and trasnformer_rag_viz.py which could either be the model definitions or visualization helpers. Clarify if these contain model code or if they are just for representation.)
- Additional RAG Experimental Model: app_rag.py, evaluate_rag.py, trasnformer_rag_viz.py.
- Responsibility: Process the natural language input, generate code, and provide model evaluation via evaluate.py, evaluate_seq2seq.py, and evaluate_rag.py.

C. Data Layer:
- Components:
- Data: CSV files (3.7K_python_dataset.csv, mbpp(formatted).csv, mbpp_conala.csv).
- Utilities: utils.py handles data preprocessing, formatting, and loading.
- Responsibility: Provide structured datasets for training, validation, and model input preprocessing.

D. Evaluation and Visualization:
- Components: evaluate.py, evaluate_seq2seq.py, evaluate_rag.py for model performance, and transformer_viz.py, trasnformer_rag_viz.py for visualizing model performance or internal workings.
- Responsibility: Measure model accuracy and aid in debugging and design validation through visual representations.

E. External Dependencies and Setup:
- Components: requirements.txt.
- Responsibility: Lists third-party libraries and tools required (e.g., libraries for machine learning, Streamlit, etc.).

##### Relationships and Data Flow
- The UI then routes the request to the corresponding model module (Seq2Seq, Transformer, or RAG).
- Prior to inference, data may need to be preprocessed (via utils.py) if required by the model.
- Model outputs (generated Python code) are then sent back to the UI.
- Separately, the evaluation modules and visualization tools receive model outputs and training logs to generate performance metrics and visual representations.

##### Important Architectural Patterns
- Modularity: Each file and module serves a distinct function (UI vs. model inference vs. evaluation), reflecting separation of concerns.
- Data Pipeline: A clear data ingest and preprocessing pipeline is evident with dedicated dataset files and utility functions.
- Extensibility: Having separate apps for different models (Seq2Seq, Transformer, RAG) suggests the architecture supports experimentation with various deep learning approaches.

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% User Input
  UQ["User Query (English input)"]:::user

  %% External Dependencies
  subgraph "External Dependencies"
    EXT["External Libraries"]:::ext
  end

  %% Frontend/Interface
  subgraph "Frontend/Interface"
    UI1["Streamlit App (app.py)"]:::interface
    UI2["Streamlit Seq2Seq (app_seq2seq.py)"]:::interface
    UI3["Streamlit RAG (app_rag.py)"]:::interface
  end

  %% Data Layer
  subgraph "Data Layer"
    DSET1["Dataset: 3.7K_python_dataset.csv"]:::data
    DSET2["Dataset: mbpp(formatted).csv"]:::data
    DSET3["Dataset: mbpp_conala.csv"]:::data
    UTIL["Data Preprocessing (utils.py)"]:::data
  end

  %% Model Layer
  subgraph "Model Layer"
    MODEL1["Baseline Model (baseline_seq2seq.py)"]:::model
    MODEL2["Transformer Model (transformer_viz.py)"]:::model
    MODEL3["RAG Model (app_rag.py entry)"]:::model
  end

  %% Evaluation & Visualization
  subgraph "Evaluation & Visualization"
    EVAL1["General Evaluation (evaluate.py)"]:::eval
    EVAL2["Seq2Seq Evaluation (evaluate_seq2seq.py)"]:::eval
    EVAL3["RAG Evaluation (evaluate_rag.py)"]:::eval
    VIS1["Transformer Visualization (transformer_viz.py)"]:::vis
    VIS2["RAG Visualization (trasnformer_rag_viz.py)"]:::vis
  end

  %% External Dependencies arrows
  EXT --- UI1
  EXT --- UI2
  EXT --- UI3
  EXT --- MODEL1
  EXT --- MODEL2
  EXT --- MODEL3
  EXT --- UTIL

  %% Data flow: Datasets to Utility
  DSET1 --> UTIL
  DSET2 --> UTIL
  DSET3 --> UTIL

  %% User Query to Frontend
  UQ --> UI1
  UQ --> UI2
  UQ --> UI3

  %% Frontend to Model Layer (based on app endpoints)
  UI1 -->|"model_inference"| MODEL2
  UI2 -->|"model_inference"| MODEL1
  UI3 -->|"model_inference"| MODEL3

  %% Data preprocessing to Models
  UTIL -->|"preprocessed_data"| MODEL1
  UTIL -->|"preprocessed_data"| MODEL2
  UTIL -->|"preprocessed_data"| MODEL3

  %% Models to Evaluation & Visualization
  MODEL1 -->|"prediction"| EVAL1
  MODEL2 -->|"prediction"| EVAL1
  MODEL3 -->|"prediction"| EVAL1

  MODEL1 -->|"details"| EVAL2
  MODEL2 -->|"details"| VIS1
  MODEL3 -->|"details"| EVAL3
  MODEL3 -->|"details"| VIS2

  %% Evaluation feedback to Frontend
  EVAL1 -->|"feedback"| UI1
  EVAL1 -->|"feedback"| UI2
  EVAL1 -->|"feedback"| UI3

  %% Styles
  classDef user fill:#ffe6e6,stroke:#cc0000,stroke-width:2px;
  classDef interface fill:#cce5ff,stroke:#004085,stroke-width:2px;
  classDef model fill:#d4edda,stroke:#155724,stroke-width:2px;
  classDef data fill:#fff3cd,stroke:#856404,stroke-width:2px;
  classDef eval fill:#e2e3e5,stroke:#818182,stroke-width:2px;
  classDef vis fill:#f8d7da,stroke:#721c24,stroke-width:2px;
  classDef ext fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;

  %% Click Events
  click UI1 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/app.py"
  click UI2 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/app_seq2seq.py"
  click UI3 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/app_rag.py"
  click MODEL1 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/baseline_seq2seq.py"
  click MODEL2 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/transformer_viz.py"
  click MODEL3 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/evaluate_rag.py"
  click UTIL "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/utils.py"
  click DSET1 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/Data/3.7K_python_dataset.csv"
  click DSET2 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/Data/mbpp(formatted).csv"
  click DSET3 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/Data/mbpp_conala.csv"
  click EVAL1 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/evaluate.py"
  click EVAL2 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/evaluate_seq2seq.py"
  click EVAL3 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/evaluate_rag.py"
  click VIS1 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/transformer_viz.py"
  click VIS2 "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/trasnformer_rag_viz.py"
  click EXT "https://github.com/sathvick-sudarsan/natural-language-to-code/blob/main/requirements.txt"
```
