# julian-reed/cs224n_final architecture

> Architecture diagram of the GitHub repository julian-reed/cs224n_final, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/julian-reed/cs224n_final
- Repository: https://github.com/julian-reed/cs224n_final

## Overview

##### Main Components
- Data Management:
- Data Directory (data/): Contains training, development, and test datasets including sonnets and CSV files.
- datasets.py: Likely responsible for data loading, preprocessing, and possibly splitting data for training and evaluation.

- Model Definition & Architecture:
- models/ directory: Contains base_gpt.py and gpt2.py for setting up the GPT-2 model.
- modules/ directory: Contains components like attention.py and gpt2_layer.py that implement key architectural elements (e.g., attention mechanism and transformer layers) of GPT-2.

- Configuration and Utility:
- config.py: Holds configuration parameters that likely control aspects like model hyperparameters or training-setup.
- utils.py: Provides helper functions used throughout the codebase.

- Optimization Techniques:
- optimizer.py and smart_optimizer.py: Implement standard and “smart” optimizer variants.
- SMART, LoRA, and SOAP adjustments are integrated through these optimization files, helping to regulate the training with different regularization techniques.

- Task-Specific Modules:
- classifier.py and smart_classifier.py: Code for classification tasks built on top of the model.
- paraphrase_detection.py and smart_paraphrase_detection.py: Modules devoted to detecting paraphrases.
- sonnet_generation.py, smart_sonnet_generation.py: Responsible for generating sonnets and related output.

- Evaluation & Prediction:
- evaluation.py: Script to assess model performance.
- predictions/ directory: Contains output files from different experiments including generated sonnets and classification/prediction results.
- prepare_submit.py: Likely packages the results for submission.

- Environment & Setup:
- env.yml, setup.sh: Define the project environment and setup instructions.
- README.md and Final Report.pdf: Provide project context, documentation, and high-level overviews that are useful for understanding overall design.

##### Relationships and Interactions
- Data flows from the “data” directory into the data management module (datasets.py) to prepare inputs for training.
- The model components (models/ and modules/) use configurations from config.py and utilities from utils.py to initialize and compute the GPT-2 architecture.
- Optimization modules (optimizer.py, smart_optimizer.py, etc.) inject techniques (LoRA, SMART, SOAP) into the training process managed by the main model scripts.
- Task-specific scripts (such as classifier.py, paraphrase_detection.py, and sonnet_generation.py) build upon the optimized model to perform tasks like classification, paraphrase detection, or sonnet generation.
- Outputs flow to the predictions/ directory and are further used by evaluation.py to assess the system's overall performance.

##### Architectural Patterns
- Modularity: The separation between models, modules, optimization, and task-specific code indicates a modular design. Each component (data processing, model building, optimization, evaluation) operates independently but communicates through defined interfaces.
- Separation of Concerns: There is a clear division between the core model architecture (in models/ and modules/), the optimization techniques, and the task-specific implementations.
- Reusability: Utility scripts and configuration files are shared across different parts of the system ensuring consistency.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        DS["Data Storage"]:::data
        DP["Datasets Module"]:::data
    end

    subgraph "Model Definition"
        subgraph "GPT-2 Core"
            BG["Base GPT"]:::model
            GT["GPT-2 Model"]:::model
        end
        subgraph "Transformer Modules"
            AT["Attention Module"]:::model
            GL["GPT-2 Layer"]:::model
        end
        CF["Config"]:::model
        UTL["Utils"]:::model
    end

    subgraph "Optimization Techniques"
        SO["Standard Optimizer"]:::opt
        SM["Smart Optimizer"]:::opt
    end

    subgraph "Task-Specific Modules"
        CL1["Classifier"]:::task
        CL2["Smart Classifier"]:::task
        PD1["Paraphrase Detection"]:::task
        PD2["Smart Paraphrase Detection"]:::task
        SG1["Sonnet Generation"]:::task
        SG2["Smart Sonnet Generation"]:::task
    end

    subgraph "Evaluation & Prediction"
        EV["Evaluation"]:::eval
        PO["Predictions Output"]:::eval
        PS["Submission Prep"]:::eval
    end

    subgraph "Environment & Setup"
        EN["Environment Definition"]:::env
        SS["Setup Script"]:::env
        RD["README"]:::env
        FR["Final Report"]:::env
    end

    DS -->|"contains"| DP
    DP -->|"preprocesses"| BG
    CF -->|"configures"| BG
    UTL -->|"supports"| BG
    AT -->|"supports"| BG
    GL -->|"supports"| BG
    BG -->|"builds"| GT
    SO -->|"optimizes(LORA,SMART,SOAP)"| BG
    SM -->|"optimizes(LORA,SMART,SOAP)"| BG
    GT -->|"feeds"| CL1
    GT -->|"feeds"| CL2
    GT -->|"feeds"| PD1
    GT -->|"feeds"| PD2
    GT -->|"feeds"| SG1
    GT -->|"feeds"| SG2
    CL1 -->|"outputs"| EV
    CL2 -->|"outputs"| EV
    PD1 -->|"outputs"| EV
    PD2 -->|"outputs"| EV
    SG1 -->|"outputs"| EV
    SG2 -->|"outputs"| EV
    EV -->|"generates"| PO
    EV -->|"packages"| PS

    click DS "https://github.com/julian-reed/cs224n_final/tree/main/data/"
    click DP "https://github.com/julian-reed/cs224n_final/blob/main/datasets.py"
    click BG "https://github.com/julian-reed/cs224n_final/blob/main/models/base_gpt.py"
    click GT "https://github.com/julian-reed/cs224n_final/blob/main/models/gpt2.py"
    click AT "https://github.com/julian-reed/cs224n_final/blob/main/modules/attention.py"
    click GL "https://github.com/julian-reed/cs224n_final/blob/main/modules/gpt2_layer.py"
    click CF "https://github.com/julian-reed/cs224n_final/blob/main/config.py"
    click UTL "https://github.com/julian-reed/cs224n_final/blob/main/utils.py"
    click SO "https://github.com/julian-reed/cs224n_final/blob/main/optimizer.py"
    click SM "https://github.com/julian-reed/cs224n_final/blob/main/smart_optimizer.py"
    click CL1 "https://github.com/julian-reed/cs224n_final/blob/main/classifier.py"
    click CL2 "https://github.com/julian-reed/cs224n_final/blob/main/smart_classifier.py"
    click PD1 "https://github.com/julian-reed/cs224n_final/blob/main/paraphrase_detection.py"
    click PD2 "https://github.com/julian-reed/cs224n_final/blob/main/smart_paraphrase_detection.py"
    click SG1 "https://github.com/julian-reed/cs224n_final/blob/main/sonnet_generation.py"
    click SG2 "https://github.com/julian-reed/cs224n_final/blob/main/smart_sonnet_generation.py"
    click EV "https://github.com/julian-reed/cs224n_final/blob/main/evaluation.py"
    click PO "https://github.com/julian-reed/cs224n_final/tree/main/predictions/"
    click PS "https://github.com/julian-reed/cs224n_final/blob/main/prepare_submit.py"
    click EN "https://github.com/julian-reed/cs224n_final/blob/main/env.yml"
    click SS "https://github.com/julian-reed/cs224n_final/blob/main/setup.sh"
    click RD "https://github.com/julian-reed/cs224n_final/blob/main/README.md"
    click FR "https://github.com/julian-reed/cs224n_final/blob/main/Final Report.pdf"

    classDef data fill:#FFD700,stroke:#DAA520,stroke-width:2px;
    classDef model fill:#ADD8E6,stroke:#00008B,stroke-width:2px;
    classDef opt fill:#90EE90,stroke:#006400,stroke-width:2px;
    classDef task fill:#FFB6C1,stroke:#C71585,stroke-width:2px;
    classDef eval fill:#FFA07A,stroke:#FF4500,stroke-width:2px;
    classDef env fill:#D3D3D3,stroke:#696969,stroke-width:2px;
```
