# davide-belli/nlp-rnn-pytorch architecture

> Architecture diagram of the GitHub repository davide-belli/nlp-rnn-pytorch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/davide-belli/nlp-rnn-pytorch
- Repository: https://github.com/davide-belli/nlp-rnn-pytorch

## Overview

##### Main Components

a. Data Pipeline
- It to a "Data Processing" component (represented in data.py)
- Three data splits: training, validation, and test sets

b. Model Architecture
- Word Embedding Layer
- RNN Layers (support for LSTM/GRU/RNN variants)
- Optional Tied Weights component
- Dropout Layers
- Output Layer (Softmax)

c. Training System
- Training Loop component (from main.py)
- Optimizer component
- Learning Rate Scheduler
- Gradient Clipping module

d. Generation System
- Text Generation component (from generate.py)
- Model Loading mechanism
- Text Output component

##### Flow and Relationships

- Data flow: Penn Treebank → Data Processing → Training Loop
- Model flow: RNN Model ↔ Training Loop
- Generation flow: Trained Model → Text Generation → Output Text
- Weight sharing (if tied): Word Embedding ↔ Output Layer

##### Important Technical Details to Highlight

- Supported RNN types: RNN_TANH, RNN_RELU, LSTM, GRU
- Configurable hyperparameters:
- Embedding size
- Hidden layer size
- Number of layers
- Dropout rate
- Batch size
- Sequence length (BPTT)

##### Additional Elements

A small panel showing:
- CUDA support indication
- Model persistence path (generated_models/)
- Performance metrics (perplexity)

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Data_Layer
        PTB[("Penn Treebank Dataset")]
        Train["Training Data"]
        Valid["Validation Data"]
        Test["Test Data"]
        PTB --> Train
        PTB --> Valid
        PTB --> Test
    end

    subgraph Processing_Layer
        DP["Data Processing"]
        Train & Valid & Test --> DP
    end

    subgraph Model_Architecture
        direction TB
        RNN["RNN Model"]
        subgraph Components
            WE["Word Embedding Layer"]
            RNNLayers["RNN Layers<br/>(LSTM/GRU/RNN)"]
            DO["Dropout Layers"]
            OL["Output Layer<br/>(Softmax)"]
            WE --> RNNLayers
            RNNLayers --> DO
            DO --> OL
        end
        TW["Tied Weights"]
        WE <--> TW
        OL <--> TW
    end

    subgraph Training_System
        TL["Training Loop"]
        OPT["Optimizer"]
        LRS["Learning Rate<br/>Scheduler"]
        GC["Gradient Clipping"]
    end

    subgraph Generation_System
        GEN["Text Generation"]
        ML["Model Loading"]
        OUT["Text Output"]
        ML --> GEN
        GEN --> OUT
    end

    subgraph Technical_Specs
        CONF["Configuration
        - RNN Types: TANH/RELU/LSTM/GRU
        - Embedding Size
        - Hidden Layer Size
        - Number of Layers
        - Dropout Rate
        - Batch Size
        - Sequence Length"]
    end

    DP --> TL
    TL --> RNN
    RNN --> GEN
    CONF -.-> RNN

    classDef data fill:#a8d1f0,stroke:#0e5ba3
    classDef processing fill:#b8e6c1,stroke:#2d8745
    classDef model fill:#d4f0b0,stroke:#789e35
    classDef training fill:#ffd1a8,stroke:#c66a1d
    classDef generation fill:#e6b8e6,stroke:#8e478e
    classDef config fill:#e0e0e0,stroke:#666666

    class PTB,Train,Valid,Test data
    class DP processing
    class RNN,WE,RNNLayers,DO,OL,TW model
    class TL,OPT,LRS,GC training
    class GEN,ML,OUT generation
    class CONF config

    click PTB "https://github.com/davide-belli/nlp-RNN-pytorch/tree/master/data/penn/"
    click Train "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/data/penn/train.txt"
    click Valid "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/data/penn/valid.txt"
    click Test "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/data/penn/test.txt"
    click DP "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/data.py"
    click RNN "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/model.py"
    click TL "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/main.py"
    click GEN "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/generate.py"
    click ML "https://github.com/davide-belli/nlp-RNN-pytorch/tree/master/generated_models/"
    click OUT "https://github.com/davide-belli/nlp-RNN-pytorch/blob/master/generated_models/LSTM_epochs6.pt"
```
