# dennybritz/cnn-text-classification-tf architecture

> Architecture diagram of the GitHub repository dennybritz/cnn-text-classification-tf, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dennybritz/cnn-text-classification-tf
- Repository: https://github.com/dennybritz/cnn-text-classification-tf

## Overview

##### Project type and purpose
- This is a Python-based machine learning pipeline that implements a Convolutional Neural Network (CNN) for sentence classification, following Kim’s “CNN for Sentence Classification” paper.
- It’s a self-contained training and evaluation tool, not a web service or microservice architecture.

##### File structure
- Top level:
- data/rt-polaritydata/… : raw dataset files (positive/negative sentiment).
- data_helpers.py     : data loading, tokenization, batching.
- text_cnn.py         : CNN model definition (embedding, conv layers, pooling, output).
- train.py            : training orchestration (argument parsing, session management, checkpointing).
- eval.py             : evaluation orchestration (loads checkpoints, runs on dev/test data).
- README.md, LICENSE, .gitignore

##### README for additional insights
- Confirms Python 3, TensorFlow > 0.12, Numpy dependencies
- Training and evaluation commands show how components interact at runtime

##### Main components and their interactions
a. Data Ingestion & Preprocessing (data_helpers.py + raw files)
→ reads rt-polarity.pos/.neg
→ builds vocabulary, converts sentences to integer sequences
→ batches data for training/eval
b. Model Definition (text_cnn.py)
→ Embedding Layer (random or pre-trained)
→ Convolution + ReLU + Max-Pooling for each filter size
→ Concatenation → Dropout → Fully Connected → Softmax
→ L2 regularization option
c. Training Pipeline (train.py)
→ Parses hyperparameters
→ Builds TensorFlow graph by instantiating TextCNN
→ Defines loss, optimizer, summaries
→ Runs tf.Session loop: feed batches → optimize → evaluate every N steps → checkpoint
d. Evaluation Pipeline (eval.py)
→ Loads saved model checkpoint
→ Feeds evaluation batches → computes accuracy/loss

##### Architectural patterns & design principles
- Layered design:
- Data layer (I/O & preprocessing)
- Model layer (tensor graph)
- Orchestration layer (training/eval scripts)
- Single-process, GPU/CPU TensorFlow execution
- Separation of concerns: model code isolated from data and from orchestration

##### Essential details not to omit
- The bidirectional dependency: train.py imports text_cnn.py and data_helpers.py
- Checkpoint directory and TensorBoard summaries as outputs of the training component
- The separation between the graph-building phase and the session-run phase in TensorFlow

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        RawDataStore[(Raw Data Store)]:::data
        DataLoader["Data Loader & Preprocessor"]:::data
        VocabBatch["Vocabulary & Batch Generator"]:::data
    end

    %% Model Layer
    subgraph "Model Layer"
        TextCNN["TextCNN Model Definition"]:::model
        subgraph "Model Internals"
            Embedding["Embedding Layer"]:::model
            ConvPool["Convolution + Pooling"]:::model
            FullyConnected["Fully Connected + Softmax"]:::model
        end
    end

    %% Orchestration Layer
    subgraph "Orchestration Layer"
        TrainCtrl["Training Controller"]:::orch
        EvalCtrl["Evaluation Controller"]:::orch
    end

    %% External Framework
    TensorFlow["TensorFlow Runtime & Device"]:::framework

    %% Documentation
    README["README.md"]:::docs
    LICENSE["LICENSE"]:::docs
    GITIGNORE[".gitignore"]:::docs

    %% Data Flow
    RawDataStore -->|"raw files"| DataLoader
    DataLoader -->|"tokenized batches"| VocabBatch
    VocabBatch -->|"input tensors"| TextCNN
    TextCNN --> Embedding
    TextCNN --> ConvPool
    TextCNN --> FullyConnected

    %% Training Flow
    TrainCtrl -->|"builds graph"| TextCNN
    TrainCtrl -->|"imports"| DataLoader
    TrainCtrl -->|"runs sessions"| TensorFlow
    TextCNN -->|"loss, gradients"| TrainCtrl
    TrainCtrl -->|"checkpoints & summaries"| TensorFlow

    %% Evaluation Flow
    EvalCtrl -->|"loads checkpoints"| TensorFlow
    EvalCtrl -->|"imports"| TextCNN
    EvalCtrl -->|"imports"| DataLoader
    EvalCtrl -->|"evaluation batches"| VocabBatch
    EvalCtrl -->|"metrics output"| TensorFlow

    %% Doc Links
    TrainCtrl -.-> README
    EvalCtrl -.-> README
    README -.-> LICENSE
    README -.-> GITIGNORE

    %% Click Events
    click RawDataStore "https://github.com/dennybritz/cnn-text-classification-tf/tree/master/data/rt-polaritydata/"
    click DataLoader "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/data_helpers.py"
    click VocabBatch "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/data_helpers.py"
    click TextCNN "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/text_cnn.py"
    click TrainCtrl "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/train.py"
    click EvalCtrl "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/eval.py"
    click README "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/README.md"
    click LICENSE "https://github.com/dennybritz/cnn-text-classification-tf/tree/master/LICENSE"
    click GITIGNORE "https://github.com/dennybritz/cnn-text-classification-tf/blob/master/.gitignore"

    %% Styles
    classDef data fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef model fill:#d4edda,stroke:#333,stroke-width:1px
    classDef orch fill:#ffe5b4,stroke:#333,stroke-width:1px
    classDef framework fill:#e2e3e5,stroke:#333,stroke-width:1px
    classDef docs fill:#f0f0f0,stroke:#666,stroke-width:1px,stroke-dasharray: 2 2
```
