# amaldo123/ngc-pc-transformers architecture

> Architecture diagram of the GitHub repository amaldo123/ngc-pc-transformers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amaldo123/ngc-pc-transformers
- Repository: https://github.com/amaldo123/ngc-pc-transformers

## Overview

NGC-PC-Transformers is a predictive-coding language-model implementation. Its user workflow tokenizes text, prepares next-token batches, trains the model, then evaluates or generates text. The model combines embedding, attention, MLP, and output components with a projection subsystem used in predictive-coding updates; attention and embedding utilities provide internal computation. The README documents the tokenize/train/evaluate/generate commands. Some connections are based on the SOURCE INDEX imports rather than complete excerpts, so unsampled implementation details are not expanded.

Main entry points: [`model.py`](https://github.com/amaldo123/ngc-pc-transformers/blob/main/model.py) (Predictive transformer), [`layers/blocks.py`](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/blocks.py) (Transformer block), [`projection/proj_block.py`](https://github.com/amaldo123/ngc-pc-transformers/blob/main/projection/proj_block.py) (Projection block), [`train.py`](https://github.com/amaldo123/ngc-pc-transformers/blob/main/train.py) (Training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflow["Language workflow"]
  node_tokenizer["Tokenizer<br/>[tokenizer.py]"]
  node_token_batches["Token batches<br/>[data_loader.py]"]
  node_trainer["Training<br/>[train.py]"]
  node_evaluator["Evaluation<br/>[eval.py]"]
  node_generator["Text generation<br/>[generation.py]"]
  node_config["Model configuration<br/>[config.py]"]
end

subgraph group_model["Predictive model"]
  node_model_core["Predictive transformer<br/>[model.py]"]
  node_embedding["Token embedding<br/>[embedding.py]"]
  node_transformer_block["Transformer block<br/>[blocks.py]"]
  node_attention["Self-attention<br/>[attention.py]"]
  node_mlp["Feed-forward MLP<br/>[mlp.py]"]
  node_output["Vocabulary output<br/>[output.py]"]
end

subgraph group_projection["Projection and learning"]
  node_projection["Predictive projection<br/>[projection.py]"]
  node_projection_block["Projection block<br/>[proj_block.py]"]
end

subgraph group_internals["Model internals"]
  node_embedding_synapse["Embedding synapse<br/>[embed_utils.py]"]
  node_attention_math["Attention computation<br/>[attention_utils.py]"]
  node_attention_dynamics["Attention rate cell<br/>[attn_ratecell.py]"]
  node_model_util["Model utilities<br/>[model_util.py]"]
  node_residual["Residual operations<br/>[residual_util.py]"]
  node_normalization["RMS normalization<br/>[rms_norm_util.py]"]
  node_rope["RoPE utilities<br/>[rope_utils.py]"]
end

node_user(("User"))
node_text["Text corpus"]

node_user -->|"runs tokenization"| node_tokenizer
node_text -.->|"tokenizes"| node_tokenizer
node_tokenizer -.->|"produces token data"| node_token_batches
node_user -->|"starts training"| node_trainer
node_token_batches -->|"supplies batches"| node_trainer
node_trainer -->|"trains"| node_model_core
node_trainer -->|"invokes evaluation"| node_evaluator
node_user -->|"runs evaluation"| node_evaluator
node_token_batches -->|"supplies batches"| node_evaluator
node_evaluator -->|"evaluates"| node_model_core
node_user -->|"runs generation"| node_generator
node_generator -->|"generates with"| node_model_core
node_generator -->|"uses tokenizer"| node_tokenizer
node_config -->|"configures"| node_trainer
node_config -->|"configures"| node_model_core
node_config -->|"configures"| node_tokenizer
node_model_core -->|"uses"| node_embedding
node_model_core -->|"uses"| node_transformer_block
node_model_core -->|"uses"| node_output
node_model_core -->|"uses"| node_projection
node_transformer_block -->|"constructs"| node_attention
node_transformer_block -->|"constructs"| node_mlp
node_transformer_block -->|"uses"| node_residual
node_transformer_block -->|"imports"| node_normalization
node_embedding -->|"uses"| node_embedding_synapse
node_attention -->|"uses"| node_attention_math
node_attention -->|"uses"| node_attention_dynamics
node_attention_math -->|"uses softmax utility"| node_model_util
node_attention_math -.->|"uses positional encoding"| node_rope
node_projection -->|"constructs"| node_projection_block
node_projection_block -->|"uses attention"| node_attention_math
node_projection_block -->|"uses dynamics"| node_attention_dynamics
node_projection_block -->|"uses"| node_mlp
node_projection_block -->|"uses reshape utility"| node_model_util

click node_text "https://github.com/amaldo123/ngc-pc-transformers/tree/main/data_preprocess/data"
click node_tokenizer "https://github.com/amaldo123/ngc-pc-transformers/blob/main/data_preprocess/tokenizer.py"
click node_token_batches "https://github.com/amaldo123/ngc-pc-transformers/blob/main/data_preprocess/data_loader.py"
click node_trainer "https://github.com/amaldo123/ngc-pc-transformers/blob/main/train.py"
click node_evaluator "https://github.com/amaldo123/ngc-pc-transformers/blob/main/eval.py"
click node_generator "https://github.com/amaldo123/ngc-pc-transformers/blob/main/generation.py"
click node_config "https://github.com/amaldo123/ngc-pc-transformers/blob/main/config.py"
click node_model_core "https://github.com/amaldo123/ngc-pc-transformers/blob/main/model.py"
click node_embedding "https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/embedding.py"
click node_transformer_block "https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/blocks.py"
click node_attention "https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/attention.py"
click node_mlp "https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/mlp.py"
click node_output "https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/output.py"
click node_projection "https://github.com/amaldo123/ngc-pc-transformers/blob/main/projection/projection.py"
click node_projection_block "https://github.com/amaldo123/ngc-pc-transformers/blob/main/projection/proj_block.py"
click node_embedding_synapse "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/embed_utils.py"
click node_attention_math "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/attention_utils.py"
click node_attention_dynamics "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/attn_ratecell.py"
click node_model_util "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/model_util.py"
click node_residual "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/residual_util.py"
click node_normalization "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/rms_norm_util.py"
click node_rope "https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/rope_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tokenizer,node_token_batches,node_trainer,node_evaluator,node_generator,node_config,node_user toneBlue
class node_model_core,node_embedding,node_transformer_block,node_attention,node_mlp,node_output toneAmber
class node_projection,node_projection_block toneMint
class node_embedding_synapse,node_attention_math,node_attention_dynamics,node_model_util,node_residual,node_normalization,node_rope toneRose
class node_text toneTeal
```

## Components

### Language workflow

- [Tokenizer](https://github.com/amaldo123/ngc-pc-transformers/blob/main/data_preprocess/tokenizer.py): `data_preprocess/tokenizer.py`
- [Token batches](https://github.com/amaldo123/ngc-pc-transformers/blob/main/data_preprocess/data_loader.py): `data_preprocess/data_loader.py`
- [Training](https://github.com/amaldo123/ngc-pc-transformers/blob/main/train.py): `train.py`
- [Evaluation](https://github.com/amaldo123/ngc-pc-transformers/blob/main/eval.py): `eval.py`
- [Text generation](https://github.com/amaldo123/ngc-pc-transformers/blob/main/generation.py): `generation.py`
- [Model configuration](https://github.com/amaldo123/ngc-pc-transformers/blob/main/config.py): `config.py`

### Predictive model

- [Predictive transformer](https://github.com/amaldo123/ngc-pc-transformers/blob/main/model.py): `model.py`
- [Token embedding](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/embedding.py): `layers/embedding.py`
- [Transformer block](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/blocks.py): `layers/blocks.py`
- [Self-attention](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/attention.py): `layers/attention.py`
- [Feed-forward MLP](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/mlp.py): `layers/mlp.py`
- [Vocabulary output](https://github.com/amaldo123/ngc-pc-transformers/blob/main/layers/output.py): `layers/output.py`

### Projection and learning

- [Predictive projection](https://github.com/amaldo123/ngc-pc-transformers/blob/main/projection/projection.py): `projection/projection.py`
- [Projection block](https://github.com/amaldo123/ngc-pc-transformers/blob/main/projection/proj_block.py): `projection/proj_block.py`

### Model internals

- [Embedding synapse](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/embed_utils.py): `utils/embed_utils.py`
- [Attention computation](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/attention_utils.py): `utils/attention_utils.py`
- [Attention rate cell](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/attn_ratecell.py): `utils/attn_ratecell.py`
- [Model utilities](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/model_util.py): `utils/model_util.py`
- [Residual operations](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/residual_util.py): `utils/residual_util.py`
- [RMS normalization](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/rms_norm_util.py): `utils/rms_norm_util.py`
- [RoPE utilities](https://github.com/amaldo123/ngc-pc-transformers/blob/main/utils/rope_utils.py): `utils/rope_utils.py`

### Other components

- **User**
- [Text corpus](https://github.com/amaldo123/ngc-pc-transformers/tree/main/data_preprocess/data): `data_preprocess/data`

## Connections

- User → Tokenizer: runs tokenization (evidence: `README.md`)
- Text corpus → Tokenizer: tokenizes
- Tokenizer → Token batches: produces token data
- User → Training: starts training (evidence: `README.md`)
- Token batches → Training: supplies batches (evidence: `train.py`)
- Training → Predictive transformer: trains (evidence: `train.py`)
- Training → Evaluation: invokes evaluation (evidence: `train.py`)
- User → Evaluation: runs evaluation (evidence: `README.md`)
- Token batches → Evaluation: supplies batches (evidence: `eval.py`)
- Evaluation → Predictive transformer: evaluates (evidence: `eval.py`)
- User → Text generation: runs generation (evidence: `README.md`)
- Text generation → Predictive transformer: generates with (evidence: `generation.py`)
- Text generation → Tokenizer: uses tokenizer (evidence: `generation.py`)
- Model configuration → Training: configures (evidence: `train.py`)
- Model configuration → Predictive transformer: configures (evidence: `model.py`)
- Model configuration → Tokenizer: configures (evidence: `data_preprocess/tokenizer.py`)
- Predictive transformer → Token embedding: uses (evidence: `model.py`)
- Predictive transformer → Transformer block: uses (evidence: `model.py`)
- Predictive transformer → Vocabulary output: uses (evidence: `model.py`)
- Predictive transformer → Predictive projection: uses (evidence: `model.py`)
- Transformer block → Self-attention: constructs (evidence: `layers/blocks.py`)
- Transformer block → Feed-forward MLP: constructs (evidence: `layers/blocks.py`)
- Transformer block → Residual operations: uses (evidence: `layers/blocks.py`)
- Transformer block → RMS normalization: imports (evidence: `layers/blocks.py`)
- Token embedding → Embedding synapse: uses (evidence: `layers/embedding.py`)
- Self-attention → Attention computation: uses (evidence: `layers/attention.py`)
- Self-attention → Attention rate cell: uses (evidence: `layers/attention.py`)
- Attention computation → Model utilities: uses softmax utility (evidence: `utils/attention_utils.py`)
- Attention computation → RoPE utilities: uses positional encoding (evidence: `utils/attention_utils.py`)
- Predictive projection → Projection block: constructs (evidence: `projection/projection.py`)
- Projection block → Attention computation: uses attention (evidence: `projection/proj_block.py`)
- Projection block → Attention rate cell: uses dynamics (evidence: `projection/proj_block.py`)
- Projection block → Feed-forward MLP: uses (evidence: `projection/proj_block.py`)
- Projection block → Model utilities: uses reshape utility (evidence: `projection/proj_block.py`)
