# meetmendapara09/attenta architecture

> Architecture diagram of the GitHub repository meetmendapara09/attenta, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/meetmendapara09/attenta
- Repository: https://github.com/meetmendapara09/attenta

## Overview

#### Purpose and entry points
Attenta is a standalone Rust crate implementing and training a Transformer language model from scratch on Indian legal-act JSON corpora. The executable entry point is `transformer/src/main.rs`; reusable crate exports and module wiring live in `transformer/src/lib.rs`. A dedicated legal-training example is `transformer/examples/train_legal.rs`.

#### Corpus and input pipeline
Legal source material is stored as JSON files under `data/`, including `data/The Code on Wages 2019.json` and `data/The Insolvency And Bankruptcy Code 2016.json`. Data ingestion and dataset preparation are centered in `transformer/src/data.rs`, while BPE-style token vocabulary/encoding is in `transformer/src/tokenizer.rs`. `transformer/src/batch.rs` groups tokenized examples into length-aware batches, forming the boundary between corpus processing and numerical training.

#### Transformer model core
`transformer/src/model.rs` composes the encoder/decoder Transformer and model-level forward behavior. Encoder stacks are implemented in `transformer/src/encoder.rs`; autoregressive decoder stacks and target-side masking behavior are in `transformer/src/decoder.rs`. These modules represent the primary sequence-to-sequence execution path over token IDs and embeddings.

#### Attention and tensor execution
Multi-head scaled dot-product attention is implemented in `transformer/src/attention.rs`. Core tensor/matrix operations, including performance-oriented matrix multiplication, reside in `transformer/src/tensor_ops.rs`. This is the main numerical boundary: higher-level model modules delegate array math and attention score transformations to these lower-level routines.

#### Training, gradients, and optimization
`transformer/src/train.rs` orchestrates the training loop: batches flow through model forward passes, loss computation, backpropagation, and parameter updates. Manual gradient propagation is implemented in `transformer/src/backward.rs`. Objective calculation, including label-smoothed cross entropy, is in `transformer/src/loss.rs`; Adam and learning-rate warmup scheduling are in `transformer/src/optim.rs`.

#### Decoding and model artifacts
Inference generation supports greedy and beam-search decoding through `transformer/src/decoder.rs` and associated model interfaces in `transformer/src/model.rs`. Training state and model parameter persistence are handled by `transformer/src/checkpoint.rs`, with committed artifacts at `transformer/checkpoint.json` and `transformer/checkpoint_avg.json`. Attention weights can be emitted or inspected through `transformer/src/visualize.rs` and `transformer/attention_weights.json`.

#### Runtime and boundaries
The system runs as a native Rust 1.70+ process via Cargo, with no PyTorch, TensorFlow, hosted service, database, or web runtime indicated. Execution may require an increased Rust stack (`RUST_MIN_STACK`) for the release binary. The only material external input is the local legal-document corpus; checkpoints and visualizations are local JSON artifacts.

Main entry points: [`transformer/src/model.rs`](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/model.rs) (Transformer model), [`transformer/src/train.rs`](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/train.rs) (Training loop), [`transformer/src/decoder.rs`](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/decoder.rs) (Decoder and generation), [`transformer/src/attention.rs`](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/attention.rs) (Multi-head attention)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Entry points"]
  node_main["CLI executable<br/>Rust binary<br/>[main.rs]"]
  node_library["Crate exports<br/>Rust library<br/>[lib.rs]"]
  node_legal_example["Legal training example<br/>Rust example<br/>[train_legal.rs]"]
end

subgraph group_data["Corpus pipeline"]
  node_corpus["Legal-act JSON corpus<br/>local JSON files"]
  node_data_prep["Dataset preparation<br/>data module<br/>[data.rs]"]
  node_tokenizer["BPE tokenizer<br/>tokenization module<br/>[tokenizer.rs]"]
  node_batching["Length-aware batching<br/>batch module<br/>[batch.rs]"]
end

subgraph group_model["Transformer core"]
  node_model_api["Transformer model<br/>model module<br/>[model.rs]"]
  node_encoder["Encoder stacks<br/>encoder module<br/>[encoder.rs]"]
  node_decoder["Decoder and generation<br/>decoder module<br/>[decoder.rs]"]
  node_attention["Multi-head attention<br/>attention module<br/>[attention.rs]"]
  node_tensor_ops["Tensor operations<br/>numerical kernel module<br/>[tensor_ops.rs]"]
end

subgraph group_training["Training runtime"]
  node_training_loop["Training loop<br/>training module<br/>[train.rs]"]
  node_loss["Label-smoothed loss<br/>objective module<br/>[loss.rs]"]
  node_backward["Manual backpropagation<br/>gradient module<br/>[backward.rs]"]
  node_optimizer["Adam with warmup<br/>optimization module<br/>[optim.rs]"]
end

subgraph group_artifacts["Local artifacts"]
  node_checkpointing["Checkpoint persistence<br/>artifact module<br/>[checkpoint.rs]"]
  node_checkpoints["Model checkpoints<br/>local JSON artifacts<br/>[checkpoint.json]"]
  node_attention_output["Attention weights<br/>local JSON artifact"]
  node_visualization["Attention visualization<br/>visualization module<br/>[visualize.rs]"]
end

node_main -->|"uses"| node_library
node_legal_example -->|"runs"| node_training_loop
node_corpus -->|"ingests"| node_data_prep
node_data_prep -->|"prepares text for"| node_tokenizer
node_tokenizer -->|"token IDs"| node_batching
node_batching -->|"batches"| node_training_loop
node_training_loop -->|"forward pass"| node_model_api
node_model_api -->|"encodes source"| node_encoder
node_model_api -->|"decodes targets"| node_decoder
node_encoder -->|"uses"| node_attention
node_decoder -->|"uses masked attention"| node_attention
node_attention -->|"score and matrix math"| node_tensor_ops
node_training_loop -->|"computes objective"| node_loss
node_loss -->|"loss gradients"| node_backward
node_backward -->|"parameter gradients"| node_optimizer
node_optimizer -->|"updates parameters"| node_model_api
node_training_loop -->|"saves/restores state"| node_checkpointing
node_checkpointing -->|"writes"| node_checkpoints
node_model_api -.->|"attention data"| node_visualization
node_visualization -->|"emits"| node_attention_output
node_decoder -.->|"generation interface"| node_model_api

click node_main "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/main.rs"
click node_library "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/lib.rs"
click node_legal_example "https://github.com/meetmendapara09/attenta/blob/main/transformer/examples/train_legal.rs"
click node_data_prep "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/data.rs"
click node_tokenizer "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/tokenizer.rs"
click node_batching "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/batch.rs"
click node_model_api "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/model.rs"
click node_encoder "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/encoder.rs"
click node_decoder "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/decoder.rs"
click node_attention "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/attention.rs"
click node_tensor_ops "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/tensor_ops.rs"
click node_training_loop "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/train.rs"
click node_loss "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/loss.rs"
click node_backward "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/backward.rs"
click node_optimizer "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/optim.rs"
click node_checkpointing "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/checkpoint.rs"
click node_checkpoints "https://github.com/meetmendapara09/attenta/blob/main/transformer/checkpoint.json"
click node_attention_output "https://github.com/meetmendapara09/attenta/blob/main/transformer/attention_weights.json"
click node_visualization "https://github.com/meetmendapara09/attenta/blob/main/transformer/src/visualize.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_library,node_legal_example toneBlue
class node_corpus,node_data_prep,node_tokenizer,node_batching toneAmber
class node_model_api,node_encoder,node_decoder,node_attention,node_tensor_ops toneMint
class node_training_loop,node_loss,node_backward,node_optimizer toneRose
class node_checkpointing,node_checkpoints,node_attention_output,node_visualization toneIndigo
```

## Components

### Entry points

- [CLI executable](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/main.rs): Rust binary · Native Cargo entry point. · `transformer/src/main.rs`
- [Crate exports](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/lib.rs): Rust library · Module wiring and reusable interfaces. · `transformer/src/lib.rs`
- [Legal training example](https://github.com/meetmendapara09/attenta/blob/main/transformer/examples/train_legal.rs): Rust example · `transformer/examples/train_legal.rs`

### Corpus pipeline

- **Legal-act JSON corpus**: local JSON files · Indian legal documents under the data directory.
- [Dataset preparation](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/data.rs): data module · `transformer/src/data.rs`
- [BPE tokenizer](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/tokenizer.rs): tokenization module · `transformer/src/tokenizer.rs`
- [Length-aware batching](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/batch.rs): batch module · Boundary from tokenized examples to numerical training batches. · `transformer/src/batch.rs`

### Transformer core

- [Transformer model](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/model.rs): model module · Composes encoder, decoder, and model forward behavior. · `transformer/src/model.rs`
- [Encoder stacks](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/encoder.rs): encoder module · `transformer/src/encoder.rs`
- [Decoder and generation](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/decoder.rs): decoder module · Autoregressive stacks, masking, greedy and beam decoding. · `transformer/src/decoder.rs`
- [Multi-head attention](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/attention.rs): attention module · Scaled dot-product attention. · `transformer/src/attention.rs`
- [Tensor operations](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/tensor_ops.rs): numerical kernel module · Matrix operations and performance-oriented multiplication. · `transformer/src/tensor_ops.rs`

### Training runtime

- [Training loop](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/train.rs): training module · Coordinates forward pass, loss, backpropagation, and updates. · `transformer/src/train.rs`
- [Label-smoothed loss](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/loss.rs): objective module · `transformer/src/loss.rs`
- [Manual backpropagation](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/backward.rs): gradient module · `transformer/src/backward.rs`
- [Adam with warmup](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/optim.rs): optimization module · `transformer/src/optim.rs`

### Local artifacts

- [Checkpoint persistence](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/checkpoint.rs): artifact module · Saves and restores training state and parameters. · `transformer/src/checkpoint.rs`
- [Model checkpoints](https://github.com/meetmendapara09/attenta/blob/main/transformer/checkpoint.json): local JSON artifacts · `transformer/checkpoint.json`
- [Attention weights](https://github.com/meetmendapara09/attenta/blob/main/transformer/attention_weights.json): local JSON artifact · Inspectable emitted attention weights. · `transformer/attention_weights.json`
- [Attention visualization](https://github.com/meetmendapara09/attenta/blob/main/transformer/src/visualize.rs): visualization module · `transformer/src/visualize.rs`

## Connections

- CLI executable → Crate exports: uses
- Legal training example → Training loop: runs
- Legal-act JSON corpus → Dataset preparation: ingests
- Dataset preparation → BPE tokenizer: prepares text for
- BPE tokenizer → Length-aware batching: token IDs
- Length-aware batching → Training loop: batches
- Training loop → Transformer model: forward pass
- Transformer model → Encoder stacks: encodes source
- Transformer model → Decoder and generation: decodes targets
- Encoder stacks → Multi-head attention: uses
- Decoder and generation → Multi-head attention: uses masked attention
- Multi-head attention → Tensor operations: score and matrix math
- Training loop → Label-smoothed loss: computes objective
- Label-smoothed loss → Manual backpropagation: loss gradients
- Manual backpropagation → Adam with warmup: parameter gradients
- Adam with warmup → Transformer model: updates parameters
- Training loop → Checkpoint persistence: saves/restores state
- Checkpoint persistence → Model checkpoints: writes
- Transformer model → Attention visualization: attention data
- Attention visualization → Attention weights: emits
- Decoder and generation → Transformer model: generation interface