cytosine-code/astrai

How cytosine-code/astrai works

AstrAI covers model construction, data preparation, training, and serving/inference. A typical path is preprocessing source records into stored datasets, loading them for training, then serving a trained model through compatible chat APIs. Training-time configuration and parallel execution are shown as internal layers. The graph uses README capabilities for unsampled evaluation and advanced model features; arrows are limited to relationships visible in excerpts or the source index.

Main entry points: train_context.py Training context · transformer.py Transformer models · strategy.py Training strategies · engine.py Inference engine

Components 27

Data preparation

Preprocess config astrai/config/preprocess_config.py
Preprocessing pipeline astrai/preprocessing/pipeline.py
Tokenizer and templates astrai/tokenize/tokenizer.py
Streaming dataset astrai/dataset/dataset.py
Dataset storage astrai/dataset/storage.py

Models and kernels

Model config astrai/config/model_config.py
Transformer models astrai/model/transformer.py
Model components astrai/model/components/linear.py
LoRA adapters astrai/model/components/lora.py
Attention backends astrai/extension/backend/attention.py
Kernel loader astrai/extension/loader.py

Training

Training context astrai/trainer/train_context.py
Training strategies astrai/trainer/strategy.py
DDP/FSDP executor astrai/parallel/executor.py
RL rollouts astrai/trainer/rollout.py
Checkpoint serialization astrai/serialization/checkpoint.py

Inference and serving

Inference engine astrai/inference/engine.py
Continuous batching astrai/inference/scheduler.py
Paged KV cache astrai/inference/cache/pool.py
Decode runtime astrai/inference/runtime/executor.py
HTTP API astrai/inference/network/app.py
Protocol adapters astrai/inference/network/protocol.py
Evaluation tools scripts/eval

Shared infrastructure

Config serialization astrai/config/base.py
Component registry astrai/factory.py

Other components

Data/model user
API client

Connections 36

Data/model user → Preprocessing pipeline: configures
Preprocessing pipeline → Preprocess config: reads config astrai/preprocessing/pipeline.py
Preprocessing pipeline → Tokenizer and templates: builds with astrai/preprocessing/pipeline.py
Preprocessing pipeline → Dataset storage: writes shards astrai/preprocessing/pipeline.py
Streaming dataset → Preprocess config: uses config astrai/dataset/dataset.py
Streaming dataset → Dataset storage: loads data astrai/dataset/dataset.py
Streaming dataset → Tokenizer and templates: uses tokenizer astrai/dataset/dataset.py
Data/model user → Training context: configures
Training context → Model config: reads config astrai/trainer/train_context.py
Training context → Transformer models: creates model
Training context → DDP/FSDP executor: creates executor astrai/trainer/train_context.py
Training context → Training strategies: creates strategy astrai/trainer/train_context.py
Training context → RL rollouts: creates generator astrai/trainer/train_context.py
Training strategies → DDP/FSDP executor: runs distributed astrai/trainer/strategy.py
Training strategies → RL rollouts: uses rollouts astrai/trainer/strategy.py
Training strategies → LoRA adapters: uses adapters astrai/trainer/train_context.py
Training context → Checkpoint serialization: loads state
Transformer models → Model config: uses config astrai/model/transformer.py
Transformer models → Model components: uses components astrai/model/transformer.py
Transformer models → Paged KV cache: uses cache
Attention backends → Kernel loader: loads operations astrai/extension/backend/attention.py
Data/model user → HTTP API: sends requests
HTTP API → Inference engine: creates engine astrai/inference/network/app.py
HTTP API → Protocol adapters: dispatches requests astrai/inference/network/app.py
Protocol adapters → Inference engine: handles generation
Inference engine → Continuous batching: creates scheduler astrai/inference/engine.py
Continuous batching → Decode runtime: executes decode astrai/inference/scheduler.py
Continuous batching → Paged KV cache: manages pages astrai/inference/scheduler.py
Continuous batching → Transformer models: runs model
Inference engine → Tokenizer and templates: uses tokenizer
API client → HTTP API: queries
Evaluation tools → Inference engine: evaluates generation
Config serialization → Preprocess config: provides base astrai/config/preprocess_config.py
Model config → Config serialization: inherits astrai/config/model_config.py
Component registry → Streaming dataset: registers components astrai/dataset/dataset.py
Component registry → Training strategies: registers strategies astrai/trainer/strategy.py

More Python diagrams: ncar/melodies-monet · nebulock-inc/agentic-detection-engineering-framework · neo4j-graphacademy/genai-workshop-graphrag · niloy-chakraborty/smart-gardening · nitin27may/e-commerce-agents

Drawn by GitDiagram on October 10, 2026 from the repository's file tree, README and source files. Read as Markdown ·