# cbfinn/maml architecture

> Architecture diagram of the GitHub repository cbfinn/maml, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cbfinn/maml
- Repository: https://github.com/cbfinn/maml

## Overview

##### Project type and purpose (what you’re diagramming)
This repo is a **research/experiment codebase implementing Model-Agnostic Meta-Learning (MAML)** for **few-shot supervised learning** (sinusoid regression + Omniglot + MiniImagenet) using **TensorFlow v1.x**.

The “system” is not a distributed service; it’s a **training/evaluation pipeline** that:
- loads/constructs **meta-learning tasks** (episodes) from datasets,
- builds a TF1 computation graph for **inner-loop adaptation** + **outer-loop meta-update**,
- runs training loops, logging and checkpointing.

Use these as your primary components:

##### Entry point / Orchestration
- **`main.py`**: CLI/config, experiment setup, builds the graph, runs training/eval loops, triggers checkpointing/logging.

##### Core algorithm implementation
- **`maml.py`**: the MAML model and meta-learning logic:
- defines the base learner network (e.g., conv net / regressor),
- constructs inner-loop update(s) (fast adaptation steps),
- constructs outer-loop meta-objective and meta-optimizer.

##### Data / Task (Episode) generation
- **`data_generator.py`**: produces few-shot tasks/episodes (support set + query set) from a dataset; typically N-way K-shot sampling logic.

##### TF graph utilities / gradients
- **`special_grads.py`**: custom gradient handling/utilities to make MAML work (common in TF1 for higher-order gradients or gradient overrides).

- **`utils.py`**: helper functions (metrics, layers, normalization, I/O convenience, etc.).

##### Data preprocessing scripts (offline pipeline)
- **`data/omniglot_resized/resize_images.py`**
- **`data/miniImagenet/proc_images.py`**
These are offline preprocessing steps to prepare datasets.

##### Artifacts / outputs
- **`logs/.../model59000.*`**: TensorFlow checkpoints and run outputs.

##### What the README implies for architecture
- TensorFlow **v1.0+** implies a **static graph** build phase (define ops) and a **session run** phase (execute).

##### Training Step Dataflow (dynamic): one meta-iteration showing inner/outer loops

This yields maximum accuracy with minimal ambiguity.

- **Experiment Runner (`main.py`)**
- **MAML Implementation (`maml.py`)**
- Inner-loop adaptation (fast weights)
- Outer-loop meta-optimization
- **Episode Generator (`data_generator.py`)**
- **Gradient Extensions (`special_grads.py`)**
- **Dataset Storage (`data/...`)**
- **Offline Preprocessing (`proc_images.py`, `resize_images.py`)**
- **Artifacts (`logs/` checkpoints, metrics)**
- **TensorFlow v1 runtime** (as the execution substrate / dependency)

- Offline Preprocessing → Dataset Storage
- Dataset Storage → Episode Generator
- Episode Generator → Experiment Runner (yields episodes/tasks)
- Experiment Runner → MAML Implementation (feeds tasks, triggers graph build + session runs)
- MAML Implementation → TensorFlow v1 Graph/Session (build/execute)
- MAML Implementation ↔ Gradient Extensions (used during graph construction/backprop)
- Experiment Runner → Artifacts (writes checkpoints/logs)
- (Optional) Artifacts → Experiment Runner (resume from checkpoint)

- “N-way K-shot episode: support/query sets”
- “Build graph (ops)”
- “Session run: meta-train step / meta-test eval”
- “Checkpoint save/restore”

##### Sample meta-batch of tasks
`data_generator.py` produces **T tasks**; each task has:
- Support set (train for inner loop)
- Query set (test for outer loop gradient)

##### For each task: Inner-loop adaptation (K steps)
- Start from meta-parameters θ
- Compute loss on support set
- Apply gradient step(s): θ’ = θ − α∇θ L_support
(If doing higher-order MAML, gradients flow through this update; otherwise first-order approximations.)

##### Compute query loss using adapted parameters
L_query(θ’) on query set

##### Aggregate across tasks → meta-objective
Sum/mean query losses across tasks in the meta-batch

##### Outer-loop meta-update
θ ← θ − β∇θ Σ L_query
(This is where `special_grads.py` may be involved.)

##### Logging + checkpointing
Write summaries, save model to `logs/...`

##### Make the “inner vs outer loop” visually unmistakable
Use two nested containers:
- Outer container: “Meta-Iteration”
- Inner repeated block: “Per-Task Adaptation”

- N-way, K-shot
- inner steps
- α (inner lr), β (meta lr)
- meta-batch size (#tasks)

##### Important architectural patterns / principles to reflect
- **Separation of concerns**
- `main.py` orchestration vs `maml.py` algorithm vs `data_generator.py` sampling
- **Static graph build + session execution (TF1)**
- A “Graph Construction” phase and a “Session Run” phase (even if both happen in one process)
- **Bilevel optimization (core of MAML)**
- inner adaptation loop and outer meta-optimization loop
- **Pluggable datasets/domains**
- Omniglot/MiniImagenet/Sinusoid are sources into the same episode generator + MAML core

##### Technologies/frameworks to label
- **Python 2/3**
- **TensorFlow v1.x** (Graph, Session, Optimizer)
- Datasets: **Omniglot**, **MiniImagenet**, **Sinusoid regression**
- Artifact format: **TF checkpoints** (`.data`, `.index`) and logs

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================
%% Diagram 1: Static Architecture (Component View)
%% =========================
subgraph D1["Diagram 1 — Training/Evaluation Pipeline Architecture (TF1.x MAML)"]
direction TB

subgraph OFF["Offline Data Preparation (one-time / batch)"]
direction TB
ODP1["Omniglot Resize Script"]:::offline
ODP2["MiniImagenet Preprocess Script"]:::offline
end

subgraph DATA["Dataset Storage (preprocessed)"]
direction TB
DSRoot[("Dataset Root")]:::data
DSO[("Omniglot Resized")]:::data
DSM[("MiniImagenet")]:::data
DSS["Sinusoid Regression (synthetic)"]:::data
end

subgraph ONLINE["Online Training/Evaluation (per run)"]
direction TB

subgraph PY["Python Modules (repo code)"]
direction TB
MAIN["Experiment Runner / Trainer\n(main.py)"]:::module

subgraph DG["Episode Construction"]
direction TB
EP["Episode / Task Sampler\n(data_generator.py)"]:::module
EPOut["Episodes (meta-batch)\nT tasks × (Support, Query)\nN-way K-shot"]:::tensor
end

subgraph MAML["MAML Implementation / Graph Builder\n(maml.py)"]
direction TB
BM["Base Model / Feature Extractor\n(convnet or regressor)"]:::module
LC["Loss Computation\n(support/query)"]:::module
IL["Inner Loop Adaptation\nfast weights θ'"]:::module
MO["Meta-Objective Aggregation\nΣ/mean L_query"]:::module
OPT["Outer Loop Meta-Optimizer\napply ∇θ"]:::module
end

SG["Gradient Ops / Higher-Order Grad Support\n(special_grads.py)"]:::module
UT["Shared Utilities\n(utils.py)"]:::module
end

subgraph TF["TensorFlow v1.x Runtime (external dependency)"]
direction TB
TFGraph["Static Graph Construction\n(ops, variables, gradients)"]:::external
TFSession["Session Run\n(train/eval execute)"]:::external
end

subgraph ART["Artifacts / Outputs"]
direction TB
LOGS[("Checkpoints + TensorBoard Logs\n(logs/)")]:::artifact
RUN[("Run Directory\n(miniimagenet 1-shot example)")]:::artifact
CKPTD[("model59000.data-00000-of-00001")]:::artifact
CKPTI[("model59000.index")]:::artifact
end

end

%% Offline to Dataset
ODP1 -->|"resize&serialize"| DSO
ODP2 -->|"decode&reformat"| DSM
DSO -->|"stored_under"| DSRoot
DSM -->|"stored_under"| DSRoot
DSS -->|"generated_as"| DSRoot

%% Dataset to Episodes
DSRoot -->|"load_dataset"| EP
DSO -->|"omniglot_source"| EP
DSM -->|"miniimagenet_source"| EP
DSS -->|"sinusoid_source"| EP
EP -->|"sample_meta_batch"| EPOut

%% Orchestration / Graph
MAIN -->|"config&invoke"| EP
MAIN -->|"build_graph(ops)"| TFGraph
MAIN -->|"feed_episodes"| MAML
EPOut -->|"support/query tensors"| MAML

%% MAML internals and TF runtime
MAML -->|"define_forward&loss"| TFGraph
SG <-->|"gradient_overrides\n(higher-order)"| TFGraph
UT -->|"layers,metrics,io_helpers"| MAML
UT -->|"helpers"| EP

TFGraph -->|"execute"| TFSession
MAIN -->|"session.run\n(meta-train/meta-test)"| TFSession
TFSession -->|"returns:losses,grads,metrics"| MAIN

%% Artifacts
MAIN -->|"write_summaries&save_ckpt"| LOGS
LOGS -->|"contains"| RUN
RUN -->|"contains"| CKPTD
RUN -->|"contains"| CKPTI
LOGS -->|"restore_ckpt(optional)"| MAIN

end

%% =========================
%% Diagram 2: Dynamic Meta-Iteration (Dataflow View)
%% =========================
subgraph D2["Diagram 2 — One Meta-Iteration Dataflow (MAML bilevel optimization)"]
direction TB

MI["Meta-Iteration (one step)\nmeta-batch size = T tasks"]:::process

S1["1) Sample T tasks\nN-way K-shot"]:::step
S2["2) For each task: Inner-loop (K steps)\nθ' = θ − α∇θ L_support"]:::step
S3["3) Evaluate query loss\nL_query(θ')"]:::step
S4["4) Aggregate meta-objective\nL_meta = mean_t L_query"]:::step
S5["5) Outer-loop meta-update\nθ ← θ − β∇θ L_meta"]:::step
S6["6) Logging + checkpointing\nsummaries + save"]:::step

MI --> S1 --> S2 --> S3 --> S4 --> S5 --> S6

subgraph REPEAT["Per-Task Adaptation (repeated for each task in meta-batch)"]
direction TB
R0["Start from meta-params θ"]:::tensor
R1["Support set (train)\ncompute L_support"]:::tensor
R2["Inner update op(s)\n(grad through update if higher-order)"]:::process
R3["Adapted params θ' (fast weights)"]:::tensor
R4["Query set (test)\ncompute L_query"]:::tensor
R0 --> R1 --> R2 --> R3 --> R4
end

S2 -->|"per_task_loop"| REPEAT
REPEAT -->|"task_query_losses"| S4
S5 -->|"uses_special_grads_if_needed"| SG2["Gradient Ops / Higher-Order Support\n(special_grads.py)"]:::module
S6 -->|"save_to"| LOGS2[("logs/ (ckpt + summaries)")]:::artifact

S1 -->|"episodes_from"| EP2["Episode / Task Sampler\n(data_generator.py)"]:::module
EP2 -->|"reads"| DS2[("data/ (datasets)")]:::data
S5 -->|"apply_optimizer_in_TF1"| TF2["TensorFlow v1.x Graph+Session"]:::external
TF2 -->|"updates θ"| R0

end

%% =========================
%% Click events (from component_mapping)
%% =========================
click MAIN "https://github.com/cbfinn/maml/blob/master/main.py"
click EP "https://github.com/cbfinn/maml/blob/master/data_generator.py"
click EP2 "https://github.com/cbfinn/maml/blob/master/data_generator.py"
click SG "https://github.com/cbfinn/maml/blob/master/special_grads.py"
click SG2 "https://github.com/cbfinn/maml/blob/master/special_grads.py"
click UT "https://github.com/cbfinn/maml/blob/master/utils.py"
click ODP1 "https://github.com/cbfinn/maml/blob/master/data/omniglot_resized/resize_images.py"
click ODP2 "https://github.com/cbfinn/maml/blob/master/data/miniImagenet/proc_images.py"
click DSRoot "https://github.com/cbfinn/maml/tree/master/data/"
click DSM "https://github.com/cbfinn/maml/tree/master/data/miniImagenet/"
click DSO "https://github.com/cbfinn/maml/tree/master/data/omniglot_resized/"
click LOGS "https://github.com/cbfinn/maml/tree/master/logs/"
click RUN "https://github.com/cbfinn/maml/tree/master/logs/miniimagenet1shot/cls_5.mbs_4.ubs_1.numstep5.updatelr0.01hidden32maxpoolbatchnorm/"
click CKPTD "https://github.com/cbfinn/maml/blob/master/logs/miniimagenet1shot/cls_5.mbs_4.ubs_1.numstep5.updatelr0.01hidden32maxpoolbatchnorm/model59000.data-00000-of-00001"
click CKPTI "https://github.com/cbfinn/maml/blob/master/logs/miniimagenet1shot/cls_5.mbs_4.ubs_1.numstep5.updatelr0.01hidden32maxpoolbatchnorm/model59000.index"
click LOGS2 "https://github.com/cbfinn/maml/tree/master/logs/"

%% =========================
%% Styles
%% =========================
classDef module fill:#E3F2FD,stroke:#1565C0,stroke-width:1.5px,color:#0D47A1
classDef offline fill:#F3E5F5,stroke:#6A1B9A,stroke-width:1.5px,color:#4A148C
classDef data fill:#E8F5E9,stroke:#2E7D32,stroke-width:1.5px,color:#1B5E20
classDef artifact fill:#FFF3E0,stroke:#EF6C00,stroke-width:1.5px,color:#E65100
classDef external fill:#ECEFF1,stroke:#455A64,stroke-width:1.5px,color:#263238,stroke-dasharray: 6 4
classDef tensor fill:#F1F8E9,stroke:#558B2F,stroke-width:1.2px,color:#33691E
classDef process fill:#E0F7FA,stroke:#00838F,stroke-width:1.5px,color:#006064
classDef step fill:#FFFDE7,stroke:#F9A825,stroke-width:1.5px,color:#F57F17
```
