How cbfinn/maml works
Project type and purpose (what you’re diagramming)
This repo is a research/experiment codebase implementing Model-Agnostic Meta-Learning (MAML) for few-shot supervised learning (sinusoid regression + Omniglot + MiniImagenet) using TensorFlow v1.x.
The “system” is not a distributed service; it’s a training/evaluation pipeline that:
• loads/constructs meta-learning tasks (episodes) from datasets,
• builds a TF1 computation graph for inner-loop adaptation + outer-loop meta-update,
• runs training loops, logging and checkpointing.
Use these as your primary components:
Entry point / Orchestration
• main.py: CLI/config, experiment setup, builds the graph, runs training/eval loops, triggers checkpointing/logging.
Core algorithm implementation
• maml.py: the MAML model and meta-learning logic:
• defines the base learner network (e.g., conv net / regressor),
• constructs inner-loop update(s) (fast adaptation steps),
• constructs outer-loop meta-objective and meta-optimizer.
Data / Task (Episode) generation
• data_generator.py: produces few-shot tasks/episodes (support set + query set) from a dataset; typically N-way K-shot sampling logic.
TF graph utilities / gradients
• special_grads.py: custom gradient handling/utilities to make MAML work (common in TF1 for higher-order gradients or gradient overrides).
• utils.py: helper functions (metrics, layers, normalization, I/O convenience, etc.).
Data preprocessing scripts (offline pipeline)
• data/omniglot_resized/resize_images.py
• data/miniImagenet/proc_images.py
These are offline preprocessing steps to prepare datasets.
Artifacts / outputs
• **logs/.../model59000.***: TensorFlow checkpoints and run outputs.
What the README implies for architecture
• TensorFlow v1.0+ implies a static graph build phase (define ops) and a session run phase (execute).
Training Step Dataflow (dynamic): one meta-iteration showing inner/outer loops
This yields maximum accuracy with minimal ambiguity.
• Experiment Runner (main.py)
• MAML Implementation (maml.py)
• Inner-loop adaptation (fast weights)
• Outer-loop meta-optimization
• Episode Generator (data_generator.py)
• Gradient Extensions (special_grads.py)
• Dataset Storage (data/...)
• Offline Preprocessing (proc_images.py, resize_images.py)
• Artifacts (logs/ checkpoints, metrics)
• TensorFlow v1 runtime (as the execution substrate / dependency)
• Offline Preprocessing → Dataset Storage
• Dataset Storage → Episode Generator
• Episode Generator → Experiment Runner (yields episodes/tasks)
• Experiment Runner → MAML Implementation (feeds tasks, triggers graph build + session runs)
• MAML Implementation → TensorFlow v1 Graph/Session (build/execute)
• MAML Implementation ↔ Gradient Extensions (used during graph construction/backprop)
• Experiment Runner → Artifacts (writes checkpoints/logs)
• (Optional) Artifacts → Experiment Runner (resume from checkpoint)
• “N-way K-shot episode: support/query sets”
• “Build graph (ops)”
• “Session run: meta-train step / meta-test eval”
• “Checkpoint save/restore”
Sample meta-batch of tasks
data_generator.py produces T tasks; each task has:
• Support set (train for inner loop)
• Query set (test for outer loop gradient)
For each task: Inner-loop adaptation (K steps)
• Start from meta-parameters θ
• Compute loss on support set
• Apply gradient step(s): θ’ = θ − α∇θ L_support
(If doing higher-order MAML, gradients flow through this update; otherwise first-order approximations.)
Compute query loss using adapted parameters
L_query(θ’) on query set
Aggregate across tasks → meta-objective
Sum/mean query losses across tasks in the meta-batch
Outer-loop meta-update
θ ← θ − β∇θ Σ L_query
(This is where special_grads.py may be involved.)
Logging + checkpointing
Write summaries, save model to logs/...
Make the “inner vs outer loop” visually unmistakable
Use two nested containers:
• Outer container: “Meta-Iteration”
• Inner repeated block: “Per-Task Adaptation”
• N-way, K-shot
• inner steps
• α (inner lr), β (meta lr)
• meta-batch size (#tasks)
Important architectural patterns / principles to reflect
• Separation of concerns
• main.py orchestration vs maml.py algorithm vs data_generator.py sampling
• Static graph build + session execution (TF1)
• A “Graph Construction” phase and a “Session Run” phase (even if both happen in one process)
• Bilevel optimization (core of MAML)
• inner adaptation loop and outer meta-optimization loop
• Pluggable datasets/domains
• Omniglot/MiniImagenet/Sinusoid are sources into the same episode generator + MAML core
Technologies/frameworks to label
• Python 2/3
• TensorFlow v1.x (Graph, Session, Optimizer)
• Datasets: Omniglot, MiniImagenet, Sinusoid regression
• Artifact format: TF checkpoints (.data, .index) and logs