How microsoft/timecraft works
Project Type and Purpose
• This is a full-stack research framework (“TimeCraft”) for synthetic time-series generation, combining three complementary modules:
a. TimeDP (cross-domain diffusion with prototype prompts)
b. BRIDGE (text-based control via multi-agent LLM pipelines)
c. TarDiff (target-aware diffusion guided by downstream model gradients)
• It also contains related baseline code (DiGA, diffusion) and data/processing utilities.
High-Level Architecture
At the highest level, TimeCraft accepts up to three inputs—few-shot time series examples, free-form text prompts, downstream task model & data—and produces synthetic time series via a unified diffusion engine enhanced by:
• Prototype Assignment Module (PAM) for cross-domain prompts
• Multi-Agent Text System (BRIDGE) for text descriptions
• Influence-Guidance Module (TarDiff) for task-aware adaptation
Main Components
A. Data Ingestion & Preprocessing
• Scripts: TimeDP/utils/prepare_datasets.py, process/*, TarDiff/data_preprocess/*
• Actions: download raw datasets → cleaning/concatenation → sliding-window segmentation → .npy/.csv outputs
B. Prototype Learning & Assignment (TimeDP)
• Modules: TimeDP/ldm/models, TimeDP/ldm/modules, TimeDP/main_train.py
• Role: learn a shared “prototype dictionary” of temporal patterns; at inference, PAM uses few-shot examples to compute domain prompts
C. Diffusion Backbone
• Modules: TimeDP/ldm, TarDiff/ldm (variants), diffusion/ldm
• Role: core denoising steps; configurable via YAML (e.g., diffusion/configs, TimeDP/configs)
D. Text-Control Subsystem (BRIDGE)
• Modules: BRIDGE/llm_agents (agent orchestration, LLM calls), BRIDGE/self_refine (iterative feedback), template_extractor.py, ts_to_text.py
• Role: generate high-quality text descriptions of time series → hybrid prompt fed into diffusion cross-attention
E. Target-Aware Guidance (TarDiff)
• Modules: TarDiff/classifier (downstream model), TarDiff/guidance_generation.py, TarDiff/train_main.py
• Role: compute gradients of downstream loss → inject into each denoising step via Guidance Module
F. Training & Orchestration
• Entrypoints: main.py (umbrella trainer), TimeDP/main_train.py, TarDiff/train_main.py, diffusion/main.py, DiGA/train.py
• Configs: environment.yml, multiple .yaml under configs/
G. Inference & Visualization
• Inputs: domain prompts, text prompts, guidance set & downstream model
• Outputs: synthetic series, evaluation metrics, visualizations
Component Interactions & Data Flow
1. Raw data → Preprocessing → segmented series files
2. For prototype branch: preprocessed series → prototype learner → prototype dictionary
3. Few-shot examples → PAM → domain prompt
4. For text branch: segmented series → ts_to_text → multi-agent LLM pipelines → text descriptions → hybrid prompt
5. Downstream branch: preprocessed series + labels → train downstream model → gradients during inference
6. Inference pipeline: (prototype prompt, text prompt, guidance gradients) → diffusion backbone → synthetic time series
7. Post-process & evaluate with metrics modules (TimeDP/utils, TarDiff/utils)
Architectural Patterns & Design Principles
• Modular monorepo: each major capability isolated under its own top-level folder
• Plugin-style YAML configs for reproducibility and easy extension
• Separation of concerns: data prep, model training, inference, evaluation, visualization
• Hybrid conditioning: cross-attention for text & prototypes + gradient steering for tasks
Technologies & Frameworks
• Python 3.x, Conda environment (environment.yml)
• PyTorch (diffusion models, classifiers, RNNs)
• HuggingFace-style LLM wrappers (BRIDGE/llm_agents/tools)
• YAML for configuration, shell scripts for quick runs (*.sh)
• External services: Zenodo data downloads, web search tools for BRIDGE