How ryanwangzf/survtrace works
Project type and purpose
• SurvTRACE is a Python library/tool for survival analysis (single-event and competing events) built on transformer architectures.
• It provides data loading, model definition, training, and evaluation utilities, plus example notebooks and data-processing scripts.
File structure
Top-level items:
• data/ – raw and processed SEER data and a script to process it (process_seer.py, seer.sl, seer_processed.csv)
• experiment_*.ipynb – example Jupyter notebooks demonstrating usage (METABRIC, SEER, support datasets)
• figure/ – screenshots for README
• requirements.txt & survtrace.yml – Python dependencies and Conda environment
• survtrace/ – core library package
– config.py – global config object (STConfig)
– dataset.py – load_data, preprocessing pipelines for METABRIC and SEER
– model.py & modeling_bert.py – SurvTraceSingle class and transformer-based architectures
– train_utils.py – Trainer class, training loop orchestration
– evaluate_utils.py – Evaluator class, evaluation metrics and procedures
– losses.py – custom loss functions for survival and competing risk
– utils.py – shared helper functions
README for additional insights
• Usage snippet shows flow: load_data → initialize model → Trainer.fit → Evaluator.eval
• Mentions dependencies (transformers, PyTorch, pandas, numpy, etc.)
• Provides guidance on obtaining and processing SEER data
a. Main components:
• External Data Sources – METABRIC CSV, SEER raw data via seerstat
• Data Processing Pipeline – process_seer.py for SEER, load_data logic in dataset.py
• Configuration – STConfig controlling data paths, hyperparameters
• Core Library Modules:
– Dataset Loader (survtrace.dataset)
– Model Definition (survtrace.model + survtrace.modeling_bert)
– Loss Computation (survtrace.losses)
– Training Engine (survtrace.train_utils)
– Evaluation Engine (survtrace.evaluate_utils)
– Utilities (survtrace.utils)
• User-facing Interfaces – Python API and example Notebooks
• Environment & Dependencies – Conda env (survtrace.yml), requirements.txt
b. Relationships and interactions:
1. External Data Sources feed into Data Processing Pipeline
2. Processed datasets are passed to Dataset Loader
3. Dataset Loader yields training/validation/test sets according to STConfig
4. Trainer uses Model Definition + Loss Computation to run training loops
5. After training, Evaluator uses trained model and test sets to compute metrics
6. Utilities support logging, metrics aggregation, checkpointing
c. Architectural patterns/design principles:
• Modular separation: each concern (config, data, model, train, eval, utils) in its own module
• Pipeline pattern: data → model → training → evaluation
• Extensibility points: custom datasets, model classes, loss functions
d. Relevant technologies/frameworks:
• Python 3.x
• PyTorch (transformers back-end)
• Hugging Face Transformers (for BERT encoder)
• Pandas, NumPy (data handling)
• Scikit-learn (potentially for metrics)
• Jupyter (experiments)
Elements
• Clear labels: e.g., “Dataset Loader”, “Trainer”, “Evaluator”
– Data flow from raw source to processed → loader → trainer → evaluator
– Control flow from configuration to modules
– Parallelograms for scripts/notebooks (orange)