# ryanwangzf/survtrace architecture

> Architecture diagram of the GitHub repository ryanwangzf/survtrace, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ryanwangzf/survtrace
- Repository: https://github.com/ryanwangzf/survtrace

## Overview

##### Project type and purpose
- SurvTRACE is a Python library/tool for survival analysis (single-event and competing events) built on transformer architectures.
- It provides data loading, model definition, training, and evaluation utilities, plus example notebooks and data-processing scripts.

##### File structure
Top-level items:
- data/ – raw and processed SEER data and a script to process it (process_seer.py, seer.sl, seer_processed.csv)
- experiment_*.ipynb – example Jupyter notebooks demonstrating usage (METABRIC, SEER, support datasets)
- figure/ – screenshots for README
- requirements.txt & survtrace.yml – Python dependencies and Conda environment
- survtrace/ – core library package
– config.py – global config object (STConfig)
– dataset.py – load_data, preprocessing pipelines for METABRIC and SEER
– model.py & modeling_bert.py – SurvTraceSingle class and transformer-based architectures
– train_utils.py – Trainer class, training loop orchestration
– evaluate_utils.py – Evaluator class, evaluation metrics and procedures
– losses.py – custom loss functions for survival and competing risk
– utils.py – shared helper functions

##### README for additional insights
- Usage snippet shows flow: load_data → initialize model → Trainer.fit → Evaluator.eval
- Mentions dependencies (transformers, PyTorch, pandas, numpy, etc.)
- Provides guidance on obtaining and processing SEER data

a. Main components:
- External Data Sources – METABRIC CSV, SEER raw data via seerstat
- Data Processing Pipeline – process_seer.py for SEER, load_data logic in dataset.py
- Configuration – STConfig controlling data paths, hyperparameters
- Core Library Modules:
– Dataset Loader (survtrace.dataset)
– Model Definition (survtrace.model + survtrace.modeling_bert)
– Loss Computation (survtrace.losses)
– Training Engine (survtrace.train_utils)
– Evaluation Engine (survtrace.evaluate_utils)
– Utilities (survtrace.utils)
- User-facing Interfaces – Python API and example Notebooks
- Environment & Dependencies – Conda env (survtrace.yml), requirements.txt

b. Relationships and interactions:
1. External Data Sources feed into Data Processing Pipeline
2. Processed datasets are passed to Dataset Loader
3. Dataset Loader yields training/validation/test sets according to STConfig
4. Trainer uses Model Definition + Loss Computation to run training loops
5. After training, Evaluator uses trained model and test sets to compute metrics
6. Utilities support logging, metrics aggregation, checkpointing

c. Architectural patterns/design principles:
- Modular separation: each concern (config, data, model, train, eval, utils) in its own module
- Pipeline pattern: data → model → training → evaluation
- Extensibility points: custom datasets, model classes, loss functions

d. Relevant technologies/frameworks:
- Python 3.x
- PyTorch (transformers back-end)
- Hugging Face Transformers (for BERT encoder)
- Pandas, NumPy (data handling)
- Scikit-learn (potentially for metrics)
- Jupyter (experiments)

##### Elements
- Clear labels: e.g., “Dataset Loader”, “Trainer”, “Evaluator”
– Data flow from raw source to processed → loader → trainer → evaluator
– Control flow from configuration to modules
– Parallelograms for scripts/notebooks (orange)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Data Sources
    subgraph "External Data Sources"
        direction TB
        DS_RAW["SEER Raw Data"]:::data
        DS_PROCESSED["seer_processed.csv"]:::data
        SL_SCRIPT["seer.sl"]:::script
    end

    %% Data Processing Pipeline
    subgraph "Data Processing Pipeline"
        direction TB
        PS_SCRIPT["process_seer.py"]:::script
        DATASET_MOD["dataset.py"]:::module
    end

    %% Configuration
    subgraph "Configuration"
        direction TB
        CONFIG["config.py"]:::module
    end

    %% Core Library Modules
    subgraph "Core Library"
        direction TB
        DatasetLoader["Dataset Loader"]:::module
        ModelCore["model.py"]:::module
        ModelBert["modeling_bert.py"]:::module
        LossComp["losses.py"]:::module
        Trainer["train_utils.py"]:::module
        Evaluator["evaluate_utils.py"]:::module
        Utils["utils.py"]:::module
    end

    %% User Interfaces
    subgraph "User Interfaces"
        direction TB
        NB1["experiment_metabric.ipynb"]:::notebook
        NB2["experiment_seer.ipynb"]:::notebook
        NB3["experiment_support.ipynb"]:::notebook
    end

    %% Environment & Dependencies
    subgraph "Environment & Dependencies"
        direction TB
        REQ["requirements.txt"]:::module
        ENV["survtrace.yml"]:::module
    end

    %% Data and Processing Flow
    DS_RAW --> PS_SCRIPT
    SL_SCRIPT --> PS_SCRIPT
    PS_SCRIPT --> DS_PROCESSED
    DS_PROCESSED --> DATASET_MOD
    DATASET_MOD --> DatasetLoader
    CONFIG --> DatasetLoader

    %% Training Flow
    DatasetLoader --> Trainer
    ModelCore --> Trainer
    ModelBert --> Trainer
    LossComp --> Trainer
    Utils --> Trainer

    %% Evaluation Flow
    Trainer --> Evaluator
    DatasetLoader --> Evaluator
    Utils --> Evaluator

    %% User Orchestration
    NB1 --> CONFIG
    NB1 --> DatasetLoader
    NB1 --> Trainer
    NB1 --> Evaluator
    NB2 --> CONFIG
    NB2 --> DatasetLoader
    NB2 --> Trainer
    NB2 --> Evaluator
    NB3 --> CONFIG
    NB3 --> DatasetLoader
    NB3 --> Trainer
    NB3 --> Evaluator

    %% Environment usage
    REQ -.-> CONFIG
    ENV -.-> CONFIG

    %% Click Events
    click SL_SCRIPT "https://github.com/ryanwangzf/survtrace/blob/main/data/seer.sl"
    click PS_SCRIPT "https://github.com/ryanwangzf/survtrace/blob/main/data/process_seer.py"
    click DS_PROCESSED "https://github.com/ryanwangzf/survtrace/blob/main/data/seer_processed.csv"
    click DATASET_MOD "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/dataset.py"
    click CONFIG "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/config.py"
    click ModelCore "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/model.py"
    click ModelBert "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/modeling_bert.py"
    click LossComp "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/losses.py"
    click Trainer "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/train_utils.py"
    click Evaluator "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/evaluate_utils.py"
    click Utils "https://github.com/ryanwangzf/survtrace/blob/main/survtrace/utils.py"
    click NB1 "https://github.com/ryanwangzf/survtrace/blob/main/experiment_metabric.ipynb"
    click NB2 "https://github.com/ryanwangzf/survtrace/blob/main/experiment_seer.ipynb"
    click NB3 "https://github.com/ryanwangzf/survtrace/blob/main/experiment_support.ipynb"
    click REQ "https://github.com/ryanwangzf/survtrace/blob/main/requirements.txt"
    click ENV "https://github.com/ryanwangzf/survtrace/blob/main/survtrace.yml"

    %% Styles
    classDef module fill:#cfe2f3,stroke:#6d9eeb,color:#000
    classDef data fill:#d9ead3,stroke:#93c47d,color:#000
    classDef script fill:#fce5cd,stroke:#f6b26b,color:#000
    classDef notebook fill:#fce5cd,stroke:#f6b26b,color:#000
```
