How ellenbet/survana works
Project type and purpose
• This is a Python-based survival analysis and biomarker‐discovery toolkit. It provides functionality for:
• Artificial data generation
• Data ingestion, merging, and preprocessing
• Model definition and fitting (e.g., Cox proportional hazards)
• Hyperparameter tuning (Optuna objectives)
• Example notebooks and result visualization
• It ships as a library (src/survana), a CLI entrypoint (main.py), and supporting docs, tests, and Docker/CI config.
File structure
• Top-level:
• Dockerfile, .dockerignore – containerization
• .pre-commit-config.yaml – code style enforcement
• mkdocs.yml and docs/ – documentation site configuration
• examples/ – Jupyter notebooks illustrating usage
• src/survana/ – core library packages
• tests/ + test_data/ – automated tests
• main.py – likely CLI start point
• Core library submodules under src/survana/:
• artificial_data_generation/ – synthetic dataset creation
• data_processing/ – data models, loaders, merger utilities
• models/ – statistical/ML models (e.g., coxph.py)
• tuning/ – tuning entrypoints for Optuna
• Supporting files: pyproject.toml for packaging, uv.lock for dependency locking.
README
• Confirms survival analysis focus and dependency on pre-commit, mypy, isort, black, flake8.
• Mentions “uv sync”—likely referencing a task runner CLI.
a. CLI / entrypoint
– main.py invokes library functions (data gen → processing → modeling → tuning)
b. Library packages
– Artificial Data Generation
– Data Processing
– Models
– Tuning
c. External systems / services
– Optuna (for hyperparameter search)
– MLflow (if used—docs mention mlflow)
– Docker (for containerized runs)
d. Data stores
– Input CSVs (e.g., test_matrix.csv, user data)
– Generated synthetic data
– Output figures (result_figs/)
e. Tests & CI
– pytest runs tests in tests/ against test_data/
Relationships and interactions
• CLI → calls Artificial Data Generation → outputs dataset
• CLI → calls Data Processing → ingests real or synthetic CSVs → normalized data models
• Processed data → passed to Models → fit and evaluate survival models
• Model fitting → optionally triggers Tuning → uses Optuna engine → iterates model fits
• Tuning results → logged to MLflow or local artifact store → final metrics and output figures
• All modules share common data models and type definitions (in data_models.py)
Architectural patterns and design principles
• Modular layering: each functional area (generation, processing, modeling, tuning) lives in its own package
• Single-responsibility: modules do one job, enabling testability
• Dependency injection via function arguments (e.g., pass DataFrame to model/trainer)
• Extensibility: adding new models or tuning objectives involves dropping in new modules under models or tuning