# bulik/ldsc architecture

> Architecture diagram of the GitHub repository bulik/ldsc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bulik/ldsc
- Repository: https://github.com/bulik/ldsc

## Overview

##### Project type and purpose
- This is a Python-based command‐line tool (“ldsc”) for genetic analyses: computing LD Scores, estimating heritability (h2), performing partitioned h2 analyses, and genetic correlation from GWAS summary statistics.
- Key entrypoints: ldsc.py (LD Score regression), munge_sumstats.py (preprocessing), make_annot.py (annotation creation).

##### File structure
- Top-level scripts: ldsc.py, munge_sumstats.py, make_annot.py, test/simulate.py.
- Core library: directory ldscore/ with modules:
- parse.py – reading and validating input files
- ldscore.py – computing LD Scores
- sumstats.py – handling summary statistic files
- regressions.py – running regression engines
- jackknife.py, irwls.py – statistical backends
- Configuration: environment.yml, requirements.txt, setup.py.
- Test suite under test/ for unit and integration tests against known data.

##### Additional insights from README
- Relies on external data (1000 Genomes–based LD Scores).
- Four major workflows: LD-Score estimation, single‐trait h2, partitioned h2, genetic correlations.
- Uses Anaconda for dependency management.

##### Core components
a. Components
- CLI Layer:
- ldsc.py
- munge_sumstats.py
- make_annot.py
- Core Library (ldscore package):
- Input Parser (parse.py)
- Data Models (sumstats.py, annotations)
- Computation Engines (ldscore.py, regressions.py, jackknife.py, irwls.py)
- Data Sources:
- User GWAS summary stats
- Reference LD Score files (external downloads)
- Annotation files (ContinuousAnnotations, make_annot)
- Environment & Dependencies
- Outputs: heritability reports, LD Score files, genetic correlation matrices.
- Testing Framework: test/ directory with fixtures and test scripts.

b. Relationships & data flow
1. User invokes a CLI script (e.g., `./ldsc.py` or `./munge_sumstats.py`).
2. Script calls parse.py to ingest input files (sumstats/Munge).
3. Data Models (sumstats.py) normalize and validate data.
4. LD Score computation in ldscore.py (may call external reference files).
5. Regression engines (regressions.py) invoke statistical routines (jackknife, irwls).
6. Results written to disk.
7. Tests call library functions and compare outputs against test fixtures.

c. Architectural patterns & principles
- Modular CLI → Library separation: Top-level scripts delegate to ldscore package.
- Single Responsibility: each module (parse, ldscore, regressions, sumstats) has clear scope.
- Data‐pipeline style for workflows: sequential stages (parse → compute → regress → output).
- Test‐driven: extensive test suite validating each module.

d. Key technologies/frameworks
- Python 2/3 (Anaconda environment)
- NumPy/SciPy (statistical routines)
- pandas (possibly for table handling)
- External data: 1000 Genomes LD Score tarballs.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CLI Layer
    subgraph "CLI Layer"
        direction TB
        LDSc["ldsc.py"]:::cli
        Munge["munge_sumstats.py"]:::cli
        AnnotCLI["make_annot.py"]:::cli
    end

    %% Core Library
    subgraph "Core Library (ldscore package)"
        direction TB
        Parser["parse.py"]:::core
        SumStats["sumstats.py"]:::core
        LDComp["ldscore.py"]:::core
        Regress["regressions.py"]:::core
        Jackknife["jackknife.py"]:::core
        IRWLS["irwls.py"]:::core
    end

    %% Data Sources & Outputs
    subgraph "Data Sources & Outputs"
        direction TB
        GWAS["User GWAS summary stats"]:::data
        RefLD["Reference LD Score files"]:::data
        ContAnno["Continuous Annotations"]:::data
        Reports["Output Reports\n(h2, LD Scores,\ncorrelation matrices)"]:::data
    end

    %% Environment & Dependencies
    subgraph "Environment & Dependencies"
        direction TB
        EnvYml["environment.yml"]:::env
        ReqTxt["requirements.txt"]:::env
        SetupPy["setup.py"]:::env
    end

    %% Testing Framework
    subgraph "Testing Framework"
        direction TB
        TestSuite["test/"]:::test
        Simulate["test/simulate.py"]:::test
    end

    %% Data Flow
    GWAS -->|raw stats| LDSc
    GWAS -->|raw stats| Munge
    ContAnno -->|annotation defs| AnnotCLI

    LDSc -->|calls parser| Parser
    Munge -->|calls parser| Parser
    AnnotCLI -->|annotation pipeline| Parser

    Parser -->|parsed data| SumStats
    SumStats -->|normalized data| LDComp
    RefLD -->|reference data| LDComp

    LDComp -->|LD matrix| Regress
    Regress -->|uses| Jackknife
    Regress -->|uses| IRWLS
    Jackknife -->|jackknife results| Regress
    IRWLS -->|IRWLS results| Regress

    Regress -->|h2 & corr estimates| LDSc
    Regress -->|partitioned h2| AnnotCLI

    LDSc -->|writes| Reports
    Munge -->|writes| Reports
    AnnotCLI -->|writes| Reports

    %% Testing Invocations
    TestSuite -->|unit tests| Parser
    TestSuite -->|unit tests| SumStats
    TestSuite -->|unit tests| LDComp
    TestSuite -->|unit tests| Regress
    Simulate -->|E2E script| LDSc

    %% Click Events
    click LDSc "https://github.com/bulik/ldsc/blob/master/ldsc.py"
    click Munge "https://github.com/bulik/ldsc/blob/master/munge_sumstats.py"
    click AnnotCLI "https://github.com/bulik/ldsc/blob/master/make_annot.py"
    click Parser "https://github.com/bulik/ldsc/blob/master/ldscore/parse.py"
    click SumStats "https://github.com/bulik/ldsc/blob/master/ldscore/sumstats.py"
    click LDComp "https://github.com/bulik/ldsc/blob/master/ldscore/ldscore.py"
    click Regress "https://github.com/bulik/ldsc/blob/master/ldscore/regressions.py"
    click Jackknife "https://github.com/bulik/ldsc/blob/master/ldscore/jackknife.py"
    click IRWLS "https://github.com/bulik/ldsc/blob/master/ldscore/irwls.py"
    click ContAnno "https://github.com/bulik/ldsc/tree/master/ContinuousAnnotations/"
    click EnvYml "https://github.com/bulik/ldsc/blob/master/environment.yml"
    click ReqTxt "https://github.com/bulik/ldsc/blob/master/requirements.txt"
    click SetupPy "https://github.com/bulik/ldsc/blob/master/setup.py"
    click TestSuite "https://github.com/bulik/ldsc/tree/master/test/"
    click Simulate "https://github.com/bulik/ldsc/blob/master/test/simulate.py"

    %% Styles
    classDef cli fill:#D0E8FF,stroke:#0366D6,color:#0366D6;
    classDef core fill:#DFFFE0,stroke:#228B22,color:#228B22;
    classDef data fill:#FFF4D0,stroke:#DAA520,color:#DAA520;
    classDef env fill:#EEEEEE,stroke:#888888,color:#555555;
    classDef test fill:#F3E0FF,stroke:#800080,color:#800080;
```
