# andifathulms/paribus architecture

> Architecture diagram of the GitHub repository andifathulms/paribus, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andifathulms/paribus
- Repository: https://github.com/andifathulms/paribus

## Overview

#### Purpose and System Shape
Paribus is a research pipeline and publication system estimating performance differences associated with competing under untested powerlifting rules. Its core product is a guarded identification workflow, not athlete-level analytics. The backend processes public competition records through reproducible stages; the frontend presents aggregate findings, methodology, and downloadable data.

Entry points are Django management commands in `backend/manage.py`, the versioned DRF API at `backend/apps/api/urls.py`, and Next.js routes under `frontend/src/app/`.

#### Runtime and Storage Boundaries
The backend is Django 5/Python 3.11 with DRF, Polars and statistical/ML libraries; it runs with PostgreSQL 16 for pipeline metadata and estimates, Redis/Celery for asynchronous work, and local parquet for row-level competition records. Deployment is containerized through `docker-compose.yml`, `backend/Dockerfile`, and `frontend/Dockerfile`.

Postgres deliberately does not become a raw-data warehouse: data paths and shared settings are centralized in `backend/apps/common/paths.py` and `backend/apps/common/config.py`.

#### 1. Acquisition, Validation, and Cleaning
The ingest domain downloads either the OpenPowerlifting corpus or synthetic data, validates its schema, and writes cleaned analytical records. Its commands establish the first pipeline boundary:
- `backend/apps/ingest/management/commands/fetch_data.py`
- `backend/apps/ingest/management/commands/validate_schema.py`
- `backend/apps/ingest/cleaning.py`

Cleaning rules are externally frozen in `config/cleaning.yml`; raw-to-clean transformation should remain reproducible and separate from downstream inference.

#### 2. Privacy-Preserving Identity Resolution
Identity resolution links records into lifter-level panels while avoiding publication of names. It combines linkage, similarity/plausibility checks, optional hand-labeled classifier candidates, and validation/reporting:
- `backend/apps/identity/linkage.py`
- `backend/apps/identity/services.py`
- `backend/apps/identity/models.py`

Published identifiers use a configured salt, and API serialization is structurally constrained from returning lifter names. Identity definitions are controlled by `config/identity.yml`; alternative identity sets feed sensitivity analysis.

#### 3. Cohort Construction and Feasibility
Cohort builders convert linked clean records into design-eligible populations, partitioning incompatible equipment and event categories rather than pooling them. They also make and persist the same-meet/RQ2 feasibility decision before estimation:
- `backend/apps/cohorts/builders.py`
- `backend/apps/cohorts/services.py`
- `backend/apps/cohorts/models.py`

Definitions are frozen via `config/cohorts.yml`, creating a boundary between population specification and statistical execution.

#### 4. Event-Study Identification Gate
The event-study stage centers switchers on first untested competition and evaluates pre-switch trends. Its recorded verdict is a hard dependency: estimation cannot proceed without it.
- `backend/apps/eventstudy/gate.py`
- `backend/apps/eventstudy/services.py`
- `backend/apps/eventstudy/management/commands/event_study.py`

This is the primary control-flow gate between cohort assembly and causal-style reporting.

#### 5. Estimation and Stored Results
Estimation executes naive, within-lifter fixed-effects, and eligible same-meet designs, with clustered lifter standard errors and intervals/sample counts required on persisted estimates:
- `backend/apps/estimation/design.py`
- `backend/apps/estimation/services.py`
- `backend/apps/estimation/models.py`

Specification definitions reside in `config/specs.yml`. Heterogeneity and spline-related analysis are adjacent but remain within the estimation domain.

#### 6. Robustness as Validity Enforcement
Robustness procedures test placebo behavior, propensity matching, federation exclusion, and identity sensitivity:
- `backend/apps/robustness/placebo.py`
- `backend/apps/robustness/propensity.py`
- `backend/apps/robustness/sensitivity.py`

The placebo command is a material quality gate: failure exits non-zero and invalidates the within-lifter headline rather than merely annotating it.

#### 7. Publication API and Dashboard
DRF exposes aggregate estimates, downloads, and health/framing information; serializers and routes enforce the no-athlete-surface boundary:
- `backend/apps/api/views.py`
- `backend/apps/api/viewsets.py`
- `backend/apps/api/downloads.py`

The Next.js frontend consumes this API through `frontend/src/lib/api.ts` and presents findings/data/methodology routes, notably `frontend/src/app/findings/page.tsx` and `frontend/src/app/data/page.tsx`. Charts visualize aggregate uncertainty and event-study outputs, not individual careers.

Main entry points: [`backend/apps/estimation/services.py`](https://github.com/andifathulms/paribus/blob/main/backend/apps/estimation/services.py) (Estimation), [`backend/apps/identity/services.py`](https://github.com/andifathulms/paribus/blob/main/backend/apps/identity/services.py) (Identity resolution), [`backend/apps/cohorts/services.py`](https://github.com/andifathulms/paribus/blob/main/backend/apps/cohorts/services.py) (Cohort construction), [`backend/apps/eventstudy/gate.py`](https://github.com/andifathulms/paribus/blob/main/backend/apps/eventstudy/gate.py) (Event-study gate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_pipeline["Research pipeline"]
  node_ingest["Ingest and clean<br/>Django commands<br/>[fetch_data.py]"]
  node_cleaning_rules["Cleaning rules<br/>frozen definition<br/>[cleaning.yml]"]
  node_identity["Identity resolution<br/>linkage service<br/>[services.py]"]
  node_identity_rules["Identity definitions<br/>frozen definition<br/>[identity.yml]"]
  node_cohorts["Cohort construction<br/>cohort service<br/>[services.py]"]
  node_cohort_rules["Cohort definitions<br/>frozen definition<br/>[cohorts.yml]"]
  node_event_gate{{"Event-study gate<br/>identification gate<br/>[gate.py]"}}
  node_estimation["Estimation<br/>[services.py]"]
  node_specs["Specifications<br/>frozen definition<br/>[specs.yml]"]
  node_robustness{{"Robustness enforcement<br/>quality gate<br/>[placebo.py]"}}
end

subgraph group_publication["Publication surface"]
  node_api["Aggregate publication API<br/>versioned DRF API<br/>[urls.py]"]
  node_downloads["Data downloads<br/>DRF download service<br/>[downloads.py]"]
  node_frontend_client["API client<br/>Next.js data client<br/>[api.ts]"]
  node_dashboard["Findings and data dashboard<br/>Next.js routes<br/>[page.tsx]"]
end

subgraph group_storage["Storage and execution"]
  node_parquet[("Competition records<br/>local parquet<br/>[paths.py]")]
  node_postgres[("Pipeline metadata and estimates<br/>PostgreSQL 16")]
  node_celery["Async workers<br/>Celery and Redis"]
end

node_cleaning_rules -.->|"governs"| node_ingest
node_ingest -->|"writes cleaned records"| node_parquet
node_parquet -->|"clean records"| node_identity
node_identity_rules -.->|"governs"| node_identity
node_identity -->|"linked panels"| node_cohorts
node_cohort_rules -.->|"governs"| node_cohorts
node_cohorts -->|"eligible switchers"| node_event_gate
node_event_gate -->|"passing verdict required"| node_estimation
node_specs -.->|"defines designs"| node_estimation
node_estimation -->|"headline estimates"| node_robustness
node_identity -->|"panel metadata"| node_postgres
node_cohorts -->|"cohorts and feasibility"| node_postgres
node_event_gate -->|"recorded verdict"| node_postgres
node_estimation -->|"persisted estimates"| node_postgres
node_robustness -->|"validity status"| node_postgres
node_celery -.->|"runs async work"| node_ingest
node_celery -.->|"runs async work"| node_estimation
node_postgres -->|"aggregate results"| node_api
node_api -->|"serves download outputs"| node_downloads
node_api -->|"aggregate API responses"| node_frontend_client
node_frontend_client -->|"findings and data"| node_dashboard

click node_ingest "https://github.com/andifathulms/paribus/blob/main/backend/apps/ingest/management/commands/fetch_data.py"
click node_cleaning_rules "https://github.com/andifathulms/paribus/blob/main/config/cleaning.yml"
click node_parquet "https://github.com/andifathulms/paribus/blob/main/backend/apps/common/paths.py"
click node_identity "https://github.com/andifathulms/paribus/blob/main/backend/apps/identity/services.py"
click node_identity_rules "https://github.com/andifathulms/paribus/blob/main/config/identity.yml"
click node_cohorts "https://github.com/andifathulms/paribus/blob/main/backend/apps/cohorts/services.py"
click node_cohort_rules "https://github.com/andifathulms/paribus/blob/main/config/cohorts.yml"
click node_event_gate "https://github.com/andifathulms/paribus/blob/main/backend/apps/eventstudy/gate.py"
click node_estimation "https://github.com/andifathulms/paribus/blob/main/backend/apps/estimation/services.py"
click node_specs "https://github.com/andifathulms/paribus/blob/main/config/specs.yml"
click node_robustness "https://github.com/andifathulms/paribus/blob/main/backend/apps/robustness/placebo.py"
click node_api "https://github.com/andifathulms/paribus/blob/main/backend/apps/api/urls.py"
click node_downloads "https://github.com/andifathulms/paribus/blob/main/backend/apps/api/downloads.py"
click node_frontend_client "https://github.com/andifathulms/paribus/blob/main/frontend/src/lib/api.ts"
click node_dashboard "https://github.com/andifathulms/paribus/blob/main/frontend/src/app/findings/page.tsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest,node_cleaning_rules,node_identity,node_identity_rules,node_cohorts,node_cohort_rules,node_event_gate,node_estimation,node_specs,node_robustness toneBlue
class node_api,node_downloads,node_frontend_client,node_dashboard toneAmber
class node_parquet,node_postgres,node_celery toneMint
```

## Components

### Research pipeline

Reproducible identification and estimation stages.

- [Ingest and clean](https://github.com/andifathulms/paribus/blob/main/backend/apps/ingest/management/commands/fetch_data.py): Django commands · Downloads, validates, and cleans competition records. · `backend/apps/ingest/management/commands/fetch_data.py`
- [Cleaning rules](https://github.com/andifathulms/paribus/blob/main/config/cleaning.yml): frozen definition · `config/cleaning.yml`
- [Identity resolution](https://github.com/andifathulms/paribus/blob/main/backend/apps/identity/services.py): linkage service · Builds privacy-preserving lifter panels and validation reports. · `backend/apps/identity/services.py`
- [Identity definitions](https://github.com/andifathulms/paribus/blob/main/config/identity.yml): frozen definition · `config/identity.yml`
- [Cohort construction](https://github.com/andifathulms/paribus/blob/main/backend/apps/cohorts/services.py): cohort service · Partitions compatible populations and records feasibility. · `backend/apps/cohorts/services.py`
- [Cohort definitions](https://github.com/andifathulms/paribus/blob/main/config/cohorts.yml): frozen definition · `config/cohorts.yml`
- [Event-study gate](https://github.com/andifathulms/paribus/blob/main/backend/apps/eventstudy/gate.py): identification gate · Records the pre-trend verdict required before estimation. · `backend/apps/eventstudy/gate.py`
- [Estimation](https://github.com/andifathulms/paribus/blob/main/backend/apps/estimation/services.py): estimation service · Runs naive, within-lifter, and eligible same-meet designs. · `backend/apps/estimation/services.py`
- [Specifications](https://github.com/andifathulms/paribus/blob/main/config/specs.yml): frozen definition · `config/specs.yml`
- [Robustness enforcement](https://github.com/andifathulms/paribus/blob/main/backend/apps/robustness/placebo.py): quality gate · Placebo failure invalidates the within-lifter headline. · `backend/apps/robustness/placebo.py`

### Publication surface

Aggregate-only API and dashboard.

- [Aggregate publication API](https://github.com/andifathulms/paribus/blob/main/backend/apps/api/urls.py): versioned DRF API · Routes and serializers exclude lifter names and athlete-level surfaces. · `backend/apps/api/urls.py`
- [Data downloads](https://github.com/andifathulms/paribus/blob/main/backend/apps/api/downloads.py): DRF download service · Publishes aggregate downloadable outputs. · `backend/apps/api/downloads.py`
- [API client](https://github.com/andifathulms/paribus/blob/main/frontend/src/lib/api.ts): Next.js data client · `frontend/src/lib/api.ts`
- [Findings and data dashboard](https://github.com/andifathulms/paribus/blob/main/frontend/src/app/findings/page.tsx): Next.js routes · Presents aggregate uncertainty, event studies, methods, and data. · `frontend/src/app/findings/page.tsx`

### Storage and execution

Runtime persistence boundaries.

- [Competition records](https://github.com/andifathulms/paribus/blob/main/backend/apps/common/paths.py): local parquet · Raw and cleaned row-level records remain outside Postgres. · `backend/apps/common/paths.py`
- **Pipeline metadata and estimates**: PostgreSQL 16 · Stores workflow metadata, verdicts, cohorts, and aggregate estimates.
- **Async workers**: Celery and Redis

## Connections

- Cleaning rules → Ingest and clean: governs
- Ingest and clean → Competition records: writes cleaned records
- Competition records → Identity resolution: clean records
- Identity definitions → Identity resolution: governs
- Identity resolution → Cohort construction: linked panels
- Cohort definitions → Cohort construction: governs
- Cohort construction → Event-study gate: eligible switchers
- Event-study gate → Estimation: passing verdict required
- Specifications → Estimation: defines designs
- Estimation → Robustness enforcement: headline estimates
- Identity resolution → Pipeline metadata and estimates: panel metadata
- Cohort construction → Pipeline metadata and estimates: cohorts and feasibility
- Event-study gate → Pipeline metadata and estimates: recorded verdict
- Estimation → Pipeline metadata and estimates: persisted estimates
- Robustness enforcement → Pipeline metadata and estimates: validity status
- Async workers → Ingest and clean: runs async work
- Async workers → Estimation: runs async work
- Pipeline metadata and estimates → Aggregate publication API: aggregate results
- Aggregate publication API → Data downloads: serves download outputs
- Aggregate publication API → API client: aggregate API responses
- API client → Findings and data dashboard: findings and data