# pscredefine/rankshift architecture

> Architecture diagram of the GitHub repository pscredefine/rankshift, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pscredefine/rankshift
- Repository: https://github.com/pscredefine/rankshift

## Overview

RankShift is a research and recommender-system repository: it measures how recommendation models behave when exposure policy changes, then includes a two-stage retrieval-and-ranking implementation with an HTTP recommendation endpoint. The research workflow runs experiments over KuaiRand logs and records metrics; the serving workflow retrieves candidates, builds point-in-time features, ranks them, and returns recommendations. The README documents several experiment capabilities whose source was not sampled; those links are shown as documented relationships rather than claims of source inspection.

Main entry points: [`results`](https://github.com/pscredefine/rankshift/tree/main/results) (Experiment results), [`07_two_stage.py`](https://github.com/pscredefine/rankshift/blob/main/07_two_stage.py) (Two-stage experiment), [`recsys/pipeline.py`](https://github.com/pscredefine/rankshift/blob/main/recsys/pipeline.py) (Recommendation pipeline), [`recsys/features.py`](https://github.com/pscredefine/rankshift/blob/main/recsys/features.py) (Feature builder)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_research["Exposure research"]
  node_signal["Signal probe<br/>[01_signal_probe.py]"]
  node_shift["Exposure evaluation"]
  node_calibration["Calibration study<br/>[03_recalibrate.py]"]
  node_ope["Off-policy evaluation<br/>[04_off_policy.py]"]
  node_neural["Neural baselines<br/>[05_two_tower.py]"]
  node_scale["Scale evaluation<br/>[06_scale_27k.py]"]
end

subgraph group_recsys["Recommender runtime"]
  node_two_stage_train["Two-stage experiment<br/>[07_two_stage.py]"]
  node_schema["Feature contracts<br/>[schema.py]"]
  node_features["Feature builder<br/>[features.py]"]
  node_retriever["Candidate retrieval<br/>[models.py]"]
  node_rankers["Ranking models<br/>[models.py]"]
  node_pipeline["Recommendation pipeline<br/>[pipeline.py]"]
  node_bundle[("Versioned bundles<br/>[pipeline.py]")]
end

subgraph group_serving["HTTP serving"]
  node_api["Recommendation API<br/>[api.py]"]
end

node_engineer(("Engineer"))
node_kuairand[("KuaiRand logs")]
node_results["Experiment results"]
node_client(("API client"))

node_engineer -->|"runs experiment"| node_signal
node_engineer -->|"runs experiment"| node_shift
node_engineer -->|"runs experiment"| node_calibration
node_engineer -->|"runs experiment"| node_ope
node_engineer -->|"runs experiment"| node_neural
node_engineer -->|"runs experiment"| node_scale
node_engineer -->|"runs experiment"| node_two_stage_train
node_kuairand -.->|"supplies logs"| node_signal
node_kuairand -->|"supplies logs"| node_shift
node_kuairand -->|"supplies logs"| node_ope
node_kuairand -->|"supplies logs"| node_neural
node_kuairand -->|"supplies logs"| node_two_stage_train
node_signal -->|"writes metrics"| node_results
node_shift -->|"writes metrics"| node_results
node_calibration -->|"writes metrics"| node_results
node_ope -->|"writes estimates"| node_results
node_neural -->|"writes metrics"| node_results
node_scale -->|"writes metrics"| node_results
node_two_stage_train -->|"writes metrics"| node_results
node_two_stage_train -->|"builds features"| node_features
node_two_stage_train -->|"fits retriever"| node_retriever
node_two_stage_train -->|"fits rankers"| node_rankers
node_features -->|"uses contracts"| node_schema
node_retriever -->|"uses encodings"| node_features
node_rankers -->|"uses encodings"| node_features
node_client -->|"requests recommendations"| node_api
node_api -->|"loads and calls"| node_pipeline
node_pipeline -->|"retrieves candidates"| node_retriever
node_pipeline -->|"builds request features"| node_features
node_pipeline -->|"scores candidates"| node_rankers
node_api -->|"loads bundle"| node_bundle
node_pipeline -->|"saves or loads"| node_bundle

click node_signal "https://github.com/pscredefine/rankshift/blob/main/01_signal_probe.py"
click node_shift "https://github.com/pscredefine/rankshift/blob/main/02_exposure_bias.py"
click node_calibration "https://github.com/pscredefine/rankshift/blob/main/03_recalibrate.py"
click node_ope "https://github.com/pscredefine/rankshift/blob/main/04_off_policy.py"
click node_neural "https://github.com/pscredefine/rankshift/blob/main/05_two_tower.py"
click node_scale "https://github.com/pscredefine/rankshift/blob/main/06_scale_27k.py"
click node_two_stage_train "https://github.com/pscredefine/rankshift/blob/main/07_two_stage.py"
click node_schema "https://github.com/pscredefine/rankshift/blob/main/recsys/schema.py"
click node_features "https://github.com/pscredefine/rankshift/blob/main/recsys/features.py"
click node_retriever "https://github.com/pscredefine/rankshift/blob/main/recsys/models.py"
click node_rankers "https://github.com/pscredefine/rankshift/blob/main/recsys/models.py"
click node_pipeline "https://github.com/pscredefine/rankshift/blob/main/recsys/pipeline.py"
click node_bundle "https://github.com/pscredefine/rankshift/blob/main/recsys/pipeline.py"
click node_api "https://github.com/pscredefine/rankshift/blob/main/recsys/api.py"
click node_results "https://github.com/pscredefine/rankshift/tree/main/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_signal,node_shift,node_calibration,node_ope,node_neural,node_scale,node_client toneBlue
class node_two_stage_train,node_schema,node_features,node_retriever,node_rankers,node_pipeline,node_bundle,node_kuairand toneAmber
class node_api toneMint
class node_engineer toneIndigo
class node_results toneTeal
```

## Components

### Exposure research

- [Signal probe](https://github.com/pscredefine/rankshift/blob/main/01_signal_probe.py): `01_signal_probe.py`
- [Exposure evaluation](https://github.com/pscredefine/rankshift/blob/main/02_exposure_bias.py): `02_exposure_bias.py`
- [Calibration study](https://github.com/pscredefine/rankshift/blob/main/03_recalibrate.py): `03_recalibrate.py`
- [Off-policy evaluation](https://github.com/pscredefine/rankshift/blob/main/04_off_policy.py): `04_off_policy.py`
- [Neural baselines](https://github.com/pscredefine/rankshift/blob/main/05_two_tower.py): `05_two_tower.py`
- [Scale evaluation](https://github.com/pscredefine/rankshift/blob/main/06_scale_27k.py): `06_scale_27k.py`

### Recommender runtime

- [Two-stage experiment](https://github.com/pscredefine/rankshift/blob/main/07_two_stage.py): `07_two_stage.py`
- [Feature contracts](https://github.com/pscredefine/rankshift/blob/main/recsys/schema.py): `recsys/schema.py`
- [Feature builder](https://github.com/pscredefine/rankshift/blob/main/recsys/features.py): `recsys/features.py`
- [Candidate retrieval](https://github.com/pscredefine/rankshift/blob/main/recsys/models.py): `recsys/models.py`
- [Ranking models](https://github.com/pscredefine/rankshift/blob/main/recsys/models.py): `recsys/models.py`
- [Recommendation pipeline](https://github.com/pscredefine/rankshift/blob/main/recsys/pipeline.py): `recsys/pipeline.py`
- [Versioned bundles](https://github.com/pscredefine/rankshift/blob/main/recsys/pipeline.py): `recsys/pipeline.py`

### HTTP serving

- [Recommendation API](https://github.com/pscredefine/rankshift/blob/main/recsys/api.py): `recsys/api.py`

### Other components

- **Engineer**
- **KuaiRand logs**
- [Experiment results](https://github.com/pscredefine/rankshift/tree/main/results): `results`
- **API client**

## Connections

- Engineer → Signal probe: runs experiment (evidence: `README.md`)
- Engineer → Exposure evaluation: runs experiment (evidence: `README.md`)
- Engineer → Calibration study: runs experiment (evidence: `README.md`)
- Engineer → Off-policy evaluation: runs experiment (evidence: `README.md`)
- Engineer → Neural baselines: runs experiment (evidence: `README.md`)
- Engineer → Scale evaluation: runs experiment (evidence: `README.md`)
- Engineer → Two-stage experiment: runs experiment (evidence: `README.md`)
- KuaiRand logs → Signal probe: supplies logs
- KuaiRand logs → Exposure evaluation: supplies logs (evidence: `02_exposure_bias.py`)
- KuaiRand logs → Off-policy evaluation: supplies logs (evidence: `04_off_policy.py`)
- KuaiRand logs → Neural baselines: supplies logs (evidence: `05_two_tower.py`)
- KuaiRand logs → Two-stage experiment: supplies logs (evidence: `07_two_stage.py`)
- Signal probe → Experiment results: writes metrics (evidence: `README.md`)
- Exposure evaluation → Experiment results: writes metrics (evidence: `02_exposure_bias.py`)
- Calibration study → Experiment results: writes metrics (evidence: `README.md`)
- Off-policy evaluation → Experiment results: writes estimates (evidence: `README.md`)
- Neural baselines → Experiment results: writes metrics (evidence: `README.md`)
- Scale evaluation → Experiment results: writes metrics (evidence: `README.md`)
- Two-stage experiment → Experiment results: writes metrics (evidence: `README.md`)
- Two-stage experiment → Feature builder: builds features (evidence: `07_two_stage.py`)
- Two-stage experiment → Candidate retrieval: fits retriever (evidence: `07_two_stage.py`)
- Two-stage experiment → Ranking models: fits rankers (evidence: `07_two_stage.py`)
- Feature builder → Feature contracts: uses contracts (evidence: `recsys/features.py`)
- Candidate retrieval → Feature builder: uses encodings (evidence: `recsys/models.py`)
- Ranking models → Feature builder: uses encodings (evidence: `recsys/models.py`)
- API client → Recommendation API: requests recommendations (evidence: `recsys/api.py`)
- Recommendation API → Recommendation pipeline: loads and calls (evidence: `recsys/api.py`)
- Recommendation pipeline → Candidate retrieval: retrieves candidates (evidence: `recsys/pipeline.py`)
- Recommendation pipeline → Feature builder: builds request features (evidence: `recsys/pipeline.py`)
- Recommendation pipeline → Ranking models: scores candidates (evidence: `recsys/pipeline.py`)
- Recommendation API → Versioned bundles: loads bundle (evidence: `recsys/api.py`)
- Recommendation pipeline → Versioned bundles: saves or loads (evidence: `recsys/pipeline.py`)
