# twitter/the-algorithm architecture

> Architecture diagram of the GitHub repository twitter/the-algorithm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/twitter/the-algorithm
- Repository: https://github.com/twitter/the-algorithm

## Overview

This repository is a collection of services, jobs, models, and shared frameworks for X recommendations. The principal workflow is constructing a For You feed: Home Mixer coordinates candidate retrieval, ranking, and filtering before serving a timeline; the README also documents Recommended Notifications. Shared data and model services support those surfaces. The source excerpts show parts of query transformation, feature hydration, similarity retrieval, and visibility evaluation; many service-to-service connections below are documented by the README rather than inspected source.

Main entry points: [`home-mixer`](https://github.com/twitter/the-algorithm/tree/main/home-mixer) (Home Mixer), [`cr-mixer`](https://github.com/twitter/the-algorithm/tree/main/cr-mixer) (CR Mixer), [`pushservice`](https://github.com/twitter/the-algorithm/tree/main/pushservice) (Pushservice), [`src/java/com/twitter/search`](https://github.com/twitter/the-algorithm/tree/main/src/java/com/twitter/search) (Search index)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_surfaces["Recommendation surfaces"]
  node_home["Home Mixer"]
  node_push["Pushservice"]
end

subgraph group_candidates["Candidate retrieval"]
  node_mixer["Tweet Mixer"]
  node_search["Search index"]
  node_user_tweet_graph[("User-tweet graph")]
  node_follow["Follow recommendations"]
  node_cr_mixer["CR Mixer"]
  node_ann["ANN query service"]
end

subgraph group_ranking["Ranking and filtering"]
  node_heavy_ranker["Heavy ranker"]
  node_light_ranker["Light ranker"]
  node_visibility["Visibility filters"]
  node_timeline_ranker["Timeline Ranker"]
end

subgraph group_shared["Shared data and models"]
  node_tweet_data[("Post data")]
  node_signals["User signals"]
  node_simclusters["SimClusters"]
  node_representation["Representation manager"]
end

subgraph group_frameworks["Serving frameworks"]
  node_product_mixer["Product Mixer"]
  node_navi["Navi model serving"]
end

node_user(("X user"))

node_user -->|"requests feed"| node_home
node_home -->|"coordinates candidates"| node_mixer
node_home -->|"gets in-network posts"| node_search
node_home -->|"gets graph candidates"| node_user_tweet_graph
node_home -->|"gets recommendations"| node_follow
node_home -->|"ranks candidates"| node_heavy_ranker
node_search -->|"ranks posts"| node_light_ranker
node_home -->|"filters content"| node_visibility
node_home -->|"gets relevance scores"| node_timeline_ranker
node_home -->|"uses framework"| node_product_mixer
node_home -->|"hydrates timeline posts"| node_tweet_data
node_home -->|"serves timeline"| node_user
node_user -->|"receives notifications"| node_push
node_push -->|"preselects candidates"| node_light_ranker
node_push -->|"ranks notifications"| node_heavy_ranker
node_cr_mixer -.->|"queries ANN"| node_ann
node_cr_mixer -.->|"retrieves similarity candidates"| node_simclusters
node_cr_mixer -.->|"uses embeddings"| node_representation
node_home -.->|"uses user signals"| node_signals
node_home -.->|"uses model serving"| node_navi

click node_home "https://github.com/twitter/the-algorithm/tree/main/home-mixer"
click node_push "https://github.com/twitter/the-algorithm/tree/main/pushservice"
click node_mixer "https://github.com/twitter/the-algorithm/tree/main/tweet-mixer"
click node_search "https://github.com/twitter/the-algorithm/tree/main/src/java/com/twitter/search"
click node_user_tweet_graph "https://github.com/twitter/the-algorithm/tree/main/src/scala/com/twitter/recos/user_tweet_graph"
click node_follow "https://github.com/twitter/the-algorithm/tree/main/follow-recommendations-service"
click node_light_ranker "https://github.com/twitter/the-algorithm/tree/main/src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird"
click node_visibility "https://github.com/twitter/the-algorithm/tree/main/visibilitylib"
click node_timeline_ranker "https://github.com/twitter/the-algorithm/tree/main/timelineranker"
click node_tweet_data "https://github.com/twitter/the-algorithm/tree/main/tweetypie"
click node_signals "https://github.com/twitter/the-algorithm/tree/main/user-signal-service"
click node_simclusters "https://github.com/twitter/the-algorithm/tree/main/src/scala/com/twitter/simclusters_v2"
click node_representation "https://github.com/twitter/the-algorithm/tree/main/representation-manager"
click node_product_mixer "https://github.com/twitter/the-algorithm/tree/main/product-mixer"
click node_navi "https://github.com/twitter/the-algorithm/tree/main/navi"
click node_cr_mixer "https://github.com/twitter/the-algorithm/tree/main/cr-mixer"
click node_ann "https://github.com/twitter/the-algorithm/blob/main/ann/src/main/scala/com/twitter/ann/service/query_server/common/QueryIndexThriftController.scala"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_home,node_push,node_user toneBlue
class node_mixer,node_search,node_user_tweet_graph,node_follow,node_cr_mixer,node_ann toneAmber
class node_heavy_ranker,node_light_ranker,node_visibility,node_timeline_ranker toneMint
class node_tweet_data,node_signals,node_simclusters,node_representation toneRose
class node_product_mixer,node_navi toneIndigo
```

## Components

### Recommendation surfaces

- [Home Mixer](https://github.com/twitter/the-algorithm/tree/main/home-mixer): `home-mixer`
- [Pushservice](https://github.com/twitter/the-algorithm/tree/main/pushservice): `pushservice`

### Candidate retrieval

- [Tweet Mixer](https://github.com/twitter/the-algorithm/tree/main/tweet-mixer): `tweet-mixer`
- [Search index](https://github.com/twitter/the-algorithm/tree/main/src/java/com/twitter/search): `src/java/com/twitter/search`
- [User-tweet graph](https://github.com/twitter/the-algorithm/tree/main/src/scala/com/twitter/recos/user_tweet_graph): `src/scala/com/twitter/recos/user_tweet_graph`
- [Follow recommendations](https://github.com/twitter/the-algorithm/tree/main/follow-recommendations-service): `follow-recommendations-service`
- [CR Mixer](https://github.com/twitter/the-algorithm/tree/main/cr-mixer): `cr-mixer`
- [ANN query service](https://github.com/twitter/the-algorithm/blob/main/ann/src/main/scala/com/twitter/ann/service/query_server/common/QueryIndexThriftController.scala): `ann/src/main/scala/com/twitter/ann/service/query_server/common/QueryIndexThriftController.scala`

### Ranking and filtering

- **Heavy ranker**
- [Light ranker](https://github.com/twitter/the-algorithm/tree/main/src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird): `src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird`
- [Visibility filters](https://github.com/twitter/the-algorithm/tree/main/visibilitylib): `visibilitylib`
- [Timeline Ranker](https://github.com/twitter/the-algorithm/tree/main/timelineranker): `timelineranker`

### Shared data and models

- [Post data](https://github.com/twitter/the-algorithm/tree/main/tweetypie): `tweetypie`
- [User signals](https://github.com/twitter/the-algorithm/tree/main/user-signal-service): `user-signal-service`
- [SimClusters](https://github.com/twitter/the-algorithm/tree/main/src/scala/com/twitter/simclusters_v2): `src/scala/com/twitter/simclusters_v2`
- [Representation manager](https://github.com/twitter/the-algorithm/tree/main/representation-manager): `representation-manager`

### Serving frameworks

- [Product Mixer](https://github.com/twitter/the-algorithm/tree/main/product-mixer): `product-mixer`
- [Navi model serving](https://github.com/twitter/the-algorithm/tree/main/navi): `navi`

### Other components

- **X user**

## Connections

- X user → Home Mixer: requests feed
- Home Mixer → Tweet Mixer: coordinates candidates (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/model/HomeFeatures.scala`)
- Home Mixer → Search index: gets in-network posts
- Home Mixer → User-tweet graph: gets graph candidates
- Home Mixer → Follow recommendations: gets recommendations (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/model/HomeFeatures.scala`)
- Home Mixer → Heavy ranker: ranks candidates
- Search index → Light ranker: ranks posts
- Home Mixer → Visibility filters: filters content
- Home Mixer → Timeline Ranker: gets relevance scores (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/query_transformer/EditedTweetsCandidatePipelineQueryTransformer.scala`)
- Home Mixer → Product Mixer: uses framework (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/query_transformer/EditedTweetsCandidatePipelineQueryTransformer.scala`)
- Home Mixer → Post data: hydrates timeline posts (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/product/for_you/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala`)
- Home Mixer → X user: serves timeline
- X user → Pushservice: receives notifications
- Pushservice → Light ranker: preselects candidates
- Pushservice → Heavy ranker: ranks notifications
- CR Mixer → ANN query service: queries ANN
- CR Mixer → SimClusters: retrieves similarity candidates (evidence: `cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala`)
- CR Mixer → Representation manager: uses embeddings
- Home Mixer → User signals: uses user signals
- Home Mixer → Navi model serving: uses model serving
