# im-xra-dev/the-algorithm architecture

> Architecture diagram of the GitHub repository im-xra-dev/the-algorithm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/im-xra-dev/the-algorithm
- Repository: https://github.com/im-xra-dev/the-algorithm

## Overview

This repository is a collection of services, jobs, models, and frameworks for constructing and serving Twitter’s Home Timeline. The principal product workflow is sourcing in-network and out-of-network candidates, ranking and filtering them, then mixing them into a timeline. The README documents major capabilities and some relationships; sampled source also shows internal behavior in candidate generation, feature hydration, and visibility evaluation. Several subsystem connections are documented only at a high level, so those links are dashed where implementation-level routing was not shown.

Main entry points: [`home-mixer`](https://github.com/im-xra-dev/the-algorithm/tree/main/home-mixer) (Home Mixer), [`cr-mixer`](https://github.com/im-xra-dev/the-algorithm/tree/main/cr-mixer) (CR Mixer), [`cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala`](https://github.com/im-xra-dev/the-algorithm/blob/main/cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala) (Unified Similarity Engine), [`visibilitylib`](https://github.com/im-xra-dev/the-algorithm/tree/main/visibilitylib) (Visibility Filters)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_timeline["Timeline Assembly"]
  node_home["Home Mixer"]
  node_timelinehydrator["Timeline Hydrator"]
end

subgraph group_candidates["Candidate Sources"]
  node_search["Search Index"]
  node_crmixer["CR Mixer"]
  node_uteg["User-Tweet Graph"]
  node_frs["Follow Recommendations"]
  node_unifiedengine["Unified Similarity Engine"]
end

subgraph group_signals["Features and Models"]
  node_simclusters["SimClusters ANN"]
  node_ann["ANN Library"]
  node_twhin["TwHIN Embeddings"]
  node_trustmodels["Safety Models"]
  node_realgraph["Real Graph"]
  node_graphfeatures["Graph Feature Service"]
  node_injector["Recos Injector"]
end

subgraph group_safety["Safety and Ranking"]
  node_light["Light Ranker"]
  node_heavy["Heavy Ranker"]
  node_visibility["Visibility Filters"]
  node_timelineranker["Timeline Ranker"]
  node_visibilityengine["Visibility Rule Engine"]
end

subgraph group_frameworks["Serving Frameworks"]
  node_productmixer["Product Mixer"]
  node_navi["Navi Serving"]
  node_twml["TWML Framework"]
  node_query["Pipeline Query"]
end

node_user(("Timeline requester"))

node_user -.->|"requests timeline"| node_home
node_home -->|"built on"| node_productmixer
node_home -.->|"sources candidates"| node_search
node_home -.->|"sources candidates"| node_crmixer
node_home -.->|"sources candidates"| node_uteg
node_home -.->|"sources candidates"| node_frs
node_crmixer -->|"coordinates retrieval"| node_unifiedengine
node_unifiedengine -->|"uses ANN candidates"| node_simclusters
node_unifiedengine -.->|"queries ANN service"| node_ann
node_crmixer -->|"retrieves candidates"| node_twhin
node_injector -->|"feeds graph inputs"| node_uteg
node_home -.->|"ranks candidates"| node_heavy
node_search -->|"ranks tweets"| node_light
node_home -.->|"filters content"| node_visibility
node_visibility -->|"evaluates rules"| node_visibilityengine
node_home -.->|"may use legacy ranking"| node_timelineranker
node_productmixer -->|"represents request"| node_query
node_home -->|"hydrates timeline tweets"| node_timelinehydrator
node_navi -.->|"serves models"| node_heavy
node_home -.->|"serves timeline"| node_user
node_trustmodels -.->|"supports safety checks"| node_visibility
node_realgraph -.->|"provides interaction signals"| node_home
node_graphfeatures -.->|"provides graph features"| node_home
node_simclusters -.->|"provides community signals"| node_home
node_twml -.->|"supports model workflows"| node_heavy

click node_home "https://github.com/im-xra-dev/the-algorithm/tree/main/home-mixer"
click node_productmixer "https://github.com/im-xra-dev/the-algorithm/tree/main/product-mixer"
click node_search "https://github.com/im-xra-dev/the-algorithm/tree/main/src/java/com/twitter/search"
click node_crmixer "https://github.com/im-xra-dev/the-algorithm/tree/main/cr-mixer"
click node_uteg "https://github.com/im-xra-dev/the-algorithm/tree/main/src/scala/com/twitter/recos/user_tweet_entity_graph"
click node_frs "https://github.com/im-xra-dev/the-algorithm/tree/main/follow-recommendations-service"
click node_simclusters "https://github.com/im-xra-dev/the-algorithm/tree/main/simclusters-ann"
click node_ann "https://github.com/im-xra-dev/the-algorithm/tree/main/ann/src/main/scala/com/twitter/ann"
click node_trustmodels "https://github.com/im-xra-dev/the-algorithm/tree/main/trust_and_safety_models"
click node_realgraph "https://github.com/im-xra-dev/the-algorithm/tree/main/src/scala/com/twitter/interaction_graph"
click node_graphfeatures "https://github.com/im-xra-dev/the-algorithm/tree/main/graph-feature-service"
click node_injector "https://github.com/im-xra-dev/the-algorithm/tree/main/recos-injector"
click node_light "https://github.com/im-xra-dev/the-algorithm/tree/main/src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird"
click node_visibility "https://github.com/im-xra-dev/the-algorithm/tree/main/visibilitylib"
click node_timelineranker "https://github.com/im-xra-dev/the-algorithm/tree/main/timelineranker"
click node_navi "https://github.com/im-xra-dev/the-algorithm/tree/main/navi/navi"
click node_twml "https://github.com/im-xra-dev/the-algorithm/tree/main/twml"
click node_timelinehydrator "https://github.com/im-xra-dev/the-algorithm/blob/main/home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala"
click node_unifiedengine "https://github.com/im-xra-dev/the-algorithm/blob/main/cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala"
click node_query "https://github.com/im-xra-dev/the-algorithm/blob/main/product-mixer/core/src/main/scala/com/twitter/product_mixer/core/pipeline/PipelineQuery.scala"
click node_visibilityengine "https://github.com/im-xra-dev/the-algorithm/blob/main/visibilitylib/src/main/scala/com/twitter/visibility/engine/VisibilityRuleEngine.scala"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_home,node_timelinehydrator toneBlue
class node_search,node_crmixer,node_uteg,node_frs,node_unifiedengine toneAmber
class node_simclusters,node_ann,node_twhin,node_trustmodels,node_realgraph,node_graphfeatures,node_injector,node_user toneMint
class node_light,node_heavy,node_visibility,node_timelineranker,node_visibilityengine toneRose
class node_productmixer,node_navi,node_twml,node_query toneIndigo
```

## Components

### Timeline Assembly

- [Home Mixer](https://github.com/im-xra-dev/the-algorithm/tree/main/home-mixer): `home-mixer`
- [Timeline Hydrator](https://github.com/im-xra-dev/the-algorithm/blob/main/home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala): `home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala`

### Candidate Sources

- [Search Index](https://github.com/im-xra-dev/the-algorithm/tree/main/src/java/com/twitter/search): `src/java/com/twitter/search`
- [CR Mixer](https://github.com/im-xra-dev/the-algorithm/tree/main/cr-mixer): `cr-mixer`
- [User-Tweet Graph](https://github.com/im-xra-dev/the-algorithm/tree/main/src/scala/com/twitter/recos/user_tweet_entity_graph): `src/scala/com/twitter/recos/user_tweet_entity_graph`
- [Follow Recommendations](https://github.com/im-xra-dev/the-algorithm/tree/main/follow-recommendations-service): `follow-recommendations-service`
- [Unified Similarity Engine](https://github.com/im-xra-dev/the-algorithm/blob/main/cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala): `cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala`

### Features and Models

- [SimClusters ANN](https://github.com/im-xra-dev/the-algorithm/tree/main/simclusters-ann): `simclusters-ann`
- [ANN Library](https://github.com/im-xra-dev/the-algorithm/tree/main/ann/src/main/scala/com/twitter/ann): `ann/src/main/scala/com/twitter/ann`
- **TwHIN Embeddings**
- [Safety Models](https://github.com/im-xra-dev/the-algorithm/tree/main/trust_and_safety_models): `trust_and_safety_models`
- [Real Graph](https://github.com/im-xra-dev/the-algorithm/tree/main/src/scala/com/twitter/interaction_graph): `src/scala/com/twitter/interaction_graph`
- [Graph Feature Service](https://github.com/im-xra-dev/the-algorithm/tree/main/graph-feature-service): `graph-feature-service`
- [Recos Injector](https://github.com/im-xra-dev/the-algorithm/tree/main/recos-injector): `recos-injector`

### Safety and Ranking

- [Light Ranker](https://github.com/im-xra-dev/the-algorithm/tree/main/src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird): `src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird`
- **Heavy Ranker**
- [Visibility Filters](https://github.com/im-xra-dev/the-algorithm/tree/main/visibilitylib): `visibilitylib`
- [Timeline Ranker](https://github.com/im-xra-dev/the-algorithm/tree/main/timelineranker): `timelineranker`
- [Visibility Rule Engine](https://github.com/im-xra-dev/the-algorithm/blob/main/visibilitylib/src/main/scala/com/twitter/visibility/engine/VisibilityRuleEngine.scala): `visibilitylib/src/main/scala/com/twitter/visibility/engine/VisibilityRuleEngine.scala`

### Serving Frameworks

- [Product Mixer](https://github.com/im-xra-dev/the-algorithm/tree/main/product-mixer): `product-mixer`
- [Navi Serving](https://github.com/im-xra-dev/the-algorithm/tree/main/navi/navi): `navi/navi`
- [TWML Framework](https://github.com/im-xra-dev/the-algorithm/tree/main/twml): `twml`
- [Pipeline Query](https://github.com/im-xra-dev/the-algorithm/blob/main/product-mixer/core/src/main/scala/com/twitter/product_mixer/core/pipeline/PipelineQuery.scala): `product-mixer/core/src/main/scala/com/twitter/product_mixer/core/pipeline/PipelineQuery.scala`

### Other components

- **Timeline requester**

## Connections

- Timeline requester → Home Mixer: requests timeline (evidence: `README.md`)
- Home Mixer → Product Mixer: built on (evidence: `README.md`)
- Home Mixer → Search Index: sources candidates (evidence: `README.md`)
- Home Mixer → CR Mixer: sources candidates (evidence: `README.md`)
- Home Mixer → User-Tweet Graph: sources candidates (evidence: `README.md`)
- Home Mixer → Follow Recommendations: sources candidates (evidence: `README.md`)
- CR Mixer → Unified Similarity Engine: coordinates retrieval (evidence: `README.md`)
- Unified Similarity Engine → SimClusters ANN: uses ANN candidates (evidence: `cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala`)
- Unified Similarity Engine → ANN Library: queries ANN service
- CR Mixer → TwHIN Embeddings: retrieves candidates (evidence: `cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala`)
- Recos Injector → User-Tweet Graph: feeds graph inputs (evidence: `README.md`)
- Home Mixer → Heavy Ranker: ranks candidates (evidence: `README.md`)
- Search Index → Light Ranker: ranks tweets (evidence: `README.md`)
- Home Mixer → Visibility Filters: filters content (evidence: `README.md`)
- Visibility Filters → Visibility Rule Engine: evaluates rules (evidence: `visibilitylib/src/main/scala/com/twitter/visibility/engine/VisibilityRuleEngine.scala`)
- Home Mixer → Timeline Ranker: may use legacy ranking (evidence: `README.md`)
- Product Mixer → Pipeline Query: represents request (evidence: `product-mixer/core/src/main/scala/com/twitter/product_mixer/core/pipeline/PipelineQuery.scala`)
- Home Mixer → Timeline Hydrator: hydrates timeline tweets (evidence: `home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala`)
- Navi Serving → Heavy Ranker: serves models (evidence: `README.md`)
- Home Mixer → Timeline requester: serves timeline (evidence: `README.md`)
- Safety Models → Visibility Filters: supports safety checks
- Real Graph → Home Mixer: provides interaction signals
- Graph Feature Service → Home Mixer: provides graph features
- SimClusters ANN → Home Mixer: provides community signals
- TWML Framework → Heavy Ranker: supports model workflows
