How twitter/the-algorithm works
This repository is a collection of services, jobs, models, and shared frameworks for X recommendations. The principal workflow is constructing a For You feed: Home Mixer coordinates candidate retrieval, ranking, and filtering before serving a timeline; the README also documents Recommended Notifications. Shared data and model services support those surfaces. The source excerpts show parts of query transformation, feature hydration, similarity retrieval, and visibility evaluation; many service-to-service connections below are documented by the README rather than inspected source.
Main entry points: home-mixer Home Mixer · cr-mixer CR Mixer · pushservice Pushservice · search Search index
Components 19
Recommendation surfaces
- Home Mixer
home-mixer - Pushservice
pushservice
Candidate retrieval
- Tweet Mixer
tweet-mixer - Search index
src/java/com/twitter/search - User-tweet graph
src/scala/com/twitter/recos/user_tweet_graph - Follow recommendations
follow-recommendations-service - CR Mixer
cr-mixer - ANN query service
ann/src/main/scala/com/twitter/ann/service/query_server/common/QueryIndexThriftController.scala
Ranking and filtering
- Heavy ranker
- Light ranker
src/python/twitter/deepbird/projects/timelines/scripts/models/earlybird - Visibility filters
visibilitylib - Timeline Ranker
timelineranker
Shared data and models
- Post data
tweetypie - User signals
user-signal-service - SimClusters
src/scala/com/twitter/simclusters_v2 - Representation manager
representation-manager
Serving frameworks
- Product Mixer
product-mixer - Navi model serving
navi
Other components
- X user
Connections 20
- X user → Home Mixer: requests feed
- Home Mixer → Tweet Mixer: coordinates candidates
home-mixer/server/src/main/scala/com/twitter/home_mixer/model/HomeFeatures.scala - Home Mixer → Search index: gets in-network posts
- Home Mixer → User-tweet graph: gets graph candidates
- Home Mixer → Follow recommendations: gets recommendations
home-mixer/server/src/main/scala/com/twitter/home_mixer/model/HomeFeatures.scala - Home Mixer → Heavy ranker: ranks candidates
- Search index → Light ranker: ranks posts
- Home Mixer → Visibility filters: filters content
- Home Mixer → Timeline Ranker: gets relevance scores
home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/query_transformer/EditedTweetsCandidatePipelineQueryTransformer.scala - Home Mixer → Product Mixer: uses framework
home-mixer/server/src/main/scala/com/twitter/home_mixer/functional_component/query_transformer/EditedTweetsCandidatePipelineQueryTransformer.scala - Home Mixer → Post data: hydrates timeline posts
home-mixer/server/src/main/scala/com/twitter/home_mixer/product/for_you/feature_hydrator/TimelineServiceTweetsQueryFeatureHydrator.scala - Home Mixer → X user: serves timeline
- X user → Pushservice: receives notifications
- Pushservice → Light ranker: preselects candidates
- Pushservice → Heavy ranker: ranks notifications
- CR Mixer → ANN query service: queries ANN
- CR Mixer → SimClusters: retrieves similarity candidates
cr-mixer/server/src/main/scala/com/twitter/cr_mixer/similarity_engine/TweetBasedUnifiedSimilarityEngine.scala - CR Mixer → Representation manager: uses embeddings
- Home Mixer → User signals: uses user signals
- Home Mixer → Navi model serving: uses model serving