How karthikrishna05/scalable-feed-system works
This Django service demonstrates pull, push, and hybrid social-feed delivery. A client reaches one of the feed endpoints: pull queries followed authors’ posts, push reads precomputed Redis post IDs and loads their posts, and hybrid combines that Redis feed with on-demand celebrity posts before sorting and serializing the result. New non-celebrity posts trigger asynchronous fan-out to followers’ Redis lists. The README documents the PostgreSQL, Redis, and Celery roles; the graph’s implementation links are limited to the provided excerpts and index.
Main entry points: views.py Push endpoint · tasks.py Fan-out task · models.py Social models · urls.py Feed routes
Components 14
Feed API
- Feed routes
feeds/urls.py - Pull endpoint
feeds/views.py - Push endpoint
feeds/views.py - Hybrid endpoint
feeds/views.py
Feed Delivery
- Post serializer
feeds/serializer.py
Async Fan-out
- Post-save signal
core/signals.py - Celery app
config/celery.py - Fan-out task
feeds/tasks.py - Celery worker
- Redis task queue
Data Stores
- Social models
core/models.py - PostgreSQL
- Redis feeds
Other components
- User client
Connections 23
- User client → Feed routes: requests feed
README.md - Feed routes → Pull endpoint: routes pull
feeds/urls.py - Feed routes → Push endpoint: routes push
feeds/urls.py - Feed routes → Hybrid endpoint: routes hybrid
feeds/urls.py - Pull endpoint → Social models: queries posts
feeds/views.py - Push endpoint → Redis feeds: reads post IDs
feeds/views.py - Push endpoint → Social models: loads posts
feeds/views.py - Hybrid endpoint → Redis feeds: reads feed IDs
feeds/views.py - Hybrid endpoint → Social models: pulls celebrity posts
feeds/views.py - Pull endpoint → Post serializer: serializes results
feeds/views.py - Push endpoint → Post serializer: serializes results
feeds/views.py - Hybrid endpoint → Post serializer: serializes results
feeds/views.py - Post-save signal → Social models: observes post saves
core/signals.py - Post-save signal → Fan-out task: queues task
core/signals.py - Celery app → Fan-out task: discovers tasks
config/celery.py - Celery worker → Fan-out task: executes task
README.md - Fan-out task → Social models: reads followers
feeds/tasks.py - Fan-out task → Redis feeds: writes feed lists
feeds/tasks.py - Celery app → Redis task queue: uses task queue
README.md - Redis task queue → Celery worker: delivers tasks
README.md - Pull endpoint → PostgreSQL: queries feed data
README.md - Hybrid endpoint → PostgreSQL: pulls celebrity posts
README.md - Push endpoint → PostgreSQL: loads post records
README.md