How mushfiq13/collabnote works
1. System shape
CollabNote is split into two runtime processes: a FastAPI API service under app/ and a separate Kafka consumer worker under consumer/. Supporting infrastructure is wired around them with PostgreSQL, MongoDB, Elasticsearch, Redis, Kafka, and Nginx, typically started together via docker-compose.yml.
2. API entrypoint and route surface
app/main.py is the HTTP entrypoint for the ASGI app. Based on the README, it exposes the main user-facing flows: authentication (/auth/signup, /auth/login), note creation (/notes), note retrieval (/notes/{id}), and search (/search).
3. Authentication and authorization boundary
app/auth.py encapsulates signup/login, password hashing, and JWT token handling. The auth flow is built on bcrypt/passlib for password storage and python-jose for Bearer tokens, with protected endpoints requiring Authorization: Bearer <token>.
4. Relational data path
app/database.py, app/models.py, and the Alembic tree define the PostgreSQL side of the system. The README makes clear this store is used for user account/profile data, while Alembic migrations manage schema evolution. This is the durable relational boundary.
5. Document storage path
app/mongodb.py represents the MongoDB integration, likely using Motor for async access. The main read/write note document lifecycle lives here: note creation stores the note document, and note reads fall back to MongoDB when Redis misses.
6. Search indexing path
app/elasticsearch.py is the search integration boundary. After a note is created, the note is indexed into Elasticsearch, which powers fuzzy search and ranked results with highlighting on GET /search?q=....
7. Caching path
app/redis_client.py is the Redis integration for hot note reads. The read flow is explicitly cache-first: GET /notes/{id} hits Redis for sub-2 ms responses when warm, and on cache miss it loads from MongoDB and populates the cache.
8. Event streaming and side effects
app/kafka_producer.py publishes domain events from the API, especially signup and create_note. These events are not the primary data source; they are the integration mechanism for asynchronous side effects and observability.
9. Background consumer
consumer/consumer.py is a separate worker that consumes Kafka events and writes them to MongoDB for logging/audit-style persistence. This is the main asynchronous branch of the architecture, distinct from the synchronous request/response path.
10. Validation and contract layer
app/schemas.py contains the Pydantic request/response models that define the API contract. Together with the database models, this separates external payload validation from persistence concerns.
11. Infrastructure and edge
nginx/nginx.conf indicates Nginx sits in front of the app as the load balancer/reverse proxy. docker-compose.yml suggests the full stack is containerized for local and deployment parity, with environment values supplied from .env.
12. Migrations, CI, and tests
alembic/ is the migration boundary for PostgreSQL schema changes. .github/workflows/ci.yml handles automated CI/CD, and tests/test_api.py appears to cover the acceptance flows described in the README, especially auth, note creation, search, and cache/read behavior.
Main entry points: main.py API app · docker-compose.yml Compose · database.py SQL store · mongodb.py MongoDB
Components 15
API service
FastAPI request path and supporting application boundaries
- API app · ASGI service exposing auth, notes, and search routes
app/main.py - Auth · Signup, login, password hashing, and bearer token handling
app/auth.py - Schemas · Request and response models for API validation
app/schemas.py - Producer · Publishes domain events such as signup and note creation
app/kafka_producer.py
Worker
Asynchronous Kafka consumer process
- Consumer · Background process consuming events and persisting audit-style data
consumer/consumer.py
Data layer
Primary persistence, cache, and search systems
- SQL store · Relational connection and user/account persistence
app/database.py - Models · Database entities and persistence mapping for relational data
app/models.py - MongoDB · Async note document persistence and read fallback
app/mongodb.py - Redis cache · Hot note read cache for low-latency lookups
app/redis_client.py - Search · Note indexing and fuzzy search with highlighting
app/elasticsearch.py
Infra
Deployment, proxy, migrations, CI, and tests
- Migrations · PostgreSQL schema evolution and version history
alembic/env.py - Nginx · Edge proxy in front of the application service
nginx/nginx.conf - Compose · Container composition for app, worker, and backing services
docker-compose.yml - CI · Automated validation workflow
.github/workflows/ci.yml - API tests · Acceptance coverage for auth, notes, search, and cache behavior
tests/test_api.py
Connections 26
- Nginx → API app: routes · Reverse proxy forwards HTTP traffic to the FastAPI app
- API app → Auth: auth · Login and signup flows use the auth boundary
- API app → Schemas: validates · Request and response payloads are validated through schemas
- API app → SQL store: users · Account and profile data use the relational store
- API app → MongoDB: notes · Note retrieval and storage use MongoDB as the document source
- API app → Redis cache: cache · Read path checks Redis before falling back to MongoDB
- API app → Search: search · Search endpoint queries Elasticsearch
- API app → Producer: events · API publishes domain events for async side effects
- Auth → SQL store: stores · User records and password hashes are persisted in PostgreSQL
- SQL store → Models: maps · ORM models define the relational schema boundary
- Migrations → SQL store: migrates · Migrations evolve the PostgreSQL schema
- Search → API app: results · Search results flow back to the API layer
- Redis cache → API app: hits · Warm cache responses serve note reads quickly
- MongoDB → API app: misses · Cache misses load note documents from MongoDB
- Producer → Consumer: streams · Kafka carries API-emitted domain events to the worker
- Consumer → MongoDB: writes · Worker persists consumed events into MongoDB
- Compose → Nginx: includes · Stack composition wires the reverse proxy into the deployment
- Compose → API app: runs · Compose starts the API service
- Compose → Consumer: runs · Compose starts the Kafka consumer worker
- Compose → SQL store: provisions · Compose brings up PostgreSQL
- Compose → MongoDB: provisions · Compose brings up MongoDB
- Compose → Search: provisions · Compose brings up Elasticsearch
- Compose → Redis cache: provisions · Compose brings up Redis
- Compose → Producer: provisions · Compose brings up Kafka
- CI → API tests: executes · CI runs the API test suite
- API tests → API app: covers · Tests exercise auth, note, search, and cache flows