How sergiocipher/lexora works
1. System shape
- Lexora is a two-tier application: a static browser frontend and a Python FastAPI backend.
- The backend is the orchestration layer that serves the UI, answers autocomplete requests, accepts search submissions, and coordinates persistence and caching.
2. Frontend boundary
- frontend/index.html, app.js, and style.css implement a single-page-style typeahead UI in vanilla JavaScript.
- The browser talks only to backend HTTP endpoints; it does not access Redis or MySQL directly.
- The UI also surfaces live operational feedback through metrics returned by the backend.
3. API and application entrypoint
- app/main.py defines the FastAPI app and the public routes.
- run.py is the runtime entrypoint used to start the service, with Uvicorn serving the backend.
- The HTTP surface is small and purpose-driven: suggestions, search submission, cache debug, metrics, and the root page.
4. Autocomplete read path
- GET /suggest is the main read path.
- A prefix is routed through a consistent hash ring to choose one Redis node.
- The backend checks Redis first; on a miss it falls back to MySQL, then repopulates cache.
- This makes Redis the latency-critical tier while MySQL remains the source of truth.
5. Distributed cache topology
- app/redis_client.py and app/consistent_hash.py define the cache routing layer.
- The project uses multiple Redis instances, started via docker-compose.yml, rather than a single cache node.
- Consistent hashing is the key architectural boundary here: it deterministically assigns prefixes to Redis nodes so the cache can scale horizontally and remain stable as nodes change.
6. Write-behind buffering path
- POST /search does not synchronously write every interaction to the database.
- Instead, the request is appended to an in-memory buffer managed in app/buffer.py.
- A background flush process periodically batches buffered updates into MySQL and refreshes related cache entries.
- This is the main durability/performance tradeoff in the system: fast ingestion with delayed persistence.
7. Persistence layer
- app/database.py holds SQLAlchemy engine/session setup, and app/models.py defines the MySQL-backed entities.
- MySQL is the durable store for the autocomplete dataset and buffered search activity.
- app/schemas.py defines the API contracts exchanged over HTTP, separating transport validation from persistence models.
8. Data ingestion / bootstrap
- app/loader.py loads the dataset from typeahed_dataset.csv into MySQL.
- This is a batch bootstrap path, separate from the live request path, and exists to pre-seed the suggestion corpus.
- In architecture terms, it is the offline data import boundary feeding the live system.
9. Runtime configuration
- app/config.py centralizes environment-based settings such as DB credentials, Redis ports, cache TTL, and buffer flush behavior.
- The system depends on runtime configuration for environment portability, especially because Redis is multi-node and MySQL is external.
10. Operational visibility
- The backend exposes /metrics for cache hits, misses, hit rate, and buffer size.
- /cache/debug provides observability into the consistent-hash routing decision for a prefix and the cache status on the selected node.
- These endpoints make the caching layer and write-behind buffer inspectable, which is important because both are distributed/async behaviors.
11. Test coverage
- tests/test_api.py, test_buffer.py, test_consistent_hash.py, and test_performance.py mirror the main architectural concerns.
- Tests validate HTTP behavior, buffering semantics, hashing correctness, and performance characteristics.
- That test split reflects the repo’s core boundaries: API, cache routing, batching, and throughput.
12. Infrastructure and deployment assumptions
- docker-compose.yml is used specifically for the Redis sidecar topology.
- The application still expects an externally available MySQL instance, so persistence is not containerized in the same way as cache.
- Overall, the architecture is a horizontally cacheable read path, an asynchronous write path, and a conventional relational source of truth.
Main entry points: main.py API Server · docker-compose.yml Redis Pool · app.js App Logic · config.py Config
Components 19
Frontend
Static browser UI
- Browser UI · Single-page typeahead interface in the browser
frontend/index.html - App Logic · Client-side interaction and API calls
frontend/app.js - Styles · UI presentation layer
frontend/style.css
Backend
FastAPI orchestration layer
- API Server · Defines HTTP routes and request flow
app/main.py - Runner · Starts the backend process
run.py - Contracts · HTTP validation and transport models
app/schemas.py - Config · Runtime configuration for DB, Redis, and buffers
app/config.py - Metrics · Exposes cache and buffer health signals
app/main.py - Cache Debug · Shows routing and selected-node cache state
app/main.py
Data Plane
Cache, buffer, and persistence
- Hash Ring · Maps prefixes to Redis nodes deterministically
app/consistent_hash.py - Redis Client · Selects and talks to the assigned Redis node
app/redis_client.py - Redis Pool · Horizontally scaled cache tier
docker-compose.yml - Write Buffer · Buffers search events before persistence
app/buffer.py - DB Layer · Engine and session management
app/database.py - Models · Persistence models for the dataset and buffered activity
app/models.py - MySQL · Durable relational store
Ops
Bootstrap and deployment
- Dataset Loader · Loads the CSV dataset into MySQL
app/loader.py - CSV Dataset · Offline corpus used to prime suggestions
typeahed_dataset.csv - Compose · Defines the Redis sidecar deployment
docker-compose.yml
Connections 22
- Browser UI → App Logic: uses · HTML shell loads the client logic
- App Logic → API Server: calls · Browser sends all API requests to the backend
- App Logic → Metrics: reads · UI surfaces live operational feedback
- Runner → API Server: starts · Uvicorn serves the FastAPI app
- API Server → Contracts: validates · Routes use request and response contracts
- API Server → Config: loads · Endpoints depend on runtime settings
- API Server → Redis Client: reads · Autocomplete requests consult cache routing
- API Server → Write Buffer: buffers · Search submissions are staged asynchronously
- API Server → DB Layer: persists · Batch flushes and cache repopulation use the DB layer
- API Server → Metrics: serves · Operational counters exposed over HTTP
- API Server → Cache Debug: serves · Debug endpoint exposes cache routing behavior
- Redis Client → Hash Ring: routes by · Prefix is assigned to a Redis node via consistent hashing
- Hash Ring → Redis Pool: selects · Deterministically picks the target cache node
- Redis Client → Redis Pool: accesses · Reads and repopulates the chosen Redis node
- Write Buffer → DB Layer: flushes to · Buffered updates are batched into MySQL
- DB Layer → Models: maps · SQLAlchemy layer operates over persistence entities
- Models → MySQL: stores · Entities are persisted in MySQL
- Dataset Loader → CSV Dataset: ingests · Bootstrap path reads the CSV corpus
- Dataset Loader → MySQL: loads · Seeder populates the durable store
- Compose → Redis Pool: defines · Deployment topology for multi-node Redis
- Config → Redis Pool: configures · Redis ports and cache settings are environment-driven
- Config → MySQL: configures · Database connectivity is environment-driven