How netflix/atlas works
Project type and purpose
• This is a multi-module Scala backend project (“Atlas”) for managing dimensional time-series data.
• It provides:
• A core engine (data model, query language, algorithms)
• Libraries for JSON serialization, chart rendering, expression evaluation, event streaming, Postgres integration
• Deployable services (web API, standalone server)
• Framework integrations (Spring, Pekko)
• Benchmarks and testkits
High-level architecture
• Client/API Tier
• Service/API Tier
• Core/Library Tier
• Data/External Tier
Main components
a. Client/API Tier
• HTTP/REST and WebSocket clients
• (Optional) Spring-based applications
b. Service/API Tier
• atlas-webapi (GraphApi, ExprApi, TagsApi, PublishApi)
• atlas-standalone (embedded WebServer)
• atlas-spring-webapi, atlas-spring-lwcapi, atlas-spring-lwc-events (Spring Boot wrappers)
• atlas-pekko (PekkoHttp server, actors, directives)
c. Core/Library Tier
• atlas-core
– model (Query, Datapoint, TimeSeries)
– algorithm (online, rolling aggregations)
– db interfaces (MemoryDatabase, StaticDatabase)
– index (TagIndex implementations)
• atlas-postgres (PostgresDatabase, PostgresService, TagIndex)
• atlas-json (JsonSupport, parser, serializers)
• atlas-chart (ChartEngine, graphing primitives, palette resources)
• atlas-eval (expression evaluator, streaming flows, grapher)
• atlas-lwc-events (LwcEventClient, DatapointEvent, subscriptions)
• atlas-lwcapi (WebSocketSessionManager, StreamsApi, SubscriptionManager)
d. External/Data Tier
• PostgreSQL database
• (Optional) static files (charts palettes)
• (Optional) external metrics stores or LWC data sources
Relationships & data flows
• Clients → atlas-webapi / atlas-standalone (HTTP + WebSocket)
• WebAPI controllers → core engine
• ExprApi → atlas-core.model + atlas-core.algorithm → database (memory or Postgres)
• GraphApi → atlas-eval → atlas-chart → PNG/JSON output
• PublishApi → atlas-core.db (AggregateCollector) → optionally push to event streams
• TagsApi → atlas-core.index → return tag lists
• atlas-lwcapi → manages WebSocket subscriptions, invokes atlas-lwc-events clients under the hood to push DatapointEvents
• atlas-postgres → implements database layer for persistent time-series and tag index
• atlas-json → used by all APIs & libraries to serialize/deserialize payloads
Architectural patterns & design principles
• Layered (API → Service → Core → Persistence)
• Modular: each concern (charting, JSON, evaluation, events, persistence) lives in its own artifact
• Dependency injection: Spring modules wire core libraries into a Boot app
• Actor model: atlas-pekko uses Pekko actors for async HTTP, WebSocket handling
Final checklist
• [ ] All major modules/services labeled
• [ ] Notes on protocols (HTTP, WebSocket, JSON, JDBC)
• [ ] Detail sub-components where they drive architecture (APIs, engines, storage)