How matrix-org/synapse works
Based on both the file tree and the README, this project is Synapse—a Matrix homeserver implementation. Its purpose is to provide a federated messaging backend that implements the Matrix protocol. The project is composed of a large set of modules in Python (with some Rust components supporting performance-critical parts) that together provide API endpoints, federation, event persistence, state handling, and worker processes.
Major system components
• Homeserver Core – This is primarily the “synapse” package (e.g. synapse/app/homeserver.py) which initializes the server, coordinates request handling, and loads the configuration.
• API and Request Handling – Modules under “synapse/http”, “synapse/rest”, and “synapse/api” manage incoming client requests, authentication, and responses.
• Federation Components – The “synapse/federation” and “synapse/http/federation” directories contain code that handles inter-server communication, federation client/server endpoints, and message relay between homeservers.
• Database and Storage Layer – The “synapse/storage” package (along with “synapse/storage/schema”) is dedicated to data persistence. This layer abstracts support for PostgreSQL and SQLite, applies schema migrations, and handles queries for events, users, and room data.
• Background Processes and Workers – There is explicit support for worker processes (e.g. synapse/app/generic_worker.py, configurations under “synapse/config/workers”, and numerous definitions in both “contrib” and “docs/systemd-with-workers”). These components coordinate tasks such as federation sending, state updates, and media handling.
• Media Repository and File Storage – Found in “synapse/media”, this component manages file uploads, media storage, thumbnailing, and cache operations.
• Authentication and Authorization Modules – Organized under “synapse/api/auth”, “synapse/handlers/auth”, and related modules that handle secure logins, password policies, and session management.
• Replication Modules – Components under “synapse/replication” implement synchronization of events, state groups, and various data streams across processes or in federated scenarios.
• Utility and Cross-Cutting Concerns – Modules under “synapse/util”, “synapse/metrics”, and “synapse/logging” provide helper functions, caching, metrics collection, and logging context support.
Relationships and interactions
• Clients (via REST endpoints) interact with Synapse’s API layer which routes requests to authentication, room management, state resolution, etc.
• A reverse proxy (external to the project but strongly recommended by the README) sits in front of Synapse, handling TLS termination and routing external HTTP(S) requests.
• The federation layer sends and receives messages to/from other Matrix homeservers; it interacts with the same persistence and event processing subsystems as the client API.
• The storage layer is accessed broadly by API handlers, federation modules, and background workers to store and retrieve events and state. Consider showing this database as “PostgreSQL/SQLite” (external services) that Synapse communicates with.
• Worker components run in parallel with the main process, handling tasks such as event persistence, push notifications, media handling, and may also use dedicated databases or queues for inter-process communication.
• Authentication and identity integration with external Identity Servers is also part of the overall ecosystem.
Architectural patterns and design principles
• Modular, layered architecture is evident; clear separation exists between the API layer, federation layer, persistence layer, and backend workers.
• There is extensive use of configuration files (“synapse/config” and various YAML/INI files) and environment-specific scripts (e.g. Docker, Debian packaging in “docker/” and “debian/” directories) to support deployment flexibility.
• Emphasis is placed on scalability and robustness by splitting key functionalities into dedicated workers and by supporting replication.
• The design uses clear callback and module-loading mechanisms (see “synapse/module_api”) to offer extensibility points.
Relevant technologies, frameworks, and libraries
• Python is the main development language, supplemented by Rust components (under “rust/”) for performance-critical areas.
• PostgreSQL and SQLite are supported as backend databases.
• Various tools for testing (extensive “tests” directories), continuous integration (directories like “.ci”, “.github/workflows”), and containerization (Docker files in the “docker” directory).
• The project also provides support for external components (reverse proxies such as nginx, Apache, etc. as mentioned in the README).
Final advice
• Do not shy from showing multiple layers: the main server process, the separate worker processes, the storage layer, and how configuration and testing/CI support the project.
This approach should help you capture the essential architectural elements of Synapse, reflecting its capabilities as a robust, federated Matrix homeserver with numerous interdependent modules and deployment considerations.