How facebook/watchman works
Main Components
• Core Watcher Engine: Represent the heart of Watchman which includes the C++ core components (found in directories such as “watchman”, “fs”, “query”, “root”, “watcher”, “telemetry”, etc.). This engine is responsible for:
• Monitoring file system events (using various OS-specific mechanisms: inotify on Linux, FSEvents on macOS, Windows specific implementations, etc.)
• Collecting and processing events.
• Interfacing with the subscription and query subsystems.
• Client Interfaces: Represent the multiple language bindings (Python, Java, Rust, Node.js, Ruby) and the CLI tool (found under “watchman/cli”, “watchman/python”, “watchman/java”, “watchman/node”, “watchman/rust”, etc.). These interact with the core engine over IPC mechanisms.
• IPC and Communication Channels: The project uses domain sockets (UNIX domain sockets in “watchman/unixsocket” and similar for Windows with named pipes) for communication between the core file watcher and its clients.
• Subscription & Query Subsystems: These handle client requests such as subscribing to changes, querying file states, and returning results (visible in directories like “watchman/query” and “watchman/cmds”).
• Build & Packaging Infrastructure: The build system (CMake, fbcode_builder and associated manifest directories) orchestrates compilation and dependency management; CI pipelines ('.github/workflows') and packaging scripts ensure cross-platform support.
• Logging & Telemetry: Components for logging (e.g., “watchman/Logging.cpp”, “watchman/telemetry”) are responsible for recording runtime events and performance metrics.
• Integration and Testing Environment: Directories named “integration”, “tests”, and “build” include various integration tests, system validation scripts, and build packaging which ensure quality and cross-platform compatibility.
Relationships and Interactions
• The Core Watcher Engine continuously monitors for file system changes.
• When a change is detected, the Core Engine processes the event (using components from fs/, query/, and root/) and dispatches it via IPC to subscribed client interfaces.
• Clients (via libraries for Python/Java/Node/etc. and CLI tools) send queries and subscribe to events, which the engine processes and responds to.
• Logging and Telemetry systems gather statistics and information during operation, providing audit trails and insights into performance.
• The Build and Packaging components (CMake scripts, fbcode_builder, CI/CD workflows) ensure that each part of the system is built and packaged correctly for various platforms.
• Integration tests simulate real-file system operations and interactions among these components to validate correct communication and functionality.
Architectural Patterns and Design Principles
• Modularity: Clearly separated components (watcher engine, client interfaces, build system, logging, IPC) make the system easier to maintain.
• Client-Server Architecture: The core engine operates as a long-running service (server), while multiple language-specific clients access its functionality through standardized IPC mechanisms.
• Abstraction for Cross-Platform Support: Different directories focus on different OS-specific implementations (in “watchman/watcher” including files for inotify, kqueue, FSEvents, etc.), abstracted away from the client interface.
• Extensibility: The presence of well-organized manifests and integration tests indicates the need to easily add new language bindings and features over time.