How fabriclabs/sensemaker works
Major Components
• Frontend:
– React components (under the “components” directory) and associated assets (in “assets”) form the interactive user interface.
– Redux actions and reducers (directories “actions” and “reducers”) managing state.
• Backend:
– Node.js server scripts including “scripts/node.js” and the several route definitions (in “routes”) that handle HTTP endpoints.
– Services (in “services”, for example “sensemaker.js”, email, fabric, etc.) implement business logic and interface with external systems.
• Data Layers:
– MySQL is used as the primary relational database (as indicated by the “knexfile.js”, migration files in “migrations”, and MySQL requirement).
– Redis (with its designated “redisActions.js” and “routes/redis”) is used for caching, queue management, and potentially session handling.
• External/Integration Layers:
– The Fabric Network is a key element. This is seen in both the README (“Fabric” section) and in integrations with external modules such as “@fabric/core”, “@fabric/http”, and “@fabric/hub”. This external network stream is used to collect data.
– Additional services (e.g., services for chat, email, blockchain actions in “actions/bitcoinActions.js”) might connect to external APIs or systems.
• Utility & Infrastructure:
– There are utility scripts (in “scripts”), helper functions (in “functions”), and tests (in “tests”) supporting operational tasks, build processes, and validations.
– Dockerfile, CI/CD configurations (.github/workflows, .travis.yml) and deployment scripts indicate a modern, containerized, and automated deployment strategy.
Relationships and Data Flows
• The end users interact with the Frontend (React UI). Their actions trigger Redux actions, which in turn may initiate API calls.
• The API calls are routed through the Node.js backend (via “routes” and “controllers”) where business logic is applied.
• Backend services (inside “services” and “actions”, plus possibly “functions”) perform data processing, especially the LLM-based pipeline and aggregate information from the Fabric Network.
• The backend communicates with the MySQL database for persistent storage (schema defined in migrations) and with Redis for caching/queue management.
• External integration with Fabric and possibly other networks (e.g., Discord, GitHub, blockchain via “bitcoinActions”) is maintained via dedicated service modules.
• Utility scripts and scheduled tasks (from “scripts”, “functions”, “migrations”, and “tests”) ensure system health and scalability.
Architectural Patterns and Key Technologies
• MVC / RESTful API Pattern: Routes serve as controllers, while the “actions” and “reducers” patterns follow a Redux-like state management approach.
• Microservice/Modular Services Design: The existence of many discrete “services” (sensemaker, fabric, email, stripe, etc.) and specialized routes shows modularity.
• Integration of LLM Pipeline: Multi-stage processing likely occurs across actions, services, and even through external processing (the “LLM Pipeline” in README).
• Technologies:
• Node.js (version 18.19.1) for the backend.
• React/Redux for the frontend.
• MySQL (8.3) and Redis Stack (7.2) for data and caching.
• Fabric Network libraries for data aggregation.
• Additional libraries (like the extensive Semantic UI library under “libraries/semantic”) for styling and UI behavior.
Instructing the Principal Software Engineer
1. Client Tier (UI components + state management)
2. Application Tier (Node.js API, route handling, business logic, LLM pipeline)
3. Data Tier (MySQL, Redis)
4. Integration Tier (Fabric Network, external services)
• User interactions initiated on the client and routed to the API.
• Backend processing which may call external services and then update the database/cache.
• Feedback loops such as asynchronous tasks (e.g., message queues via Redis) or notifications to the frontend.
• Do not shy away from breaking down the system into as many logical components as possible – separation of concerns is critical given the complexity of actions, routes, and services.