How openspug/spug works
1. Repository shape
- This is a split-stack project: spug_api is the backend control plane, and spug_web is a React single-page admin console.
- The product is an agentless operations platform: the backend talks to managed hosts primarily over SSH, while the browser talks to the backend over HTTP and WebSocket.
2. Backend framework boundary
- spug_api is a Django 2.2 application with the usual split between settings.py, urls.py, wsgi.py, and asgi.py.
- The presence of consumer/ and spug/routing.py shows that the backend also serves real-time channels for terminal/output streams, not just REST APIs.
3. Core domain subsystems
- The backend is organized into Django apps by domain: host, exec, deploy, schedule, monitor, alarm, config, repository, file, account, setting, notify, and home.
- Each app typically contains its own models.py, views.py, urls.py, and supporting utils.py or helper modules, so the architecture is modular but still monolithic at deploy time.
4. Host management and SSH operations
- apps/host is the inventory and grouping layer: host CRUD, group management, import/sync, and host detail views.
- Low-level host interaction is centralized in libs/ssh.py, libs/channel.py, and related helpers, which are used by terminal, file transfer, command execution, and deployment flows.
5. Interactive execution and file transfer
- apps/exec implements batch command execution and file transfer workflows.
- It has dedicated background execution pieces (executors.py, transfer.py) and a worker command (management/commands/runworker.py), indicating asynchronous job execution with streamed output back to the UI.
6. Deployment and repository management
- apps/repository manages source repositories and repository metadata.
- apps/deploy coordinates application release flows, including deployment requests, helper utilities, and workflow/state logic.
- libs/gitlib.py and filesystem paths under spug_api/repos/ suggest local checkout/build staging is part of the deployment pipeline.
7. Scheduling and monitoring loops
- apps/schedule and apps/monitor each have their own scheduler/executor modules plus dedicated startup commands (runscheduler.py, runmonitor.py).
- This implies separate background processes for periodic task orchestration and monitoring checks rather than running everything in the web request process.
8. Alerting and notifications
- apps/alarm models alarm rules, and apps/notify is the delivery/notification layer.
- libs/mail.py and libs/push.py show external notification channels, which likely receive events from monitoring, scheduling, or deployment failures.
9. Configuration center
- apps/config manages application/environment/service configuration, including diffing, version/history, and rendering helpers.
- On the frontend this is one of the larger feature areas, which matches the backend being a distinct domain with its own data model and views.
10. Identity, permissions, and system settings
- apps/account and apps/setting handle users, roles, permissions, login history, security settings, and integration settings.
- libs/ldap.py indicates optional LDAP integration for enterprise authentication.
- This layer is the main authorization boundary for all other modules.
11. Frontend structure
- spug_web mirrors backend domains one-for-one in src/pages/, with separate page folders for host, exec, deploy, schedule, monitor, alarm, config, system, and ssh.
- Shared UI composition lives in src/layout/ and reusable widgets in src/components/.
- src/libs/http.js is the API client layer, and src/routes.js defines the navigation structure.
12. Client-server interaction pattern
- Most pages use page-local store.js files, suggesting each feature page owns its own data fetching and state wiring.
- The UI appears to load data via REST endpoints and subscribe to live execution/terminal updates via WebSocket where needed.
13. Runtime/process model
- The backend is not a single process: there are explicit startup scripts for API, worker, scheduler, monitor, and WebSocket services under spug_api/tools/.
- Docker and supervisor configs in docs/docker/ and tools/supervisor-spug.ini show the app is designed to run as multiple cooperating services behind Nginx and Redis.
14. Storage and operational boundaries
- Persistent data is split between the database-backed Django models, local filesystem areas for logs/repo/build artifacts/transfer files, and external systems like SSH targets and notification providers.
- spug_api/logs/, spug_api/repos/, and spug_api/storage/transfer/ indicate that runtime artifacts are intentionally kept outside the request path.
15. Architecture summary for a diagram
- A good diagram would show: browser SPA → Django API → domain apps/models → background workers/schedulers/monitors → SSH-managed hosts and external services.
- Add a separate real-time path from backend consumers/WebSocket routing back to the browser for terminal output, exec logs, and task progress.
Main entry points: apps Domain apps · exec Exec · deploy Deploy · libs Core libs
Components 22
Backend
Django control plane, domain apps, and runtime workers.
- Django app · Main Django project wiring settings, URLs, WSGI, ASGI, and routing.
spug_api/spug - URL map · Top-level request routing into domain apps and APIs.
spug_api/spug/urls.py - Settings · Global Django and deployment configuration.
spug_api/spug/settings.py - Domain apps · Django apps for host, exec, deploy, schedule, monitor, alarm, config, repository, file, account, setting, notify, home, app, and APIs.
spug_api/apps - Hosts · Host inventory, grouping, import/sync, and host detail operations.
spug_api/apps/host - Exec · Batch command execution and file transfer workflows with streamed output.
spug_api/apps/exec - Deploy · Deployment requests, workflow coordination, and release state handling.
spug_api/apps/deploy - Config · Application, environment, and service configuration management with versioning and diffs.
spug_api/apps/config - Schedule/Monitor · Periodic scheduling and monitoring loops with dedicated executors and startup commands.
spug_api/apps/schedule - Alarm/Notify · Alarm rules and notification delivery hooks.
spug_api/apps/alarm - Identity/Settings · Users, roles, permissions, security settings, login history, and integration settings.
spug_api/apps/account - Core libs · Shared SSH, git, LDAP, mail, push, channel, parser, and middleware helpers.
spug_api/libs
Frontend
React SPA and shared UI layers.
- SPA · Browser-based operations console for all major product domains.
spug_web/src/App.js - Routes · Declares client-side navigation and page entry points.
spug_web/src/routes.js - Layout · Shared shell for header, sidebar, notifications, and page framing.
spug_web/src/layout/index.js - HTTP client · Central browser-to-backend request layer.
spug_web/src/libs/http.js - Feature pages · Domain-oriented page folders for host, exec, deploy, schedule, monitor, alarm, config, system, ssh, and home.
spug_web/src/pages
Runtime
Background processes and real-time channels.
- WebSocket consumer · Terminal and output streaming handlers for browser-connected live updates.
spug_api/consumer/consumers.py - Workers · Separate runtime processes for API, worker, scheduler, monitor, and WebSocket services.
spug_api/tools - Repo storage · Local checkout, build, logs, and transfer storage used by repository and deployment flows.
spug_api/repos
External systems
Managed hosts and third-party services the platform integrates with.
- Managed hosts · Remote machines operated through SSH rather than an installed agent.
- External services · Redis, LDAP, email/push providers, and other integrated services.
Connections 29
- SPA → Routes: navigates · Client-side route structure drives the SPA.
- SPA → Layout: uses · Pages render inside the shared application shell.
- SPA → HTTP client: calls API · All feature pages communicate with backend endpoints through the HTTP client.
- Routes → Feature pages: maps to · Navigation points into domain page modules.
- Layout → Feature pages: hosts · Page content is framed by common UI chrome.
- Feature pages → HTTP client: fetches data · Feature pages load and mutate backend state through the API layer.
- Django app → URL map: routes · Top-level Django URL configuration dispatches requests to apps.
- Django app → Settings: configures · Project entrypoint loads framework settings.
- URL map → Domain apps: dispatches to · Requests are routed into domain apps and API modules.
- Domain apps → Hosts: contains · Host inventory and grouping features.
- Domain apps → Exec: contains · Command execution and transfer features.
- Domain apps → Deploy: contains · Deployment workflow features.
- Domain apps → Config: contains · Configuration management features.
- Domain apps → Schedule/Monitor: contains · Scheduling and monitoring features.
- Domain apps → Alarm/Notify: contains · Alerting and notification features.
- Domain apps → Identity/Settings: contains · Identity, permissions, and system settings.
- Hosts → Core libs: uses · Host operations rely on shared SSH and helper code.
- Exec → WebSocket consumer: streams · Execution and transfer output is pushed over real-time channels.
- Exec → Core libs: uses · Batch execution and file transfer depend on SSH/channel helpers.
- Deploy → Repo storage: stages in · Deployment work uses local repository/build artifacts.
- Deploy → Core libs: uses · Release workflows rely on git and filesystem helpers.
- Config → Core libs: uses · Config rendering and comparison reuse shared helpers.
- Schedule/Monitor → Workers: runs as · Scheduling and monitoring are executed by dedicated background processes.
- WebSocket consumer → Workers: served by · WebSocket endpoints run as a separate service process.
- Identity/Settings → External services: auth/integrates · Authentication, login, and security settings integrate with external providers.
- Alarm/Notify → External services: notifies via · Alerts can be delivered through mail or push integrations.
- Exec → Managed hosts: executes on · Commands and transfers target managed hosts over SSH.
- Deploy → Managed hosts: deploys to · Release flows operate against remote hosts.
- Hosts → Managed hosts: tracks · Inventory records represent managed hosts.