How absurdcoder/app_recovery_tool works
Project type and purpose
• This is a full-stack Django web application (“Dynamic Recovery Tool”) that provides a visual workflow builder and execution engine to automate data recovery and migration across multiple infrastructure systems.
• Key purpose: allow users to define recovery workflows via a UI, store them (as YAML and in the DB), and execute them against targets such as Kafka, MQ, databases, service buses, and file systems.
File structure
• Top level:
• manage.py, recovery_tool/ (project settings), workflow_builder/ (core app), workflow_yamls/ (persisted YAMLs), db.sqlite3
• start-project.sh, .gitignore, README.md
• In recovery_tool/:
• settings.py, urls.py, wsgi.py, asgi.py → Django configuration & entry points
• In workflow_builder/:
• models.py → WorkflowStep, WorkflowExecution, etc.
• views.py, urls.py, forms.py, admin.py → MVC/Web layer
• action_handlers.py, event_handlers.py, workflow_executor.py → execution logic & connectors
• migrations/ → DB schema evolution
• templates/workflow_builder/ → HTML templates for builder, execution status, logs
• templatetags/custom_filters.py → UI helper filters
• workflow_yamls/: example YAML files autogenerated from workflows
README for additional insights
• Confirms Django 4.2 + Python 3.8+, uses SQLite by default but configurable
• Features: visual builder, multiple infra connectors, YAML generation, real-time tracking, extensibility
• Lists supported event types and infra types (Kafka, MQ, DB, Service Bus, File System)
• Development notes: how to add new event or infra types by extending models.py and handlers
Main components
a. User Interface (Web Browser)
• UI templates + JavaScript for workflow builder, execution dashboard, real-time updates
b. Django Application Server
• ASGI/WSGI entrypoint → URL router → Views → Forms → Models
• Core “workflow_builder” app
c. Database
• Django ORM persistence (Workflow, WorkflowStep, Execution, Logs)
d. Workflow Executor Service
• workflow_executor.py orchestrating step execution
• action_handlers.py / event_handlers.py implementing connector logic
e. YAML Store
• filesystem folder “workflow_yamls/” holding YAML configs for versioning
f. External Infrastructure Systems
• Kafka cluster
• MQ (e.g., IBM MQ)
• SQL/NoSQL databases
• Service Bus (Azure/AWS)
• File System
Relationships and interactions
• Browser → HTTP(S) → Django URL Router → Views → ORM → DB
• View “Create Workflow” writes both DB and generates YAML file in workflow_yamls/
• “Execute Workflow” request → View triggers workflow_executor → reads workflow steps from DB or YAML → for each step:
• invokes corresponding handler in action_handlers/event_handlers
• handler communicates with external infra (via network protocols or SDKs)
• updates step status in DB
• Real-time execution status: Browser polls or uses WebSocket (ASGI) → Django → Views → DB → UI
• Admin interface (optional) interacting directly with Django Admin
Architectural patterns and design principles
• MVC (Model-View-Controller) for the web layer
• Modular “workflow_builder” app encapsulating UI, models, execution logic
• Plugin-style handlers: event_handlers and action_handlers for extensibility
• YAML as externalized configuration for portability and version control
• Separation of concerns: UI, orchestration engine, persistence, and connectors are distinct
Detailed elements to capture
• HTTP request flow from browser to Django views and back
• DB CRUD flow for workflows, steps, and execution logs
• Executor loop invoking handlers and updating status
• Per-step connections to external systems
• Real-time update mechanism (polling or WebSocket channel)