How fractal-analytics-platform/fractal-server works
Main Components
a. Core Server Components:
• FastAPI Application Layer (evident from the app structure)
• Authentication System (auth routes and security modules)
• Database Layer (SQLAlchemy ORM, based on models structure)
• Task Runner System (with multiple executors)
• API Versioning (v1 and v2 endpoints)
b. External Interfaces:
• REST API Endpoints
• OAuth Integration
• SLURM Integration
• SSH Connectivity
• Database Connection
Component Relationships
Top Level (Client Interface):
• REST API (v1, v2) <-> Authentication Layer
Middle Level (Business Logic):
• Route Handlers <-> Task Management <-> Runner System
Bottom Level (Infrastructure):
• Database <-> Models
• File System <-> Task Executors
• External Services (SLURM, SSH)
Specific Implementation Details to Highlight
a. API Structure (fractal_server/app/routes/):
• Separate v1 and v2 API versions
• Admin, auth, and core API routes
• Task collection endpoints
b. Task Execution System (fractal_server/app/runner/):
• Local executor
• SLURM executor
• SSH integration
• Task lifecycle management
c. Data Models (fractal_server/app/models/):
• Project structure
• Workflow management
• Task definitions
• User management
Key Workflows to Illustrate
a. Task Execution Flow:
1. API Request
2. Authentication
3. Task Validation
4. Runner Selection
5. Execution
6. Status Updates
7. Result Storage
b. Data Processing Pipeline:
1. Input Reception
2. Task Distribution
3. Parallel Processing
4. Result Collection
5. Storage Management
Important Design Patterns
• Repository Pattern (for database access)
• Factory Pattern (for runner creation)
• Strategy Pattern (for different execution methods)
• Observer Pattern (for task status updates)
Technical Details
• PostgreSQL Database
• FastAPI Framework
• Alembic Migrations
• OAuth2 Authentication
• SLURM Integration
• SSH Connectivity
• Task Queue System
Infrastructure Layer (execution environment)
• Component types
• Connection types (sync/async)
• Data flow directions
• Security boundaries
• Version distinctions (v1/v2)
This architecture should emphasize the system's ability to handle:
• Parallel processing
• Multiple execution environments
• Versioned APIs
• Security requirements
• Scalability concerns
Remember to maintain a balance between detail and clarity, focusing on the key architectural decisions while avoiding overwhelming technical minutiae.