How openfreeenergy/alchemiscale works
Project Type and Purpose
• Type: A modular, full-stack scientific execution system (library + CLI + services)
• Purpose: Orchestrate and scale free-energy calculations across HPC, cloud, and volunteer platforms
File Structure (Top-Level Components)
• alchemiscale/cli.py + cli_utils.py
– Command-line interface entry point
• alchemiscale/base
– Core API definitions and client abstractions
• alchemiscale/compute
– Compute API, client, manager, service, settings → Schedules and runs jobs
• alchemiscale/interface
– External‐facing API and client (e.g., REST or gRPC)
• alchemiscale/storage
• alchemiscale/strategist
– Strategy automation service reads state and generates new experiments
• alchemiscale/security
– Authentication, authorization models
• alchemiscale/models.py
– Shared data models & schemas
• Settings & migrations
– Configuration plus versioned DB migrations
• Docker definitions
– alchemiscale-compute & alchemiscale-server containers
README & Docs
• Confirms high-throughput free energy workflows
• References HPC, cloud, Folding@Home integration
• Indicates RESTful API layers and CLI usage
Main Components & Interactions
a. Components
1. User & CLI
2. Interface Service (REST/gRPC)
3. Compute Service & Manager
4. Strategist Service
5. Storage Layer
– Object Store (sim files, results)
– State Store (job metadata)
6. Security Module (AuthN/Z)
7. External Execution Backends (HPC schedulers, cloud VMs, FAH)
8. Database (for state, migrations)
b. Relationships / Data Flow
• User invokes CLI → CLI uses Interface Client → Interface Service
• Interface Service authenticates user → stores job request in State Store
• Compute Manager polls State Store → dispatches tasks to external backends
• External backend runs simulations → uploads results to Object Store
• Compute Service updates State Store on completion
• Strategist Service subscribes to State Store changes → generates next tasks
• All components respect security policies via security.auth
c. Architectural Patterns
• Microservices style: distinct Interface, Compute, Strategist services
• Client/Server: base.client and clients in each module
• Event-driven or polling (Compute Manager / Strategist poll state store)
• Separation of concerns: storage vs compute vs orchestration
d. Key Technologies & Libraries
• Python 3, pyproject.toml with dependencies
• Flask/FastAPI or gRPC (in interface.api)
• Async job execution patterns
• Object store adapters (alchemiscale.storage.objectstore)
• State store adapters (e.g., Redis, RDBMS via statestore)
• Docker for containerization
Detailed Elements to Capture
• Layered view: Presentation (CLI), API, Orchestration, Storage, External Execution
• Shared modules (models.py, security) perched centrally to indicate reuse
• Configuration & migrations as a side module connected to State Store
• Docker-compose deployment grouping of compute & server