How grycap/oscar works
Project Type & Purpose
• OSCAR is an open-source platform that implements a serverless computing model for data-processing applications.
• It supports event-driven execution where file uploads trigger containerized jobs on an elastic Kubernetes cluster.
• The focus is on integrating various backends (including Knative, OpenFaaS, and a “fake” backend for tests), external storage (MinIO, Amazon S3, etc.), and providing a centralized management API (OSCAR Manager) plus a UI.
Understanding the File Structure
• Top-Level Directories:
• “pkg”: Contains core logic partitioned into modules for backends, handlers (for creating, deleting, status, etc.), resource management, and utility functions (including authentication with OIDC and multitenancy).
• “dashboard” and “ui”: Likely represent the OSCAR Manager’s interface and visual control.
• “deploy”: Contains scripts and configurations (Ansible playbooks and Kubernetes YAML files) for deploying OSCAR on clusters, indicating that deployment automation and orchestration are integral.
• “docs”: Houses comprehensive documentation, API docs, and architectural images (e.g., docs/images/oscar-components.png) that illustrate the OSCAR architecture.
• “examples”: Provides several example workflows demonstrating OSCAR’s integration with various use cases—this hints at extensibility and multiple entry points (sync, async, etc.).
• “tools” and “templates”: Support for building utility tools (for downloading binaries) and service templates, reinforcing the modular design.
• Other Notable Files:
• Dockerfiles in various locations emphasize containerization.
• Go source files (main.go and modules under pkg) highlight that the core system is implemented in Go.
• CI/CD pipelines and configuration files in .github suggest active integration and deployment practices.
Core Components
• OSCAR Manager:
• Acts as the centerpiece, managing service creation, updates, and deletion.
• Connects directly to the Kubernetes cluster where jobs are executed.
• Kubernetes Cluster:
• Deployed using deployment scripts (Ansible, kind, Helm, etc.) and orchestrates OSCAR components.
• Mention integration with Clues (elasticity manager) and IM (Infrastructure Manager) for dynamic infrastructure management.
• Backends Integration:
• Different serverless backends (Knative, OpenFaaS, and a “fake” backend for testing).
• Pkg/backends modules as interchangeable components that the OSCAR Manager interacts with.
• Data Storage:
• How file uploads trigger job execution and later the storage of results.
• User Interfaces & External Clients:
• OSCAR UI (web-based graphical interface)
• Possibly a CLI (as hinted in docs/oscar-cli.md) for user interactions.
• Event & Invocation Flow:
• Event triggers (file uploads or HTTP requests) that invoke synchronous or asynchronous containerized applications.
• Data flow—from user or external system to the OSCAR Manager and then to the Kubernetes-based execution.
Interactions and Data Flows
• From “User/Client” → “Storage Backend” → “Event Trigger” → “OSCAR Manager”
• Then, “OSCAR Manager” forwards tasks to appropriate “Backend” (Knative/OpenFaaS)
• Finally, "Jobs/Containers" interact with “MinIO” or other storage for input/output.
• API and control layers (for example, handlers in pkg/handlers) that process requests and manage job lifecycle (create, read, update, delete).
Architectural Patterns & Technologies
• Microservices and Modular Design: Show that services (handlers, backends, resource management) are decoupled, each having its own Go package.
• Containerization: Emphasize Docker usage and container orchestration via Kubernetes.
• Infrastructure as Code: Include deployment automation using Ansible, Kubernetes YAML files, and CI/CD workflows from the .github directory.