How kms7530/salsa works
Main Components
• API Layer & Deployment:
• api.py – Implements a FastAPI server exposing the service.
• service.py and BentoML-related files (bentofile.yaml, models folder) – Used for deploying models with BentoML.
• Inference & Model Services:
• inferences folder contains wrappers for VLM, OCR, and LLM (files like lang.py, ocr.py, vision_lang.py).
• models folder holds model service code to be used by BentoML.
• Preprocessing:
• preprocess folder – Contains scripts for data pre-processing for images, videos, and prompt generation.
• Pipeline & Orchestration:
• pipelines folder – Manages pipeline execution (e.g., video_section.py) and orchestrates components.
• Code Parsing & Generation:
• parser folder – Includes a code generator and parser (code_generator.py, code_parser.py) to process BentoML-related code.
• Configuration & Prompts:
• config folder – Houses configuration files (bentofile.yaml, config_template.py).
• prompts folder – Contains prompt templates using Jinja for text generation.
Relationships and Interactions
• The user interacts with the system primarily via two deployment approaches:
• FastAPI (api.py) for real-time API requests.
• BentoML service (service.py along with bentofile.yaml) for model serving.
• When a request is received:
• The API may trigger a pipeline (from the pipelines folder) that uses preprocess modules to prepare data.
• Inferences are then performed via the sub-modules in the inferences folder and models deployed via BentoML.
• The Parser module (in the parser folder) is used for generating or parsing code (Bako code), potentially as part of the model service orchestration.
• The configuration files in the config folder and prompt templates in the prompts folder customize behavior at runtime or during deployment.
Architectural Patterns and Design Principles
• Separation of Concerns:
• Each functionality (preprocessing, inference, deployment, pipelining, and parsing) is isolated in its own directory.
• Modular Design:
• Components such as the API, model inferences, and preprocessing can be updated independently.
• Deployment Flexibility:
• Two deployment approaches (FastAPI and BentoML) highlight the system’s flexibility in serving models.
• Extensibility:
• Pipelines and parser modules indicate that additional functionalities or new services (e.g., new inference modules or processing stages) can be integrated without major changes to the architecture.
Additional Details
• Depicting potential interactions not only during runtime (data flow) but also during development (code generation/parsing by the parser module).