# joachimz/dsba-platform architecture

> Architecture diagram of the GitHub repository joachimz/dsba-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/joachimz/dsba-platform
- Repository: https://github.com/joachimz/dsba-platform

## Overview

##### Project Type and Purpose
- The DSBA Platform is a toy MLOps platform meant for educational purposes. It supports various operations like data ingestion, preprocessing, model training, evaluation, prediction, and managing a model registry. It further supports multiple interfaces – a CLI, an API (with Docker support), and interactive notebooks for experimentation.

##### Key Components
a. Core DSBA Library: The main business logic is under the "src/dsba" directory.
- Components include:
- Data Ingestion (modules for files and databases)
- Preprocessing
- Model Training, Evaluation, Prediction
- Model Registry for handling trained models
b. API Component: Under "src/api", including its Dockerfile which indicates containerized deployment for exposing functionality as a service.
c. CLI Component: Located in "src/cli", exposing commands like model listing and predictions.
d. Notebooks: Under "src/notebooks" providing interactive examples (e.g., model_training_example.ipynb) to demonstrate usage.
e. Tests: A dedicated "tests" folder with unit and integration tests to verify functionality.

##### Relationships and Interactions
- The CLI and API both interface with the DSBA core library. They invoke functionalities (e.g., training, prediction, registry management) offered by the library.
- The CLI can be visualized as a command-line front-end that directly calls DSBA functions.
- The API (possibly containerized via Docker) exposes endpoints which internally call the DSBA library functions.
- Notebooks serve as examples; they import and execute DSBA modules similar to how CLI/API do.
- The tests are layered to validate each module independently (unit tests) and check interactions between modules (integration tests).

##### Architectural Patterns and Design Principles
- Separation of Concerns: Data ingestion, model functions, and registry management are clearly compartmentalized.
- Modularity: Different interfaces (CLI, API, notebooks) rely on the same core DSBA functionality, promoting code reuse.
- Containerization: The existence of Dockerfiles (e.g., Dockerfile.api) highlights isolation and deployment patterns commonly seen in microservice-style architectures.
- Environment Configuration: Using environment variables (like DSBA_MODELS_ROOT_PATH) to externalize configuration, typical in modern applications.

##### Technologies and Frameworks
- Python (version 3.12)
- Modern dependency management via pyproject.toml (instead of setup.py)
- Testing framework (pytest) for validations
- Docker for containerizing the API component
- The potential use of machine learning libraries (for instance, xgboost is mentioned in the README, as well as a potential fallback to scikit-learn)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Interfaces Section
    subgraph "Interfaces"
        CLI["CLI Component"]:::interface
        API["API Component"]:::interface
        Notebooks["Notebooks (Interactive)"]:::interface
    end

    %% Core DSBA Library Section
    subgraph "DSBA Core Library"
        DSA_Core["DSBA Core Library"]:::core
        subgraph "Modules"
            Data_Inestion["Data Ingestion"]:::module
            Preprocessing["Preprocessing"]:::module
            Model_Training["Model Training"]:::module
            Model_Evaluation["Model Evaluation"]:::module
            Model_Prediction["Model Prediction"]:::module
            Model_Registry["Model Registry"]:::module
        end
    end

    %% Data Ingestion Submodules
    Data_Ingestion -->|"has"| Data_Ingestion_Files["Files Module"]:::module
    Data_Ingestion -->|"has"| Data_Ingestion_Databases["Databases Module"]:::module

    %% Deployment Section
    subgraph "Deployment"
        Docker["Docker Container (API Deployment)"]:::deployment
    end

    %% Testing Section
    subgraph "Testing"
        Tests["Tests Suite"]:::testing
    end

    %% Relationships from Interfaces to Core Library
    CLI -->|"calls"| DSA_Core
    API -->|"calls"| DSA_Core
    Notebooks -->|"uses"| DSA_Core

    %% Relationships within DSBA Core Library
    DSA_Core -->|"includes"| Data_Ingestion
    DSA_Core -->|"includes"| Preprocessing
    DSA_Core -->|"includes"| Model_Training
    DSA_Core -->|"includes"| Model_Evaluation
    DSA_Core -->|"includes"| Model_Prediction
    DSA_Core -->|"includes"| Model_Registry

    %% API Deployment Mapping
    API ---|"wrappedBy"| Docker

    %% Testing relationship
    Tests -->|"validates"| DSA_Core

    %% Click Events for DSBA Core Library Components
    click DSA_Core "https://github.com/joachimz/dsba-platform/tree/main/src/dsba"
    click Data_Ingestion "https://github.com/joachimz/dsba-platform/tree/main/src/dsba/data_ingestion"
    click Data_Ingestion_Files "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/data_ingestion/files.py"
    click Data_Ingestion_Databases "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/data_ingestion/databases.py"
    click Preprocessing "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/preprocessing.py"
    click Model_Training "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/model_training.py"
    click Model_Evaluation "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/model_evaluation.py"
    click Model_Prediction "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/model_prediction.py"
    click Model_Registry "https://github.com/joachimz/dsba-platform/blob/main/src/dsba/model_registry.py"

    %% Click Events for API and Deployment Components
    click API "https://github.com/joachimz/dsba-platform/blob/main/src/api/api.py"
    click Docker "https://github.com/joachimz/dsba-platform/blob/main/src/api/Dockerfile.api"

    %% Click Event for CLI Component
    click CLI "https://github.com/joachimz/dsba-platform/tree/main/src/cli/dsba_cli"

    %% Click Event for Notebooks Component
    click Notebooks "https://github.com/joachimz/dsba-platform/blob/main/src/notebooks/model_training_example.ipynb"

    %% Click Event for Testing Suite
    click Tests "https://github.com/joachimz/dsba-platform/tree/main/tests"

    %% Styles
    classDef core fill:#f9f9f9,stroke:#333,stroke-width:2px;
    classDef module fill:#e6ffe6,stroke:#333,stroke-width:1px;
    classDef interface fill:#cce5ff,stroke:#333,stroke-width:1px;
    classDef deployment fill:#fff0cc,stroke:#333,stroke-width:2px;
    classDef testing fill:#ffd6d6,stroke:#333,stroke-width:1px;
```
