# mstabc/forlaps architecture

> Architecture diagram of the GitHub repository mstabc/forlaps, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mstabc/forlaps
- Repository: https://github.com/mstabc/forlaps

## Overview

Based on our analysis of both the file structure and the README, the project can be seen as a dual-purpose machine learning framework that encompasses two major components:

1. Reinforcement Learning (RL) Pipeline – found in the "OfflineRL and FORLAPS" folder and associated files:
- This component handles three main operations:
- Offline RL (training without data augmentation)
- Augmented RL (enhanced training with synthetic data through data augmentation)
- Evaluation of RL models using test datasets (involving Parquet files and CSV evaluations)
- Key files include:
- main.py – orchestrates the RL pipeline including data handling and RL training/evaluation
- config.py – holds configuration parameters such as RL hyperparameters and data paths
- offline_rl.py – contains the core offline RL implementation
- data_augmentation_processor.py – processes synthetic data creation for augmented RL
- rl_evaluation.py – manages model evaluation using test inputs
- Additional utility scripts (agent_plots_save.py and utils.py) that help with ancillary tasks, logging, or plotting

##### LSTM and KNN Models Component – encapsulated within the "lstm_KNN" directory
- This section focuses on training a Multi-Task LSTM model for sequence prediction and a KNN model for identifying similar process traces.
- It involves:
- Data preprocessing (data_preprocessing.py)
- Training modules (lstm_model.py and knn_model.py)
- Evaluation logic (lstm_evaluation.py)
- A main.py that orchestrates these sequences similar to the RL workflow
- Utility scripts to support these tasks (utils.py)
- The README also points out that parameters like epochs, learning rate, batch size, and hidden dimension are configurable via command line args.

Additionally, there is a top-level dataset_processor.py and an evaluation_dataset directory that appear to support both workflows by preparing and evaluating data.

A. Main Components to Include:
1. Reinforcement Learning Module:
- Configuration Manager (config.py)
- Data Ingestion & Processing (data handling scripts, e.g., part of main.py and dataset_processor.py)
- Offline RL Engine (offline_rl.py)
- Data Augmentation Processor (data_augmentation_processor.py)
- Evaluation Engine (rl_evaluation.py)
- Utility Tools (agent_plots_save.py, utils.py)
2. LSTM & KNN Model Training Module:
- Data Preprocessing (data_preprocessing.py)
- LSTM Model (lstm_model.py)
- KNN Model (knn_model.py)
- Evaluation Process (lstm_evaluation.py)
- Main Orchestration (main.py for this directory)
- Helper Libraries (utils.py)
3. Shared or External Elements:
- Dataset sources and evaluation datasets (evaluation_dataset folder, dataset_processor.py)
- Command-line interface which acts as an entry point for both pipelines (instructions in README on running the script with arguments)

B. Relationships and Interactions:
- From “User/CLI Input” to “Main Orchestrator” (for either RL or LSTM/KNN pipelines)
- Within RL module, data flows from input files → data augmentation processor (if applicable) → offline RL engine → evaluation engine → file outputs (CSV/Parquet)
- Within LSTM/KNN module, data flows from preprocessing → respective model training steps → evaluation → outputs

C. Architectural Patterns and Principles:
- Separation of concerns: distinct modules for data processing, model training, and evaluation.
- Parameterization and modularity – many parameters are configurable via command-line arguments.

D. Relevant Technologies and Frameworks:
- Python is the primary language.
- The design hints at using typical ML libraries (such as those for RL, LSTM, or KNN) even though the precise libraries might not be explicitly mentioned.
- Data persistence and evaluation formats (CSV and Parquet) should be noted.
- The design should also indicate that the system is intended to be run via CLI with various configurable options.

1. For the RL pipeline:
- Clearly separate different operation modes (offline, augmented, and evaluation)
- A node for the CLI that routes arguments to main.py which then directs the process accordingly.
2. For the LSTM/KNN pipeline:
- Data preprocessing, model training (both LSTM and KNN), and evaluation sequence.
- Use of training parameters (epochs, learning rate, batch size, hidden dim) as configurable option nodes influencing the training process.

- Optionally, include a “Shared Resources” block pointing out data directories (evaluation_dataset) and common utility scripts.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Global Entry Point
    CLI["CLI Input (command-line)"]:::cli

    %% Shared Components Subgraph
    subgraph "Shared Components"
        DS["Common Dataset Processor (dataset_processor.py)"]:::shared
        ED["Evaluation Dataset (evaluation_dataset/)"]:::shared
    end

    %% Reinforcement Learning Pipeline Subgraph
    subgraph "Reinforcement Learning Pipeline"
        RL_Main["Main Orchestration (main.py)"]:::rl
        RL_Config["Configuration Manager (config.py)"]:::rl
        RL_Data["Data Ingestion & Processing (data_handler_processor.py)"]:::rl
        Offline_RL["Offline RL Engine (offline_rl.py)"]:::rl
        Data_Aug["Data Augmentation Processor (data_augmentation_processor.py)"]:::rl
        Eval_RL["Evaluation Engine (rl_evaluation.py)"]:::rl
        RL_Utils["Utility Tools (agent_plots_save.py, utils.py)"]:::rl
    end

    %% LSTM & KNN Module Subgraph
    subgraph "LSTM & KNN Module"
        LSTM_Main["Main Orchestration (main.py)"]:::lstm
        Data_Pre["Data Preprocessing (data_preprocessing.py)"]:::lstm
        LSTM_Model["LSTM Model (lstm_model.py)"]:::lstm
        KNN_Model["KNN Model (knn_model.py)"]:::lstm
        Eval_LSTM["Evaluation Process (lstm_evaluation.py)"]:::lstm
        LSTM_Utils["Helper Utilities (utils.py)"]:::lstm
    end

    %% Relationships for RL Pipeline
    CLI -->|"starts"| RL_Main
    RL_Main -->|"loads"| RL_Config
    RL_Main -->|"initiates"| RL_Data
    DS -->|"provides data"| RL_Data
    RL_Data -->|"triggers"| Offline_RL
    RL_Data -->|"triggers"| Data_Aug
    Offline_RL -->|"outputs to"| Eval_RL
    Data_Aug -->|"enhances"| Eval_RL
    ED -->|"feeds"| Eval_RL
    Eval_RL -->|"logs via"| RL_Utils

    %% Relationships for LSTM & KNN Pipeline
    CLI -->|"starts"| LSTM_Main
    LSTM_Main -->|"initiates"| Data_Pre
    DS -->|"provides data"| Data_Pre
    Data_Pre -->|"trains"| LSTM_Model
    Data_Pre -->|"trains"| KNN_Model
    LSTM_Model -->|"evaluated by"| Eval_LSTM
    KNN_Model -->|"evaluated by"| Eval_LSTM
    ED -->|"feeds"| Eval_LSTM
    Eval_LSTM -->|"logs via"| LSTM_Utils

    %% Styling Classes
    classDef cli fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef rl fill:#ADD8E6,stroke:#0000FF,stroke-width:2px;
    classDef lstm fill:#98FB98,stroke:#008000,stroke-width:2px;
    classDef shared fill:#FFDAB9,stroke:#FF8C00,stroke-width:2px;

    %% Click Events for Reinforcement Learning Pipeline
    click RL_Main "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/main.py"
    click RL_Config "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/config.py"
    click RL_Data "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/data_handler_processor.py"
    click Offline_RL "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/offline_rl.py"
    click Data_Aug "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/data_augmentation_processor.py"
    click Eval_RL "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/rl_evaluation.py"
    click RL_Utils "https://github.com/mstabc/forlaps/blob/main/OfflineRL and FORLAPS/agent_plots_save.py"

    %% Click Events for LSTM & KNN Module
    click LSTM_Main "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/main.py"
    click Data_Pre "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/data_preprocessing.py"
    click LSTM_Model "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/lstm_model.py"
    click KNN_Model "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/knn_model.py"
    click Eval_LSTM "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/lstm_evaluation.py"
    click LSTM_Utils "https://github.com/mstabc/forlaps/blob/main/lstm_KNN/utils.py"

    %% Click Events for Shared Components
    click DS "https://github.com/mstabc/forlaps/blob/main/dataset_processor.py"
    click ED "https://github.com/mstabc/forlaps/tree/main/evaluation_dataset/"
```
