# mlshukai/pamiq-core architecture

> Architecture diagram of the GitHub repository mlshukai/pamiq-core, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mlshukai/pamiq-core
- Repository: https://github.com/mlshukai/pamiq-core

## Overview

##### Main Components
- Central Library (pamiq_core): This is the heart of the project. Within this module, several submodules exist:
– Console: Handles system status reporting and API interfaces (found in src/pamiq_core/console).
– Data: Manages data structures and buffers (including implementations for random and sequential replacement buffers, in src/pamiq_core/data and its impls).
– Interaction: Implements the agent/environment interfaces, wrappers, and event handling (located in src/pamiq_core/interaction).
– Model: Defines model containers and interfaces (see src/pamiq_core/model).
– Torch: Integrates PyTorch functionality for the neural network backend (in src/pamiq_core/torch).
– Trainer: Contains modules for training algorithms and orchestration (in src/pamiq_core/trainer).
– Threads: Provides threaded operations and thread control classes (in src/pamiq_core/threads).
– Utils: Provides various helper functions such as reflection and schedulers (in src/pamiq_core/utils).
– Supporting Files: Additional modules, like “state_persistence” and “time”, which support internal state handling and timing functions.

##### Relationships and Interactions
- Core Library as Central Hub: Show pamiq_core as the central component.
- Data Flow:
– Data Module delivers buffers and containers to both Trainer and Model components.
– Trainer uses the Torch module for model updates and training routines.
- Interaction Layer:
– Interaction module links agents and environments, feeding data into Trainer and Model.
– Console provides a user interface (CLI and web API) for monitoring and interacting with the system.
- Concurrency:
– Threads module coordinates with other modules to support concurrent training, data handling and interaction processing.
- Utility Support:
– Utils underlies multiple components to provide cross-cutting concerns like logging, configuration via reflection, and scheduling.

##### Architectural Patterns and Design Principles
- Modularity: Each submodule (data, interaction, trainer, etc.) is designed to be independent yet orchestrated by the core engine.
- Clear Separation of Concerns: The architecture separates responsibilities, e.g., data management is decoupled from model training.
- Integration with PyTorch: The “torch” module serves as an abstraction layer for deep learning, indicating a design optimized for ML workloads.
- Test-Driven Structure: The parallel tests folder structure signals that each module has corresponding tests ensuring component isolation and correctness.

##### Including Relevant Technologies
- Python as the primary language with type hinting (evidenced by the .py files and py.typed).
- PyTorch (for neural network model implementation).
- Docker and docker-compose: Indicate containerization and standardized deployment practices.
- CI/CD (GitHub Actions workflows) shown in .github/workflows for testing and deployment (publishing to PyPI).

##### Putting It All Together
- Optionally, include a layer that shows external interfaces (Console/web API, CLI) interacting with the core library.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Central Library Node
    A["pamiq_core"]:::core

    %% External Interface
    B["Console (CLI/web API)"]:::interface

    %% Core Modules Subgraph
    subgraph "Core Modules"
        C["Data (Buffers & Containers)"]:::core
        D["Interaction (Agent/Environment)"]:::core
        E["Model (Model Containers)"]:::core
        F["Trainer (Training Orchestration)"]:::core
        G["Torch (PyTorch Integration)"]:::core
    end

    %% Support Modules Subgraph
    subgraph "Support Modules"
        H["Threads (Concurrency Control)"]:::support
        I["Utils (Helper Functions)"]:::support
        J["State Persistence"]:::support
        K["Time Handling"]:::support
    end

    %% Primary Connections from Central Library
    A -->|"provides UI"| B
    A -->|"integrates"| C
    A -->|"integrates"| D
    A -->|"integrates"| E
    A -->|"integrates"| F
    A -->|"integrates"| G

    %% Inter-module data flows within Core Modules
    C -->|"buffers_to"| F
    C -->|"data_to"| E
    D -->|"events_to"| F
    D -->|"data_to"| E
    F -->|"updates_via"| G

    %% Support Modules relationships
    H -->|"supports_concurrent"| C
    H -->|"supports_concurrent"| D
    H -->|"supports_concurrent"| F

    I -->|"helpers_for"| C
    I -->|"helpers_for"| D
    I -->|"helpers_for"| E
    I -->|"helpers_for"| F

    J -->|"state_persistence"| A
    K -->|"timing_support"| A

    %% Click Events
    click A "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core"
    click B "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/console"
    click C "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/data"
    click D "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/interaction"
    click E "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/model"
    click F "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/trainer"
    click G "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/torch"
    click H "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/threads"
    click I "https://github.com/mlshukai/pamiq-core/tree/main/src/pamiq_core/utils"
    click J "https://github.com/mlshukai/pamiq-core/blob/main/src/pamiq_core/state_persistence.py"
    click K "https://github.com/mlshukai/pamiq-core/blob/main/src/pamiq_core/time.py"

    %% Styles
    classDef core fill:#AED6F1,stroke:#2471A3,stroke-width:2px;
    classDef interface fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef support fill:#A9DFBF,stroke:#28B463,stroke-width:2px;
```
