# lileipisces/pod architecture

> Architecture diagram of the GitHub repository lileipisces/pod, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lileipisces/pod
- Repository: https://github.com/lileipisces/pod

## Overview

##### Major Components
- Data Sources & Input:
– The three domains (Sports, Beauty, Toys) serve as separate data sources.
– Domain-related shell scripts (sports.sh, beauty.sh, toys.sh) act as entry points that trigger experiments for their respective datasets.
- Data Processing & Preprocessing:
– Although not shown as dedicated folders, several Python files (e.g., utils.py, templates.py, seq.py) likely support data transformations, prompt templates, and preprocessing routines.
- Model Training & Distillation:
– pretrain.py serves as the module that orchestrates model pretraining using prompt distillation techniques.
– module.py (and potentially exp.py) encapsulate the core model definition and experiment logic.
- Evaluation:
– Files like bleu.py, rouge.py, and topn.py indicate the calculation of evaluation metrics (BLEU, ROUGE, Top-N metrics) to assess recommendation performance.
- Logging & Experiment Monitoring:
– The presence of domain-specific log files (sports.log, beauty.log, toys.log) shows that the experiments are monitored and output is logged for later review.

##### Establish Relationships and Data Flow
- Entry Points (sports.sh, beauty.sh, toys.sh) initiate domain-specific experiment runs.
- These scripts likely call the centralized experiment script (exp.py) which then directs data preprocessing functions (utils.py, templates.py, seq.py).
- The preprocessed data is fed into the model pipeline defined in module.py and carried through pretrain.py which handles model training with libraries such as PyTorch and transformers.
- After training, evaluation is performed with scripts (bleu.py, rouge.py, topn.py) that compute quality metrics on model outputs.
- Each stage generates log files (e.g., sports.log) that capture runtime data and performance metrics.

##### Architectural Patterns and Design Principles
- The architecture is modular: distinct scripts for data handling, training, and evaluation encourage separation of concerns.
- Experiment reusability is implied by having domain-specific shell scripts which exploit a common codebase and shared modules.
- There is a clear data flow: input (Amazon data) → preprocessing → model training via prompt distillation → evaluation and logging.

##### Technologies and Libraries to Highlight
- Python 3.6, PyTorch 1.6, transformers 4.18.0 (for model distillation and LLM-based tasks).
- External data sources as provided in the README for Sports, Beauty, and Toys.
- Any custom utilities (from utils.py, templates.py, seq.py) that assist in processing or model interfacing.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources & Input
    subgraph "Data Sources & Input"
        Beauty["Beauty Script (beauty.sh)"]:::dataInput
        Sports["Sports Script (sports.sh)"]:::dataInput
        Toys["Toys Script (toys.sh)"]:::dataInput
    end

    %% Model Training & Distillation
    subgraph "Model Training & Distillation"
        Exp["Experiment Controller (exp.py)"]:::model
        Module["Model Module (module.py)"]:::model
        Pretrain["Pretrain (pretrain.py)"]:::model
    end

    %% Data Processing & Preprocessing
    subgraph "Data Processing & Preprocessing"
        Utils["Utilities (utils.py)"]:::processing
        Templates["Templates (templates.py)"]:::processing
        Seq["Sequence Tools (seq.py)"]:::processing
    end

    %% Evaluation
    subgraph "Evaluation"
        Bleu["BLEU (bleu.py)"]:::evaluation
        Rouge["ROUGE (rouge.py)"]:::evaluation
        TopN["Top-N Metrics (topn.py)"]:::evaluation
    end

    %% Logging & Monitoring
    subgraph "Logging & Monitoring"
        LogBeauty["Beauty Log (beauty.log)"]:::logging
        LogSports["Sports Log (sports.log)"]:::logging
        LogToys["Toys Log (toys.log)"]:::logging
    end

    %% External Dependencies
    External["External: PyTorch & Transformers"]:::external

    %% Connections
    %% Data Sources call the Experiment Controller
    Beauty -->|"triggers"| Exp
    Sports -->|"triggers"| Exp
    Toys -->|"triggers"| Exp

    %% Experiment Controller invokes Preprocessing routines
    Exp -->|"calls"| Utils
    Exp -->|"calls"| Templates
    Exp -->|"calls"| Seq

    %% Preprocessing feeds into Model Training components
    Utils -->|"prepares data for"| Module
    Utils -->|"prepares data for"| Pretrain
    Templates -->|"formats data for"| Module
    Templates -->|"formats data for"| Pretrain
    Seq -->|"structures input for"| Module
    Seq -->|"structures input for"| Pretrain

    %% External dependencies support model training
    External -->|"supports"| Exp
    External -->|"supports"| Module
    External -->|"supports"| Pretrain

    %% Model Training outputs are evaluated
    Module -->|"outputs to"| Bleu
    Module -->|"outputs to"| Rouge
    Module -->|"outputs to"| TopN
    Pretrain -->|"triggers evaluation"| Bleu
    Pretrain -->|"triggers evaluation"| Rouge
    Pretrain -->|"triggers evaluation"| TopN

    %% Domain-specific logging
    Beauty ---|"logs"| LogBeauty
    Sports ---|"logs"| LogSports
    Toys ---|"logs"| LogToys
    Bleu ---|"records metrics in"| LogBeauty
    Rouge ---|"records metrics in"| LogSports
    TopN ---|"records metrics in"| LogToys

    %% Click Events
    click Beauty "https://github.com/lileipisces/pod/blob/master/beauty.sh"
    click Sports "https://github.com/lileipisces/pod/blob/master/sports.sh"
    click Toys "https://github.com/lileipisces/pod/blob/master/toys.sh"
    click LogBeauty "https://github.com/lileipisces/pod/blob/master/beauty.log"
    click LogSports "https://github.com/lileipisces/pod/blob/master/sports.log"
    click LogToys "https://github.com/lileipisces/pod/blob/master/toys.log"
    click Utils "https://github.com/lileipisces/pod/blob/master/utils.py"
    click Templates "https://github.com/lileipisces/pod/blob/master/templates.py"
    click Seq "https://github.com/lileipisces/pod/blob/master/seq.py"
    click Pretrain "https://github.com/lileipisces/pod/blob/master/pretrain.py"
    click Module "https://github.com/lileipisces/pod/blob/master/module.py"
    click Exp "https://github.com/lileipisces/pod/blob/master/exp.py"
    click Bleu "https://github.com/lileipisces/pod/blob/master/bleu.py"
    click Rouge "https://github.com/lileipisces/pod/blob/master/rouge.py"
    click TopN "https://github.com/lileipisces/pod/blob/master/topn.py"

    %% Styles
    classDef dataInput fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef processing fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef model fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef evaluation fill:#FFA07A,stroke:#000,stroke-width:2px;
    classDef logging fill:#D3D3D3,stroke:#000,stroke-width:2px;
    classDef external fill:#F0E68C,stroke:#000,stroke-width:2px;
```
