# lamm-mit/preflexor architecture

> Architecture diagram of the GitHub repository lamm-mit/preflexor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lamm-mit/preflexor
- Repository: https://github.com/lamm-mit/preflexor

## Overview

##### Project’s Main Pillars
- Core Functionality: PRefLexOR uses a hybrid approach interweaving Odds Ratio Preference Optimization (ORPO) and Direct Preference Optimization (DPO) to enable recursive reasoning. It is primarily focused on self-refining language models with iterative feedback loops.
- Use Cases: The project is aimed at enhancing reasoning capabilities; it even includes a materials design application that leverages a dynamically generated knowledge graph via retrieval-augmented generation (RAG).
- External Dependencies: It integrates with libraries like transformers, trl, Hugging Face model repositories, and optional components like flash attention for efficiency.

##### Break Down the File Structure into Components
- PRefLexOR Package:
- Contains the core source files:
- active_trainer.py (triggers training sessions with preference optimization loops)
- inference.py (handles model inference including thinking/reflection token processing)
- utils.py (likely includes utility functions shared across training/inference)
- __init__.py marks it as a package.
- Notebooks and Scripts:
- Several Jupyter notebooks (e.g., PRefLexOR_train_Phase-I-and-II.ipynb, the inference notebooks) serve as interactive examples or experiments that demonstrate training, inference, and graph visualization.
- Additional Scripts:
- Graph-PRefLexOR_train_Phase-I-and-II.ipynb and 3D_graph_visualization.py, which hint at graph-based visualization of the generated knowledge graphs.
- Packaging and Configuration:
- Files like setup.py, requirements.txt, LICENSE, and README indicate standard packaging and dependency management.

##### Architectural Components and Their Interactions
- Training Modules:
- Separate trainers for the ORPO (Structured Thought Integration Training) and the DPO (Independent Reasoning Development) phases.
- These include iterative training loops where the model generates, refines, and reintegrates reasoning outputs.
- Data pipelines that dynamically generate and update training datasets.
- Inference Engine:
- Module responsible for converting prompts into agentic and reflective answers.
- Relies on special tokens (<|thinking|> and <|/thinking|>) to drive iterative reasoning.
- Graph and Visualization Component:
- External Systems:
- Integration with Hugging Face's transformers, model hub push (for saving models), and possibly external libraries/components like flash attention.
- Feedback & Logging:
- Logging callbacks and saving of training logs/datasets to track rewards, losses, and performance so as to improve model tuning over iterations.

##### Final Touches
- Component has a concise description.
- From initial prompt input → processing in the Inference Module → token-driven iterative reasoning (active_trainer and inference) → training loop → dataset updates → logging/save.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Engine
    subgraph "Core Engine"
        UP["User Prompt"]:::input
        IE["Inference Engine<br/>(Thinking Tokens)"]:::inference
        subgraph "Training Engine"
            OT["ORPO Trainer<br/>(Structured Thought Integration)"]:::training
            DT["DPO Trainer<br/>(Efficient Exact Optimization)"]:::training
            UF["Utility Functions"]:::utils
            DL["Dataset & Logging"]:::data
        end
        UP -->|"processes"| IE
        IE -->|"feeds"| OT
        IE -->|"feeds"| DT
        UF -->|"supports"| OT
        UF -->|"supports"| DT
        OT -->|"updates"| DL
        DT -->|"updates"| DL
        DL -->|"iterative feedback"| OT
        DL -->|"iterative feedback"| DT
    end

    %% Graph Visualization & Knowledge Graph
    subgraph "Graph Visualization & Knowledge Graph"
        GN["Graph Notebook"]:::viz
        GV["3D Graph Visualizer"]:::viz
    end
    DL -->|"visualizes"| GN
    DL -->|"visualizes"| GV

    %% Interactive Examples & Experiments
    subgraph "Interactive Examples & Experiments"
        TEN["Training Example Notebook"]:::notebook
        ITN1["Inference Example: Thinking-Reflection Notebook"]:::notebook
        ITN2["Inference Example: Thinking Notebook"]:::notebook
    end
    UP -->|"triggers"| TEN
    IE -->|"demonstrated in"| ITN1
    IE -->|"demonstrated in"| ITN2

    %% Packaging & External Configuration
    subgraph "Packaging & Config"
        ST["setup.py"]:::packaging
        RT["requirements.txt"]:::packaging
    end

    %% External Dependencies
    subgraph "External Dependencies"
        TL["Transformers Library"]:::external
        HF["Hugging Face Model Hub"]:::external
        FA["Flash Attention (Optional)"]:::external
    end
    TL --- IE
    TL --- OT
    TL --- DT
    HF --- IE
    HF --- OT
    HF --- DT
    FA --- IE

    %% Click Events
    click OT "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR/active_trainer.py"
    click DT "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR/active_trainer.py"
    click IE "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR/inference.py"
    click UF "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR/utils.py"
    click GN "https://github.com/lamm-mit/preflexor/blob/main/Graph-PRefLexOR_train_Phase-I-and-II.ipynb"
    click GV "https://github.com/lamm-mit/preflexor/blob/main/3D_graph_visualization.py"
    click TEN "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR_train_Phase-I-and-II.ipynb"
    click ITN1 "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR_inference_thinking-reflection.ipynb"
    click ITN2 "https://github.com/lamm-mit/preflexor/blob/main/PRefLexOR_inference_thinking.ipynb"
    click ST "https://github.com/lamm-mit/preflexor/blob/main/setup.py"
    click RT "https://github.com/lamm-mit/preflexor/blob/main/requirements.txt"

    %% Class Definitions and Styles
    classDef input fill:#f9f,stroke:#333,stroke-width:2px;
    classDef inference fill:#a2d5c6,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    classDef training fill:#d4a5a5,stroke:#333,stroke-width:2px,rx:10,ry:10;
    classDef utils fill:#a3c1da,stroke:#333,stroke-width:2px;
    classDef data fill:#ffe699,stroke:#333,stroke-width:2px,shape:parallelogram;
    classDef viz fill:#c3f7d4,stroke:#333,stroke-width:2px,shape:hexagon;
    classDef notebook fill:#ffcc99,stroke:#333,stroke-width:2px;
    classDef packaging fill:#f7c6c7,stroke:#333,stroke-width:2px;
    classDef external fill:#d0e1f9,stroke:#333,stroke-width:2px,shape:hexagon;
```
