# pravko-solutions/flashlearn architecture

> Architecture diagram of the GitHub repository pravko-solutions/flashlearn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pravko-solutions/flashlearn
- Repository: https://github.com/pravko-solutions/flashlearn

## Overview

##### Project type and purpose
- FlashLearn is an open-source Python library (packaged on PyPI) that lets data engineers and ML developers build multi-step LLM pipelines just like standard ML transformers.
- It provides an easy “fit/predict”-style interface (learn_skill, GeneralSkill, ClassificationSkill, etc.) with JSON definitions for each step, and orchestrates parallel requests to arbitrary OpenAI-compatible LLM clients.

##### File structure
- Top-level packaging & config: `pyproject.toml`, `requirements.txt`, `dist/`, `flashlearn.egg-info/`
- Core orchestration: `flashlearn/core/flash_client.py` + `orchestration.py` + `core/tests/`
- “Skills” modules: `flashlearn/skills/` (base_skill, classification, learn_skill, general_skill) + JSON validations + `toolkit/` generators + tests
- Utilities: `flashlearn/utils/` (token_utils, demo_data, logging) + tests
- Examples: dozens of domain-specific example pipelines under `examples/`
- CI & docs: `.github/`, README, CODE_OF_CONDUCT, CONTRIBUTING, SECURITY

##### Key insights from README
- High-level flow: Your Data → Load/Learn Skill → Create Tasks → Run Tasks → Structured JSON Results → Downstream Steps
- Pluggable client support: OpenAI, LiteLLM, Ollama, DeepSeek, etc.
- Emphasis on “JSON in, JSON out,” 95% test coverage, parallel execution up to 1000 calls/min.

##### Detail and essential elements
- Both code modules and runtime components (in-memory orchestrator vs. remote API)
- Extension points: directory `flashlearn/skills/toolkit/` and how new JSON definitions plug in
- Optional persistence layer (saving `.json` skill definitions in user’s file system)
- Call out test coverage (95%) and CI integration as supporting processes

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Layer"
        UserCode["Python Script/Notebook/ETL"]:::user
        ExampleCode["Examples"]:::user
        ExampleAgent["Agent Patterns"]:::user
        ExampleBrowser["Browser Price Matching"]:::user
    end

    subgraph "FlashLearn Library"
        subgraph "Core Modules"
            Orchestrator["Core Orchestrator"]:::core
            FlashClient["Flash Client (LLM adapter)"]:::core
            TaskRunner["Parallel Task Runner"]:::core
        end
        subgraph "Skill Modules"
            BaseSkill["Base Skill Factory"]:::skills
            LearnSkill["LearnSkill"]:::skills
            GeneralSkill["GeneralSkill"]:::skills
            ClassificationSkill["ClassificationSkill"]:::skills
            DiscoverLabelsSkill["DiscoverLabelsSkill"]:::skills
            Toolkit["Toolkit (generate/validate)"]:::skills
        end
        subgraph "Utilities"
            Logging["Logging Utils"]:::utils
            TokenUtils["Token Utils"]:::utils
            DemoData["Demo Data"]:::utils
        end
    end

    subgraph "LLM Clients"
        OpenAI["OpenAI Adapter"]:::external
        LiteLLM["LiteLLM Adapter"]:::external
        Ollama["Ollama Adapter"]:::external
        DeepSeek["DeepSeek Adapter"]:::external
    end

    ExternalAPIs["External LLM APIs"]:::external
    Persistence["Persistence (JSONL/FileSystem/DB)"]:::persistence
    Downstream["Downstream Systems (DBs, Dashboards)"]:::external

    UserCode -->|calls skill| BaseSkill
    UserCode -->|or via| LearnSkill
    UserCode -->|or via| GeneralSkill
    BaseSkill -->|produces JSON tasks| Orchestrator
    LearnSkill -->|extends| BaseSkill
    GeneralSkill -->|extends| BaseSkill
    ClassificationSkill -->|extends| BaseSkill
    DiscoverLabelsSkill -->|extends| BaseSkill
    Orchestrator -->|orchestrates| TaskRunner
    TaskRunner -->|parallel calls| FlashClient
    FlashClient -->|requests| OpenAI
    FlashClient -->|requests| LiteLLM
    FlashClient -->|requests| Ollama
    FlashClient -->|requests| DeepSeek
    OpenAI --> ExternalAPIs
    LiteLLM --> ExternalAPIs
    Ollama --> ExternalAPIs
    DeepSeek --> ExternalAPIs
    ExternalAPIs -->|responses| FlashClient
    FlashClient -->|aggregates| TaskRunner
    TaskRunner -->|returns results| Orchestrator
    Orchestrator -->|returns structured JSON| UserCode
    Orchestrator -->|optional save| Persistence
    Orchestrator -->|optional send| Downstream

    %% Utility usages
    Logging -.-> Orchestrator
    Logging -.-> FlashClient
    TokenUtils -.-> FlashClient
    DemoData -.-> UserCode

    click Orchestrator "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/core/orchestration.py"
    click FlashClient "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/core/flash_client.py"
    click BaseSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/base_skill.py"
    click BaseSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/base_data_skill.py"
    click LearnSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/learn_skill.py"
    click GeneralSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/general_skill.py"
    click ClassificationSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/classification.py"
    click DiscoverLabelsSkill "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/discover_labels.py"
    click Toolkit "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/toolkit/generate_toolkit.py"
    click Toolkit "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/skills/toolkit/validate_json_files.py"
    click Logging "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/utils/logging_utils.py"
    click TokenUtils "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/utils/token_utils.py"
    click DemoData "https://github.com/pravko-solutions/flashlearn/blob/main/flashlearn/utils/demo_data.py"
    click ExampleCode "https://github.com/pravko-solutions/flashlearn/tree/main/examples/Code only examples/"
    click ExampleAgent "https://github.com/pravko-solutions/flashlearn/tree/main/examples/Agent Patterns/"
    click ExampleBrowser "https://github.com/pravko-solutions/flashlearn/tree/main/examples/browser_use_price_matching/"

    classDef user fill:#FFA500,stroke:#333,stroke-width:1px;
    classDef core fill:#87CEEB,stroke:#333,stroke-width:1px;
    classDef skills fill:#87CEEB,stroke:#333,stroke-width:1px;
    classDef utils fill:#87CEEB,stroke:#333,stroke-width:1px;
    classDef external fill:#98FB98,stroke:#333,stroke-width:1px;
    classDef persistence fill:#D3D3D3,stroke:#333,stroke-width:1px;
```
