# lff8888/audiosort_tflm architecture

> Architecture diagram of the GitHub repository lff8888/audiosort_tflm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lff8888/audiosort_tflm
- Repository: https://github.com/lff8888/audiosort_tflm

## Overview

##### Project Type and Purpose
- This is a collection of TinyML demos for microcontrollers:
- “ArduinoSineFunction”: a TensorFlow Lite for Microcontrollers (TFLM) sine-wave generator on Arduino.
- “AudioSortCode”: end-to-end audio classification on Arduino/ESP32 (data acquisition → feature extraction → model training → model deployment → on-device inference).

##### High-Level Components
A. Offline (Development) Pipeline
1. Raw Audio Data & Samples
– Stored in “AudioSortCode/SamplingAndMelSpectrumAndTrain/sample” (e.g., .mp3, .txt).
2. Preprocessing & Feature Extraction
– Python scripts:
- datasetSplit.py (split datasets)
- melSpectrumReader.py (compute mel spectrogram)
- aiModuleTrain.py (train TensorFlow model)
3. Model Training
– TensorFlow training code in aiModuleTrain.py
– Produces a TensorFlow .tflite model file
4. Model Conversion & Header Generation
– Conversion script (inside aiModuleTrain.py or external TFLite converter)
– Outputs audio_classification_model.tflite
– Converts .tflite → C array → audio_classification_model.h

B. On-Device (Runtime) Pipeline
1. Microcontroller Firmware
– Arduino sketches:
- SamplingAndMelSpectrumAndTrain.ino (optional on-device training/demo)
- SamplingAndRecognize.ino (audio capture & inference)
- TFLM_TEST.ino (simple inference demo)
– Includes generated model header (audio_classification_model.h)
– Includes TFLM library and Arduino APIs
2. Hardware Platforms
– Arduino UNO / Nano (8-bit)
– ESP32 / ESP32-S3 (more compute, camera integration)
3. Peripherals
– Microphone module (I²S or analog)
– Optional camera module (ESP32-S3 face detection demo)
4. Output
– Serial console or LEDs indicating classified audio class

C. Arduino Sine Function Demo
1. TinyML “model.h” provides sine parameters
2. ArduinoSineFunction.ino reads model and outputs sine wave via DAC/PWM
3. Demonstrates TFLM integration for non-ML DSP

##### Relationships & Data Flow
1) Offline lane (data → preprocessing → training → model conversion)
2) On-device lane (firmware + model → microcontroller → classification/sine output)
- Raw data → Python preprocessing → Trained model
- Trained model → TFLite conversion → C header
- C header + Sketch → Compiled firmware → Hardware → Inference output

##### Architectural Patterns & Principles
- Pipeline pattern separating offline and runtime concerns
- Modularization: Python scripts are decoupled from Arduino sketches
- Reusability: single .h model file reused across demos
- Minimal dependencies on microcontroller: only TFLM and Arduino core

##### Relevant Technologies & Libraries
- Python 3.x, TensorFlow, TensorFlow Lite Converter
- TensorFlow Lite for Microcontrollers (TFLM)
- Arduino IDE / PlatformIO, C++
- ESP-IDF or Arduino‐ESP32 for ESP32 variants

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Offline Pipeline
    subgraph "Offline Pipeline"
        direction TB
        RawData["Raw Audio Data & Samples"]:::dataStore
        datasetSplit["datasetSplit.py"]:::offline
        melReader["melSpectrumReader.py"]:::offline
        train["aiModuleTrain.py"]:::offline
        LiteModel["audio_classification_model.tflite"]:::dataStore
        Header["audio_classification_model.h"]:::dataStore
    end

    RawData -->|provides data to| datasetSplit
    datasetSplit -->|computes mel features| melReader
    melReader -->|feeds features to| train
    train -->|outputs| LiteModel
    LiteModel -->|converted to C array| Header

    %% On-Device Runtime
    subgraph "On-Device Runtime"
        direction TB
        Sketch1["SamplingAndMelSpectrumAndTrain.ino"]:::ondevice
        Sketch2["SamplingAndRecognize.ino"]:::ondevice
        Sketch3["TFLM_TEST.ino"]:::ondevice
        ModelHeader["audio_classification_model.h"]:::dataStore
        TFLM["TFLM Library & Arduino APIs"]:::ondevice
        MCU["Microcontroller (Arduino/ESP32)"]:::hardware
        Microphone["Microphone Module"]:::hardware
        Camera["Camera Module (optional)"]:::hardware
        Output["Serial Console / LEDs"]:::hardware
    end

    Header -->|included in| Sketch1
    Header -->|included in| Sketch2
    Header -->|included in| Sketch3
    Sketch1 -->|uses| TFLM
    Sketch2 -->|uses| TFLM
    Sketch3 -->|uses| TFLM
    TFLM -->|runs on| MCU
    MCU -->|captures audio via| Microphone
    MCU -->|captures image via| Camera
    MCU -->|drives| Output

    %% Sine Demo
    subgraph "Sine Demo"
        direction TB
        SineSketch["ArduinoSineFunction.ino"]:::sine
        SineModel["model.h"]:::dataStore
        DemoDocs["TFLMArduinoSineFunction DEMO.md"]:::sine
        DAC["DAC/PWM Output"]:::hardware
    end

    SineModel -->|provides params to| SineSketch
    SineSketch -->|outputs sine wave via| DAC

    %% Click Events
    click RawData "https://github.com/lff8888/audiosort_tflm/tree/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/"
    click datasetSplit "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/datasetSplit.py"
    click melReader "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/melSpectrumReader.py"
    click train "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/aiModuleTrain.py"
    click LiteModel "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/audio_classification_model.tflite"
    click Header "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/sample/audio_classification_model.h"
    click Sketch1 "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndMelSpectrumAndTrain/SamplingAndMelSpectrumAndTrain.ino"
    click Sketch2 "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/SamplingAndRecognize/SamplingAndRecognize.ino"
    click Sketch3 "https://github.com/lff8888/audiosort_tflm/blob/main/AudioSortCode/TFLM_TEST/TFLM_TEST.ino"
    click SineSketch "https://github.com/lff8888/audiosort_tflm/blob/main/ArduinoSineFunction/ArduinoSineFunction.ino"
    click SineModel "https://github.com/lff8888/audiosort_tflm/blob/main/ArduinoSineFunction/model.h"
    click DemoDocs "https://github.com/lff8888/audiosort_tflm/blob/main/ArduinoSineFunction/TFLMArduinoSineFunction DEMO.md"

    %% Styles
    classDef offline fill:#cce5ff,stroke:#999,stroke-width:1px
    classDef ondevice fill:#d4edda,stroke:#999,stroke-width:1px
    classDef sine fill:#fff3cd,stroke:#999,stroke-width:1px
    classDef dataStore fill:#e2e3e5,stroke:#999,stroke-width:1px,stroke-dasharray: 5 5
    classDef hardware fill:#f8d7da,stroke:#999,stroke-width:1px,stroke-dasharray: 2 2
```
