# leonardltk/shazam-an-industrial-strength-audio-search-algorithm- architecture

> Architecture diagram of the GitHub repository leonardltk/shazam-an-industrial-strength-audio-search-algorithm-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-
- Repository: https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-

## Overview

##### Project Type and Purpose
- This is a standalone, batch‐oriented audio‐fingerprinting/search tool inspired by Shazam. It ingests a database of WAV files, builds and updates a lookup table (LUT) of audio “hashes,” then processes query WAVs in parallel to find matches, and finally combines and reports statistics.

##### Top-Level Components
- A. Configuration & Orchestration
- path.sh – environment setup (paths, thresholds)
- utils/conf.py – Shazam_conf class (thresholds for short/long audio)
- B. Database Management
- utils/Hashing.py – initial hash‐generation pipeline
- utils/Hashing_update.py – incremental add/delete of hashes
- data/database/*.scp & wav/database/*.wav – source audio lists + files
- Output: persistent LUT (e.g., serialized Python dict)
- C. Query Processing
- split (bash) – divides query SCP into $nj shards
- utils/Matching.py – per‐shard fingerprint matching against LUT
- Logs: Matching.<jid>.log
- D. Results Aggregation
- utils/Combine.py – merges per‐job match outputs
- utils/GetStats.py – computes match statistics (precision, recall)
- Final outputs: qry2db, db2qry, statistical report

##### Data Flow & Interactions
- Raw audio files (wav/database/*.wav) → wav.scp → Hashing.py
→ LUT written to disk
- For updates: wav_add.scp or wav_delete.scp → Hashing_update.py → LUT updated
- Query audio list (data/query_*/wav.scp) → split into shards → Matching.py (parallel jobs)
→ per‐job match logs → Combine.py → combined match list → GetStats.py → stats report

##### Architectural Patterns & Principles
- Pipeline / Batch processing: each stage reads files, performs computation, writes outputs
- Modular scripts: each Python tool handles one responsibility (build, update, match, combine, stats)
- Config‐driven thresholds: Shazam_conf centralizes tunable parameters
- Parallelism: queries are split and matched in parallel worker processes

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration & Orchestration
    subgraph "Configuration & Orchestration"
        PS["path.sh"]:::orch
        CF["conf.py"]:::orch
        SP["split.sh"]:::orch
    end

    %% Database Builder & Updater
    subgraph "Database Builder & Updater"
        subgraph Builder
            WSC["wav.scp"]:::data
            WDIR["wav/database/*.wav"]:::data
            HS["Hashing.py: Build LUT"]:::compute
        end
        subgraph Updater
            WADD["wav_add.scp"]:::data
            WDEL["wav_delete.scp"]:::data
            HU["Hashing_update.py: Update LUT"]:::compute
        end
        LUT["LUT (hashes.dict)"]:::data
        CORE["_Shazam_.py"]:::compute
    end

    %% Query Split & Matching Workers
    subgraph "Query Split & Matching Workers"
        QL["data/query_long/wav.scp"]:::data
        QS["data/query_short/wav.scp"]:::data
        SPL2["split into $nj shards"]:::orch
        MW["Matching.py Workers × $nj"]:::compute
        LOGS["Matching.<jid>.log"]:::data
    end

    %% Combine & Statistics
    subgraph "Combine & Statistics"
        CB["Combine.py"]:::compute
        CM["combined_matches.lst"]:::data
        GS["GetStats.py"]:::compute
        REP["Final Report"]:::data
    end

    %% Data Flows
    WSC -->|reads| HS
    WDIR -->|reads| HS
    HS -->|writes| LUT
    WADD -->|reads| HU
    WDEL -->|reads| HU
    HU -->|updates| LUT

    QL -->|input| SPL2
    QS -->|input| SPL2
    SP -->|invokes| SPL2
    SPL2 -->|shards| MW
    MW -->|logs| LOGS
    LOGS -->|input| CB
    CB -->|writes| CM
    CM -->|input| GS
    GS -->|writes| REP

    %% Configuration imports
    CF --> HS
    CF --> HU
    CF --> MW
    CF --> CB
    CF --> GS

    %% Core routines usage
    CORE --> HS
    CORE --> MW

    %% Click Events
    click PS "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/path.sh"
    click CF "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/conf.py"
    click HS "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/Hashing.py"
    click WSC "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/data/database/wav.scp"
    click WDIR "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/tree/master/wav/database/"
    click HU "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/Hashing_update.py"
    click WADD "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/data/database/wav_add.scp"
    click WDEL "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/data/database/wav_delete.scp"
    click CORE "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/_Shazam_.py"
    click MW "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/Matching.py"
    click QL "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/data/query_long/wav.scp"
    click QS "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/data/query_short/wav.scp"
    click CB "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/Combine.py"
    click GS "https://github.com/leonardltk/shazam-an-industrial-strength-audio-search-algorithm-/blob/master/utils/GetStats.py"

    %% Styles
    classDef data fill:#cfe2f3,stroke:#07689f,color:#000;
    classDef compute fill:#d9ead3,stroke:#38761d,color:#000;
    classDef orch fill:#fff2cc,stroke:#b45f06,color:#000
```
