# nkhlkp/song-identifier-v2 architecture

> Architecture diagram of the GitHub repository nkhlkp/song-identifier-v2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nkhlkp/song-identifier-v2
- Repository: https://github.com/nkhlkp/song-identifier-v2

## Overview

This is a **Python audio recognition system** packaged as a reusable library (`audio_fingerprint/`) with **two entrypoints**:
- **Web app**: `app.py` (Flask) serving `templates/index.html` + `static/app.js|style.css`, exposing REST endpoints (`/add`, `/identify`, `/hum`, `/songs`, `/stats`, …).
- **CLI**: `main.py` (batch operations: add/identify/list/stats/remove/clear; supports choosing a DB file via `--db`).

Core capability is split into two pipelines:
##### Hum-to-search melody extraction + embedding similarity (via Spotify basic-pitch).

Use the file tree as the authoritative map for component boundaries:

##### Entrypoints / Interfaces
- `app.py` → **Flask Web Server + REST API**
- `main.py` → **CLI Application**
- `templates/index.html` + `static/app.js` + `static/style.css` → **Browser UI**

##### Core library (the real “engine”)
Inside `audio_fingerprint/`:
- `audio.py` → audio loading & preprocessing (mono, resample, bandpass); likely invokes **pydub/ffmpeg** underneath
- `spectrogram.py` → STFT / spectrogram generation
- `peaks.py` → peak detection on spectrogram
- `fingerprint.py` → hash generation (anchor/target/time delta)
- `matcher.py` → lookup + time-offset histogram coherence scoring + ranking
- `database.py` → fingerprint store + metadata + persistence (README indicates “database file” option from CLI)
- `melody.py` → melody extraction (basic-pitch) + embedding + cosine similarity matching

##### basic-pitch inference (library/model)

Goal: show the internal modules as separate components (the repo makes this easy).

##### Fingerprinting pipeline components (exact match)
- `audio.py` **AudioLoader/Preprocessor**
- `spectrogram.py` **Spectrogram/STFT**
- `peaks.py` **PeakDetector**
- `fingerprint.py` **FingerprintHasher**
- `database.py` **FingerprintDatabase (hash index + songs + stats)**
- `matcher.py` **FingerprintMatcher (offset histogram + ranker)**
- `pipeline.py` **Pipeline Orchestrator** (optional but helpful: shows “glue”)

Audio file/bytes
→ Preprocess
→ STFT
→ Peaks
→ Hashes
→ (Add) store hashes into DB index
OR
→ (Identify) lookup hashes in DB index → coherence scoring → ranked results

##### Melody pipeline components (hum-to-search)
- `melody.py` **MelodyExtractor (basic-pitch)**
- `melody.py` or `database.py` **MelodyEmbeddingStore** (where embeddings live)
- `melody.py` **SimilarityMatcher (cosine similarity)**

**Flow:**
Hum audio
→ basic-pitch notes
→ 128-d embedding
→ compare vs stored embeddings
→ ranked matches

##### Web: “Add Song”
Browser (`static/app.js`)
→ `POST /add` (Flask `app.py`)
→ `pipeline.py` (or high-level API)
→ `audio.py` (decode via FFmpeg)
→ `spectrogram.py` → `peaks.py` → `fingerprint.py`
→ `database.py` store (hash→(song_id, t), metadata)
→ `melody.py` extract embedding (calls basic-pitch) [if enabled during add]
→ `database.py` store melody embedding
→ response (song_id, counts, status)

##### Web: “Hum to Search”
Browser mic recording
→ `POST /hum`
→ `melody.py` (FFmpeg decode if needed)
→ basic-pitch inference
→ embedding
→ cosine similarity vs DB embeddings
→ results

##### Fingerprint Hash Index
- Key: `hash` (anchor_freq, target_freq, dt packed)
- Value: list of `(song_id, timestamp)`
- This is the performance-critical index.

##### Song Catalog / Metadata
- `song_id`, title, artist, duration, fingerprint_count, etc.

##### Melody Embeddings Store
- `song_id` → `128-d normalized vector`

##### Web UI boundary: don’t forget the frontend’s real role
From the repo:
- `templates/index.html` is the single-page UI shell.
- `static/app.js` is effectively the “frontend controller” that:
- records mic audio (browser APIs),
- uploads files (multipart/form-data),
- renders match results, library list, stats.

- That *recording happens client-side*, then audio blob is uploaded.

##### Diagramming conventions (to make it “principal-engineer readable”)
Use consistent notation and over-separate components (as requested), but keep it legible:

- Rectangles: services/modules
- Blue: Interfaces (Browser, CLI, REST)
- Green: Fingerprinting pipeline modules
- Purple: Melody pipeline modules
- Orange: Storage/indices
- “audio bytes (wav/mp3)”, “spectrogram (freq×time)”, “peaks (f,t)”, “hashes (uint32)”, “embedding (128f)”, “results (ranked matches)”

##### two dataflow lanes
- Lane A: Fingerprint add + identify
- Lane B: Melody add + hum query

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =========================================================
%% A) System Context (C4 L1)
%% =========================================================
subgraph "A) System Context (C4 Level 1)"
direction TB

U["User"]:::external

subgraph "Client Side"
direction TB
B["Browser (Web UI)"]:::interface
T["Terminal (CLI User)"]:::interface
end

S["Audio Fingerprinting System (Python)"]:::system

FF["FFmpeg (external executable)"]:::externalDep
BP["basic-pitch (model inference)"]:::externalDep
FS[("File System\nAudio files + DB file")]:::storage
MIC["OS Microphone"]:::external

U -->|"uses"| B
U -->|"uses"| T

B -->|"HTTP(S)\nREST calls"| S
T -->|"invokes\nlocal process"| S

S -->|"decode/convert"| FF
S -->|"melody inference"| BP
S -->|"read/write\n(audio + DB file)"| FS
B -->|"records\n(Web Audio API)"| MIC
end

%% =========================================================
%% B) Containers (C4 L2)
%% =========================================================
subgraph "B) Containers (C4 Level 2)"
direction TB

subgraph "Browser Container"
direction TB
IDX["index.html (UI shell)"]:::interface
JS["app.js (controller)\nWeb Audio API + fetch"]:::interface
CSS["style.css (styling)"]:::interface
end

subgraph "Python Process (single-machine typical runtime)"
direction TB
WEB["Flask Web Server\n(app.py)"]:::interface
CLI["CLI Entrypoint\n(main.py)"]:::interface
CORE["Core Library\n(audio_fingerprint package)"]:::engine
end

DB[("DB file + in-memory indices")]:::storage
FF2["FFmpeg"]:::externalDep
BP2["basic-pitch inference"]:::externalDep

IDX -->|"loads"| JS
IDX -->|"loads"| CSS

JS -->|"HTTP GET /\nserves UI"| WEB
JS -->|"POST /add\nmultipart audio"| WEB
JS -->|"POST /identify\naudio bytes"| WEB
JS -->|"POST /hum\nmic blob"| WEB
JS -->|"GET /songs"| WEB
JS -->|"GET /stats"| WEB

WEB -->|"calls library APIs"| CORE
CLI -->|"calls library APIs"| CORE

CORE -->|"persist/load"| DB
CORE -->|"decode via wrapper"| FF2
CORE -->|"melody model calls"| BP2
end

%% =========================================================
%% C) Components (C4 L3) - Core Engine
%% =========================================================
subgraph "C) Component Diagram (C4 Level 3) — audio_fingerprint engine room"
direction TB

P["pipeline.py\nPipeline Orchestrator (glue)"]:::engine

subgraph "Fingerprinting Pipeline (exact snippet match)"
direction TB
AUDIO["audio.py\nAudioLoader/Preprocessor\n(mono,resample,bandpass)\nFFmpeg via wrapper"]:::fp
SPEC["spectrogram.py\nSTFT/Spectrogram"]:::fp
PEAK["peaks.py\nPeakDetector"]:::fp
HASH["fingerprint.py\nFingerprintHasher\n(anchor,target,dt)"]:::fp
MATCH["matcher.py\nFingerprintMatcher\n(offset histogram + ranking)"]:::fp
end

subgraph "Melody Pipeline (hum-to-search)"
direction TB
MELODY["melody.py\nMelodyExtractor + Embedding + Similarity\n(basic-pitch + cosine sim)"]:::mel
end

subgraph "Storage Layer (logical stores; may be one DB file)"
direction TB
DBMOD["database.py\nPersistence + Indices API"]:::storageMod
IDXHASH[("Fingerprint Hash Index\nhash -> [(song_id,t)]")]:::storage
CAT[("Song Catalog / Metadata\nsong_id,title,artist,...")]:::storage
EMB[("Melody Embeddings\nsong_id -> 128-d vector")]:::storage
end

%% Orchestration edges
P -->|"decode/preprocess\n(audio bytes)"| AUDIO
P -->|"stft\n(spectrogram freq×time)"| SPEC
P -->|"peak picking\n(peaks f,t)"| PEAK
P -->|"hashing\n(hashes uint32)"| HASH
P -->|"store/lookup\n(hashes)"| DBMOD
P -->|"match candidates"| MATCH
P -->|"melody extract\n(hum audio)"| MELODY

%% Fingerprint pipeline flow
AUDIO -->|"PCM frames"| SPEC
SPEC -->|"spectrogram"| PEAK
PEAK -->|"peak list"| HASH
HASH -->|"hashes + time offsets"| DBMOD
DBMOD -->|"hash hits\n(song_id,t)"| MATCH
MATCH -->|"ranked matches\n(confidence)"| P

%% Melody pipeline flow
MELODY -->|"embedding (128f)"| DBMOD
DBMOD -->|"embedding set"| MELODY

%% Storage internals
DBMOD -->|"writes/reads"| IDXHASH
DBMOD -->|"writes/reads"| CAT
DBMOD -->|"writes/reads"| EMB
end

%% =========================================================
%% D) Sequence Diagrams (critical flows)
%% =========================================================
subgraph "D) Sequence Diagrams"
direction TB

subgraph "D1) Web: Add Song (/add)"
direction TB
seq1["Sequence: Add Song"]:::seq
end

subgraph "D2) Web/CLI: Identify Snippet (fingerprint)"
direction TB
seq2["Sequence: Identify Snippet"]:::seq
end

subgraph "D3) Web: Hum to Search (/hum)"
direction TB
seq3["Sequence: Hum Search"]:::seq
end
end

%% --- Sequence details as callouts (kept vertical)
seq1 -->|"Browser uploads\n(audio bytes wav/mp3)"| WEB
WEB -->|"POST /add handler"| P
P -->|"decode/mono/resample"| AUDIO
AUDIO -->|"calls decode"| FF2
P -->|"spectrogram"| SPEC
P -->|"peaks"| PEAK
P -->|"hashes"| HASH
P -->|"store hashes\nhash->(song_id,t)\n+ metadata"| DBMOD
P -->|"optional: melody embedding"| MELODY
MELODY -->|"model inference"| BP2
P -->|"store embedding"| DBMOD
WEB -->|"response\n(song_id,counts,status)"| JS

seq2 -->|"client provides\nsnippet audio bytes"| WEB
WEB -->|"POST /identify or CLI command"| P
P -->|"fingerprint pipeline\n(audio->stft->peaks->hashes)"| HASH
P -->|"lookup hashes"| DBMOD
DBMOD -->|"hash hits"| MATCH
MATCH -->|"offset histogram + rank"| P
WEB -->|"ranked results\n+ confidence"| JS

seq3 -->|"mic recording blob"| JS
JS -->|"POST /hum"| WEB
WEB -->|"hum handler"| P
P -->|"decode if needed"| AUDIO
AUDIO -->|"calls decode"| FF2
P -->|"basic-pitch notes\n+ embedding"| MELODY
MELODY -->|"inference"| BP2
MELODY -->|"cosine similarity\nvs stored embeddings"| DBMOD
WEB -->|"ranked melody matches"| JS

%% =========================================================
%% Quality (optional)
%% =========================================================
subgraph "Quality / Tests (non-runtime)"
direction TB
TFP["tests/test_fingerprint.py"]:::quality
TBP["test_bp.py"]:::quality
end

%% =========================================================
%% Click Events (from component mapping)
%% =========================================================
click WEB "https://github.com/nkhlkp/song-identifier-v2/blob/main/app.py"
click CLI "https://github.com/nkhlkp/song-identifier-v2/blob/main/main.py"
click IDX "https://github.com/nkhlkp/song-identifier-v2/blob/main/templates/index.html"
click JS "https://github.com/nkhlkp/song-identifier-v2/blob/main/static/app.js"
click CSS "https://github.com/nkhlkp/song-identifier-v2/blob/main/static/style.css"

click CORE "https://github.com/nkhlkp/song-identifier-v2/tree/main/audio_fingerprint/"
click AUDIO "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/audio.py"
click SPEC "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/spectrogram.py"
click PEAK "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/peaks.py"
click HASH "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/fingerprint.py"
click MATCH "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/matcher.py"
click DBMOD "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/database.py"
click MELODY "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/melody.py"
click P "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/pipeline.py"

click TFP "https://github.com/nkhlkp/song-identifier-v2/blob/main/tests/test_fingerprint.py"
click TBP "https://github.com/nkhlkp/song-identifier-v2/blob/main/test_bp.py"

%% Also include package init if represented implicitly
%% (node added for click coverage without cluttering names)
INIT["__init__.py (package init)"]:::engine
click INIT "https://github.com/nkhlkp/song-identifier-v2/blob/main/audio_fingerprint/__init__.py"
INIT -.->|"exports/public API"| CORE

%% =========================================================
%% Styles
%% =========================================================
classDef interface fill:#1f77b4,stroke:#0b3556,color:#ffffff,stroke-width:1.2px
classDef system fill:#0b4f6c,stroke:#063447,color:#ffffff,stroke-width:1.4px
classDef engine fill:#2a6f97,stroke:#144a66,color:#ffffff,stroke-width:1.2px

classDef fp fill:#2e8b57,stroke:#1f5f3c,color:#ffffff,stroke-width:1.2px
classDef mel fill:#6a4c93,stroke:#3f2d59,color:#ffffff,stroke-width:1.2px

classDef storage fill:#f4a261,stroke:#9a5b1f,color:#1b1b1b,stroke-width:1.2px
classDef storageMod fill:#e76f51,stroke:#9c3e2b,color:#ffffff,stroke-width:1.2px

classDef external fill:#444444,stroke:#222222,color:#ffffff,stroke-width:1.2px
classDef externalDep fill:#6c757d,stroke:#343a40,color:#ffffff,stroke-width:1.2px,stroke-dasharray: 6 4

classDef quality fill:#9aa0a6,stroke:#5f6368,color:#111111,stroke-width:1px,stroke-dasharray: 4 4
classDef seq fill:#ffd166,stroke:#9c7a12,color:#111111,stroke-width:1.2px
```
