# chrisapton/music-kg-project architecture

> Architecture diagram of the GitHub repository chrisapton/music-kg-project, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chrisapton/music-kg-project
- Repository: https://github.com/chrisapton/music-kg-project

## Overview

##### Project type and purpose
- This is a data‐driven application combining a data‐ingestion pipeline with a graph database and a web UI.
- Purpose: scrape music metadata, build a sampling knowledge graph in Neo4j, and let end users explore it via a Streamlit app.

##### File structure
- Top-level folders:
- scrapers – three scraper modules that pull data from external sources (e.g., MusicBrainz, WhoSampled, AcousticBrainz)
- knowledge-graph – contains the ETL layer and Neo4j configuration and import scripts
- Website – the Streamlit application that queries Neo4j and renders results
- Config and orchestration: Docker Compose for Neo4j, Python scripts for data loading.

##### README for additional insights
- Neo4j is launched via Docker Compose and populated by a Python loader script
- The Streamlit UI is a standalone component that connects to the running Neo4j instance

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Ingestion Layer
    subgraph "Data Ingestion"
        direction TB
        ES["External Data Sources"]:::external
        subgraph "Scrapers"
            direction TB
            AB["AcousticBrainz Scraper"]:::ingestion
            MB["MusicBrainz Scraper"]:::ingestion
            WS["WhoSampled Scraper"]:::ingestion
        end
    end

    %% ETL Layer
    ETL["ETL Layer"]:::ingestion

    %% Storage Layer
    Neo4j["Neo4j Graph DB"]:::storage

    %% Presentation Layer
    UI["Streamlit Web UI"]:::presentation

    %% Data Flow
    ES --> AB
    ES --> MB
    ES --> WS
    AB --> ETL
    MB --> ETL
    WS --> ETL
    ETL --> Neo4j
    UI --> Neo4j

    %% Click Events
    click AB "https://github.com/chrisapton/music-kg-project/blob/main/scrapers/acousticbrainz/audio_features.py"
    click MB "https://github.com/chrisapton/music-kg-project/blob/main/scrapers/musicbrainz/web_scraper.ipynb"
    click WS "https://github.com/chrisapton/music-kg-project/blob/main/scrapers/whosampled/whosampled/spiders/whosampled_spider.py"
    click ETL "https://github.com/chrisapton/music-kg-project/tree/main/knowledge-graph/etl/"
    click Neo4j "https://github.com/chrisapton/music-kg-project/blob/main/knowledge-graph/neo4j/docker-compose.yml"
    click UI "https://github.com/chrisapton/music-kg-project/blob/main/Website/Home.py"

    %% Styling
    classDef ingestion fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef storage fill:#90EE90,stroke:#000,stroke-width:1px
    classDef presentation fill:#FFA500,stroke:#000,stroke-width:1px
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px
```
