# ethannbrown/soundstages architecture

> Architecture diagram of the GitHub repository ethannbrown/soundstages, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ethannbrown/soundstages
- Repository: https://github.com/ethannbrown/soundstages

## Overview

soundstages is a research pipeline that collects music metadata and audio features, cleans and joins them into an analysis dataset, evaluates popularity models across artists and quantiles, and produces findings figures. Its main workflow runs from artist discovery through track-feature collection, preparation, modeling, evaluation, and reporting. The graph retains the documented collection and modeling branches, including both linear and quantile/XGBoost analyses. The README establishes the stage sequence; source excerpts and the index support selected internal calls. Unexcerpted implementation details are not inferred.

Main entry points: [`src/collect/artists.py`](https://github.com/ethannbrown/soundstages/blob/main/src/collect/artists.py) (Artist discovery), [`src/model/prep.py`](https://github.com/ethannbrown/soundstages/blob/main/src/model/prep.py) (Model table and folds), [`src/model/folds.py`](https://github.com/ethannbrown/soundstages/blob/main/src/model/folds.py) (Fold utilities), [`src/prepare/clean_tracks.py`](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/clean_tracks.py) (Track cleaning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_collection["Data collection"]
  node_artist_discovery["Artist discovery<br/>[artists.py]"]
  node_listener_collection["Listener counts<br/>[listeners.py]"]
  node_genre_resolution["Genre resolution<br/>[resolve.py]"]
  node_genre_rules["Genre mapping<br/>[genres.py]"]
  node_genre_assignment["Genre rules<br/>[rule.py]"]
  node_id_resolution["Track ID resolution<br/>[ids.py]"]
  node_album_collection["Album collection<br/>[albums.py]"]
  node_track_collection["Track collection<br/>[tracks.py]"]
  node_feature_collection["Audio features<br/>[features.py]"]
end

subgraph group_preparation["Dataset preparation"]
  node_artist_merge["Artist merge<br/>[merge.py]"]
  node_album_cleaning["Album cleaning<br/>[clean_albums.py]"]
  node_track_cleaning["Track cleaning<br/>[clean_tracks.py]"]
  node_text_normalization["Text normalization<br/>[text.py]"]
  node_dataset_build["Dataset build<br/>[build.py]"]
end

subgraph group_modeling["Modeling"]
  node_model_prep["Model table and folds<br/>[prep.py]"]
  node_fold_logic["Fold utilities<br/>[folds.py]"]
  node_baseline_models["Baselines<br/>[baselines.py]"]
  node_linear_models["Linear models<br/>[linear.py]"]
  node_quantile_models["Quantile regression<br/>[quantile.py]"]
  node_era_quantiles["Era quantiles<br/>[quantile_era.py]"]
  node_boost_models["Quantile XGBoost<br/>[boost.py]"]
end

subgraph group_evaluation["Evaluation and reporting"]
  node_evaluation["Prediction evaluation<br/>[evaluate.py]"]
  node_figures["Results figures<br/>[figures.py]"]
end

node_researcher(("Researcher"))
node_every_noise["Every Noise"]
node_spotify["Spotify pages"]
node_reccobeats["ReccoBeats API"]

node_every_noise -->|"provides artists"| node_artist_discovery
node_spotify -->|"provides artists"| node_artist_discovery
node_artist_discovery -->|"collects listeners"| node_listener_collection
node_artist_discovery -->|"resolves genres"| node_genre_resolution
node_artist_discovery -->|"uses rules"| node_genre_assignment
node_artist_discovery -->|"uses mapping"| node_genre_rules
node_genre_resolution -->|"uses mapping"| node_genre_rules
node_artist_discovery -->|"merges dataset"| node_artist_merge
node_listener_collection -->|"merges dataset"| node_artist_merge
node_genre_resolution -->|"merges dataset"| node_artist_merge
node_artist_merge -->|"resolves IDs"| node_id_resolution
node_reccobeats -->|"resolves IDs"| node_id_resolution
node_id_resolution -->|"collects albums"| node_album_collection
node_reccobeats -->|"serves albums"| node_album_collection
node_album_collection -->|"cleans albums"| node_album_cleaning
node_album_cleaning -->|"collects tracks"| node_track_collection
node_reccobeats -->|"serves tracks"| node_track_collection
node_track_collection -->|"cleans tracks"| node_track_cleaning
node_track_cleaning -->|"uses text rules"| node_text_normalization
node_track_cleaning -->|"reads album years"| node_album_cleaning
node_track_cleaning -->|"collects features"| node_feature_collection
node_reccobeats -->|"serves features"| node_feature_collection
node_album_cleaning -->|"joins dataset"| node_dataset_build
node_track_cleaning -->|"joins dataset"| node_dataset_build
node_feature_collection -->|"joins features"| node_dataset_build
node_dataset_build -->|"prepares model table"| node_model_prep
node_model_prep -->|"assigns folds"| node_fold_logic
node_model_prep -->|"fits baselines"| node_baseline_models
node_model_prep -->|"fits regressions"| node_linear_models
node_model_prep -->|"fits quantiles"| node_quantile_models
node_model_prep -->|"fits by era"| node_era_quantiles
node_model_prep -->|"fits boosted models"| node_boost_models
node_baseline_models -->|"uses folds"| node_fold_logic
node_linear_models -->|"uses folds"| node_fold_logic
node_quantile_models -->|"uses folds"| node_fold_logic
node_era_quantiles -->|"uses folds"| node_fold_logic
node_boost_models -->|"uses folds"| node_fold_logic
node_baseline_models -->|"writes predictions"| node_evaluation
node_linear_models -->|"writes predictions"| node_evaluation
node_evaluation -->|"reports scores"| node_figures
node_boost_models -->|"writes score and SHAP results"| node_figures
node_researcher -->|"runs pipeline"| node_artist_discovery

click node_artist_discovery "https://github.com/ethannbrown/soundstages/blob/main/src/collect/artists.py"
click node_listener_collection "https://github.com/ethannbrown/soundstages/blob/main/src/collect/listeners.py"
click node_genre_resolution "https://github.com/ethannbrown/soundstages/blob/main/src/collect/resolve.py"
click node_genre_rules "https://github.com/ethannbrown/soundstages/blob/main/src/collect/genres.py"
click node_genre_assignment "https://github.com/ethannbrown/soundstages/blob/main/src/collect/rule.py"
click node_artist_merge "https://github.com/ethannbrown/soundstages/blob/main/src/prepare/merge.py"
click node_id_resolution "https://github.com/ethannbrown/soundstages/blob/main/src/collect/ids.py"
click node_album_collection "https://github.com/ethannbrown/soundstages/blob/main/src/collect/albums.py"
click node_album_cleaning "https://github.com/ethannbrown/soundstages/blob/main/src/prepare/clean_albums.py"
click node_track_collection "https://github.com/ethannbrown/soundstages/blob/main/src/collect/tracks.py"
click node_track_cleaning "https://github.com/ethannbrown/soundstages/blob/main/src/prepare/clean_tracks.py"
click node_text_normalization "https://github.com/ethannbrown/soundstages/blob/main/src/prepare/text.py"
click node_feature_collection "https://github.com/ethannbrown/soundstages/blob/main/src/collect/features.py"
click node_dataset_build "https://github.com/ethannbrown/soundstages/blob/main/src/prepare/build.py"
click node_model_prep "https://github.com/ethannbrown/soundstages/blob/main/src/model/prep.py"
click node_fold_logic "https://github.com/ethannbrown/soundstages/blob/main/src/model/folds.py"
click node_baseline_models "https://github.com/ethannbrown/soundstages/blob/main/src/model/baselines.py"
click node_linear_models "https://github.com/ethannbrown/soundstages/blob/main/src/model/linear.py"
click node_quantile_models "https://github.com/ethannbrown/soundstages/blob/main/src/model/quantile.py"
click node_era_quantiles "https://github.com/ethannbrown/soundstages/blob/main/src/model/quantile_era.py"
click node_boost_models "https://github.com/ethannbrown/soundstages/blob/main/src/model/boost.py"
click node_evaluation "https://github.com/ethannbrown/soundstages/blob/main/src/model/evaluate.py"
click node_figures "https://github.com/ethannbrown/soundstages/blob/main/src/report/figures.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_artist_discovery,node_listener_collection,node_genre_resolution,node_genre_rules,node_genre_assignment,node_id_resolution,node_album_collection,node_track_collection,node_feature_collection toneBlue
class node_artist_merge,node_album_cleaning,node_track_cleaning,node_text_normalization,node_dataset_build toneAmber
class node_model_prep,node_fold_logic,node_baseline_models,node_linear_models,node_quantile_models,node_era_quantiles,node_boost_models,node_reccobeats toneMint
class node_evaluation,node_figures toneRose
class node_researcher,node_every_noise,node_spotify toneIndigo
```

## Components

### Data collection

- [Artist discovery](https://github.com/ethannbrown/soundstages/blob/main/src/collect/artists.py): `src/collect/artists.py`
- [Listener counts](https://github.com/ethannbrown/soundstages/blob/main/src/collect/listeners.py): `src/collect/listeners.py`
- [Genre resolution](https://github.com/ethannbrown/soundstages/blob/main/src/collect/resolve.py): `src/collect/resolve.py`
- [Genre mapping](https://github.com/ethannbrown/soundstages/blob/main/src/collect/genres.py): `src/collect/genres.py`
- [Genre rules](https://github.com/ethannbrown/soundstages/blob/main/src/collect/rule.py): `src/collect/rule.py`
- [Track ID resolution](https://github.com/ethannbrown/soundstages/blob/main/src/collect/ids.py): `src/collect/ids.py`
- [Album collection](https://github.com/ethannbrown/soundstages/blob/main/src/collect/albums.py): `src/collect/albums.py`
- [Track collection](https://github.com/ethannbrown/soundstages/blob/main/src/collect/tracks.py): `src/collect/tracks.py`
- [Audio features](https://github.com/ethannbrown/soundstages/blob/main/src/collect/features.py): `src/collect/features.py`

### Dataset preparation

- [Artist merge](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/merge.py): `src/prepare/merge.py`
- [Album cleaning](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/clean_albums.py): `src/prepare/clean_albums.py`
- [Track cleaning](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/clean_tracks.py): `src/prepare/clean_tracks.py`
- [Text normalization](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/text.py): `src/prepare/text.py`
- [Dataset build](https://github.com/ethannbrown/soundstages/blob/main/src/prepare/build.py): `src/prepare/build.py`

### Modeling

- [Model table and folds](https://github.com/ethannbrown/soundstages/blob/main/src/model/prep.py): `src/model/prep.py`
- [Fold utilities](https://github.com/ethannbrown/soundstages/blob/main/src/model/folds.py): `src/model/folds.py`
- [Baselines](https://github.com/ethannbrown/soundstages/blob/main/src/model/baselines.py): `src/model/baselines.py`
- [Linear models](https://github.com/ethannbrown/soundstages/blob/main/src/model/linear.py): `src/model/linear.py`
- [Quantile regression](https://github.com/ethannbrown/soundstages/blob/main/src/model/quantile.py): `src/model/quantile.py`
- [Era quantiles](https://github.com/ethannbrown/soundstages/blob/main/src/model/quantile_era.py): `src/model/quantile_era.py`
- [Quantile XGBoost](https://github.com/ethannbrown/soundstages/blob/main/src/model/boost.py): `src/model/boost.py`

### Evaluation and reporting

- [Prediction evaluation](https://github.com/ethannbrown/soundstages/blob/main/src/model/evaluate.py): `src/model/evaluate.py`
- [Results figures](https://github.com/ethannbrown/soundstages/blob/main/src/report/figures.py): `src/report/figures.py`

### Other components

- **Researcher**
- **Every Noise**
- **Spotify pages**
- **ReccoBeats API**

## Connections

- Every Noise → Artist discovery: provides artists (evidence: `README.md`)
- Spotify pages → Artist discovery: provides artists (evidence: `README.md`)
- Artist discovery → Listener counts: collects listeners (evidence: `README.md`)
- Artist discovery → Genre resolution: resolves genres (evidence: `README.md`)
- Artist discovery → Genre rules: uses rules (evidence: `src/collect/artists.py`)
- Artist discovery → Genre mapping: uses mapping (evidence: `src/collect/artists.py`)
- Genre resolution → Genre mapping: uses mapping (evidence: `src/collect/resolve.py`)
- Artist discovery → Artist merge: merges dataset (evidence: `README.md`)
- Listener counts → Artist merge: merges dataset (evidence: `README.md`)
- Genre resolution → Artist merge: merges dataset (evidence: `README.md`)
- Artist merge → Track ID resolution: resolves IDs (evidence: `README.md`)
- ReccoBeats API → Track ID resolution: resolves IDs (evidence: `README.md`)
- Track ID resolution → Album collection: collects albums (evidence: `README.md`)
- ReccoBeats API → Album collection: serves albums (evidence: `README.md`)
- Album collection → Album cleaning: cleans albums (evidence: `README.md`)
- Album cleaning → Track collection: collects tracks (evidence: `README.md`)
- ReccoBeats API → Track collection: serves tracks (evidence: `README.md`)
- Track collection → Track cleaning: cleans tracks (evidence: `README.md`)
- Track cleaning → Text normalization: uses text rules (evidence: `src/prepare/clean_tracks.py`)
- Track cleaning → Album cleaning: reads album years (evidence: `src/prepare/clean_tracks.py`)
- Track cleaning → Audio features: collects features (evidence: `README.md`)
- ReccoBeats API → Audio features: serves features (evidence: `README.md`)
- Album cleaning → Dataset build: joins dataset (evidence: `README.md`)
- Track cleaning → Dataset build: joins dataset (evidence: `README.md`)
- Audio features → Dataset build: joins features (evidence: `README.md`)
- Dataset build → Model table and folds: prepares model table (evidence: `README.md`)
- Model table and folds → Fold utilities: assigns folds (evidence: `src/model/prep.py`)
- Model table and folds → Baselines: fits baselines (evidence: `README.md`)
- Model table and folds → Linear models: fits regressions (evidence: `README.md`)
- Model table and folds → Quantile regression: fits quantiles (evidence: `README.md`)
- Model table and folds → Era quantiles: fits by era (evidence: `README.md`)
- Model table and folds → Quantile XGBoost: fits boosted models (evidence: `README.md`)
- Baselines → Fold utilities: uses folds (evidence: `src/model/baselines.py`)
- Linear models → Fold utilities: uses folds (evidence: `src/model/linear.py`)
- Quantile regression → Fold utilities: uses folds (evidence: `src/model/quantile.py`)
- Era quantiles → Fold utilities: uses folds (evidence: `src/model/quantile_era.py`)
- Quantile XGBoost → Fold utilities: uses folds (evidence: `src/model/boost.py`)
- Baselines → Prediction evaluation: writes predictions (evidence: `README.md`)
- Linear models → Prediction evaluation: writes predictions (evidence: `README.md`)
- Prediction evaluation → Results figures: reports scores (evidence: `README.md`)
- Quantile XGBoost → Results figures: writes score and SHAP results (evidence: `src/report/figures.py`)
- Researcher → Artist discovery: runs pipeline (evidence: `README.md`)
