# iain-mcdonald/pyssed architecture

> Architecture diagram of the GitHub repository iain-mcdonald/pyssed, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/iain-mcdonald/pyssed
- Repository: https://github.com/iain-mcdonald/pyssed

## Overview

##### Project Type and Purpose
- This is an open-source scientific CLI tool (PySSED) for constructing and fitting Spectral Energy Distributions (SEDs) of evolved stars, with an auxiliary interactive 3D tomography web app (“gtomo”).
- Purpose: ingest raw catalog and filter data, generate model grids, run fitting routines via Python scripts (pyssed.py, makemodel.py), and output SED data, plots, and ancillary products. The “gtomo” submodule provides a containerized web frontend/backend for exploring 3D dust cubes.

##### Top-Level Architecture Layers
- Data Layer
- data/ – raw observational catalogs (/*.dat), filter definitions (XML), distance tables, target lists
- models/ – raw model grids (bt-settle), reduce scripts
- Configuration Layer
- src/*.default, src/*.ness, src/*.test – interchangeable setup files for filters, catalogues, ancillary parameters, and overall program settings
- Core Processing Layer (CLI)
- src/pyssed.py – main driver for SED fitting
- src/makemodel.py – builds model grids based on configuration
- src/reject.* – filtering/rejection logic
- src/*.dat, *.scr – helper data and scripts
- Computation & Cache Layer
- proc/phot and proc/anc – processed NumPy arrays, match tables, intermediate SED data
- proc/last_cmd.npy – tracking execution state
- Output Layer
- output/anc, output/ebvdata, output/png, output/sed – final products: ancillary tables, reddening data, PNG plots, SED files
- Web App Submodule (“gtomo”)
- src/gtomo/Dockerfile + docker-compose.yml + .gitlab-ci.yml – containerized service definition and CI integration
- src/gtomo/sda – Python backend (likely Flask or similar) and static assets (HTML/CSS/CSV) for the interactive interface
- src/gtomo/_data/app_data – precomputed HDF5 cubes and scripts for converting between HDF5 and FITS

##### Relationships & Data Flows
- CLI Execution Flow
1. User invokes `python3 makemodel.py …` or `python3 pyssed.py …`
2. Script loads appropriate config (filters.*, catalogues.*, ancillary.*)
3. Reads raw data from data/ and models/
4. Generates or loads processed arrays under proc/
5. Performs fitting, writes results to output/
- Web App Flow
1. Docker Compose spins up backend service (reading from src/gtomo/_data/app_data)
2. Frontend assets fetch data via REST endpoints in src/gtomo/sda
3. Interactive visualizations served in browser

##### Architectural Patterns & Principles
- Modular CLI with pluggable configs: allows multiple “setups” (default, ness, test) by swapping simple files
- Data-pipeline pattern: raw → process (NumPy) → output
- Separation of concerns: data storage, computation, and output directories
- Containerization for web component: Dockerfile + docker-compose + CI

- Raw Data Store (data/, models/)
- Config Manager (setup.* files, filters.*, catalogues.*, ancillary.*)
- CLI Processor
- Processing Cache (proc/)
- Output Repository (output/)
- Web Service (“gtomo”)
- Subcomponents of Web Service:
- Container Orchestration (docker-compose.yml)
- Backend API (src/gtomo/sda/*.py)
- Data Layer (src/gtomo/_data/app_data)
- Frontend Static Assets (HTML/CSS under sda/assets)
- CI/CD (.gitlab-ci.yml)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        direction TB
        RD(("Raw Data Store")):::data
        DM(("Model Grids")):::data
        CD(("Filter Definitions")):::data
    end

    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        ANC[/"ancillary.default<br>ancillary.ness"/]:::config
        CAT[/"catalogues.default<br>catalogues.ness<br>catalogues.test"/]:::config
        FIL[/"filters.default<br>filters.ness<br>filters.test"/]:::config
        CON[/"constraints.default"/]:::config
        SET[/"setup.default<br>setup.ness<br>setup.ness2<br>setup.test"/]:::config
    end

    %% Core Processing Layer
    subgraph "CLI Processing Layer"
        direction TB
        SED["pyssed.py"]:::proc
        MOD["makemodel.py"]:::proc
        REJ["reject.default"]:::proc
        HLP[/"outputheaders.scr<br>shorten-model.scr"/]:::proc
    end

    %% Computation & Cache Layer
    subgraph "Computation & Cache Layer"
        direction TB
        CACHE(("proc/")):::data
        PHOT(("proc/phot/")):::data
        ANCACHE(("proc/anc/")):::data
        STATE(("proc/last_cmd.npy")):::data
    end

    %% Output Layer
    subgraph "Output Layer"
        direction TB
        OUTANC(("output/anc/")):::output
        OUTEBV(("output/ebvdata/")):::output
        OUTPNG(("output/png/")):::output
        OUTSED(("output/sed/")):::output
    end

    %% Web Service Submodule
    subgraph "Web Service (gtomo)" 
        direction TB
        ORCH[/"Dockerfile<br>docker-compose.yml<br>.gitlab-ci.yml"/]:::web
        API["Backend API (sda/*.py)"]:::web
        WDATA(("app_data/")):::data
        ASSETS[/"Static Assets (HTML/CSS)"/]:::web
    end

    %% Dependencies & Flows
    RD -->|reads| SED
    DM -->|reads| MOD
    CD -->|provides| SED
    ANC -->|provides| SED
    CAT -->|provides| SED
    FIL -->|provides| SED
    CON -->|provides| SED
    SET -->|configures| SED
    SET -->|configures| MOD

    SED -->|writes| CACHE
    MOD -->|writes| CACHE
    CACHE --> PHOT
    CACHE --> ANCACHE
    CACHE --> STATE
    PHOT -->|feeds| SED
    ANCACHE -->|feeds| SED

    PHOT -->|to| OUTANC
    PHOT -->|to| OUTEBV
    SED -->|plots| OUTPNG
    SED -->|exports| OUTSED

    %% Web flows
    WDATA -->|serves data to| API
    API -->|serves| ASSETS
    ORCH -->|deploys| API

    %% Click Events
    click RD "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/data/"
    click CD "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/data/filters/"
    click DM "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/models/bt-settl/"
    click ANC "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/ancillary.default"
    click ANC "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/ancillary.ness"
    click CAT "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/catalogues.default"
    click CAT "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/catalogues.ness"
    click CAT "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/catalogues.test"
    click FIL "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/filters.default"
    click FIL "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/filters.ness"
    click FIL "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/filters.test"
    click CON "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/constraints.default"
    click SET "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/setup.default"
    click SET "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/setup.ness"
    click SET "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/setup.ness2"
    click SET "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/setup.test"
    click SED "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/pyssed.py"
    click MOD "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/makemodel.py"
    click REJ "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/reject.default"
    click HLP "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/outputheaders.scr"
    click HLP "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/shorten-model.scr"
    click CACHE "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/proc/"
    click PHOT "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/proc/phot/"
    click ANCACHE "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/proc/anc/"
    click STATE "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/proc/last_cmd.npy"
    click OUTANC "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/output/anc/"
    click OUTEBV "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/output/ebvdata/"
    click OUTPNG "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/output/png/"
    click OUTSED "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/output/sed/"
    click ORCH "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/src/gtomo/Dockerfile"
    click ORCH "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/gtomo/docker-compose.yml"
    click ORCH "https://github.com/iain-mcdonald/pyssed/blob/v1.1.20250129/src/gtomo/.gitlab-ci.yml"
    click API "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/src/gtomo/sda/"
    click ASSETS "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/src/gtomo/sda/assets/"
    click WDATA "https://github.com/iain-mcdonald/pyssed/tree/v1.1.20250129/src/gtomo/_data/app_data/"

    %% Styles
    classDef data fill:#bbdefb,stroke:#2196f3,stroke-width:2px;
    classDef proc fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
    classDef config fill:#fff9c4,stroke:#fbc02d,stroke-width:2px;
    classDef output fill:#ffe0b2,stroke:#ef6c00,stroke-width:2px;
    classDef web fill:#e1bee7,stroke:#8e24aa,stroke-width:2px;
```
