How iain-mcdonald/pyssed works
Project Type and Purpose
• This is an open-source scientific CLI tool (PySSED) for constructing and fitting Spectral Energy Distributions (SEDs) of evolved stars, with an auxiliary interactive 3D tomography web app (“gtomo”).
• Purpose: ingest raw catalog and filter data, generate model grids, run fitting routines via Python scripts (pyssed.py, makemodel.py), and output SED data, plots, and ancillary products. The “gtomo” submodule provides a containerized web frontend/backend for exploring 3D dust cubes.
Top-Level Architecture Layers
• Data Layer
• data/ – raw observational catalogs (/*.dat), filter definitions (XML), distance tables, target lists
• models/ – raw model grids (bt-settle), reduce scripts
• Configuration Layer
• src/*.default, src/*.ness, src/*.test – interchangeable setup files for filters, catalogues, ancillary parameters, and overall program settings
• Core Processing Layer (CLI)
• src/pyssed.py – main driver for SED fitting
• src/makemodel.py – builds model grids based on configuration
• src/reject.* – filtering/rejection logic
• src/*.dat, *.scr – helper data and scripts
• Computation & Cache Layer
• proc/phot and proc/anc – processed NumPy arrays, match tables, intermediate SED data
• proc/last_cmd.npy – tracking execution state
• Output Layer
• output/anc, output/ebvdata, output/png, output/sed – final products: ancillary tables, reddening data, PNG plots, SED files
• Web App Submodule (“gtomo”)
• src/gtomo/Dockerfile + docker-compose.yml + .gitlab-ci.yml – containerized service definition and CI integration
• src/gtomo/sda – Python backend (likely Flask or similar) and static assets (HTML/CSS/CSV) for the interactive interface
• src/gtomo/_data/app_data – precomputed HDF5 cubes and scripts for converting between HDF5 and FITS
Relationships & Data Flows
• CLI Execution Flow
1. User invokes python3 makemodel.py … or python3 pyssed.py …
2. Script loads appropriate config (filters.*, catalogues.*, ancillary.*)
3. Reads raw data from data/ and models/
4. Generates or loads processed arrays under proc/
5. Performs fitting, writes results to output/
• Web App Flow
1. Docker Compose spins up backend service (reading from src/gtomo/_data/app_data)
2. Frontend assets fetch data via REST endpoints in src/gtomo/sda
3. Interactive visualizations served in browser
Architectural Patterns & Principles
• Modular CLI with pluggable configs: allows multiple “setups” (default, ness, test) by swapping simple files
• Data-pipeline pattern: raw → process (NumPy) → output
• Separation of concerns: data storage, computation, and output directories
• Containerization for web component: Dockerfile + docker-compose + CI
• Raw Data Store (data/, models/)
• Config Manager (setup.* files, filters.*, catalogues.*, ancillary.*)
• CLI Processor
• Processing Cache (proc/)
• Output Repository (output/)
• Web Service (“gtomo”)
• Subcomponents of Web Service:
• Container Orchestration (docker-compose.yml)
• Backend API (src/gtomo/sda/*.py)
• Data Layer (src/gtomo/_data/app_data)
• Frontend Static Assets (HTML/CSS under sda/assets)
• CI/CD (.gitlab-ci.yml)