How tirthankar4/elemental-maps-using-class-data works
Project Type and Purpose
• This is a batch-oriented scientific data-processing pipeline (not a web service or full-stack app).
• It automates analysis of Chandrayaan-2 CLASS L1 XRF data, extracting solar-flare windows, performing spectral corrections, and producing sub-pixel elemental ratio maps.
Top-Level Components
A. Data Ingestion & Flare Sorting
– Inputs: CLASS L1 files, flare catalogs (Hinode/XRT), SPICE kernels
– Scripts: fetch-flares.py, fetch-flares_hxrt.py
B. Spectral Aggregation & Background Generation
– Scripts: addClassSpectra.py (aggregates 8s spectra), gen_bg.py (nighttime background)
C. Spectral Correction & Peak Detection
– Libraries: HEASoft / PyXSPEC, OGIP ARF/RMF
– Scripts: arfCorrection2.py, responseCorrectedSpecrtum*.py, catalogue.py (GMM peak detection)
D. Ratio Calculation & Mapping
– Scripts: fluxRatios/* (ratio computation: Mg/Si, Al/Si, …), colorMap.py (projection, plotting)
E. Sub-Pixel Resolution Enhancement
– Scripts: SubpixelResolution/drizzle.py, interlacing_*.py, processSubsets.py
External Services & Infrastructure
• HPC Cluster / SLURM scheduler
– Job scripts: fluxRatios/SLURM_JOB.sh, fluxRatios/python_job.sh
• Geo/Spatial Libraries
– GeoPandas, Cartopy (for geospatial reprojection)
• Visualization
– Matplotlib, Blender (3D), plus a future web-based 2D/3D viewer
Data Stores & Artifacts
• Raw Data Store: CLASS L1 repository (5 TB)
• Intermediate FITS files (8s spectra aggregates, background files)
• Final Products: GeoTIFFs or image tiles of elemental ratios (4.125 km resolution)
• Metadata: logs, catalogue outputs, notebooks (mareDetection subset)
Relationships & Data Flow
Ingestion → Aggregation → Correction → Mapping → Subpixel
• Bidirectional or feedback loops if any (e.g., parameter tuning between Correction ↔ Catalogue)
• Python scripts under “Compute Layer” and external libraries under “Dependency Layer.”
Architectural Patterns & Design Principles
• Pipeline (staged, batch) pattern: each stage transforms and enriches data
• Modular scripts: loosely coupled, can run independently or on HPC
• Extensibility: new elements or machine-learning modules can slot into Correction or Mapping stages
• Reproducibility: SLURM jobs encapsulate environment and resource specs