# matteobachetti/toaextractor architecture

> Architecture diagram of the GitHub repository matteobachetti/toaextractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/matteobachetti/toaextractor
- Repository: https://github.com/matteobachetti/toaextractor

## Overview

##### Project type and purpose
- **Type**: Open-source Python library/tool built on the Astropy ecosystem.
- **Purpose**: Ingest pulsar event data, compute Times Of Arrival (TOAs) and timing residuals, then produce summary tables and plots.

##### File structure
- **Top-level**:
- setup.py, pyproject.toml, setup.cfg → packaging / installation
- tox.ini, .github/workflows → CI/CD
- README.rst, docs/ → user documentation
- **Core package**: `toa_extractor/`
- **pipeline.py** – orchestrates the main TOA-extraction workflow
- **plotting.py** – plotting routines for residuals and profiles
- **summary.py** – collates results into tables/reports
- **data_setup.py**, `data/` – built-in templates and example ECVS catalogs
- **utils/** – helper modules (config management, data slicing, FFT fitting, folding, profile fitting)
- **Tests**: `toa_extractor/tests/` and `toa_extractor/utils/tests/` for unit and integration tests.

##### Extract insights from the README
- Emphasis on Astropy as a foundational dependency.
- Core functionality: reading event files, template profiles, folding, FFT-based fitting, TOA/residual calculation.

a. **User Interface / Entry Points**
– CLI or Python API calls (e.g. `toa_extractor.pipeline.run(...)`)
b. **Data Inputs**
– Raw event files (FITS, .evt, .evt.gz)
– Template definitions (ECSV, YAML)
c. **Processing Engine**
– `pipeline.py`: high-level workflow
– `data_setup.py`: loads templates & data catalogs
– **Utils sub-modules**:
· `data_manipulation.py` (slice/filter events)
· `fold.py` (phase-folding routines)
· `fftfit.py`, `fit_crab_profiles.py` (profile fitting)
· `config.py` (runtime configuration)
d. **Results / Output**
– TOA tables (ECSV, ASCII) via `summary.py`
– Residual plots via `plotting.py`
e. **External Dependencies**
– Astropy (I/O, table structures, FITS handling)
– NumPy, SciPy (numerical routines)
– Matplotlib (plotting)
f. **Quality & Packaging**
– Test suite (pytest)
– CI workflows (.github/workflows)
– Documentation build (Sphinx under docs/)

##### Relationships and data flows
- **CI & Docs** can be shown as side modules that consume source code but do not affect runtime flow.

##### Architectural patterns and design principles
- **Modular design**: clear separation between orchestration (`pipeline.py`), utilities, and I/O
- **Plug-in style** for templates & data via `data/` and config files → extensibility
- **Test isolation**: each utils module has its own test folder

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "User Interface / Entry Points"
        UI["CLI / API Entry Point"]:::core
    end

    subgraph "Data Inputs"
        RawEvt(("Raw event files\n(FITS/.evt/.evt.gz)")):::data
        Templates(("Templates & Catalogs (ECSV/YAML)")):::data
    end

    subgraph "Processing Engine"
        Pipeline["pipeline.py\nOrchestrator"]:::core
        DataSetup["data_setup.py\nLoad templates & catalogs"]:::core
        Config["config.py\nRuntime configuration"]:::core
        DataManip["data_manipulation.py\nEvent slicing/filtering"]:::core
        Fold["fold.py\nPhase-folding routines"]:::core
        FFTFit["fftfit.py\nFFT-based profile fitting"]:::core
        CrabFit["fit_crab_profiles.py\nCrab-profile fitting"]:::core
        Crab["crab.py\nAdditional crab helpers"]:::core
    end

    subgraph "Results / Output"
        Summary["summary.py\nGenerate TOA tables"]:::core
        Plotting["plotting.py\nGenerate residual/profile plots"]:::core
    end

    subgraph "External Dependencies"
        Astropy["Astropy"]:::external
        NumPy["NumPy / SciPy"]:::external
        Matplotlib["Matplotlib"]:::external
    end

    subgraph "Quality & Continuous Integration"
        TestsCore["toa_extractor/tests/"]:::ci
        TestsUtils["toa_extractor/utils/tests/"]:::ci
        CI[".github/workflows"]:::ci
    end

    subgraph "Documentation"
        Docs["docs/ (Sphinx)"]:::docs
        Readme["README.rst"]:::docs
    end

    %% Data Flow
    UI -->|invokes| Pipeline
    Pipeline -->|loads data| DataSetup
    DataSetup --> Templates
    DataSetup --> RawEvt
    Pipeline --> Config
    Pipeline --> DataManip
    DataManip --> Fold
    Fold --> FFTFit
    Fold --> CrabFit
    CrabFit --> Summary
    FFTFit --> Summary
    Pipeline --> Summary
    Summary --> Plotting

    %% Dependencies
    Pipeline -.->|uses| Astropy
    DataManip -.->|uses| NumPy
    FFTFit -.->|uses| SciPy
    Plotting -.->|uses| Matplotlib

    %% CI & Docs
    CI -.-> Pipeline
    CI -.-> DataSetup
    CI -.-> Config
    CI -.-> DataManip
    CI -.-> Fold
    CI -.-> FFTFit
    CI -.-> CrabFit
    CI -.-> Summary
    CI -.-> Plotting
    TestsCore -.-> Pipeline
    TestsCore -.-> Summary
    TestsUtils -.-> DataManip
    TestsUtils -.-> Fold
    TestsUtils -.-> FFTFit
    Docs -.-> Pipeline
    Docs -.-> Summary
    Readme -.-> UI

    %% Click Events
    click Pipeline "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/pipeline.py"
    click DataSetup "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/data_setup.py"
    click Config "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/config.py"
    click DataManip "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/data_manipulation.py"
    click Fold "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/fold.py"
    click FFTFit "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/fftfit.py"
    click CrabFit "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/fit_crab_profiles.py"
    click Crab "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/utils/crab.py"
    click Summary "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/summary.py"
    click Plotting "https://github.com/matteobachetti/toaextractor/blob/main/toa_extractor/plotting.py"
    click Templates "https://github.com/matteobachetti/toaextractor/tree/main/toa_extractor/data/"
    click TestsCore "https://github.com/matteobachetti/toaextractor/tree/main/toa_extractor/tests/"
    click TestsUtils "https://github.com/matteobachetti/toaextractor/tree/main/toa_extractor/utils/tests/"
    click CI "https://github.com/matteobachetti/toaextractor/tree/main/.github/workflows/"
    click Docs "https://github.com/matteobachetti/toaextractor/tree/main/docs/"
    click Readme "https://github.com/matteobachetti/toaextractor/blob/main/README.rst"

    %% Styles
    classDef core fill:#E0F7FA,stroke:#039BE5,color:#000;
    classDef data fill:#FFF3E0,stroke:#FB8C00,color:#000;
    classDef external fill:#E8F5E9,stroke:#43A047,color:#000;
    classDef ci fill:#ECEFF1,stroke:#90A4AE,color:#000;
    classDef docs fill:#F3E5F5,stroke:#8E24AA,color:#000;
```
