# ctlearn-project/ctlearn architecture

> Architecture diagram of the GitHub repository ctlearn-project/ctlearn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ctlearn-project/ctlearn
- Repository: https://github.com/ctlearn-project/ctlearn

## Overview

##### Project Overview and Type
- CTLearn is a specialized Python package for deep learning-based event reconstruction from IACT data. It loads data from various telescopes (e.g., CTAO, FACT, H.E.S.S., LST-1, MAGIC, VERITAS) and integrates with external projects like ctapipe and DL1DataHandler.
- The package targets scientific data analysis and model training/inference, and thus its architecture is centered around modular data processing and machine learning workflows.

##### File Structure
- The top-level directories indicate key components:
- The "ctlearn" directory contains core modules, divided into:
▪ core: Houses modules such as attention, loader, and model. This indicates the heart of the data handling, processing, and model management.
▪ resources: Contains data resource files (e.g., the LST-1 HDF5 file) that support specific telescope subarray descriptions.
▪ tools: Contains scripts (e.g., predict_model.py, train_model.py) providing command-line interfaces for training and inference operations.
▪ __init__.py and utils.py that coordinate the package’s initialization and utility functions.
- The "docs" directory is dedicated to Sphinx-generated documentation, indicating that architectural details and usage instructions are also documented.
- Directories like "notebooks" and "scripts" provide additional usage examples and data processing scripts.
- GitHub workflows, Dockerfile, and conda-related files highlight the integration testing, CI/CD, and container/deployment approaches.

##### Key Components
- Core Modules:
- Data Loader (ctlearn/core/loader.py): Responsible for loading and preprocessing input CT data.
- Model Definition (ctlearn/core/model.py): Contains the definition and architecture of the deep learning model.
- Attention Mechanisms (ctlearn/core/attention.py): Implements specific deep learning operations.
- Tools/Operational Interfaces:
- Training Script (train_model.py): Handles the training routines and model optimization.
- Prediction Tools (predict_model.py, predict_LST1.py): Manage inference tasks.
- Resources:
- Data Files (e.g., LST-1_SubarrayDescription.h5): Support metadata or calibration data.
- Testing and Utilities:
- Test Suite (ctlearn/core/tests): Contains unit tests for core functionalities.
- Utility Functions (ctlearn/utils.py): General helper functions facilitating various operations.
- External Components:
- Integration with external libraries such as ctapipe and DL1DataHandler for data processing.
- Dependency management using Conda (environment.yml, GitHub conda workflows) and containerization (Dockerfile).

##### Architectural Relationships and Data Flow
- The loaded data then flows into the Model module where pre-processing, feature extraction, and deep learning model training occur.
- The Tools layer interfaces with the core modules to expose functionalities such as training, predicting, and evaluating.
- Testing and utility modules serve to validate and support the functionality of the core system, ensuring robustness.
- External integrations (ctapipe, DL1DataHandler) are depicted as separate blocks interfacing with the Data Loader and pre-processing layers.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "External Integrations"
        Ext["External Data Sources (ctapipe,DL1DataHandler)"]:::cloud
    end

    subgraph "Core Modules"
        Loader["Data Loader"]:::core
        Model["Deep Learning Model"]:::core
        Attention["Attention Mechanisms"]:::core
        Ext -->|"loads data"| Loader
        Loader -->|"preprocesses data"| Model
        Loader -->|"applies"| Attention
        Attention -->|"enhances"| Model
    end

    subgraph "Operational Tools"
        Train["Training Script"]:::ops
        Pred1["Prediction Tool (predict_model.py)"]:::ops
        Pred2["Prediction Tool (predict_LST1.py)"]:::ops
        Model -->|"trains model"| Train
        Model -->|"runs inference"| Pred1
        Model -->|"runs inference"| Pred2
    end

    subgraph "Supporting & Resources"
        Resources["Data Resource Files"]:::resources
        Utils["Utility Functions"]:::resources
        Tests["Testing Suite"]:::resources
        Model -->|"uses metadata"| Resources
        Model -->|"validated by"| Tests
        Model -->|"assisted by"| Utils
    end

    subgraph "Documentation"
        Docs["Documentation"]:::docs
    end

    subgraph "DevOps & Deployment"
        Env["Environment Configuration"]:::devo
        Docker["Dockerfile"]:::devo
        GitHub["CI/CD & GitHub Workflows"]:::devo
    end

    click Loader "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/core/loader.py"
    click Model "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/core/model.py"
    click Attention "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/core/attention.py"
    click Train "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/tools/train_model.py"
    click Pred1 "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/tools/predict_model.py"
    click Pred2 "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/tools/predict_LST1.py"
    click Resources "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/resources/LST-1_SubarrayDescription.h5"
    click Tests "https://github.com/ctlearn-project/ctlearn/tree/master/ctlearn/core/tests"
    click Utils "https://github.com/ctlearn-project/ctlearn/blob/master/ctlearn/utils.py"
    click Docs "https://github.com/ctlearn-project/ctlearn/tree/master/docs/source"
    click Env "https://github.com/ctlearn-project/ctlearn/blob/master/environment.yml"
    click Docker "https://github.com/ctlearn-project/ctlearn/tree/master/Dockerfile"
    click GitHub "https://github.com/ctlearn-project/ctlearn/blob/master/.github"

    classDef cloud fill:#cce5ff,stroke:#003366,stroke-width:2px;
    classDef core fill:#ffdddd,stroke:#cc0000,stroke-width:2px;
    classDef ops fill:#ddffdd,stroke:#228B22,stroke-width:2px;
    classDef resources fill:#ffffcc,stroke:#999900,stroke-width:2px;
    classDef docs fill:#e6e6ff,stroke:#6666ff,stroke-width:2px;
    classDef devo fill:#ffe6cc,stroke:#ff9900,stroke-width:2px;
```
