# afm-spm/topostats architecture

> Architecture diagram of the GitHub repository afm-spm/topostats, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/afm-spm/topostats
- Repository: https://github.com/afm-spm/topostats

## Overview

##### Project Context and Purpose
- TopoStats is an AFM (Atomic Force Microscopy) image analysis tool designed for batch processing and statistical analysis of images. Its core purpose is to read, filter, process, and extract meaningful measurements from AFM scans.
- The project is distributed as a Python package via PyPI, supports command-line execution (e.g. “topostats process”), and provides extensive documentation (via ReadTheDocs) and interactive examples (Jupyter notebooks).

##### Main Components
- Command-Line Interface (CLI) & Entry Point:
- Represented by scripts like topostats/__main__.py and tests like tests/test_entry_point.py. The CLI accepts arguments (e.g. configuration file path) and triggers the processing pipeline.
- Configuration Management:
- The project uses YAML configuration files (default and user-provided) to validate and set runtime parameters. Modules such as the default_config.yaml, simple_config.yaml, and summary_config.yaml illustrate this.
- Core Processing Pipeline:
- The main processing logic is contained within the “topostats” package. Submodules include:
▸ IO: Reading input image files, parsing scan formats.
▸ Filters: Pre-processing and filtering of images.
▸ Grains and Grain Statistics (grains.py, grainstats.py): Detecting features and extracting statistical measures.
▸ Processing and Run Modules: Modules like processing.py and run_modules.py that manage the workflow end-to-end.
▸ Plotting and Visualization: Modules like plotting.py, plottingfuncs.py, and the associated YAML for plotting_dictionary to generate visual summaries.
▸ Tracing & Measurement Components: The “topostats/tracing” package (and related measure submodules) handles complex image tracing, including disordered/ordered tracing, nodestats, splining, and even DNA curvature analysis.
▸ Logging: The logs module (topostats/logs) captures runtime events.
- External Interfaces:
- Jupyter Notebooks and tests directories indicate support for interactive use and regression verification.
- Documentation and continuous integration (GitHub workflows, ReadTheDocs configuration) serve as supporting infrastructure.

##### Relationships and Interactions Between Components
- The CLI invokes the entry point, which reads the default or user-supplied YAML configuration.
- The configuration manager sets parameters that guide the behavior of subsequent modules.
- The IO module imports scan files (from local directories), passes data to the processing pipeline.
- Filters, grains, and tracing modules work sequentially (or according to a set pipeline) to process images, extract features, and perform statistical analysis.
- Results are then passed to the plotting and logging modules, generating output files, images, and log entries.
- Testing and CI/CD components (tests, GitHub workflows) validate each part of the pipeline through unit and regression tests.

##### Architectural Patterns and Design Principles
- The structure suggests a modular, layered design where each module is responsible for a specific part of the processing pipeline.
- There is a clear separation of concerns: configuration & CLI handling, image processing logic, visualization, and logging are distinct.
- Using a plug-in or pipeline model appears to be a central idea, where components (e.g., filters, tracing algorithms) can be independently modified or extended.
- Emphasis on testability is evident through the extensive tests directory and regression test outputs.

##### Including Specific Technologies and Frameworks
- Python (general language and package structure).
- YAML for configuration and plotting dictionary.
- Pytest (for tests) and CI integration via GitHub workflows.
- Documentation is built using Sphinx (docs/ and the .readthedocs.yaml file).

##### Final Advice
- Be as detailed as possible to capture the design’s modular nature – separate each of the functional areas into distinct components.
- Use the project’s folder separation (topostats, docs, tests, notebooks) as a guide to delineate which areas are part of the core system and which are peripheral (testing, docs, CI/CD).

## Diagram (Mermaid)

```mermaid
graph TD
    %% User Interface & Configuration Layer
    subgraph "User Interface & Configuration"
        CLI_main("CLI Entry: __main__.py"):::ui
        CLI_entry("CLI Entry: entry_point.py"):::ui
        Test_CLI("Test: test_entry_point.py"):::tests
        Config_default("Config: default_config.yaml"):::ui
        Config_simple("Config: simple_config.yaml"):::ui
        Config_summary("Config: summary_config.yaml"):::ui
    end

    %% Processing Core Layer
    subgraph "Processing Core"
        IO("IO Module (io.py)"):::core
        Filters("Filters Module (filters.py)"):::core
        Grains("Grains (grains.py)"):::core
        GrainStats("Grain Stats (grainstats.py)"):::core
        Processing("Processing Module (processing.py)"):::core
        RunModules("Run Modules (run_modules.py)"):::core
        Plotting("Plotting Module (plotting.py)"):::core
        PlotFuncs("Plotting Functions (plottingfuncs.py)"):::core
        PlotDict("Plotting Dictionary (plotting_dictionary.yaml)"):::core
        Logging("Logging (logs.py)"):::aux

        %% Tracing & Measurement Subgraph
        subgraph "Tracing & Measurement"
            %% Tracing Modules
            DisorderedTracing("Disordered Tracing (disordered_tracing.py)"):::core
            OrderedTracing("Ordered Tracing (ordered_tracing.py)"):::core
            DNACurvature("DNA Curvature (dnacurvature.py)"):::core
            Nodestats("Nodestats (nodestats.py)"):::core
            Pruning("Pruning (pruning.py)"):::core
            Skeletonize("Skeletonize (skeletonize.py)"):::core
            Splining("Splining (splining.py)"):::core
            TracingFuncs("Tracing Functions (tracingfuncs.py)"):::core

            %% Measurement Modules
            Curvature("Measurement: Curvature (curvature.py)"):::core
            Feret("Measurement: Feret (feret.py)"):::core
            Geometry("Measurement: Geometry (geometry.py)"):::core
            HeightProfiles("Measurement: Height Profiles (height_profiles.py)"):::core
        end
    end

    %% Testing & Auxiliary Layer
    subgraph "Testing"
        Test_Filters("Test: Filters (test_filters.py)"):::tests
        Test_Filters_Min("Test: Filters Minicircle (test_filters_minicircle.py)"):::tests
        Test_Grains("Test: Grains (test_grains.py)"):::tests
        Test_GrainStats("Test: Grain Stats (test_grainstats.py)"):::tests
        Test_GrainStats_Min("Test: Grain Stats Minicircle (test_grainstats_minicircle.py)"):::tests
        Test_Processing("Test: Processing (test_processing.py)"):::tests
        Test_RunModules("Test: Run Modules (test_run_modules.py)"):::tests
        Test_Plotting("Test: Plotting (test_plotting.py)"):::tests
        Test_PlotFuncs("Test: Plotting Funcs (test_plottingfuncs.py)"):::tests
        Test_Tracing("Tracing Tests"):::tests
        Test_Measure("Measurement Tests"):::tests
    end

    %% Infrastructure Layer
    subgraph "Infrastructure"
        Notebooks("Jupyter Notebooks (notebooks/)"):::infra
        Documentation("Documentation (docs/ & .readthedocs.yaml)"):::infra
        CI("CI Workflows (.github/workflows/)"):::infra
    end

    %% Connections between layers
    CLI_main -->|"reads"| Config_default
    CLI_entry -->|"initiates"| Config_default
    Config_default -->|"configures"| IO

    IO -->|"passes data"| Filters
    Filters -->|"processes"| Grains
    Grains -->|"analyzes"| GrainStats
    GrainStats -->|"triggers"| DisorderedTracing
    DisorderedTracing -->|"feeds"| OrderedTracing
    OrderedTracing -->|"refines"| Splining
    Splining -->|"advances"| Processing

    %% Measurements flow
    GrainStats -->|"extracts"| Curvature
    GrainStats -->|"extracts"| Feret
    GrainStats -->|"extracts"| Geometry
    GrainStats -->|"extracts"| HeightProfiles
    Curvature -->|"merges"| Processing
    Feret -->|"merges"| Processing
    Geometry -->|"merges"| Processing
    HeightProfiles -->|"merges"| Processing

    Processing -->|"orchestrates"| RunModules
    Processing -->|"outputs"| Plotting
    Processing -->|"logs"| Logging
    Plotting -->|"calls"| PlotFuncs
    PlotFuncs -->|"uses"| PlotDict

    %% Testing connections (dashed lines)
    Test_Filters -.-> Filters
    Test_Filters_Min -.-> Filters
    Test_Grains -.-> Grains
    Test_GrainStats -.-> GrainStats
    Test_GrainStats_Min -.-> GrainStats
    Test_Processing -.-> Processing
    Test_RunModules -.-> RunModules
    Test_Plotting -.-> Plotting
    Test_PlotFuncs -.-> PlotFuncs
    Test_Tracing -.-> DisorderedTracing
    Test_Measure -.-> Curvature

    %% Infrastructure connections (dashed lines)
    Notebooks --- CLI_main
    Documentation --- CLI_main
    CI --- Test_CLI

    %% Styles
    classDef ui fill:#FFFACD,stroke:#DAA520,stroke-width:2px;
    classDef core fill:#B0E0E6,stroke:#4682B4,stroke-width:2px;
    classDef tests fill:#FFE4B5,stroke:#CD853F,stroke-width:2px;
    classDef aux fill:#98FB98,stroke:#2E8B57,stroke-width:2px;
    classDef infra fill:#FFC0CB,stroke:#FF0000,stroke-width:2px;

    %% Click Events for CLI & Entry Point
    click CLI_main "https://github.com/afm-spm/topostats/blob/main/topostats/__main__.py"
    click CLI_entry "https://github.com/afm-spm/topostats/blob/main/topostats/entry_point.py"
    click Test_CLI "https://github.com/afm-spm/topostats/blob/main/tests/test_entry_point.py"

    %% Click Events for Configuration Management
    click Config_default "https://github.com/afm-spm/topostats/blob/main/topostats/default_config.yaml"
    click Config_simple "https://github.com/afm-spm/topostats/blob/main/topostats/simple_config.yaml"
    click Config_summary "https://github.com/afm-spm/topostats/blob/main/topostats/summary_config.yaml"

    %% Click Events for IO Module
    click IO "https://github.com/afm-spm/topostats/blob/main/topostats/io.py"

    %% Click Events for Filters Module
    click Filters "https://github.com/afm-spm/topostats/blob/main/topostats/filters.py"
    click Test_Filters "https://github.com/afm-spm/topostats/blob/main/tests/test_filters.py"
    click Test_Filters_Min "https://github.com/afm-spm/topostats/blob/main/tests/test_filters_minicircle.py"

    %% Click Events for Grains and Grain Statistics
    click Grains "https://github.com/afm-spm/topostats/blob/main/topostats/grains.py"
    click GrainStats "https://github.com/afm-spm/topostats/blob/main/topostats/grainstats.py"
    click Test_Grains "https://github.com/afm-spm/topostats/blob/main/tests/test_grains.py"
    click Test_GrainStats "https://github.com/afm-spm/topostats/blob/main/tests/test_grainstats.py"
    click Test_GrainStats_Min "https://github.com/afm-spm/topostats/blob/main/tests/test_grainstats_minicircle.py"

    %% Click Events for Processing and Run Modules
    click Processing "https://github.com/afm-spm/topostats/blob/main/topostats/processing.py"
    click RunModules "https://github.com/afm-spm/topostats/blob/main/topostats/run_modules.py"
    click Test_Processing "https://github.com/afm-spm/topostats/blob/main/tests/test_processing.py"
    click Test_RunModules "https://github.com/afm-spm/topostats/blob/main/tests/test_run_modules.py"

    %% Click Events for Plotting and Visualization
    click Plotting "https://github.com/afm-spm/topostats/blob/main/topostats/plotting.py"
    click PlotFuncs "https://github.com/afm-spm/topostats/blob/main/topostats/plottingfuncs.py"
    click PlotDict "https://github.com/afm-spm/topostats/blob/main/topostats/plotting_dictionary.yaml"
    click Test_Plotting "https://github.com/afm-spm/topostats/blob/main/tests/test_plotting.py"
    click Test_PlotFuncs "https://github.com/afm-spm/topostats/blob/main/tests/test_plottingfuncs.py"

    %% Click Events for Tracing & Measurement Components
    click DisorderedTracing "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/disordered_tracing.py"
    click OrderedTracing "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/ordered_tracing.py"
    click DNACurvature "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/dnacurvature.py"
    click Nodestats "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/nodestats.py"
    click Pruning "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/pruning.py"
    click Skeletonize "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/skeletonize.py"
    click Splining "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/splining.py"
    click TracingFuncs "https://github.com/afm-spm/topostats/blob/main/topostats/tracing/tracingfuncs.py"
    click Curvature "https://github.com/afm-spm/topostats/blob/main/topostats/measure/curvature.py"
    click Feret "https://github.com/afm-spm/topostats/blob/main/topostats/measure/feret.py"
    click Geometry "https://github.com/afm-spm/topostats/blob/main/topostats/measure/geometry.py"
    click HeightProfiles "https://github.com/afm-spm/topostats/blob/main/topostats/measure/height_profiles.py"

    %% Click Events for Logging
    click Logging "https://github.com/afm-spm/topostats/blob/main/topostats/logs/logs.py"
    click Documentation "https://github.com/afm-spm/topostats/tree/main/docs/"
    click CI "https://github.com/afm-spm/topostats/tree/main/.github/workflows/"
    click Notebooks "https://github.com/afm-spm/topostats/tree/main/notebooks/"
```
