How afm-spm/topostats works
Project Context and Purpose
• TopoStats is an AFM (Atomic Force Microscopy) image analysis tool designed for batch processing and statistical analysis of images. Its core purpose is to read, filter, process, and extract meaningful measurements from AFM scans.
• The project is distributed as a Python package via PyPI, supports command-line execution (e.g. “topostats process”), and provides extensive documentation (via ReadTheDocs) and interactive examples (Jupyter notebooks).
Main Components
• Command-Line Interface (CLI) & Entry Point:
• Represented by scripts like topostats/__main__.py and tests like tests/test_entry_point.py. The CLI accepts arguments (e.g. configuration file path) and triggers the processing pipeline.
• Configuration Management:
• The project uses YAML configuration files (default and user-provided) to validate and set runtime parameters. Modules such as the default_config.yaml, simple_config.yaml, and summary_config.yaml illustrate this.
• Core Processing Pipeline:
• The main processing logic is contained within the “topostats” package. Submodules include:
▸ IO: Reading input image files, parsing scan formats.
▸ Filters: Pre-processing and filtering of images.
▸ Grains and Grain Statistics (grains.py, grainstats.py): Detecting features and extracting statistical measures.
▸ Processing and Run Modules: Modules like processing.py and run_modules.py that manage the workflow end-to-end.
▸ Plotting and Visualization: Modules like plotting.py, plottingfuncs.py, and the associated YAML for plotting_dictionary to generate visual summaries.
▸ Tracing & Measurement Components: The “topostats/tracing” package (and related measure submodules) handles complex image tracing, including disordered/ordered tracing, nodestats, splining, and even DNA curvature analysis.
▸ Logging: The logs module (topostats/logs) captures runtime events.
• External Interfaces:
• Jupyter Notebooks and tests directories indicate support for interactive use and regression verification.
• Documentation and continuous integration (GitHub workflows, ReadTheDocs configuration) serve as supporting infrastructure.
Relationships and Interactions Between Components
• The CLI invokes the entry point, which reads the default or user-supplied YAML configuration.
• The configuration manager sets parameters that guide the behavior of subsequent modules.
• The IO module imports scan files (from local directories), passes data to the processing pipeline.
• Filters, grains, and tracing modules work sequentially (or according to a set pipeline) to process images, extract features, and perform statistical analysis.
• Results are then passed to the plotting and logging modules, generating output files, images, and log entries.
• Testing and CI/CD components (tests, GitHub workflows) validate each part of the pipeline through unit and regression tests.
Architectural Patterns and Design Principles
• The structure suggests a modular, layered design where each module is responsible for a specific part of the processing pipeline.
• There is a clear separation of concerns: configuration & CLI handling, image processing logic, visualization, and logging are distinct.
• Using a plug-in or pipeline model appears to be a central idea, where components (e.g., filters, tracing algorithms) can be independently modified or extended.
• Emphasis on testability is evident through the extensive tests directory and regression test outputs.
Including Specific Technologies and Frameworks
• Python (general language and package structure).
• YAML for configuration and plotting dictionary.
• Pytest (for tests) and CI integration via GitHub workflows.
• Documentation is built using Sphinx (docs/ and the .readthedocs.yaml file).
Final Advice
• Be as detailed as possible to capture the design’s modular nature – separate each of the functional areas into distinct components.
• Use the project’s folder separation (topostats, docs, tests, notebooks) as a guide to delineate which areas are part of the core system and which are peripheral (testing, docs, CI/CD).