# complat/chemotion-converter-app architecture

> Architecture diagram of the GitHub repository complat/chemotion-converter-app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/complat/chemotion-converter-app
- Repository: https://github.com/complat/chemotion-converter-app

## Overview

##### Project Type and Purpose
- chemotion-converter-app is a Python-based conversion service and CLI utility built on Flask. It ingests instrument-specific raw data files, parses them via dynamically registered “reader” plugins, normalizes into an internal dataset/model, then serializes into standardized output formats (JCAMP, ZIP, JSON) via “writer” plugins. It also manages profile definitions with versioned migrations.
- It runs in two modes:
- HTTP API mode: Flask app behind Gunicorn/Nginx
- CLI mode: direct invocation via `python -m converter_app`

##### High-Level File-Structure Analysis
- Top-level:
- converter_app/: core application
– app.py, router.py: Flask entrypoints and request routing
– converters.py, datasets.py, models.py: service-layer conversion logic
– readers/: one submodule per file format, plus a dynamic registration helper
– writers/: output serializers
– profile_migration/: scripts & utilities for profile versioning
– profiles/: JSON/YAML profile definitions
– options.py, utils.py: configuration & shared helpers
- etc/: systemd service + NGINX sample config for production
- requirements/: pinned dependencies
- scripts/: ad‐hoc profile update scripts
- test_manager/: end-to-end test harness (loads sample files, invokes readers & writers, verifies outputs)
- pytest.ini, .pylintrc, GitHub workflows: CI configuration
- No separate “frontend” directory: it’s purely a backend service + CLI.

##### Key README Insights
- Uses Flask for HTTP API, Gunicorn + NGINX in production.
- Configuration via .env (FLASK_APP, HTPASSWD_PATH, Gunicorn port).
- New readers and migrations are generated via CLI commands.
- No traditional database; profiles and user credentials (htpasswd) are file-based.

a. External Layer
- Clients
– HTTP clients (curl, Postman, web front-end)
– CLI users (shell scripts, `python -m converter_app`)
- Reverse Proxy
– NGINX (listening on 80/443, proxy /api/v1 → Gunicorn)
b. Application Layer
- Gunicorn WSGI workers
- Flask App
– app.py: request entrypoint
– router.py: dispatch to converter service
- Conversion Service
– Reader Registry: dynamically discovers reader plugins
– Readers: format-specific parsers under converter_app/readers/
– Dataset & Model Layer: unified in-memory representation
– Writers: format serializers under converter_app/writers/
– Profile Migration Engine: applies migration scripts to profiles
c. Persistence & Configuration
- File System
– Profiles directory (versioned profiles JSON)
– Logs and runtime data (tmpfiles.d)
– htpasswd file for HTTP basic auth
- Environment Variables
– .env.dev / .env.prod for FLASK_APP, GUNICORN_PORT, HTPASSWD_PATH, etc.
d. Dev/Test Automation
- CI pipelines: PyTest + Pylint via GitHub Actions
- test_manager: sample-file harness validating readers & writers

##### Relationships & Data Flow
1. HTTP client → NGINX → Gunicorn → Flask app → router
2. Router invokes Conversion Service with input file
3. Service picks matching Reader → parses file → builds Dataset
4. Dataset passed to Writer → serializes output → Flask response returned
5. CLI path bypasses HTTP layers; directly loads Conversion Service
6. Profiles loaded from file system and migrated as needed

##### Architectural Patterns & Principles
- Plugin Pattern: readers & writers auto-registered via base classes
- Layered Architecture: presentation (Flask/CLI), service (converter logic), persistence (file-based profiles)
- Config-Driven: behavior controlled by environment variables and .env files
- Single Responsibility: each reader/writer handles one format
- Extensibility: new file formats added without core changes

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Layer
    subgraph "External Clients"
        direction TB
        Clients["HTTP Clients"]:::external
        CLIUsers["CLI Users"]:::external
    end

    NGINX["NGINX (Reverse Proxy)"]:::external
    Clients -->|HTTP Request| NGINX
    NGINX -->|proxy /api/v1| Gunicorn["Gunicorn WSGI Workers"]:::app
    CLIUsers -->|CLI Invocation| CLIEntry["CLI Entry Point"]:::app

    %% Application Layer
    subgraph "Application Layer"
        direction TB
        FlaskApp["Flask App\n(app.py)"]:::app
        Router["Request Router\n(router.py)"]:::app
        ConverterCore["Conversion Service Core\n(converters.py)"]:::app
        DatasetModel["Dataset & Model\n(datasets.py, models.py)"]:::app
        ReaderReg["Reader Registry\n(registration helper)"]:::app
        Readers["Reader Plugins"]:::app
        WriterBase["Writer Base\n(writers/base.py)"]:::app
        Writers["Writer Plugins"]:::app
        ProfileMig["Profile Migration Engine"]:::app

        Gunicorn --> FlaskApp
        FlaskApp --> Router
        Router --> ConverterCore
        ConverterCore --> ReaderReg
        ReaderReg --> Readers
        ConverterCore --> DatasetModel
        ConverterCore --> WriterBase
        WriterBase --> Writers
        ConverterCore --> ProfileMig
        ConverterCore -->|returns output| FlaskApp
        FlaskApp -->|HTTP Response| Gunicorn
    end

    %% Persistence & Configuration
    subgraph "Persistence & Config"
        direction TB
        ProfilesFS[(Profiles FS)]:::persistence
        HTPasswd[(htpasswd File)]:::persistence
        Tmpfiles[(Runtime Logs & Data)]:::persistence
        EnvVars["Environment Files\n(.env.dev, .env.prod)"]:::persistence
        ProfilesFS --> ProfileMig
        ProfileMig --> ProfilesFS
        FlaskApp --> HTPasswd
        FlaskApp --> EnvVars
        ConverterCore --> Tmpfiles
    end

    %% Dev/Test Automation
    subgraph "CI & Test Automation"
        direction TB
        GitHubActions["GitHub Actions\n(pylint.yml, pytest.yml)"]:::ci
        PytestIni["pytest.ini"]:::ci
        Pylintrc[".pylintrc"]:::ci
        TestManager["Test Manager\n(test_manager/)"]:::ci
        GitHubActions --> TestManager
        TestManager -->|validates readers & writers| ConverterCore
    end

    %% Click Events
    click FlaskApp "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/app.py"
    click Router "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/router.py"
    click ConverterCore "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/converters.py"
    click DatasetModel "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/datasets.py"
    click DatasetModel "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/models.py"
    click ReaderReg "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/readers/helper/registration.py"
    click Readers "https://github.com/complat/chemotion-converter-app/tree/master/converter_app/readers/"
    click WriterBase "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/writers/base.py"
    click Writers "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/writers/jcamp.py"
    click Writers "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/writers/jcampzip.py"
    click ProfileMig "https://github.com/complat/chemotion-converter-app/tree/master/converter_app/profile_migration/"
    click ProfileMig "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/profile_migration/20250429144529_migration.py"
    click ProfileMig "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/profile_migration/20250429151003_migration.py"
    click ProfileMig "https://github.com/complat/chemotion-converter-app/tree/master/converter_app/profile_migration/utils/"
    click ProfilesFS "https://github.com/complat/chemotion-converter-app/tree/master/converter_app/profiles/"
    click CLIEntry "https://github.com/complat/chemotion-converter-app/blob/master/converter_app/__main__.py"
    click EnvVars "https://github.com/complat/chemotion-converter-app/blob/master/.env.dev"
    click EnvVars "https://github.com/complat/chemotion-converter-app/blob/master/.env.prod"
    click NGINX "https://github.com/complat/chemotion-converter-app/tree/master/etc/nginx/sites-available/default"
    click NGINX "https://github.com/complat/chemotion-converter-app/blob/master/etc/systemd/system/chemotion-converter.service"
    click Tmpfiles "https://github.com/complat/chemotion-converter-app/blob/master/etc/tmpfiles.d/chemotion-converter.conf"
    click GitHubActions "https://github.com/complat/chemotion-converter-app/blob/master/.github/workflows/pylint.yml"
    click GitHubActions "https://github.com/complat/chemotion-converter-app/blob/master/.github/workflows/pytest.yml"
    click PytestIni "https://github.com/complat/chemotion-converter-app/blob/master/pytest.ini"
    click Pylintrc "https://github.com/complat/chemotion-converter-app/blob/master/.pylintrc"
    click TestManager "https://github.com/complat/chemotion-converter-app/tree/master/test_manager/"
    click TestManager "https://github.com/complat/chemotion-converter-app/blob/master/test_manager/cli_manager.py"
    click TestManager "https://github.com/complat/chemotion-converter-app/blob/master/test_manager/utils.py"
    click TestManager "https://github.com/complat/chemotion-converter-app/blob/master/test_manager/utils_test.py"
    click TestManager "https://github.com/complat/chemotion-converter-app/tree/master/test_static/"

    %% Styles
    classDef external fill:#AED6F1,stroke:#2E86C1,stroke-width:2px
    classDef app fill:#ABEBC6,stroke:#27AE60,stroke-width:2px
    classDef persistence fill:#F9E79F,stroke:#B7950B,stroke-width:2px
    classDef ci fill:#D5D8DC,stroke:#566573,stroke-width:2px
```
