How complat/chemotion-converter-app works
Project Type and Purpose
• chemotion-converter-app is a Python-based conversion service and CLI utility built on Flask. It ingests instrument-specific raw data files, parses them via dynamically registered “reader” plugins, normalizes into an internal dataset/model, then serializes into standardized output formats (JCAMP, ZIP, JSON) via “writer” plugins. It also manages profile definitions with versioned migrations.
• It runs in two modes:
• HTTP API mode: Flask app behind Gunicorn/Nginx
• CLI mode: direct invocation via python -m converter_app
High-Level File-Structure Analysis
• Top-level:
• converter_app/: core application
– app.py, router.py: Flask entrypoints and request routing
– converters.py, datasets.py, models.py: service-layer conversion logic
– readers/: one submodule per file format, plus a dynamic registration helper
– writers/: output serializers
– profile_migration/: scripts & utilities for profile versioning
– profiles/: JSON/YAML profile definitions
– options.py, utils.py: configuration & shared helpers
• etc/: systemd service + NGINX sample config for production
• requirements/: pinned dependencies
• scripts/: ad‐hoc profile update scripts
• test_manager/: end-to-end test harness (loads sample files, invokes readers & writers, verifies outputs)
• pytest.ini, .pylintrc, GitHub workflows: CI configuration
• No separate “frontend” directory: it’s purely a backend service + CLI.
Key README Insights
• Uses Flask for HTTP API, Gunicorn + NGINX in production.
• Configuration via .env (FLASK_APP, HTPASSWD_PATH, Gunicorn port).
• New readers and migrations are generated via CLI commands.
• No traditional database; profiles and user credentials (htpasswd) are file-based.
a. External Layer
• Clients
– HTTP clients (curl, Postman, web front-end)
– CLI users (shell scripts, python -m converter_app)
• Reverse Proxy
– NGINX (listening on 80/443, proxy /api/v1 → Gunicorn)
b. Application Layer
• Gunicorn WSGI workers
• Flask App
– app.py: request entrypoint
– router.py: dispatch to converter service
• Conversion Service
– Reader Registry: dynamically discovers reader plugins
– Readers: format-specific parsers under converter_app/readers/
– Dataset & Model Layer: unified in-memory representation
– Writers: format serializers under converter_app/writers/
– Profile Migration Engine: applies migration scripts to profiles
c. Persistence & Configuration
• File System
– Profiles directory (versioned profiles JSON)
– Logs and runtime data (tmpfiles.d)
– htpasswd file for HTTP basic auth
• Environment Variables
– .env.dev / .env.prod for FLASK_APP, GUNICORN_PORT, HTPASSWD_PATH, etc.
d. Dev/Test Automation
• CI pipelines: PyTest + Pylint via GitHub Actions
• test_manager: sample-file harness validating readers & writers
Relationships & Data Flow
1. HTTP client → NGINX → Gunicorn → Flask app → router
2. Router invokes Conversion Service with input file
3. Service picks matching Reader → parses file → builds Dataset
4. Dataset passed to Writer → serializes output → Flask response returned
5. CLI path bypasses HTTP layers; directly loads Conversion Service
6. Profiles loaded from file system and migrated as needed
Architectural Patterns & Principles
• Plugin Pattern: readers & writers auto-registered via base classes
• Layered Architecture: presentation (Flask/CLI), service (converter logic), persistence (file-based profiles)
• Config-Driven: behavior controlled by environment variables and .env files
• Single Responsibility: each reader/writer handles one format
• Extensibility: new file formats added without core changes