# jdisset/kmap architecture

> Architecture diagram of the GitHub repository jdisset/kmap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jdisset/kmap
- Repository: https://github.com/jdisset/kmap

## Overview

##### Main Components
- Command Line Interface (CLI) & Input Handling: This component is responsible for reading command line arguments (indicated by flags like –k, –i, –o, –a, etc.), loading configuration and input file lists.
- Sequence File Parser: A module that reads fasta sequence files, performs any preprocessing (like prepending “start” and appending “stop” symbols), and passes the sequences on for kmer analysis.
- Kmer Counter / Matrix Builder: The core logic that identifies each kmer and counts its occurrences. This component builds a matrix where each row corresponds to an input sequence file and each column corresponds to letters (from the chosen alphabet).
- Data Persistence & Database Layer: Integration with RocksDB (built via integrated CMake scripts and triggered on build) manages the storage of intermediate kmer data. It ensures efficient read/write operations, especially for large sequence datasets.
- Conversion / Post-Processing Unit (kconv): Once kmap has computed counts in a sparse format, kconv takes the output data and formats or consolidates it into the final CSV (or another user-specified format).
- Python Integration Layer (pykmap): Via the included pybind11 framework, this module exposes parts of the C++ API to Python, allowing for scripting or integration with Python-based tools.

##### Out Relationships and Interactions
- The CLI module passes parameters to the Sequence Parser.
- Parsed sequence data flows into the Kmer Counter which uses internal helper functions (likely defined in header files like alphabet.hpp, export.hpp, and utils.hpp).
- As counting progresses, the Database Layer (with RocksDB support, as well as considerations for libBz2 and compression libraries configured via CMake) handles data storage and retrieval.
- Once counting is finished, the Kconv component uses the stored data to generate the final output file.
- Optionally, the Python Integration Layer (pykmap) bridges the core C++ functionality to Python. This module is likely built separately using its own CMakeLists.txt under pykmap/cpp.
- The build process, driven by CMake (using the top-level CMakeLists.txt and additional cmake scripts), orchestrates building all modules and pulling in dependencies (like RocksDB).

##### Architectural Patterns & Design Principles
- Modular Design: Each processing step (parsing, counting, conversion) is separated into independent source files and modules.
- Extensibility: The separate “pykmap” module and use of pybind11 indicate a clear design intention to expose functionality for further expansion, e.g., to embed in Python workflows.
- Use of External Libraries & Dependency Management: The use of RocksDB and several third-party header-only libraries is managed via CMake, making dependency resolution transparent.
- Pipeline Processing: The workflow follows a clear pipeline (Input -> Processing -> Data Storage -> Output Conversion).

## Diagram (Mermaid)

```mermaid
flowchart TD
    Build["Build & Configuration\n(CMakeLists.txt, cmake/)"]:::build
    CLI["CLI & Input Module\n(Handles command-line arguments, file input)"]:::core
    Parser["Sequence File Parser\n(Reads fasta files and preprocesses sequences)"]:::core
    Counter["Kmer Counter / Matrix Builder\n(Counts kmers and builds kmer matrix)"]:::core
    subgraph "Data Persistence Layer"
        DB_Core["RocksDB Core\n(rocksdb.cpp, rocksdb.hpp)"]:::external
        DB_Merge["Merge Engine\n(rocksdbmerge.cpp, rocksdbmerge.h)"]:::external
    end
    Conv["Conversion/Post-Processing Unit\n(Formats and consolidates kmer data)"]:::core
    Output["Final Output\n(CSV or user-specified format)"]:::core
    PyInt["Python Integration Layer\n(Exposes C++ API via pybind11)"]:::external

    %% Data flow arrows
    Build --> CLI
    CLI -->|"raw_sequences"| Parser
    Parser -->|"preprocessed_sequences"| Counter
    Counter -->|"kmer_count_data"| DB_Core
    Counter -->|"kmer_count_data"| DB_Merge
    DB_Core -->|"retrieved_data"| Conv
    DB_Merge -->|"retrieved_data"| Conv
    Conv -->|"formatted_output"| Output
    %% Python integration side channel
    PyInt ---|"accesses_API"| Counter
    Build --> PyInt

    %% Click Events
    click CLI "https://github.com/jdisset/kmap/blob/main/main.cpp"
    click Parser "https://github.com/jdisset/kmap/blob/main/seri.cpp"
    click Counter "https://github.com/jdisset/kmap/blob/main/main.cpp"
    click DB_Core "https://github.com/jdisset/kmap/blob/main/rocksdb.cpp"
    click DB_Core "https://github.com/jdisset/kmap/blob/main/rocksdb.hpp"
    click DB_Merge "https://github.com/jdisset/kmap/blob/main/rocksdbmerge.cpp"
    click DB_Merge "https://github.com/jdisset/kmap/blob/main/rocksdbmerge.h"
    click Conv "https://github.com/jdisset/kmap/blob/main/convdb.cpp"
    click PyInt "https://github.com/jdisset/kmap/tree/main/pykmap/cpp/"
    click Build "https://github.com/jdisset/kmap/blob/main/CMakeLists.txt"
    click Build "https://github.com/jdisset/kmap/tree/main/cmake/"

    %% Styles
    classDef core fill:#AED6F1,stroke:#2980B9,stroke-width:2px;
    classDef external fill:#F9E79F,stroke:#F1C40F,stroke-width:2px;
    classDef build fill:#A9DFBF,stroke:#27AE60,stroke-width:2px;
```
