# tshatrov/ichiran architecture

> Architecture diagram of the GitHub repository tshatrov/ichiran, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tshatrov/ichiran
- Repository: https://github.com/tshatrov/ichiran

## Overview

##### Main Components
- Core Processing Engine: Several Lisp source files (e.g., characters.lisp, romanize.lisp, cli.lisp, ichiran.lisp, and many dict-*.lisp files) form the heart of the system. These contain the algorithms for romanization, segmentation, and dictionary management.
- Dictionary Management: Multiple modules (like dict-counters.lisp, dict-custom.lisp, dict-errata.lisp, dict-fix.lisp, dict-grammar.lisp, dict-load.lisp, etc.) handle importing, processing, and managing the JMdictDB data and other dictionary sources.
- Data Sources: The “data” directory contains CSV and XML files (e.g., romaji-map.csv, extra.xml, gyoseiku.csv, jichitai.csv) that supply essential mapping and supplementary information.
- Database Layer: The project utilizes a Postgres database (as indicated by docker/postgres-dockerfile, docker/postgres-initdb scripts, and the instructions to initialize or recreate the database). This is the backend for storing and querying dictionary data.
- Dockerized Deployment: The Docker-related files (e.g., .dockerignore, docker-compose.yml, docker scripts) show that the system is containerized. There are specific containers for SBCL (the Common Lisp runtime), for running the CLI, and for managing the Postgres database.
- Testing & Maintenance: Files such as tests.lisp and maintenance commands in the README indicate automated test suites and DB initialization/maintenance utilities.
- (Future) Web Interface: Though still experimental and under development, the README refers to a web interface currently accessible at ichi.moe.

##### Relationships and Interactions
- The CLI and (future) web interface act as front-ends that receive user input.
- These front-ends interact with the core processing engine, which processes text (e.g., romanizing and segmenting input) by calling various utility and dictionary management modules.
- The engine consults the database for extra information (word meanings, readings, etc.) via connection modules (e.g., conn.lisp) and potentially via designated functions in settings.lisp.
- Docker orchestrates these components by running separate containers (e.g., SBCL container for the Lisp code and a Postgres container for the database). The docker-compose setup defines and links these containers together so that inter-component communication is seamless.
- Maintenance scripts and test suites serve as auxiliary elements ensuring that the database is properly initialized and that the functionality of the text-processing modules is validated.

##### Architectural Patterns & Design Principles
- Separation of Concerns: The system is modular with clearly delineated responsibilities (text processing vs. dictionary management vs. database operations vs. container orchestration).
- Loose Coupling & Clear Dependencies: The core processing components depend on specific modules (e.g., dictionary loading) and external data (CSV, XML, and database), but they are orchestrated through well-defined ASDF systems.
- Containerization: Docker file configuration reinforces a micro-service-like environment where each service (database and the Lisp runtime) remains isolated yet interconnected.
- Extensibility: With a clean division between core algorithm modules and I/O components (CLI, web interface), extending the web interface or adding new language processing features should be straightforward.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interfaces
    subgraph "User Interfaces"
        UI_CLI["CLI – Input Processor"]
        UI_Web["Web Interface (Experimental)"]
    end

    %% Core Processing Engine
    subgraph "Core Processing Engine"
        %% Processing Modules
        subgraph "Processing Modules"
            Lisp1["characters.lisp"]
            Lisp2["romanize.lisp"]
            Lisp3["deromanize.lisp"]
            Lisp4["ichiran.lisp"]
            Lisp5["numbers.lisp"]
            Lisp6["kanji.lisp"]
        end
        %% Dictionary Management
        subgraph "Dictionary Management Modules"
            Dict1["dict-counters.lisp"]
            Dict2["dict-custom.lisp"]
            Dict3["dict-errata.lisp"]
            Dict4["dict-fix.lisp"]
            Dict5["dict-grammar.lisp"]
            Dict6["dict-load.lisp"]
            Dict7["dict-split.lisp"]
            Dict8["dict.lisp"]
        end
    end

    %% Data Sources
    subgraph "Data Sources"
        Data1["romaji-map.csv"]
        Data2["extra.xml"]
        Data3["gyoseiku.csv"]
        Data4["jichitai.csv"]
    end

    %% Database Layer
    subgraph "Database Layer"
        Conn["conn.lisp"]
        DBInit["DB Init (ichiran-db.sh)"]
        DB["Postgres Database"]
    end

    %% Docker & Deployment
    subgraph "Docker & Deployment"
        DockerIgnore[".dockerignore"]
        DockerCompose["docker-compose.yml"]
        DockerScripts["docker/ichiran-scripts"]
        DockerPostgres["docker/postgres-dockerfile"]
        DockerSBCL["docker/sbcl-dockerfile"]
        DockerSettings["docker/settings.lisp"]
    end

    %% Testing & Maintenance
    subgraph "Testing & Maintenance"
        Tests["tests.lisp"]
    end

    %% Connections between major blocks
    UI_CLI -->|"input"| Lisp4
    UI_Web -->|"input"| Lisp4

    %% Core Engine interactions with Data Sources and Database
    Lisp4 -->|"processes"| Data1
    Lisp4 -->|"processes"| Data2
    Lisp4 -->|"processes"| Data3
    Lisp4 -->|"processes"| Data4

    %% Core Processing interacts with Database Layer
    Lisp4 -->|"lookup"| Conn
    Conn -->|"queries"| DB
    DBInit -->|"initializes"| DB

    %% Docker orchestrates components
    DockerScripts -->|"orchestrates"| Lisp4
    DockerSBCL -->|"runs"| Lisp4
    DockerPostgres -->|"runs"| DB

    %% Docker also manages settings
    DockerCompose -->|"configures"| DockerScripts
    DockerSettings -->|"configures"| DockerScripts

    %% Testing & Maintenance interacts with Core and Database
    Tests -->|"verifies"| Lisp4
    Tests -->|"verifies"| DB

    %% Styling
    class UI_CLI,UI_Web ui
    class Lisp1,Lisp2,Lisp3,Lisp4,Lisp5,Lisp6 core
    class Dict1,Dict2,Dict3,Dict4,Dict5,Dict6,Dict7,Dict8 dict
    class Data1,Data2,Data3,Data4 data
    class Conn,DB,DBInit db
    class DockerIgnore,DockerCompose,DockerScripts,DockerPostgres,DockerSBCL,DockerSettings docker
    class Tests test

    %% Click Events for Core Processing Engine
    click Lisp1 "https://github.com/tshatrov/ichiran/blob/master/characters.lisp"
    click Lisp2 "https://github.com/tshatrov/ichiran/blob/master/romanize.lisp"
    click Lisp3 "https://github.com/tshatrov/ichiran/blob/master/deromanize.lisp"
    click Lisp4 "https://github.com/tshatrov/ichiran/blob/master/ichiran.lisp"
    click Lisp5 "https://github.com/tshatrov/ichiran/blob/master/numbers.lisp"
    click Lisp6 "https://github.com/tshatrov/ichiran/blob/master/kanji.lisp"

    %% Click Events for Dictionary Management Modules
    click Dict1 "https://github.com/tshatrov/ichiran/blob/master/dict-counters.lisp"
    click Dict2 "https://github.com/tshatrov/ichiran/blob/master/dict-custom.lisp"
    click Dict3 "https://github.com/tshatrov/ichiran/blob/master/dict-errata.lisp"
    click Dict4 "https://github.com/tshatrov/ichiran/blob/master/dict-fix.lisp"
    click Dict5 "https://github.com/tshatrov/ichiran/blob/master/dict-grammar.lisp"
    click Dict6 "https://github.com/tshatrov/ichiran/blob/master/dict-load.lisp"
    click Dict7 "https://github.com/tshatrov/ichiran/blob/master/dict-split.lisp"
    click Dict8 "https://github.com/tshatrov/ichiran/blob/master/dict.lisp"

    %% Click Event for User Interface
    click UI_CLI "https://github.com/tshatrov/ichiran/blob/master/cli.lisp"

    %% Click Events for Data Sources
    click Data1 "https://github.com/tshatrov/ichiran/blob/master/data/romaji-map.csv"
    click Data2 "https://github.com/tshatrov/ichiran/blob/master/data/sources/extra.xml"
    click Data3 "https://github.com/tshatrov/ichiran/blob/master/data/sources/gyoseiku.csv"
    click Data4 "https://github.com/tshatrov/ichiran/blob/master/data/sources/jichitai.csv"

    %% Click Events for Database Layer
    click Conn "https://github.com/tshatrov/ichiran/blob/master/conn.lisp"
    click DBInit "https://github.com/tshatrov/ichiran/blob/master/docker/postgres-initdb/ichiran-db.sh"

    %% Click Events for Docker & Deployment
    click DockerIgnore "https://github.com/tshatrov/ichiran/blob/master/.dockerignore"
    click DockerCompose "https://github.com/tshatrov/ichiran/blob/master/docker-compose.yml"
    click DockerScripts "https://github.com/tshatrov/ichiran/tree/master/docker/ichiran-scripts"
    click DockerPostgres "https://github.com/tshatrov/ichiran/tree/master/docker/postgres-dockerfile"
    click DockerSBCL "https://github.com/tshatrov/ichiran/tree/master/docker/sbcl-dockerfile"
    click DockerSettings "https://github.com/tshatrov/ichiran/blob/master/docker/settings.lisp"

    %% Click Event for Testing & Maintenance
    click Tests "https://github.com/tshatrov/ichiran/blob/master/tests.lisp"

classDef ui fill:#cce5ff,stroke:#333,stroke-width:2px;
classDef core fill:#b3d9b3,stroke:#333,stroke-width:2px;
classDef dict fill:#ffffcc,stroke:#333,stroke-width:2px;
classDef data fill:#ffcc99,stroke:#333,stroke-width:2px;
classDef db fill:#ffd699,stroke:#333,stroke-width:2px;
classDef docker fill:#d9ead3,stroke:#333,stroke-width:2px;
classDef test fill:#f4cccc,stroke:#333,stroke-width:2px;
```
