# knot-fit-but/ner architecture

> Architecture diagram of the GitHub repository knot-fit-but/ner, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/knot-fit-but/ner
- Repository: https://github.com/knot-fit-but/ner

## Overview

##### Main Components
- NER Core Engine – Found in the “ner” directory, this component appears to be the orchestrator. It holds configuration files, language-specific modules (e.g., “ner/lang_modules/cs” and “ner/lang_modules/en”), and components to run the actual NER processes.
- Automata Module – Inside the “automata” directory you see many Python scripts (e.g., KB2namelist.py, filter_namelist.py) and shell scripts (create_cedar.sh, deploy.sh). This is likely responsible for automata-based processing like scanning text using deterministic automata techniques to match names or entity patterns.
- Figa Component – The “figa” directory contains C++ code (with files like figa.cpp, figa.h, and specialized modules such as figa_cedar.cpp) that indicates implementation of a fast pattern-matching component (likely using trie or DARTS-based structures). This module would be important for performance-critical sections of entity recognition.
- Name Recognizer Module – In “name_recognizer”, you see Python scripts together with data files (lists such as BLIST, onlyfirst, etc.) that handle the extraction of names from text. It may operate as an independent pipeline or a complementary engine to the NER core.
- Supporting Libraries – The “libs” directory contains several utility modules, including entity loaders, language modules for processing names or word types (organized per language) and nationalities data support. These are shared resources that help standardize processing across different components.

##### Relationships and Interactions
- The NER Core Engine is the main entry point that orchestrates entity recognition by interacting with the other modules.
- The Automata Module provides efficient text scanning services (possibly for pre-processing or searching through large text corpora) which can feed results to the NER core.
- Figa is integrated for its high-performance pattern matching (through its C++ implementation) and may be called from Python or used as an external service.
- The Name Recognizer module complements the NER by handling specific lists of names and applying additional heuristics or filters.
- The Supporting Libraries supply essential functions and data (such as language-specific configurations, entity loading, and nationality data) that are reused by all other components.

##### Architectural Patterns and Design Principles
- Modularity and Separation of Concerns: Each submodule (NER core, automata, figa, name_recognizer, and libs) is self-contained yet interoperates with the others. This separation helps in maintaining and extending specific parts independently.
- Reusability through Libraries: The “libs” directory indicates a design choice to maximize code reuse across different parts of the system by isolating common functionality.

##### Final Advice
- Don’t be afraid to separate components into many nodes if it helps illustrate dependencies and flow among the modules. This is particularly important in a project that mixes high-level orchestration with performance-oriented low-level modules.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% NER Core Engine Subgraph
    subgraph "NER Core Engine"
        ner_configs("Configurations (configs.py)"):::python
        ner_entity("Entity Handler (entity.py)"):::python
        ner_kb("KB Daemon (kb_daemon.py)"):::python
        subgraph "Language Modules"
            ner_cs("Czech Module"):::python
            ner_en("English Module"):::python
        end
    end

    %% Automata Processing Module Subgraph
    subgraph "Automata Processing Module"
        automata_scripts("Main Scripts (automata/src)"):::python
        automata_lang("Lang Modules (automata/src/lang_modules)"):::python
        automata_tests("Testing Suite (automata/tests)"):::python
    end

    %% Figa – Fast Matching Engine Subgraph
    subgraph "Figa – Fast Matching Engine"
        figa_cpp("Core Engine (figa/src/figa.cpp)"):::cpp
        figa_cedar("Cedar Module (figa/src/figa_cedar.cpp)"):::cpp
    end

    %% Name Recognizer Module Subgraph
    subgraph "Name Recognizer Module"
        name_recognizer("Recognizer Engine (name_recognizer/name_recognizer.py)"):::python
        name_data("Data Files (name_recognizer/data)"):::python
    end

    %% Supporting Libraries & Utilities Subgraph
    subgraph "Supporting Libraries & Utilities"
        libs_core("Libraries Core"):::libs
        lib_entities("Entity Loaders (libs/entities)"):::libs
        lib_lang("Language Modules (libs/lang_modules)"):::libs
        lib_nationalities("Nationalities Data (libs/nationalities)"):::libs
    end

    %% Relationships among core components
    ner_configs -->|"leverages"| automata_scripts
    ner_configs -->|"offloads"| figa_cpp
    ner_configs -->|"complements"| name_recognizer

    %% All core components utilize supporting libraries
    ner_configs -->|"utilizes"| libs_core
    automata_scripts -->|"utilizes"| libs_core
    figa_cpp -->|"utilizes"| libs_core
    name_recognizer -->|"utilizes"| libs_core

    %% Additional intra-module relationships
    automata_scripts -->|"includes"| automata_lang
    automata_scripts -->|"validated by"| automata_tests
    figa_cpp -->|"integrated with"| figa_cedar

    %% Click Events for NER Core Engine
    click ner_configs "https://github.com/knot-fit-but/ner/blob/master/ner/configs.py"
    click ner_entity "https://github.com/knot-fit-but/ner/blob/master/ner/entity.py"
    click ner_kb "https://github.com/knot-fit-but/ner/blob/master/ner/kb_daemon.py"
    click ner_cs "https://github.com/knot-fit-but/ner/tree/master/ner/lang_modules/cs"
    click ner_en "https://github.com/knot-fit-but/ner/tree/master/ner/lang_modules/en"

    %% Click Events for Automata Processing Module
    click automata_scripts "https://github.com/knot-fit-but/ner/tree/master/automata/src/"
    click automata_lang "https://github.com/knot-fit-but/ner/tree/master/automata/src/lang_modules/"
    click automata_tests "https://github.com/knot-fit-but/ner/tree/master/automata/tests"

    %% Click Events for Figa – Fast Matching Engine
    click figa_cpp "https://github.com/knot-fit-but/ner/blob/master/figa/src/figa.cpp"
    click figa_cedar "https://github.com/knot-fit-but/ner/blob/master/figa/src/figa_cedar.cpp"

    %% Click Events for Name Recognizer Module
    click name_recognizer "https://github.com/knot-fit-but/ner/blob/master/name_recognizer/name_recognizer.py"
    click name_data "https://github.com/knot-fit-but/ner/tree/master/name_recognizer/data"

    %% Click Events for Supporting Libraries & Utilities
    click libs_core "https://github.com/knot-fit-but/ner/tree/master/libs/"
    click lib_entities "https://github.com/knot-fit-but/ner/tree/master/libs/entities/"
    click lib_lang "https://github.com/knot-fit-but/ner/tree/master/libs/lang_modules/"
    click lib_nationalities "https://github.com/knot-fit-but/ner/tree/master/libs/nationalities/"

    %% Styles
    classDef python fill:#DDEBF7,stroke:#6BA5D7,stroke-width:2px;
    classDef cpp fill:#FCE4D6,stroke:#D8A100,stroke-width:2px;
    classDef libs fill:#E2EFDA,stroke:#3C8031,stroke-width:2px;
```
