# yanlinf/casent architecture

> Architecture diagram of the GitHub repository yanlinf/casent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yanlinf/casent
- Repository: https://github.com/yanlinf/casent

## Overview

##### Project type and purpose
- CASENT is an open-source NLP library and command-line toolkit for ultra-fine entity typing and extraction.
- It wraps a seq2seq T5 backbone with calibration, label‐space management (UFET and WikiData ontologies), dataset utilities, and optional span detection.

##### File structure
- Top‐level “casent/” directory contains the core Python package:
- app.py – REST/CLI entrypoint for serving predictions
- entity_typing_t5.py – Predictor classes (API and CLI wrappers)
- calibration.py / dataset.py / trie.py – Model calibration and dataset handling
- utils/ – Helpers for LLM backends, NER utils, SPARQL queries, visualization
- “scripts/” holds orchestration scripts: data download, benchmarking, evaluation, serving.
- “ontology_data/” holds static label vocabularies and mappings (JSON, CSV).
- Training/inference/evaluation drivers at top level: train_t5.py, predict_t5.py, evaluate_ufet_prediction.py.

##### README for insights
- Core components: T5 seq2seq model, UFET and WikiData ontologies, span detector (optional).
- Usage modes: raw API calls, CLI scripts, demonstration server.
- External dependencies: HuggingFace model hub, SPARQL endpoint for ontology lookups, stanza parser.

##### Main components and their interactions
a. Core Library (“casent/” Python package)
- Model Module: wraps T5ForConditionalGeneration with calibration layer
- Dataset Module: loads UFET/WikiData labels, builds prefix‐trie for decoding
- Utils Module:
– llm_backend: abstracts HF/LLM calls
– ner_utils & ufet_utils: preprocessing and postprocessing
– wikidata_sparql & ontology loaders: external ontology services
– visualization: output formatting
b. CLI & API Layer
- train_t5.py: orchestrates data loading → model instantiation → training loop
- predict_t5.py: loads checkpoint → runs batched inference → dumps JSON
- evaluate_ufet_prediction.py: computes metrics on predictions
- scripts/serve_casent_app.sh & casent/app.py: spins up REST service
c. Data & Ontology Layer
- ontology_data/: static JSON/CSV used at runtime for label mapping
- scripts/download_ufet.sh: fetches raw UFET data
d. External Services
- HuggingFace Hub: pre-trained T5 checkpoints
- Wikidata SPARQL endpoint: optional RDF lookups

Relationships:
– Training pipeline pulls raw data → dataset module → core model → checkpoint
– Inference pipeline loads checkpoint → predictor → utils postprocess → output
– Serving pipeline wraps inference in REST endpoints

##### Architectural patterns & design principles
- Layered design:
- Data Layer (ontology_data, scripts)
- Core Logic Layer (casent package)
- Interface Layer (CLI scripts, REST via app.py)
- Single‐pass autoregressive decoding with calibrated confidence
- Modular utilities for ontology and parser extensibility

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Layer
    subgraph "Data & Ontology Layer"
        OData[(Ontology Data Store)]:::data
        Scripts["Orchestration & Evaluation Scripts"]:::interface
    end

    OData -->|load labels| Loader
    Scripts -->|download| OData

    %% Core Logic Layer
    subgraph "Core Logic Layer"
        Loader["Dataset Loader & Prefix-Trie Builder"]:::core
        Model["T5 Model"]:::core
        Calib["Calibration Layer"]:::core
        Predictor["Predictor (casent.entity_typing_t5)"]:::core
        subgraph "Utilities"
            LLMB["llm_backend"]:::core
            NERU["ner_utils"]:::core
            UFETU["ufet_utils"]:::core
            VISU["visualization_utils"]:::core
            ONTO["wikidata_ontology"]:::core
            SPARQ["wikidata_sparql"]:::core
            HELP["helpers"]:::core
        end
    end

    Loader -->|train/predict| Model
    Model <-->|calibrate| Calib
    Model -->|generate| Predictor
    Predictor -->|post-process| LLMB
    Predictor --> NERU
    Predictor --> UFETU
    Predictor --> VISU
    Predictor --> ONTO
    Predictor --> SPARQ
    Predictor --> HELP

    %% Interface Layer
    subgraph "Interface Layer"
        CLITrain["train_t5.py"]:::interface
        CLIPredict["predict_t5.py"]:::interface
        CLIEval["evaluate_ufet_prediction.py"]:::interface
        REST["REST Service (casent/app.py)"]:::interface
        Shell["serve_casent_app.sh"]:::interface
        Output["Output JSON/REST Response"]:::interface
    end

    CLITrain -->|train| Model
    CLIPredict -->|predict| Predictor
    CLIEval -->|evaluate| Predictor
    REST -->|REST call| Predictor
    Shell --> REST
    Predictor -->|response| Output

    %% External Services
    subgraph "External Services"
        HF[(HuggingFace Hub)]:::ext
        SPEND[(SPARQL Endpoint)]:::ext
        STANZA[(Stanza Parser)]:::ext
    end

    Predictor -->|fetch model| HF
    Predictor -->|query ontology| SPEND
    Predictor -->|parse text| STANZA

    %% Click Events
    click OData "https://github.com/yanlinf/casent/tree/main/ontology_data/"
    click Loader "https://github.com/yanlinf/casent/blob/main/casent/dataset.py"
    click Loader "https://github.com/yanlinf/casent/blob/main/casent/trie.py"
    click Model "https://github.com/yanlinf/casent/blob/main/casent/entity_typing_t5.py"
    click Calib "https://github.com/yanlinf/casent/blob/main/casent/calibration.py"
    click CLITrain "https://github.com/yanlinf/casent/blob/main/train_t5.py"
    click CLIPredict "https://github.com/yanlinf/casent/blob/main/predict_t5.py"
    click CLIEval "https://github.com/yanlinf/casent/blob/main/evaluate_ufet_prediction.py"
    click REST "https://github.com/yanlinf/casent/blob/main/casent/app.py"
    click Shell "https://github.com/yanlinf/casent/blob/main/scripts/serve_casent_app.sh"
    click LLMB "https://github.com/yanlinf/casent/blob/main/casent/utils/llm_backend.py"
    click NERU "https://github.com/yanlinf/casent/blob/main/casent/utils/ner_utils.py"
    click UFETU "https://github.com/yanlinf/casent/blob/main/casent/utils/ufet_utils.py"
    click VISU "https://github.com/yanlinf/casent/blob/main/casent/utils/visualization_utils.py"
    click ONTO "https://github.com/yanlinf/casent/blob/main/casent/utils/wikidata_ontology.py"
    click SPARQ "https://github.com/yanlinf/casent/blob/main/casent/utils/wikidata_sparql.py"
    click HELP "https://github.com/yanlinf/casent/blob/main/casent/utils/helpers.py"
    click Scripts "https://github.com/yanlinf/casent/blob/main/scripts/benchmark.py"
    click Scripts "https://github.com/yanlinf/casent/blob/main/scripts/download_ufet.sh"
    click Scripts "https://github.com/yanlinf/casent/blob/main/scripts/evaluate_chatgpt_ufet.py"
    click Scripts "https://github.com/yanlinf/casent/blob/main/scripts/evaluate_lite.py"
    click Scripts "https://github.com/yanlinf/casent/blob/main/scripts/plot_beam_size.py"

    %% Styles
    classDef data fill:#fff2cc,stroke:#333,stroke-width:1px;
    classDef core fill:#e0ffe0,stroke:#333,stroke-width:1px;
    classDef interface fill:#e0f0ff,stroke:#333,stroke-width:1px;
    classDef ext fill:#efefef,stroke:#333,stroke-width:1px;
```
