# roshan-research/hazm architecture

> Architecture diagram of the GitHub repository roshan-research/hazm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/roshan-research/hazm
- Repository: https://github.com/roshan-research/hazm

## Overview

##### Project Type and Core Components

a. Main Components to Include:
- Core NLP Processors (center)
- Data Resources (bottom)
- Corpus Readers (left)
- Model Management (right)
- External Integrations (top)

##### Component Relationships

- Text Preprocessing
- Normalizer
- Tokenizers (Word/Sentence)
- Token Splitter
- Language Analysis
- POS Tagger
- Lemmatizer
- Stemmer
- Chunker
- Advanced Processing
- Dependency Parser
- Named Entity Recognition
- Word/Sentence Embedding

- Static Data Files
- abbreviations.dat
- verbs.dat
- words.dat
- stopwords.dat
- Pre-trained Models
- Word Embeddings
- POS Tagger Models
- Parser Models
- Chunker Models

C. Corpus Readers (use parallelograms):
- Readers by type:
- Treebank Readers
- Text Corpus Readers
- Specialized Corpus Readers

##### Technical Details

Annotations showing:
- Python version compatibility
- Key dependencies
- Performance metrics (as shown in README)
- Model sizes and requirements

##### Additional Elements

- Evaluation metrics (85.6% for DependencyParser, 98.8% for POSTagger, etc.)
- Integration with other language versions (JHazm, NHazm)
- GitHub Actions workflow integration

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Core["Core NLP Pipeline"]
        direction TB
        subgraph Preprocess["Text Preprocessing"]
            direction TB
            Norm["Normalizer"]
            WTok["Word Tokenizer"]
            STok["Sentence Tokenizer"]
            TSplit["Token Splitter"]
            INorm["Informal Normalizer"]
        end

        subgraph LangAnalysis["Language Analysis"]
            direction TB
            POS["POS Tagger"]
            Lem["Lemmatizer"]
            Stem["Stemmer"]
            Chunk["Chunker"]
        end

        subgraph Advanced["Advanced Processing"]
            direction TB
            DepParse["Dependency Parser"]
            NER["Named Entity Recognition"]
            Embed["Word/Sentence Embedding"]
            SeqTag["Sequence Tagger"]
        end
    end

    subgraph Data["Data Resources"]
        direction TB
        subgraph StaticData["Static Data Files"]
            direction TB
            Abbrev[("abbreviations.dat")]
            Verbs[("verbs.dat")]
            Words[("words.dat")]
            Stop[("stopwords.dat")]
            IVerbs[("iverbs.dat")]
            IWords[("iwords.dat")]
        end

        subgraph Models["Pre-trained Models"]
            direction TB
            WEmbed[("Word Embeddings")]
            POSModel[("POS Models")]
            ParseModel[("Parser Models")]
            ChunkModel[("Chunker Models")]
        end
    end

    subgraph Corpus["Corpus Readers"]
        direction TB
        Tree["Treebank Reader"]
        Dad["Dadegan Reader"]
        Wiki["Wikipedia Reader"]
        Plain["Plain Text Reader"]
        NERRead["NER Reader"]
    end

    subgraph Utils["Utility Functions"]
        direction TB
        Util["Utils"]
    end

    %% Relationships
    Corpus --> Preprocess
    Preprocess --> LangAnalysis
    LangAnalysis --> Advanced
    StaticData --> LangAnalysis
    Models --> Advanced
    Utils --> Core

    %% Click Events
    click Norm "https://github.com/roshan-research/hazm/blob/master/hazm/normalizer.py"
    click WTok "https://github.com/roshan-research/hazm/blob/master/hazm/word_tokenizer.py"
    click STok "https://github.com/roshan-research/hazm/blob/master/hazm/sentence_tokenizer.py"
    click TSplit "https://github.com/roshan-research/hazm/blob/master/hazm/token_splitter.py"
    click INorm "https://github.com/roshan-research/hazm/blob/master/hazm/informal_normalizer.py"
    click POS "https://github.com/roshan-research/hazm/blob/master/hazm/pos_tagger.py"
    click Lem "https://github.com/roshan-research/hazm/blob/master/hazm/lemmatizer.py"
    click Stem "https://github.com/roshan-research/hazm/blob/master/hazm/stemmer.py"
    click Chunk "https://github.com/roshan-research/hazm/blob/master/hazm/chunker.py"
    click DepParse "https://github.com/roshan-research/hazm/blob/master/hazm/dependency_parser.py"
    click NER "https://github.com/roshan-research/hazm/blob/master/hazm/ner.py"
    click Embed "https://github.com/roshan-research/hazm/blob/master/hazm/embedding.py"
    click SeqTag "https://github.com/roshan-research/hazm/blob/master/hazm/sequence_tagger.py"
    click StaticData "https://github.com/roshan-research/hazm/tree/master/hazm/data"
    click Tree "https://github.com/roshan-research/hazm/blob/master/hazm/corpus_readers/treebank_reader.py"
    click Dad "https://github.com/roshan-research/hazm/blob/master/hazm/corpus_readers/dadegan_reader.py"
    click Wiki "https://github.com/roshan-research/hazm/blob/master/hazm/corpus_readers/wikipedia_reader.py"
    click Plain "https://github.com/roshan-research/hazm/blob/master/hazm/corpus_readers/persian_plain_text_reader.py"
    click NERRead "https://github.com/roshan-research/hazm/blob/master/hazm/corpus_readers/ner_reader.py"
    click Util "https://github.com/roshan-research/hazm/blob/master/hazm/utils.py"

    %% Styling
    classDef core fill:#e1f5fe,stroke:#01579b
    classDef preprocess fill:#e8f5e9,stroke:#1b5e20
    classDef advanced fill:#fff3e0,stroke:#e65100
    classDef data fill:#f3e5f5,stroke:#4a148c
    classDef utils fill:#eceff1,stroke:#263238

    class Core,Preprocess,LangAnalysis,Advanced core
    class Data,StaticData,Models data
    class Corpus preprocess
    class Utils utils

    %% Legend
    subgraph Legend
        direction TB
        L1["Core Components"]:::core
        L2["Data Resources"]:::data
        L3["Corpus Readers"]:::preprocess
        L4["Utilities"]:::utils
    end
```
