How roshan-research/hazm works
Project Type and Core Components
a. Main Components to Include:
• Core NLP Processors (center)
• Data Resources (bottom)
• Corpus Readers (left)
• Model Management (right)
• External Integrations (top)
Component Relationships
• Text Preprocessing
• Normalizer
• Tokenizers (Word/Sentence)
• Token Splitter
• Language Analysis
• POS Tagger
• Lemmatizer
• Stemmer
• Chunker
• Advanced Processing
• Dependency Parser
• Named Entity Recognition
• Word/Sentence Embedding
• Static Data Files
• abbreviations.dat
• verbs.dat
• words.dat
• stopwords.dat
• Pre-trained Models
• Word Embeddings
• POS Tagger Models
• Parser Models
• Chunker Models
C. Corpus Readers (use parallelograms):
• Readers by type:
• Treebank Readers
• Text Corpus Readers
• Specialized Corpus Readers
Technical Details
Annotations showing:
• Python version compatibility
• Key dependencies
• Performance metrics (as shown in README)
• Model sizes and requirements
Additional Elements
• Evaluation metrics (85.6% for DependencyParser, 98.8% for POSTagger, etc.)
• Integration with other language versions (JHazm, NHazm)
• GitHub Actions workflow integration