# asyrofist/extractreq architecture

> Architecture diagram of the GitHub repository asyrofist/extractreq, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/asyrofist/extractreq
- Repository: https://github.com/asyrofist/extractreq

## Overview

##### Project type and purpose
- This is a Python-based NLP library/tool (not a full-stack web app).
- It provides a set of “extractreq” modules for extracting requirement-dependency information from text, using multiple engines (spaCy, Stanford CoreNLP, custom part-extractor, triplet extractor).
- It packages itself as a pip-installable library and exposes a simple API/CLI for batch processing of SRS or GRD spreadsheets.

##### File structure
- Top-level packaging and metadata:
– setup.py, pyproject.toml, requirements.txt → packaging and dependency definitions
– dist/ → built releases (wheels, tar.gz)
- Source code under src/extractreq:
– modul_ekspart.py → core “evolutionary requirement part” extractor
– modul_spacySent.py → clause splitter using spaCy
– modul_stanfordSent.py → clause splitter using Stanford CoreNLP
– modul_triplet.py → SPO triplet extractor
– usecase_modul1.py, usecase_modul2.py, usecase_modul3.py → end-to-end examples/pipelines
- docs/ and notebook/ → Sphinx documentation and Jupyter notebooks for experimenting and tutorials
- CI/config: .travis.yml, .readthedocs.yaml

##### Architecture insights from README
- The library exposes classes (partOf, stanford_clause, spacyClause, extractNlp)
- You feed it input files (e.g., “grd.xlsx”, “test.xlsx”, “indeks_parameter”) or raw sentences
- Internally it dispatches to one of the four extractor modules, which rely on either local Python code (modul_ekspart, modul_triplet) or external NLP engines (spaCy model, Stanford CoreNLP server).
- Outputs are various representations (clauses, triplets, dependency graphs).

a. Main components
- Client/User layer – scripts or notebooks that import and invoke extractreq API
- API Layer – extractreq package entry points (partOf, spacyClause, stanford_clause, extractNlp)
- Core Modules – modul_ekspart, modul_spacySent, modul_stanfordSent, modul_triplet
- External NLP Engines – spaCy language model, Stanford CoreNLP server
- Input Data – SRS/GRD Excel files or raw text
- Output Data – clause lists, SPO triplets, dependency reports
b. Relationships and interactions
1. Client calls extractreq API →
2. API Layer orchestrates chosen module →
3. Module processes Input Data; if it’s spaCy or Stanford, it calls that engine (local library or REST/gRPC) →
4. Module returns processed structures →
5. API Layer formats/writes Outputs.
c. Architectural patterns and principles
- Modular “plug-in” pattern: each extractor implements a common interface for extensibility
- Separation of concerns: I/O (Excel/text) is separate from NLP processing
- Wrapper pattern: external engines are wrapped behind a uniform Python class
d. Key technologies
- Python 3.7+, setuptools/pyproject.toml (packaging)
- spaCy, Stanford CoreNLP, NLTK (via modul_triplet)
- Sphinx (docs), Jupyter (notebooks)
- pytest or CodeClimate (test/quality badges in README)

##### Essential elements
– Library code (light blue)
– External dependencies (orange)
– Data artifacts (green)
– User interface/notebook (gray)

##### Detailed capture of architectural elements
- Example flow for “partOf().extractPart(…)” and for “extractNlp().triplet_extraction(…)” to illustrate branching.
- Config points (Excel input file path, SRS index) and output modes.
- Optionally, add a side-panel showing docs/ and notebook/ as “Developer & Experimentation Tools.”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User & Developer Tools
    subgraph "User & Developer Tools"
        direction TB
        Notebooks["Jupyter Notebooks"]:::ui
        Scripts["Client Scripts"]:::ui
        Docs["Sphinx Docs"]:::ui
    end

    %% Input Data
    DataIn["Excel Files / Raw Text"]:::data

    %% API Layer
    subgraph "API Layer (extractreq)"
        direction TB
        APIInit["__init__.py"]:::library
    end

    %% Core Modules
    subgraph "Extractreq Modules"
        direction TB
        Ekspart["modul_ekspart.py"]:::library
        SpacySent["modul_spacySent.py"]:::library
        StanfordSent["modul_stanfordSent.py"]:::library
        Triplet["modul_triplet.py"]:::library
    end

    %% Use Case Examples
    subgraph "Example Pipelines"
        direction TB
        UC1["usecase_modul1.py"]:::library
        UC2["usecase_modul2.py"]:::library
        UC3["usecase_modul3.py"]:::library
    end

    %% External Engines
    SpaCyModel["spaCy Language Model"]:::external
    CoreNLP["Stanford CoreNLP Server"]:::external

    %% Output Data
    DataOut["Clause Lists / SPO Triplets / Reports"]:::data

    %% Packaging & CI
    subgraph "Packaging & CI"
        direction TB
        Setup["setup.py"]:::library
        Pyproject["pyproject.toml"]:::library
        Requirements["requirements.txt"]:::library
        Dist["dist/"]:::data
        Travis[".travis.yml"]:::library
        Readthedocs[".readthedocs.yaml"]:::library
    end

    %% Flows
    Notebooks -->|uses| APIInit
    Scripts -->|calls API| APIInit
    Docs -->|references API| APIInit
    DataIn -->|input| APIInit
    APIInit -->|dispatches| Ekspart
    APIInit -->|dispatches| SpacySent
    APIInit -->|dispatches| StanfordSent
    APIInit -->|dispatches| Triplet

    Ekspart -->|processes| DataIn
    SpacySent -->|processes| DataIn
    StanfordSent -->|processes| DataIn
    Triplet -->|processes| DataIn

    SpacySent -->|calls| SpaCyModel
    StanfordSent -->|calls| CoreNLP

    Ekspart -->|returns results| APIInit
    SpacySent -->|returns clauses| APIInit
    StanfordSent -->|returns clauses| APIInit
    Triplet -->|returns triplets| APIInit

    APIInit -->|formats/writes| DataOut

    APIInit --> UC1
    APIInit --> UC2
    APIInit --> UC3

    %% Packaging flow
    Setup -->|defines package| Pyproject
    Pyproject --> Requirements
    Setup -->|builds| Dist
    Travis -->|CI for| Setup
    Readthedocs -->|builds docs| Docs

    %% Click Events
    click Notebooks "https://github.com/asyrofist/extractreq/tree/main/notebook/"
    click Docs "https://github.com/asyrofist/extractreq/tree/main/docs/"
    click APIInit "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/__init__.py"
    click Ekspart "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/modul_ekspart.py"
    click SpacySent "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/modul_spacySent.py"
    click StanfordSent "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/modul_stanfordSent.py"
    click Triplet "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/modul_triplet.py"
    click UC1 "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/usecase_modul1.py"
    click UC2 "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/usecase_modul2.py"
    click UC3 "https://github.com/asyrofist/extractreq/blob/main/src/extractreq/usecase_modul3.py"
    click Setup "https://github.com/asyrofist/extractreq/blob/main/setup.py"
    click Pyproject "https://github.com/asyrofist/extractreq/blob/main/pyproject.toml"
    click Requirements "https://github.com/asyrofist/extractreq/blob/main/requirements.txt"
    click Dist "https://github.com/asyrofist/extractreq/tree/main/dist/"
    click Travis "https://github.com/asyrofist/extractreq/blob/main/.travis.yml"
    click Readthedocs "https://github.com/asyrofist/extractreq/blob/main/.readthedocs.yaml"

    %% Styles
    classDef library fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef data fill:#90EE90,stroke:#000,stroke-width:1px
    classDef ui fill:#D3D3D3,stroke:#000,stroke-width:1px
```
