How asyrofist/extractreq works
Project type and purpose
• This is a Python-based NLP library/tool (not a full-stack web app).
• It provides a set of “extractreq” modules for extracting requirement-dependency information from text, using multiple engines (spaCy, Stanford CoreNLP, custom part-extractor, triplet extractor).
• It packages itself as a pip-installable library and exposes a simple API/CLI for batch processing of SRS or GRD spreadsheets.
File structure
• Top-level packaging and metadata:
– setup.py, pyproject.toml, requirements.txt → packaging and dependency definitions
– dist/ → built releases (wheels, tar.gz)
• Source code under src/extractreq:
– modul_ekspart.py → core “evolutionary requirement part” extractor
– modul_spacySent.py → clause splitter using spaCy
– modul_stanfordSent.py → clause splitter using Stanford CoreNLP
– modul_triplet.py → SPO triplet extractor
– usecase_modul1.py, usecase_modul2.py, usecase_modul3.py → end-to-end examples/pipelines
• docs/ and notebook/ → Sphinx documentation and Jupyter notebooks for experimenting and tutorials
• CI/config: .travis.yml, .readthedocs.yaml
Architecture insights from README
• The library exposes classes (partOf, stanford_clause, spacyClause, extractNlp)
• You feed it input files (e.g., “grd.xlsx”, “test.xlsx”, “indeks_parameter”) or raw sentences
• Internally it dispatches to one of the four extractor modules, which rely on either local Python code (modul_ekspart, modul_triplet) or external NLP engines (spaCy model, Stanford CoreNLP server).
• Outputs are various representations (clauses, triplets, dependency graphs).
a. Main components
• Client/User layer – scripts or notebooks that import and invoke extractreq API
• API Layer – extractreq package entry points (partOf, spacyClause, stanford_clause, extractNlp)
• Core Modules – modul_ekspart, modul_spacySent, modul_stanfordSent, modul_triplet
• External NLP Engines – spaCy language model, Stanford CoreNLP server
• Input Data – SRS/GRD Excel files or raw text
• Output Data – clause lists, SPO triplets, dependency reports
b. Relationships and interactions
1. Client calls extractreq API →
2. API Layer orchestrates chosen module →
3. Module processes Input Data; if it’s spaCy or Stanford, it calls that engine (local library or REST/gRPC) →
4. Module returns processed structures →
5. API Layer formats/writes Outputs.
c. Architectural patterns and principles
• Modular “plug-in” pattern: each extractor implements a common interface for extensibility
• Separation of concerns: I/O (Excel/text) is separate from NLP processing
• Wrapper pattern: external engines are wrapped behind a uniform Python class
d. Key technologies
• Python 3.7+, setuptools/pyproject.toml (packaging)
• spaCy, Stanford CoreNLP, NLTK (via modul_triplet)
• Sphinx (docs), Jupyter (notebooks)
• pytest or CodeClimate (test/quality badges in README)
Essential elements
– Library code (light blue)
– External dependencies (orange)
– Data artifacts (green)
– User interface/notebook (gray)
Detailed capture of architectural elements
• Example flow for “partOf().extractPart(…)” and for “extractNlp().triplet_extraction(…)” to illustrate branching.
• Config points (Excel input file path, SRS index) and output modes.
• Optionally, add a side-panel showing docs/ and notebook/ as “Developer & Experimentation Tools.”