# abchapman93/rehoused architecture

> Architecture diagram of the GitHub repository abchapman93/rehoused, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abchapman93/rehoused
- Repository: https://github.com/abchapman93/rehoused

## Overview

##### Project type and purpose
- This is a Python NLP library (“rehoused_nlp”) built on spaCy/medSpaCy.
- Purpose: process clinical text to classify patient housing stability at two levels (document and patient), driven by rule-based concept tagging, context detection, sectionization, postprocessing, then document-level classification and patient-level aggregation.

##### File structure
- Top-level:
- setup.py, package root (“rehoused_nlp”), notebooks folder, tests, resources (images), README.
- Core code in rehoused_nlp/:
- tokenizer.py, utils.py, constants.py, build_nlp entry point, document classifier, postprocessor subpackage, resources subpackage.
- Resources:
- JSON/TSV rule files under concept_tagger, context, sectionizer, target_matcher, configs.
- Notebooks: demonstrate each pipeline stage (entity extraction, attribute detection, postprocessing, classification, longitudinal scoring).
- Tests: unit tests for each component.

##### Extract insights from the README
- The pipeline stages map to notebook sequence (00–05).
- Dependencies: spaCy ≥3.1.3,<3.2.0, medSpaCy 0.2.0.1.
- Two usage patterns: document-level via build_nlp() + visualize_doc_classification(), patient-level via calculate_rehoused(df).

##### Main components
a. Input Layer
- Raw clinical text documents (scripts, notebooks, API).
b. NLP Pipeline Components (spacy pipeline)
1. Tokenizer (tokenizer.py)
2. Concept Tagger (medSpaCy Matcher loaded with JSON patterns)
3. Context Component (rules from context/*.json)
4. Sectionizer (section rules from sectionizer/*.json)
5. Target Matcher (target_matcher/*.json)
6. Attribute Detection (postprocessing via postprocessing_functions.py, patterns, rules)
c. Document Classification
- rehoused_document_classifier.py: takes pipeline annotations, assigns STABLY_HOUSED/UNSTABLY_HOUSED/UNKNOWN.
d. Patient-Level Aggregator
- calculate_rehoused(df) in utils: groups by patient, applies time-series logic, outputs Relative Housing Stability score (0–1).
e. Resources Loader
- callbacks.py, constants.py: load JSON/TSV rule files into pipeline.
f. Output
- spaCy Doc object with custom extensions, classification labels, visualization in notebooks.
- pandas DataFrame with ReHouSED scores.

##### Relationships & data flows
- Side-component: Resources Loader feeds rule files into each pipeline component.

##### Architectural patterns
- Plugin-based spaCy pipeline (each stage is a component).
- Configuration-driven (JSON/TSV rule files decouple logic).
- Modular separation: preprocessing, entity/concept extraction, context, postprocessing, classification, aggregation.
- Notebook-driven demonstration (not part of runtime service but useful for examples).

##### Technologies & frameworks
- Python 3.7/3.8
- spaCy 3.x
- medSpaCy 0.2.0.1
- pandas (for patient aggregation)
- JSON/TSV resource files (concept patterns, context rules, section rules, target matcher templates)

##### Elements
- Clear labels: e.g., “Tokenizer”, “Concept Tagger”, “Context Detector”, etc.
- Blue for core code modules
- Green for resource files
- Orange for pipeline components
- Yellow for entry points (notebooks/API)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Entry Points"
        NB["Notebooks / API"]:::entry
    end
    click NB "https://github.com/abchapman93/rehoused/blob/main/notebooks/00-Overview.ipynb"

    NB --> BI["build_nlp()"]:::entry
    click BI "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/__init__.py"

    subgraph "Resources Loader"
        RL["callbacks.py"]:::core
        CC["constants.py"]:::core
    end
    click RL "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/resources/callbacks.py"
    click CC "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/constants.py"

    subgraph "Resource Configurations"
        CT["Concept Patterns"]:::resources
        CX["Context Rules"]:::resources
        SZ["Section Rules"]:::resources
        TM["Target Matcher Templates"]:::resources
    end
    click CT "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/resources/concept_tagger/*.json"
    click CX "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/resources/context/*.json"
    click SZ "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/resources/sectionizer/*.json"
    click TM "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/resources/target_matcher/*.json"

    RL --> CT
    RL --> CX
    RL --> SZ
    RL --> TM
    CC --> CT
    CC --> CX
    CC --> SZ
    CC --> TM

    subgraph "spaCy Pipeline"
        TKN["Tokenizer"]:::pipeline
        CTG["Concept Tagger"]:::pipeline
        CXT["Context Detector"]:::pipeline
        SEC["Sectionizer"]:::pipeline
        TMT["Target Matcher"]:::pipeline
        PPR["Postprocessor"]:::pipeline
    end
    click TKN "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/tokenizer.py"
    click PPR "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/rehoused_postprocessor/postprocessing_functions.py"

    BI --> TKN --> CTG --> CXT --> SEC --> TMT --> PPR

    PPR --> DC["Document Classifier"]:::core
    click DC "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/rehoused_document_classifier.py"

    DC --> DOCDOC["spaCy Doc (with labels)"]:::output
    DOCDOC --> DF["pandas DataFrame"]:::storage

    DF --> AGG["calculate_rehoused()"]:::core
    click AGG "https://github.com/abchapman93/rehoused/blob/main/rehoused_nlp/utils.py"

    AGG --> POUT["Patient-Level Scores"]:::output

    classDef entry fill:#f9f,stroke:#333,stroke-width:1px
    classDef core fill:#bbf,stroke:#333,stroke-width:1px
    classDef resources fill:#bfb,stroke:#333,stroke-width:1px
    classDef pipeline fill:#fb7,stroke:#333,stroke-width:1px
    classDef output fill:#ffb,stroke:#333,stroke-width:1px
    classDef storage fill:#ddf,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
```
