# larsiusprime/openavmkit architecture

> Architecture diagram of the GitHub repository larsiusprime/openavmkit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/larsiusprime/openavmkit
- Repository: https://github.com/larsiusprime/openavmkit

## Overview

##### Project type and purpose
- OpenAVMKit is a Python library and toolkit for mass appraisal of real estate, providing data cleaning, enrichment, modeling, inference, and report generation.
- It ships as a PyPI package, includes CLI-style pipelines, Jupyter notebooks for interactive workflows, and connectors to cloud storage/APIs.

##### File structure
- Top-level:
– openavmkit/ ← core Python package
– notebooks/ ← example pipelines & interactive notebooks
– docs/ ← user and API documentation
– docker-compose.yml, multistage-dockerfile, entrypoint.sh ← containerization
– tests/ ← unit and integration tests
- Inside openavmkit/:
– core modules: cleaning.py, modeling.py, inference.py, pipeline.py, etc.
– cloud connectors: cloud/azure.py, huggingface.py, sftp.py
– synthetic data: synthetic/…
– studies & analyses: horizontal_equity_study.py, ratio_study.py, etc.
– resources: JSON settings templates, report markdown/CSS
– utilities: reusable helpers (census API, plotting, geometry)

##### README for technical stack
- Pure Python (>=3.x)
- Optional cloud APIs: Azure, SFTP, Hugging Face, US Census, OpenStreetMap
- Jupyter notebooks for examples
- Dockerized for reproducible environments

##### Main components & their interactions
a. Data Inputs
– Raw property records (CSV, database dumps)
– Census API calls
– OpenStreetMap enrichment
– Synthetic data generator (for testing/benchmarks)
b. Data Cleaning & Enrichment
– openavmkit.cleaning & cleaning pipeline notebooks
– Filters, data dictionary lookups, geometry routines
c. Feature Engineering & Modeling
– openavmkit.calculations, modeling.py, tuning.py
– Training statistical or ML models
d. Inference & Projection
– openavmkit.inference & projection.py
– Batch scoring of properties
e. Quality Control & Studies
– openavmkit.quality_control, horizontal_equity_study, ratio_study, sales_scrutiny_study
f. Report Generation
– openavmkit.reports → Markdown → HTML/PDF via CSS in resources/reports/
g. Cloud Storage & Deployment
– openavmkit.cloud.* interfaces push/pull data and artifacts
– Docker container bundles the runtime, entrypoint.sh orchestrates pipelines
h. User Interfaces
– CLI/pipeline scripts invoked from entrypoint.sh or directly
– Jupyter notebooks for exploratory use

##### Architectural patterns & principles
- Modular design: each stage (cleaning, modeling, reporting) is its own module
- Inversion of control: pipeline.py orchestrates modules via configuration JSON
- Extensibility: cloud connectors and synthetic data plug into base classes
- Reusability: utilities shared across modules

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Inputs & Outputs
    RawData["CSV/DB"]:::external
    CensusAPI["US Census API"]:::external
    OSMAPI["OpenStreetMap API"]:::external
    SyntheticGen["Synthetic Data Generator"]:::external
    CloudStorage["Cloud Storage"]:::external

    %% Core Engine
    subgraph "Core Engine"
        Cleaning["Data Cleaning"]:::core
        Enrichment["Data Enrichment"]:::core
        FeatureEng["Feature Engineering"]:::core
        Modeling["Model Training & Tuning"]:::core
        Inference["Inference & Projection"]:::core
        Projection["Projection"]:::core
        QC["Quality Control & Studies"]:::core
        Reports["Report Generation"]:::core
    end

    %% Utilities & Connectors
    subgraph "Utilities & Connectors"
        CensusUtil["Census Utilities"]:::util
        OSMUtil["OSM Utilities"]:::util
        GeoUtil["Geometry Utilities"]:::util
        AzureConn["Azure Connector"]:::util
        SFTPConn["SFTP Connector"]:::util
        HFConn["HuggingFace Connector"]:::util
    end

    %% Interfaces & Infrastructure
    subgraph "Interfaces & Infrastructure"
        CLI["CLI / Orchestration"]:::infra
        Notebooks["Jupyter Notebooks"]:::infra
        Docker["Docker Container"]:::infra
    end

    %% Testing & QA
    Tests["Tests & QA"]:::test

    %% Data Flow
    RawData --> Cleaning
    CensusAPI --> Cleaning
    CensusUtil --> Cleaning
    GeoUtil --> Cleaning
    Cleaning --> FeatureEng
    FeatureEng --> Modeling
    Modeling --> Inference
    Inference --> Projection
    Projection --> Reports
    QC --> Reports
    Reports --> CloudStorage

    OSMAPI --> Enrichment
    Enrichment --> FeatureEng
    OSMUtil --> Enrichment

    %% Connectivity
    AzureConn --> CloudStorage
    SFTPConn --> CloudStorage
    HFConn --> CloudStorage
    CloudConns-->CloudStorage

    %% Interfaces triggers
    Docker --> CLI
    CLI --> Cleaning
    CLI --> FeatureEng
    CLI --> Modeling
    CLI --> Inference
    CLI --> Reports
    Notebooks --> Cleaning
    Notebooks --> FeatureEng
    Notebooks --> Modeling

    %% Testing links
    Tests --> Cleaning
    Tests --> FeatureEng
    Tests --> Modeling
    Tests --> Inference
    Tests --> Reports

    %% Click Events
    click CensusAPI "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/utilities/census.py"
    click OSMAPI "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/utilities/openstreetmap.py"
    click SyntheticGen "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/synthetic/generate.py"
    click Cleaning "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/cleaning.py"
    click Enrichment "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/filters.py"
    click FeatureEng "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/calculations.py"
    click Modeling "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/modeling.py"
    click Modeling "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/tuning.py"
    click Inference "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/inference.py"
    click Projection "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/projection.py"
    click QC "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/quality_control.py"
    click Reports "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/reports.py"
    click AzureConn "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/cloud/azure.py"
    click SFTPConn "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/cloud/sftp.py"
    click HFConn "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/cloud/huggingface.py"
    click CLI "https://github.com/larsiusprime/openavmkit/blob/master/openavmkit/pipeline.py"
    click Notebooks "https://github.com/larsiusprime/openavmkit/blob/master/notebooks/pipeline/*.ipynb"
    click Tests "https://github.com/larsiusprime/openavmkit/blob/master/tests/*.py"

    %% Styles
    classDef external fill:#9f6,stroke:#333,stroke-width:1px
    classDef core fill:#69f,stroke:#333,stroke-width:1px
    classDef util fill:#bbb,stroke:#333,stroke-width:1px
    classDef infra fill:#fc6,stroke:#333,stroke-width:1px
    classDef test fill:#f96,stroke:#333,stroke-width:1px
```
