# sandialabs/pvops architecture

> Architecture diagram of the GitHub repository sandialabs/pvops, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sandialabs/pvops
- Repository: https://github.com/sandialabs/pvops

## Overview

##### Project type and purpose
- pvops is an open-source Python library (not a web app) for fusing text-based metadata with time-series performance data from photovoltaic sites.
- Its purpose is to provide pipelines for:
- Text processing and classification (pvops/text)
- Converting text events into time stamps (pvops/text2time)
- Time-series preprocessing and modeling (pvops/timeseries)
- I-V curve analysis and simulation (pvops/iv)

##### File structure
- Top-level directories:
- pvops/         ← source code, organized by module
- pvops/tests/   ← unit/fixture tests
- tutorials/     ← example notebooks + data
- docs/          ← Sphinx documentation
- Inside pvops/, main submodules:
- text/          ← NLP, classification, utilities
- text2time/     ← text-to-timestamp logic, visualization
- timeseries/    ← TS preprocessing, model implementations
- iv/            ← I-V data extractors, physics utils, simulators
- Cross-cutting utilities:
- Configuration & CI:
- setup.py, requirements.txt, noxfile.py, .github/workflows for build/test, .readthedocs.yml

##### README for additional insights
- Confirms module breakdown and links to in-depth docs/tutorials on ReadTheDocs.
- No runtime external services (DBs or APIs), this is purely a library with data inputs/outputs.

a. Main components:
- Data Inputs
– Raw text logs/notes (tutorials/example_data, stopwords.txt)
– Raw production time-series (CSV example data)
– I-V measurement files
- Core Modules
1. Text Processing (pvops/text)
2. Text2Time Converter (pvops/text2time)
3. Time-Series Processor & Models (pvops/timeseries)
4. I-V Analyzer & Simulator (pvops/iv)
- Outputs
– Cleaned time-series with event timestamps
– Model predictions/metrics
- Documentation & CI
– Sphinx docs (docs/)
– Automated tests (pvops/tests/, GitHub Actions)

b. Relationships & interactions:
1. Raw text flows into Text Processing ⇒ cleaned tokens, classifications
2. Classified events feed into Text2Time ⇒ timestamped events aligned with time-series
3. Time-series preprocessor ingests raw CSV + timestamped events ⇒ feature matrices
4. Processed data goes into Models (in timeseries/models and iv/models) ⇒ predictions
5. Visualization modules produce plots at each stage
6. Tests validate each submodule in isolation
7. Tutorials/instruction notebooks orchestrate end-to-end pipelines

c. Architectural patterns / design principles:
- Modular design: clearly delineated subpackages by domain (Single Responsibility)
- Pluggable models: timeseries/models and iv/models define interchangeable algorithms
- Test harness: pytest structure in pvops/tests to ensure stability
- Documentation-driven: Sphinx source under docs/ auto-published

d. Technologies, frameworks, libraries:
- Python 3.x, setuptools (setup.py), pip install
- pytest for tests, nox for multi-session testing
- Sphinx for docs, Jupyter notebooks for tutorials
- numpy, pandas, scikit-learn (implied), matplotlib/seaborn

##### Elements
- Clear labels: name each module (text, text2time, timeseries, iv)
- “Core Library”
- “Data”
- “Documentation & CI”
- “Tutorials & Examples”

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Sources
    subgraph "Data Sources" 
        direction TB
        Data_Text["Raw Text Logs"]:::data
        Data_TS["Raw Time-Series CSV"]:::data
        Data_IV["I-V Measurement Files"]:::data
    end

    %% Core Library
    subgraph "Core Library"
        direction TB

        subgraph "Text Processing Module" 
            direction TB
            Text_Preproc["preprocess()"]:::text
            Text_Classify["classify()"]:::text
            Text_Visualize["visualize()"]:::text
            Text_NLP["nlp_utils"]:::text
            Text_Utils["utils"]:::text
            Text_Stopwords["stopwords.txt"]:::text
        end

        subgraph "Text2Time Converter Module"
            direction TB
            T2T_Preproc["preprocess()"]:::text2time
            T2T_Utils["utils"]:::text2time
            T2T_Visualize["visualize()"]:::text2time
        end

        subgraph "Time-Series Processor & Models"
            direction TB
            TS_Preproc["preprocess()"]:::timeseries
            subgraph "Model Registry" 
                direction TB
                TS_AIT["AIT.py"]:::timeseries
                TS_IEC["iec.py"]:::timeseries
                TS_Lin["linear.py"]:::timeseries
                TS_Surv["survival.py"]:::timeseries
            end
        end

        subgraph "I-V Analyzer & Simulator"
            direction TB
            IV_Extract["extractor.py"]:::iv
            IV_Preproc["preprocess()"]:::iv
            IV_Physics["physics_utils"]:::iv
            IV_Sim["simulator.py"]:::iv
            IV_TS_Sim["timeseries_simulator"]:::iv
            subgraph "IV Model Registry"
                direction TB
                IV_NN["nn.py"]:::iv
            end
            IV_Util["utils"]:::iv
        end
    end

    %% Outputs
    subgraph "Outputs"
        direction TB
        Out_TS["Cleaned Time-Series with Events"]:::output
        Out_Pred["Model Predictions/Metrics"]:::output
        Out_Vis["Visualizations"]:::output
    end

    %% Documentation & CI
    subgraph "Documentation & CI"
        direction TB
        Docs["Sphinx Documentation"]:::docs
        Tests["Unit & Fixture Tests"]:::tests
        CI["GitHub Actions & CI"]:::docs
    end

    %% Tutorials & Examples
    subgraph "Tutorials & Examples"
        direction TB
        Tut_Text["tutorial_textmodule.ipynb"]:::tutorials
        Tut_T2T["tutorial_text2time_module.ipynb"]:::tutorials
        Tut_TS["tutorial_timeseries.ipynb"]:::tutorials
        Tut_IV["tutorial_iv_simulator.ipynb"]:::tutorials
        Example_Data["example_data/*.csv"]:::tutorials
    end

    %% Data Flow
    Data_Text --> Text_Preproc
    Text_Preproc --> Text_Classify
    Text_Classify --> T2T_Preproc
    T2T_Preproc -->|"timestamps"| TS_Preproc
    Data_TS --> TS_Preproc
    TS_Preproc --> TS_AIT
    TS_Preproc --> TS_IEC
    TS_Preproc --> TS_Lin
    TS_Preproc --> TS_Surv
    TS_AIT --> Out_Pred
    TS_IEC --> Out_Pred
    TS_Lin --> Out_Pred
    TS_Surv --> Out_Pred

    Data_IV --> IV_Extract
    IV_Extract --> IV_Preproc
    IV_Preproc --> IV_Physics
    IV_Physics --> IV_Sim
    IV_Sim --> IV_TS_Sim
    IV_TS_Sim --> IV_NN
    IV_NN --> Out_Pred

    Text_Visualize --> Out_Vis
    T2T_Visualize --> Out_Vis
    TS_Preproc --> Out_Vis
    IV_Preproc --> Out_Vis

    Out_TS --> Out_Vis

    %% Docs & CI flow
    Docs --> Tests
    Tests --> CI

    %% Tutorials link to modules and data
    Tut_Text --> Text_Preproc
    Tut_Text --> Text_Classify
    Tut_T2T --> T2T_Preproc
    Tut_TS --> TS_Preproc
    Tut_IV --> IV_Extract
    Tut_IV --> IV_Sim
    Example_Data --> Data_Text
    Example_Data --> Data_TS

    %% Click Events
    click Text_Preproc "https://github.com/sandialabs/pvops/blob/master/pvops/text/preprocess.py"
    click Text_Classify "https://github.com/sandialabs/pvops/blob/master/pvops/text/classify.py"
    click Text_NLP "https://github.com/sandialabs/pvops/blob/master/pvops/text/nlp_utils.py"
    click Text_Utils "https://github.com/sandialabs/pvops/blob/master/pvops/text/utils.py"
    click Text_Visualize "https://github.com/sandialabs/pvops/blob/master/pvops/text/visualize.py"
    click Text_Stopwords "https://github.com/sandialabs/pvops/blob/master/pvops/text/stopwords.txt"

    click T2T_Preproc "https://github.com/sandialabs/pvops/blob/master/pvops/text2time/preprocess.py"
    click T2T_Utils "https://github.com/sandialabs/pvops/blob/master/pvops/text2time/utils.py"
    click T2T_Visualize "https://github.com/sandialabs/pvops/blob/master/pvops/text2time/visualize.py"

    click TS_Preproc "https://github.com/sandialabs/pvops/blob/master/pvops/timeseries/preprocess.py"
    click TS_AIT "https://github.com/sandialabs/pvops/blob/master/pvops/timeseries/models/AIT.py"
    click TS_IEC "https://github.com/sandialabs/pvops/blob/master/pvops/timeseries/models/iec.py"
    click TS_Lin "https://github.com/sandialabs/pvops/blob/master/pvops/timeseries/models/linear.py"
    click TS_Surv "https://github.com/sandialabs/pvops/blob/master/pvops/timeseries/models/survival.py"

    click IV_Extract "https://github.com/sandialabs/pvops/blob/master/pvops/iv/extractor.py"
    click IV_Preproc "https://github.com/sandialabs/pvops/blob/master/pvops/iv/preprocess.py"
    click IV_Physics "https://github.com/sandialabs/pvops/blob/master/pvops/iv/physics_utils.py"
    click IV_Sim "https://github.com/sandialabs/pvops/blob/master/pvops/iv/simulator.py"
    click IV_TS_Sim "https://github.com/sandialabs/pvops/blob/master/pvops/iv/timeseries_simulator.py"
    click IV_Util "https://github.com/sandialabs/pvops/blob/master/pvops/iv/utils.py"
    click IV_NN "https://github.com/sandialabs/pvops/blob/master/pvops/iv/models/nn.py"

    click Tests "https://github.com/sandialabs/pvops/tree/master/pvops/tests/"
    click Docs "https://github.com/sandialabs/pvops/tree/master/docs/"

    click Tut_Text "https://github.com/sandialabs/pvops/blob/master/tutorials/tutorial_textmodule.ipynb"
    click Tut_T2T "https://github.com/sandialabs/pvops/blob/master/tutorials/tutorial_text2time_module.ipynb"
    click Tut_TS "https://github.com/sandialabs/pvops/blob/master/tutorials/tutorial_timeseries.ipynb"
    click Tut_IV "https://github.com/sandialabs/pvops/blob/master/tutorials/tutorial_iv_simulator.ipynb"
    click Example_Data "https://github.com/sandialabs/pvops/tree/master/tutorials/example_data/"

    %% Styles
    classDef data fill:#FFFBCC,stroke:#333,stroke-width:1px
    classDef text fill:#E3F6F5,stroke:#1B9AAA,stroke-width:2px
    classDef text2time fill:#FFE8D6,stroke:#C14953,stroke-width:2px
    classDef timeseries fill:#E2EAFc,stroke:#0B3954,stroke-width:2px
    classDef iv fill:#F0F3BD,stroke:#606C38,stroke-width:2px
    classDef output fill:#D5E8D4,stroke:#82B366,stroke-width:2px
    classDef docs fill:#FDEBD0,stroke:#D35400,stroke-width:2px
    classDef tests fill:#FADBD8,stroke:#C0392B,stroke-width:2px
    classDef tutorials fill:#EBF5FB,stroke:#2980B9,stroke-width:2px
```
