# kawasakiharuto712/review_priority architecture

> Architecture diagram of the GitHub repository kawasakiharuto712/review_priority, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kawasakiharuto712/review_priority
- Repository: https://github.com/kawasakiharuto712/review_priority

## Overview

##### Project Type and Purpose
- Type: A Python-based data-pipeline + machine-learning CLI tool.
- Purpose: Automate data collection from OpenStack’s Gerrit, preprocess and extract features, apply inverse reinforcement learning (IRL) to predict code‐review priorities.

##### File Structure
- Top-level:
- main.py – CLI orchestrator
- Dockerfile / docker-compose.yml – containerization
- pyproject.toml, requirements.txt – packaging & dependencies
- src/ – all application code, subdivided into:
◦ collectors/ – Gerrit data fetchers (openstack.py, release_collector.py)
◦ preprocessing/ – review comment cleaning & keyword extraction
◦ features/ – functions computing 16 metrics (bug, change, developer, etc.)
◦ learning/ – IRL model (irl_models.py) and time-series evaluation
◦ config/ – application settings (gerrymanderconfig.ini, path.py)
◦ utils/ – shared helpers (constants, language identifier)
- tests/ – pytest coverage for all modules

##### README Insights
- Workflow stages: Collect → Preprocess → Feature Extraction → IRL Learning → Predict
- Data folders (logical): raw (openstack/), processed/, results/
- Outputs: JSON (irl_analysis_*.json), Pickle (irl_model_*.pkl)

##### Main Components and Their Interactions
a. Data Sources & Ingest
- External: OpenStack Gerrit API
- Collector Module: src/collectors/openstack.py, release_collector.py
b. Preprocessing Pipeline
- review_comment_processor.py → cleans & tokenizes comments
- Noise filtering (bot messages)
c. Feature Extraction
- src/features/*.py → computes bug, change, test, developer, project, refactoring, review metrics
d. Machine-Learning / IRL
- src/learning/irl_models.py – defines reward functions & solver
- temporal_model_evaluation.py, temporal_weight_analysis.py – time-series analysis
e. Orchestration & Configuration
- main.py – entry point, subcommands (status, full, collect, preprocess, train, analyze)
- src/config/gerrymanderconfig.ini & path.py – credentials & file paths
f. Utilities
- src/utils – shared constants, language detection
g. Persistence & Outputs
- data/raw (logical) → data/processed → data/results/*.json/.pkl
h. Containerization & Deployment
- Dockerfile – image build
- docker-compose.yml – service definition
i. Testing
- pytest suites under tests/

##### Architectural Patterns & Design Principles
- Modular Pipeline: clear separation of concerns—ingest, ETL, feature, learning
- CLI Orchestration: single entry-point main.py using subcommands
- Configuration Management: centralized in config module & INI file
- Layered Flow: one-way data flow from collectors → preprocess → features → learning
- Test-Driven: every module has corresponding tests

## Diagram (Mermaid)

```mermaid
flowchart TD
  direction TB

  API(["Gerrit API"]):::external

  subgraph "Docker Container"
    direction TB

    CLI["CLI Orchestrator (main.py)"]:::cli
    click CLI "https://github.com/kawasakiharuto712/review_priority/blob/main/main.py"

    subgraph Collectors
      direction TB
      OS["openstack.py"]:::module
      RC["release_collector.py"]:::module
      click OS "https://github.com/kawasakiharuto712/review_priority/blob/main/src/collectors/openstack.py"
      click RC "https://github.com/kawasakiharuto712/review_priority/blob/main/src/collectors/release_collector.py"
    end

    subgraph Preprocessing
      direction TB
      PCP["review_comment_processor.py"]:::module
      click PCP "https://github.com/kawasakiharuto712/review_priority/blob/main/src/preprocessing/review_comment_processor.py"
    end

    subgraph Features
      direction TB
      BM["bug_metrics.py"]:::module
      CM["change_metrics.py"]:::module
      DM["developer_metrics.py"]:::module
      PM["project_metrics.py"]:::module
      RM["refactoring_metrics.py"]:::module
      RevM["review_metrics.py"]:::module
      click BM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/bug_metrics.py"
      click CM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/change_metrics.py"
      click DM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/developer_metrics.py"
      click PM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/project_metrics.py"
      click RM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/refactoring_metrics.py"
      click RevM "https://github.com/kawasakiharuto712/review_priority/blob/main/src/features/review_metrics.py"
    end

    subgraph Learning
      direction TB
      IRL["irl_models.py"]:::module
      TME["temporal_model_evaluation.py"]:::module
      TWA["temporal_weight_analysis.py"]:::module
      click IRL "https://github.com/kawasakiharuto712/review_priority/blob/main/src/learning/irl_models.py"
      click TME "https://github.com/kawasakiharuto712/review_priority/blob/main/src/learning/temporal_model_evaluation.py"
      click TWA "https://github.com/kawasakiharuto712/review_priority/blob/main/src/learning/temporal_weight_analysis.py"
    end

    subgraph Config
      direction TB
      GINI["gerrymanderconfig.ini"]:::config
      PATH["path.py"]:::config
      click GINI "https://github.com/kawasakiharuto712/review_priority/blob/main/src/config/gerrymanderconfig.ini"
      click PATH "https://github.com/kawasakiharuto712/review_priority/blob/main/src/config/path.py"
    end

    subgraph Utils
      direction TB
      CONST["constants.py"]:::util
      LANG["lang_identifiyer.py"]:::util
      click CONST "https://github.com/kawasakiharuto712/review_priority/blob/main/src/utils/constants.py"
      click LANG "https://github.com/kawasakiharuto712/review_priority/blob/main/src/utils/lang_identifiyer.py"
    end

    subgraph DataStores
      direction TB
      RAW["Raw Data Store"]:::datastore
      PROC["Processed Data Store"]:::datastore
      FEAT["Feature Store"]:::datastore
      RES["Results Store"]:::datastore
    end

    API -->|"HTTP REST API"| OS
    OS -->|"JSON"| RAW
    RC -->|"JSON"| RAW
    RAW -->|"CSV/JSON"| PCP
    PCP -->|"CSV/JSON"| PROC
    PROC -->|"DataFrame"| BM
    PROC -->|"DataFrame"| CM
    PROC -->|"DataFrame"| DM
    PROC -->|"DataFrame"| PM
    PROC -->|"DataFrame"| RM
    PROC -->|"DataFrame"| RevM
    BM -->|"Features"| FEAT
    CM -->|"Features"| FEAT
    DM -->|"Features"| FEAT
    PM -->|"Features"| FEAT
    RM -->|"Features"| FEAT
    RevM -->|"Features"| FEAT
    FEAT -->|"Feature Array"| IRL
    IRL -->|"Pickle/JSON"| RES
    TME -->|"Analysis JSON"| RES
    TWA -->|"Analysis JSON"| RES

    CLI -->|"trigger"| Collectors
    CLI -->|"trigger"| Preprocessing
    CLI -->|"trigger"| Features
    CLI -->|"trigger"| Learning
    CLI -->|"load"| Config
    CLI -->|"use"| Utils

  end

  subgraph "Testing Boundary"
    direction TB
    TC1["tests/collectors"]:::test
    TC2["tests/preprocessing"]:::test
    TF["tests/features"]:::test
    TL["tests/learning"]:::test
    TI["tests/test_integration.py"]:::test
    click TC1 "https://github.com/kawasakiharuto712/review_priority/tree/main/tests/collectors/"
    click TC2 "https://github.com/kawasakiharuto712/review_priority/tree/main/tests/preprocessing/"
    click TF "https://github.com/kawasakiharuto712/review_priority/tree/main/tests/features/"
    click TL "https://github.com/kawasakiharuto712/review_priority/tree/main/tests/learning/"
    click TI "https://github.com/kawasakiharuto712/review_priority/blob/main/tests/test_integration.py"
  end

  subgraph "Packaging & Deployment"
    direction TB
    PYPROJ["pyproject.toml"]:::pack
    REQ["requirements.txt"]:::pack
    DOCKERF["Dockerfile"]:::pack
    COMPOSE["docker-compose.yml"]:::pack
    click PYPROJ "https://github.com/kawasakiharuto712/review_priority/blob/main/pyproject.toml"
    click REQ "https://github.com/kawasakiharuto712/review_priority/blob/main/requirements.txt"
    click DOCKERF "https://github.com/kawasakiharuto712/review_priority/tree/main/Dockerfile"
    click COMPOSE "https://github.com/kawasakiharuto712/review_priority/blob/main/docker-compose.yml"
  end

  classDef external fill:#d3d3d3,stroke:#333,stroke-width:1px;
  classDef cli fill:#f4a261,stroke:#333,stroke-width:1px;
  classDef module fill:#2a9d8f,stroke:#333,stroke-width:1px;
  classDef datastore fill:#2b9348,stroke:#333,stroke-width:1px,stroke-dasharray:5 5;
  classDef config fill:#e9c46a,stroke:#333,stroke-width:1px;
  classDef util fill:#8ecae6,stroke:#333,stroke-width:1px;
  classDef test fill:#b0b0b0,stroke:#333,stroke-width:1px,stroke-dasharray:2 2;
  classDef pack fill:#264653,stroke:#fff,stroke-width:1px;
```
