How kawasakiharuto712/review_priority works
Project Type and Purpose
• Type: A Python-based data-pipeline + machine-learning CLI tool.
• Purpose: Automate data collection from OpenStack’s Gerrit, preprocess and extract features, apply inverse reinforcement learning (IRL) to predict code‐review priorities.
File Structure
• Top-level:
• main.py – CLI orchestrator
• Dockerfile / docker-compose.yml – containerization
• pyproject.toml, requirements.txt – packaging & dependencies
• src/ – all application code, subdivided into:
◦ collectors/ – Gerrit data fetchers (openstack.py, release_collector.py)
◦ preprocessing/ – review comment cleaning & keyword extraction
◦ features/ – functions computing 16 metrics (bug, change, developer, etc.)
◦ learning/ – IRL model (irl_models.py) and time-series evaluation
◦ config/ – application settings (gerrymanderconfig.ini, path.py)
◦ utils/ – shared helpers (constants, language identifier)
• tests/ – pytest coverage for all modules
README Insights
• Workflow stages: Collect → Preprocess → Feature Extraction → IRL Learning → Predict
• Data folders (logical): raw (openstack/), processed/, results/
• Outputs: JSON (irl_analysis_*.json), Pickle (irl_model_*.pkl)
Main Components and Their Interactions
a. Data Sources & Ingest
• External: OpenStack Gerrit API
• Collector Module: src/collectors/openstack.py, release_collector.py
b. Preprocessing Pipeline
• review_comment_processor.py → cleans & tokenizes comments
• Noise filtering (bot messages)
c. Feature Extraction
• src/features/*.py → computes bug, change, test, developer, project, refactoring, review metrics
d. Machine-Learning / IRL
• src/learning/irl_models.py – defines reward functions & solver
• temporal_model_evaluation.py, temporal_weight_analysis.py – time-series analysis
e. Orchestration & Configuration
• main.py – entry point, subcommands (status, full, collect, preprocess, train, analyze)
• src/config/gerrymanderconfig.ini & path.py – credentials & file paths
f. Utilities
• src/utils – shared constants, language detection
g. Persistence & Outputs
• data/raw (logical) → data/processed → data/results/*.json/.pkl
h. Containerization & Deployment
• Dockerfile – image build
• docker-compose.yml – service definition
i. Testing
• pytest suites under tests/
Architectural Patterns & Design Principles
• Modular Pipeline: clear separation of concerns—ingest, ETL, feature, learning
• CLI Orchestration: single entry-point main.py using subcommands
• Configuration Management: centralized in config module & INI file
• Layered Flow: one-way data flow from collectors → preprocess → features → learning
• Test-Driven: every module has corresponding tests