How claudiobors/robycode works
Project Type and Purpose
• This is a Python-based machine learning experimentation toolkit (“roby”) focused on three main analysis modes: referral, robustness, and uncertainty.
• It provides core engines for running neural-network experiments (ReferralNN, RobustnessNN, UncertaintyNN), collecting results, computing metrics, and generating visualizations.
High-Level File Structure Analysis
• Root scripts and batch files:
• run_roby.py – main entry point
• download_mnist.py – dataset fetcher
• *.bat files – Windows wrappers for common tasks (build, test, run)
• Configuration:
• json config/ – JSON templates driving experiment parameters (referral, robustness, uncertainty)
• Core library (roby/):
• Alterations.py, EnsembleTools.py, CloudTools.py, Utility.py – shared services (data transforms, cloud integration, utility functions)
• EnvironmentRTest.py, Reporter.py – environment management and logging
• ReferralNN.py, RobustnessNN.py, UncertaintyNN.py – three specialized experiment engines
• *_Results.py – result collectors for each mode
• Metrics and visualization:
• roby/metrics/ – modular metrics implementations (accuracies, calibration, curves, utility, area-based)
• roby/uncertainty/measures.py – uncertainty-specific measures
• roby/viz/ – plotting routines for referral, robustness
Identifying Main Components
a. Runner Layer (CLI & scripts):
– run_roby.py, batch wrappers → orchestrates the workflow.
b. Configuration Layer:
– JSON config files → parameter definitions for each analysis mode.
c. Data Layer:
– download_mnist.py → data ingestion; Alterations → data augmentation.
d. Core Processing Layer:
– ReferralNN, RobustnessNN, UncertaintyNN → load data + config, execute model training/evaluation.
– Utility, EnsembleTools, CloudTools → shared operations (model ensemble, cloud compute offloading).
e. Metrics Layer:
– roby/metrics + roby/uncertainty/measures → compute performance metrics post-run.
f. Visualization & Reporting Layer:
– roby/viz/referral.py, robustness.py → produce plots; Reporter → generate HTML logs.
g. Output Artifacts:
– log_*.html files, saved model checkpoints, result data structures.
Relationships & Data Flow
• Start: Runner reads CLI arguments → loads appropriate JSON config.
• Data ingestion: download_mnist → data repository → Alterations preprocesses data.
• Processing: Core engines (ReferralNN, etc.) consume data + config → train/evaluate models.
• Shared tools: Utilities, Ensembles, CloudTools invoked as needed.
• Metrics: Engines emit raw predictions → metrics layer computes statistics.
• Reporting: Reporter + viz modules generate HTML/logs/plots.
Architectural Patterns & Design Principles
• Layered architecture – clear separation of concerns (configuration, data, processing, metrics, viz).
• Modular plug-in structure – adding a new analysis mode or metric only requires dropping in a new module under roby/.
• Dependency injection via JSON config – experiment parameters externalized.
• Single-responsibility classes – each NN mode and results class handles its own logic.