How anhtuanhsgs/colorrl works
Project Type and Purpose
• This is ColorRL, a research‐grade machine‐learning project implementing a reinforcement‐learning approach (“Reinforced Coloring”) for end‐to‐end instance segmentation of 2D and 3D biomedical imagery (e.g., CVPPP plant leaves, CREMI neuron data, zebrafish volumes).
• The README and notebooks show pipelines for data preparation, RL environment setup, training an agent (policy network), model checkpointing, deployment for inference, and post‐hoc evaluation.
File‐Structure Analysis
• Top‐level directories:
• Data/ – raw and preprocessed data folders (CVPPP, CREMI), plus scripts (ExtractData.py) to convert and organize images/labels.
• Utils/ – Python utilities and Jupyter notebooks for data augmentation (img_aug_func.py), synthetic data generation (SynData.ipynb), metrics, logging (Logger.py), and other helper functions.
• models/ – definitions of neural architectures (UNet, AttUNet variants, DeepLabV3, GCN‐UNet, Conv‐GRU/LSTM modules, etc.) and the main policy model API.
• evaluation/ – Jupyter notebooks and helper scripts for computing segmentation metrics and visualizing results on different datasets.
• run_scrips/ – collection of bash scripts orchestrating training and deployment across datasets (cvppp_train.sh, 256_cremi_train.sh, 3d_zebrafish_train.sh, etc.).
• Entrypoint scripts at root: main.py (configures environment, data paths, hyperparameters), train.py (starts RL training loop), deploy.py (loads a checkpoint and runs inference), load.py (dataset loading), test.py/test_models.py (unit/functional tests), shared_optim.py, rewards.py, player_util.py.
• logs/ – hierarchical subdirectories capturing prior training runs, organized by dataset, date, and model variant, containing scripts, environment snapshots, logs, and saved checkpoints.
a. Main Components
• Data Ingestion & Preprocessing: Data/ExtractData.py, load.py, Utils/img_aug_func.py, Utils/utils.py
• Policy Network (Agent): models/main.py + models/*.py define CNN‐based actor networks (UNet, AttUNet variants, etc.), sometimes with temporal modules (Conv‐GRU/LSTM)
• Training Loop: train.py orchestrates episodes, interacts with environment, updates policy via shared_optim.py (optimizer and learning‐rate schedule), logs to TensorBoard (Logger.py) and disk (logs/)
• Deployment / Inference: deploy.py loads a saved checkpoint (via --load flag configured in run_scrips/*.sh), wraps the environment for inference, saves predicted instance masks to deploy/
• Evaluation: evaluation/utils.py and evaluation/metrics.py, plus Jupyter notebooks in evaluation/ compute IoU, boundary‐F1, etc., and generate figures.
b. Relationships & Data Flow
1. Bash script (run_scrips/*.sh) → launches main.py/train.py/deploy.py with command‐line args
2. main.py or train.py calls load.py to list Data/A & Data/B images and applies augmentations via Utils/img_aug_func.py
3. environment.py exposes step() and reset() to the agent; retrieves images and labels from Data/ via load.py
4. train.py loops: reset env → for t in horizon: policy_network(state) → action → env.step(action) → reward → store transition → after each batch, call shared_optim optimizer to update policy parameters
5. Checkpoints saved under trained_models/ (implied by README) and logs written to logs/ and TensorBoard via Logger.py
6. deploy.py loads checkpoint, runs episodes on test images, outputs predicted masks to deploy/
7. evaluation notebooks read deploy/ results and use evaluation/metrics.py to compute performance.
c. Architectural Patterns & Design Principles
• Modular directory layout separating data, model definitions, utilities, scripts, and evaluation
• Gym‐style RL environment abstraction promoting plug‐and‐play of different model architectures
• Separation of concerns: policy networks in models/, optimization logic in shared_optim.py, reward logic in rewards.py, data I/O in load.py, experimentation orchestration via shell scripts.
• Logging & reproducibility: logs/ structure that mirrors run_scrips parameters, checkpoint versioning, environment snapshots.
d. Key Technologies & Frameworks
• PyTorch 1.x for neural network models and training
• OpenAI Gym pattern (custom env in environment.py)
• NumPy and SciPy for data handling
• TIFF, PNG I/O via PIL or tifffile (implied)
• Bash for experiment orchestration
• Jupyter notebooks for exploratory evaluation
• TensorBoard for training visualization