How rwchakra/conbias works
Project Type and Purpose
• This is a research/ML framework intended for visual dataset debiasing. It diagnoses spurious concept co-occurrence biases by representing datasets as concept graphs and then employs a clique-based sampling method for data augmentation.
• The design supports multiple visual datasets (Waterbirds, UrbanCars, Coco-GB) and integrates both pre-processing (graph and clique generation) and training/evaluation routines for deep learning (e.g., backing off a base ResNet model).
File Structure
• At the top level, there are configuration files (LICENSE, README) and two primary directories:
• metadata – Contains precomputed artifacts such as clique dictionaries (in .pkl and .json formats), CSV files for co-occurrence matrices, and JSON files listing concept data.
• src – Contains the core Python scripts responsible for key tasks:
○ co_occurrence.py and co_occurrence_cliques.py: Scripts to compute co-occurrence of concepts from visual data.
○ concept_graph.py: Likely responsible for generating and handling the concept graph of the dataset.
○ concept_sampler.py: Samples or generates balanced cliques for data augmentation.
○ train.py and evaluate.py: Handle model training as well as evaluation (including out-of-distribution evaluation).
○ dataloaders.py: Sets up data ingestion pipelines (this script may need to be modified to point at the correct dataset location).
○ cutmix_proc.py: Possibly implements augmentation (or a variant of CutMix) to combine images, integrating with the concept balancing.
• The overall structure indicates a pipeline beginning with data input (and metadata), processing (concept graph & cliques), and finally training/evaluation.
How to Map the System Components
a. Main Components:
• External Data Sources:
• Metadata Components:
• Precomputed files for cliques and concept mappings (stored in the metadata folder). These act as reference data for the augmentation pipeline.
• Data Processing Modules (Preprocessing):
• Scripts for co-occurrence computation (co_occurrence.py, co_occurrence_cliques.py).
• Concept graph creation (concept_graph.py).
• Concept sampling/clique formation (concept_sampler.py).
• Augmentation processing (cutmix_proc.py).
• Training and Evaluation:
• Training module (train.py) that uses the processed data along with metadata for boosting performance via data augmentation.
• Evaluation module (evaluate.py) that supports both standard and OOD testing.
• Data Loader:
• dataloaders.py to abstract how datasets are loaded into the training/evaluation pipeline.
b. Relationships and Interactions:
• Data flows from external datasets through the dataloaders into the training pipeline.
• Preprocessing (via co-occurrence and concept graph generation scripts) reads the metadata files, refines and balances the visual concepts.
• The output of concept sampling (balanced cliques in JSON) feeds into the training script to perform data augmentation.
• Training results (checkpoints for the ResNet baseline) are then used by the evaluation module.
• All modules interact via command-line interfaces, where arguments (like dataset names, checkpoint paths, augmentation flag, etc.) dictate execution flow.
c. Architectural Patterns and Design Principles:
• Modularity: Separate scripts for distinct tasks (graph generation, sampling, training, evaluation) indicate a modular design.
• Pipeline-based processing: Data flows sequentially from ingestion (via dataloaders) to preprocessing and training.
• CLI-based Execution: The use of Python scripts with command-line arguments supports simplicity and adaptability across datasets.
• Data Augmentation by Balancing: Emphasis on clique-based augmentation leveraging metadata ensures that data pre-processing is tightly coupled with the augmentation strategy for debiasing.
d. Relevant Technologies, Frameworks, and Libraries:
• Python is the primary language with common dependencies likely including libraries for deep learning (PyTorch or TensorFlow, though the specific library isn’t directly mentioned) and data processing (NumPy, Pandas, etc.).
• Preprocessing and file serialization rely on formats like .pkl (pickle), JSON, and CSV.
• The training scripts reference a base ResNet model, indicating the use of a deep learning framework and possibly GPU-accelerated training.
Instructing the Principal Software Engineer
• From the Data Loader, branch flows into both the Training Pipeline and the Preprocessing modules (which then feed results back into the Training Pipeline for augmentation).
• That the Training Pipeline outputs model checkpoints that are subsequently consumed by the Evaluation Pipeline.
• Rectangles for modules (like “Concept Graph Builder”),