How jan1na/proki-hackathon-24 works
Project type and purpose
• This is not a web app or microservice mesh but a command-line ML pipeline for image-based robotic gripper placement.
• It takes CSV task files, runs a deep-learning segmentation model, post-processes the masks, then runs a simulated-annealing optimizer and writes out solutions.
File structure
• Top level:
• Dockerfile, setup.py, requirements.txt → packaging & deployment
• data/ → placeholder input images & masks
• readme_images/ → static assets for documentation
• solution/ → main pipeline code + model checkpoint
• evaluate/ & my_evaluate/ → evaluation scripts & ground-truth/task CSVs
• solution/ substructure:
• main.py → CLI entrypoint
• deeplabv3_resnet101_preprocessing.py & preprocessing_utils.py → image prep
• DeepLabV3Plus_finetuning.ipynb → model fine-tuning notebook (Colab)
• simulated_annealing.py → optimizer
• visualizer.py → result rendering
• Configuration/build: setup.py for pip install; optional Dockerfile for containerization
README insights
• Three-stage pipeline:
1. Image segmentation via a fine-tuned DeepLabV3-Plus (ResNet101)
2. Post-processing (threshold + erosion) to get binary masks
3. Simulated annealing for gripper (x,y,θ) placement
• CLI usage: python solution/main.py <input tasks.csv> <output solutions.csv>
• External dependency: large model checkpoint stored on Google Drive
Main components
a. CLI Orchestrator (main.py) – reads input CSV, coordinates services, writes output CSV
b. Segmentation Service – loads fine-tuned DeepLabV3Plus model, performs inference on images
c. Preprocessing Module – resizes, normalizes, thresholds & erodes masks
d. Optimization Engine – runs simulated annealing to choose best gripper pose
e. Visualization & Evaluation – optional, via visualizer.py and evaluate/*.py for ground-truth comparisons
f. Data Stores & Artifacts – image assets, model checkpoint, input task CSVs, output CSVs
g. External Services – Google Drive for model storage (or S3); GPU resource for training/inference
Relationships & data flow
• The CLI Orchestrator triggers Segmentation → outputs raw mask
• Preprocessing Module consumes raw mask → produces binary mask
• Optimization Engine consumes binary mask + gripper geometry → produces (x,y,θ)
• Visualization can tap into binary mask + pose → render overlay
• Evaluation scripts consume output CSV + ground_truth.csv → compute metrics
Architectural patterns & design principles
• Pipeline Pattern: clear linear stages (segmentation → post-processing → optimization)
• Modularity: each stage separated into its own Python module
• Separation of concerns: CLI logic separate from model code and optimization logic
Detail level
• Technology/tool names inside or next to modules (e.g., “DeepLabV3-Plus (PyTorch)”, “SciPy SimAnneal”)
• Essential edges only—don’t recreate every util function, but do show major modules