How dali92002/htrbymatching works
Project Type and Purpose
• This is a research‐oriented machine‐learning project implementing a few‐shot handwriting recognition system in PyTorch. It combines object‐detection (Faster R-CNN style) with a metric‐learning / matching module for recognizing ciphered manuscript characters in low‐resource scenarios.
File Structure Analysis
Top-Level:
• alphabet/…: support‐set character images grouped by cipher (borg, copiale, vatican).
• lines/…: query‐set line images arranged per cipher.
• configs.py, htrmatching.yml: environment and hyperparameter settings.
• generate_data.py, load_data.py, utils.py, htr_utils.py, transforms.py: data preparation, augmentation, helper routines.
• train.py, train_progressive.py: entry points for standard vs. progressive few‐shot training.
• test.py: inference script that processes query lines against a trained model.
• src/: core model modules (Faster R-CNN backbone, RPN, ROI heads, matching layers, training/engine loop, COCO evaluation utilities).
Architecture Insights from README
• The system consists of two main stages:
a) Few‐Shot Matching Model (based on Faster R-CNN backbone + metric head)
b) Progressive Fine‐Tuning Pipeline (multi‐stage few‐shot adaptation)
• Input: small support set (“shots”) from alphabet/… + query lines (lines/…).
• Output: detection + classification of cipher characters in query lines.
Main Components & Their Relationships
A. Data Layer
• Support Dataset Loader
– Reads alphabet/<cipher>/<char>/sample.png
– Applies transforms (transforms.py)
• Query Dataset Loader
– Reads lines/<cipher>/line_*.png
– Applies line‐level transforms
• Configuration Parser (configs.py, htrmatching.yml)
B. Model Layer (src/)
• Backbone CNN (e.g., ResNet from generalized_rcnn.py)
• RPN (src/rpn.py) → region proposals for character candidates
• ROI Heads (src/roi_heads.py) → pool features per proposal
• Metric Matching Head (src/util_fc.py) → computes similarity between support features and query ROI features
• Loss & Training Engine (src/engine.py) → handles forward/backward, integrates coco_utils.py, coco_eval.py for metrics
C. Training & Fine‐Tuning
• train.py → standard few‐shot training from scratch or fine‐tuning with omniglot‐pretrained weights
• train_progressive.py → multi‐stage curriculum learning
D. Inference
Architectural Patterns & Design Principles
• Modular MVC‐style separation:
• Models and heads under src/
• Data preparation in separate scripts
• Configuration isolated from code
• Reuse of standard object‐detection pipelines (Faster R‐CNN) combined with custom matching layers
• Few‐shot training orchestrated by passing support and query batches through shared feature extractor