How korepupavankumar/business-entity-resolution works
This is a business-entity resolution application: it trains a classifier from labeled records and then matches Source 1 entities against Sources 2 and 3. The main run is orchestrated by pipeline.py: load and normalize records, generate candidates, train or apply the model, and write candidate and match files. Training also builds labeled feature pairs using ground truth; prediction scores candidate pairs against a threshold. The README provides only the product name, so feature details and links here are based on the sampled source; unsampled subsystem wiring is omitted.
Main entry points: pipeline.py Workflow runner · train_model.py Classifier training · predict.py Candidate scoring · blocking.py Candidate blocking
Components 16
Run orchestration
- Workflow runner
pipeline.py
Matching stages
- Record normalization
normalization.py - Candidate blocking
blocking.py - Blocking index
blocking.py - Candidate scoring
predict.py
Model training
- Classifier training
train_model.py - Pair features
features.py - Trained model
Data and evaluation
- TSV loader
data_loader.py - Ground-truth mapping
ground_truth.py - Match evaluation
evaluate.py - Result writer
output_writer.py
Other components
- Operator
- Training TSVs
- Test TSVs
- Output TSVs
Connections 20
- Operator → Workflow runner: starts run
- Training TSVs → TSV loader: provides records
- Test TSVs → TSV loader: provides records
- Workflow runner → TSV loader: loads data
- Workflow runner → Record normalization: normalizes records
- Workflow runner → Candidate blocking: generates candidates
- Candidate blocking → Blocking index: reads and writes
- Workflow runner → Ground-truth mapping: maps labels
- Workflow runner → Classifier training: trains classifier
- Classifier training → Pair features: builds pair features
- Classifier training → Ground-truth mapping: uses labels
- Classifier training → Candidate blocking: builds source indexes
- Classifier training → Trained model: saves model
- Workflow runner → Trained model: loads model
- Workflow runner → Candidate scoring: scores candidates
- Candidate scoring → Pair features: computes features
- Candidate scoring → Trained model: scores with model
- Workflow runner → Match evaluation: evaluates blocking
- Workflow runner → Result writer: writes results
- Result writer → Output TSVs: writes TSVs