How praneeth132006/entitymatch-pipeline works
EntityMatch is an offline business-entity resolution pipeline: it retrieves likely cross-source record pairs, trains and applies a CatBoost classifier, validates the resulting files, and packages a reproducible submission. The main workflow is driven by CLI commands in src/pipeline.py; the native C++ retriever performs blocking and feature construction. The README also documents methodology generation and reproduction from a ZIP. Source excerpts are partial, so the graph omits internal data handoffs and calls that cannot be established from the available evidence.
Main entry points: pipeline.py CLI pipeline · retrieve.cpp Candidate retrieval · output Candidate pairs · build_document.py Methodology generation
Components 16
Pipeline orchestration
- CLI pipeline
src/pipeline.py - Output validation
src/pipeline.py - Submission packaging
src/pipeline.py - Pipeline reproduction
src/pipeline.py
Entity matching
- Candidate retrieval
src/retrieve.cpp - Model training
src/pipeline.py - Trained model
- Candidate scoring
src/pipeline.py
Results and packaging
- Candidate pairs
output - Matching results
output - Validation report
output - Submission ZIP
output
Methodology documents
- Methodology generation
src/build_document.py - Methodology files
output
Other components
- Operator
- Input datasets
Connections 23
- Operator → CLI pipeline: runs commands
- CLI pipeline → Candidate retrieval: invokes retrieval
- Input datasets → Candidate retrieval: scanned by
- Candidate retrieval → Model training: provides training pairs
- CLI pipeline → Model training: runs training
- Model training → Trained model: saves model
- CLI pipeline → Candidate scoring: runs prediction
- Trained model → Candidate scoring: scores with
- Input datasets → Candidate scoring: provides test data
- Candidate scoring → Candidate pairs: writes candidates
- Candidate scoring → Matching results: writes matches
- CLI pipeline → Output validation: runs validation
- Input datasets → Output validation: checks target IDs
- Candidate pairs → Output validation: checked by
- Matching results → Output validation: checked by
- Output validation → Validation report: writes report
- CLI pipeline → Submission packaging: runs packaging
- Submission packaging → Submission ZIP: creates ZIP
- CLI pipeline → Methodology generation: dispatches generation
- Methodology generation → Methodology files: writes documents
- CLI pipeline → Pipeline reproduction: runs reproduction
- Pipeline reproduction → Candidate scoring: reruns prediction
- Pipeline reproduction → Output validation: reruns validation