How atharvak-2209/amazon-ml-challenge-26 works
This repository implements business-entity resolution: it normalizes business records, retrieves candidate pairs, engineers pairwise evidence, scores likely matches, and produces entity-level submission results including non-matches. The main runtime route is explicit in src/pipeline.py; the README also describes training, evaluation, and phased experimentation. Several phase modules are unsampled, so their internal connections are omitted rather than inferred.
Main entry points: src/pipeline.py End-to-end pipeline · phase8/pipeline.py Final submission pipeline · pair_features.py Pair feature assembly · normalize.py Normalization
Components 24
Record preparation
- Normalization
src/preprocessing/normalize.py - Address parsing
src/preprocessing/address_parser.py
Candidate matching
- TF-IDF blocking
src/blocking/tfidf_blocking.py - Exact blocking
src/blocking/exact_blocking.py - Neighborhood blocking
src/blocking/token_blocking.py - Dense ANN blocking
src/blocking/faiss_blocking.py - Pair feature assembly
src/features/pair_features.py - Name features
src/features/name_features.py - Address features
src/features/address_features.py
Scoring and decisions
- Model training
src/models/train_xgb.py - Pair prediction
src/models/predict.py - Decision engine
src/decision/decision_engine.py - Submission assembly
src/decision/submission_pipeline.py - Singleton filling
src/decision/singleton.py - F0.5 evaluation
src/evaluation/evaluate_f05.py
Pipelines and experiments
- End-to-end pipeline
src/pipeline.py - Blocking experiments
src/phase3/pipeline.py - Feature pipeline
src/features/phase4_pipeline.py - Decision pipeline
src/phase6/pipeline.py - Experiment runners
src/phase7/combined_runner.py - Final submission pipeline
src/phase8/pipeline.py
Other components
- Pipeline user
- Entity records
- Matching and candidate files
Connections 32
- Pipeline user → End-to-end pipeline: runs
src/pipeline.py - Entity records → End-to-end pipeline: loads
src/pipeline.py - End-to-end pipeline → Normalization: normalizes
src/pipeline.py - End-to-end pipeline → TF-IDF blocking: generates candidates
src/pipeline.py - End-to-end pipeline → Exact blocking: generates candidates
src/pipeline.py - End-to-end pipeline → Neighborhood blocking: generates candidates
src/pipeline.py - End-to-end pipeline → Pair feature assembly: builds features
src/pipeline.py - Pair feature assembly → Name features: extracts name signals
src/features/pair_features.py - Pair feature assembly → Address features: extracts address signals
src/features/pair_features.py - End-to-end pipeline → Pair prediction: scores pairs
src/pipeline.py - End-to-end pipeline → Singleton filling: fills unmatched entities
src/pipeline.py - End-to-end pipeline → Matching and candidate files: writes outputs
src/pipeline.py - Model training → Pair prediction: produces model
README.md - Blocking experiments → Normalization: uses normalization
src/phase3/pipeline.py - Blocking experiments → Pair feature assembly: uses features
src/phase3/pipeline.py - Feature pipeline → Normalization: uses normalization
src/features/phase4_pipeline.py - Feature pipeline → Pair feature assembly: builds features
src/features/phase4_pipeline.py - Decision pipeline → Decision engine: applies decisions
src/phase6/pipeline.py - Decision pipeline → F0.5 evaluation: evaluates results
- Decision engine → F0.5 evaluation: evaluates decisions
src/decision/decision_engine.py - Submission assembly → Singleton filling: fills singletons
src/decision/submission_pipeline.py - Experiment runners → Model training: experiments with models
- Final submission pipeline → Normalization: normalizes records
src/phase8/pipeline.py - Final submission pipeline → TF-IDF blocking: generates candidates
src/phase8/pipeline.py - Final submission pipeline → Exact blocking: generates candidates
src/phase8/pipeline.py - Final submission pipeline → Neighborhood blocking: generates candidates
src/phase8/pipeline.py - Final submission pipeline → Pair feature assembly: builds features
src/phase8/pipeline.py - Final submission pipeline → Pair prediction: scores pairs
src/phase8/pipeline.py - Final submission pipeline → Decision engine: applies decisions
src/phase8/pipeline.py - Final submission pipeline → Matching and candidate files: writes submission files
- End-to-end pipeline → Address parsing: parses addresses
src/pipeline.py - Dense ANN blocking → Entity records: retrieves candidates