How sameerbandi/tenx-data-engineer-starter works
This repository derives August 2026 month-end advisor positions from legacy files, compares them with migrated-platform positions, and writes a reconciliation report. The user starts the workflow with python run.py; the entry point configures and runs the DuckDB pipeline, then validates the report. The main stages are ingestion, cleaning and valuation, position derivation, reconciliation, and report output. The graph reflects documented inputs and shown calls; some SQL details and report-writing implementation were not excerpted.
Main entry points: pipeline.py Pipeline coordinator · run.py Command runner · migrated Migrated positions · legacy Legacy CSV data
Components 12
Entry point
- Command runner
run.py
Position processing
- Pipeline coordinator
pipeline.py - Raw ingestion
pipeline.py - Cleaning and pricing
pipeline.py - Position derivation
pipeline.py
Reconciliation
- Position reconciliation
pipeline.py - Report writer
pipeline.py
Inputs and output
- Legacy CSV data
data/legacy - Migrated positions
data/migrated - Reconciliation report
out/recon_report.json - Report schema
schema/recon_report.schema.json
Other components
- Operator
Connections 14
- Operator → Command runner: runs command
README.md - Command runner → Pipeline coordinator: starts pipeline
run.py - Pipeline coordinator → Raw ingestion: calls ingestion
pipeline.py - Pipeline coordinator → Cleaning and pricing: calls cleaning
pipeline.py - Pipeline coordinator → Position derivation: calls derivation
pipeline.py - Pipeline coordinator → Position reconciliation: calls reconciliation
pipeline.py - Pipeline coordinator → Report writer: calls writer
pipeline.py - Raw ingestion → Legacy CSV data: reads CSVs
pipeline.py - Raw ingestion → Migrated positions: reads positions
pipeline.py - Cleaning and pricing → Position derivation: provides cleaned data
pipeline.py - Position derivation → Position reconciliation: provides positions
pipeline.py - Position reconciliation → Migrated positions: compares against
README.md - Report writer → Reconciliation report: writes report
README.md - Command runner → Report schema: validates report
run.py