How improve-ai/tracker-trainer works
a. Client → Tracking Endpoint → Kinesis Firehose
b. Firehose → S3 Raw Data
c. Ingest Lambda → Processed Data in S3
d. Training Pipeline:
• Processed Data → SageMaker Training Job
• Training Job → Model Artifacts
• Model Artifacts → Model Serving Layer
Key Components to Highlight
a. XGBoost Training Pipeline:
• Feature encoding (feature_encoder.py)
• Decision training (decision_trainer.py)
• Model checkpointing (checkpoint.py)
b. Data Processing:
• Firehose record processing
• Parquet conversions
• Feature selection and flattening
Technical Details
• Serverless components (Lambda, Firehose)
• File formats (Parquet, XGB, MLModel)
• Public endpoints vs private infrastructure
Architecture Notes
• Serverless nature of the system
• Multi-region capability if applicable
• Scaling points
• Integration points for different SDK clients (iOS, Android, Python)