How xmootoo/optrade works
Main high-level components
• Data Ingestion Layer
– ThetaData API integration (optrade/data, optrade/dev/download_universe.py)
– Universe selection (optrade/data/universe.py)
• Featurization Layer
– Raw → ML-ready transformation (optrade/data/features.py, transform_features)
– TTE, datetime, volatility features
• Dataset Layer
– Single-contract: ForecastingDataset (optrade/data/forecasting.py)
– Multi-contract: ContractDataset + get_forecasting_dataset
• Modeling Layer
– Classical models (scikit-learn & sktime wrappers in opttrade/models/classical)
– Deep learning models (patchtst, emforecaster, etc. in opttrade/models/pytorch)
• Experimentation & Logging Layer
– Experiment class (optrade/exp/forecasting.py)
– scikit-learn experiments (optrade/exp/evaluate.py)
– Neptune logging & offline logging
• Orchestration & CLI
– optrade/main.py (entry point)
– dev utilities (grid_search, run scripts)
– jobs folder (YAML configs, batch run.sh scripts)
Interactions and data flows
1. ThetaData API → Data Ingestion
2. Ingested raw DataFrame → Featurization
3. Feature DataFrame → Dataset constructors → PyTorch DataLoader
4. DataLoader → Model training (Experiment)
5. Model → Metrics → Logging (Neptune/offline) → Results
6. Experiment artifacts → Disk (models, logs)
• Control flows:
– CLI triggers ingestion / featurization / experiment
– Job scheduler (.yaml + run.sh) triggers batch experiments
Architectural patterns & design principles
• Modular layered architecture (Data → Features → Dataset → Model → Experiment)
• Separation of concerns: ingestion vs. featurization vs. modeling vs. logging
• Extensibility points:
– New models in opttrade/models/pytorch
– New features in opttrade/data/features
– Custom experiments via optrade/exp
Technologies & frameworks
• Python 3.10–3.12
• ThetaData API (external service)
• pandas for tabular data
• PyTorch (torch.utils.data.DataLoader)
• scikit-learn, sktime
• Neptune (optional)
• YAML for job configs