How jedaniel777-tech/ai-investing works
1. Repository purpose
- This is a Python command-line toolkit for constructing and testing quantitative stock portfolios.
- The core workflow is: choose a stock universe, fetch historical prices, build multiple optimized portfolios, then backtest and/or forward-test them against a benchmark index.
2. Primary entrypoint and orchestration
- portfolio_manager.py is the main coordinator described in the README.
- It accepts CLI flags such as is_test, future_bars, data_granularity_minutes, history_to_use, apply_noise_filtering, market_index, only_long, eigen_portfolio_number, and stocks_file_path.
- This file likely drives the full pipeline: input parsing, data loading, strategy execution, and results reporting.
3. Input boundary: stock universe files
- The stocks/ folder contains plain-text ticker lists like stocks.txt and all_stocks.txt.
- These files define the investable universe and are the main user-controlled input to the system.
- The architecture is universe-driven rather than market-wide; the strategies only operate on the symbols listed here.
4. Market data ingestion
- data_loader.py is the data access layer.
- The README states it loads historical price data from Yahoo Finance via yfinance.
- It likely normalizes time granularity, trims history based on the requested window, and splits the data into training and optional out-of-sample test ranges.
5. Strategy subsystem
- The strategies/ package contains the portfolio construction algorithms:
- eigen_portfolio_strategy.py
- minimum_variance_strategy.py
- maximum_sharpe_ratio_strategy.py
- genetic_algo_strategy.py
- These strategies all consume the same market data/returns inputs but differ in how they optimize portfolio weights and risk/reward characteristics.
6. Shared strategy utilities and dispatch
- strategies/strategy_helper_functions.py appears to hold reusable math and preprocessing helpers used by multiple strategies.
- strategy_manager.py is the abstraction layer that selects and coordinates the implemented strategies, so the orchestrator does not need to know strategy-specific details.
7. Optimization and risk modeling
- The strategies rely on return series, covariance estimation, and portfolio optimization.
- The README highlights “noise filtering” using random matrix theory, which affects covariance estimation before optimization.
- only_long and eigen_portfolio_number are important control parameters that materially change the optimization boundary conditions.
8. Evaluation subsystem
- backtester.py is responsible for measuring how generated portfolios perform over historical and forward periods.
- It compares strategy returns against the chosen benchmark index such as SPY or QQQ.
- This is the main validation boundary between “portfolio generation” and “performance assessment.”
9. Forward simulation
- simulator.py provides a Monte Carlo style future-price/return simulation based on historical returns.
- This is a separate analysis path from backtesting: backtesting uses actual historical out-of-sample data, while simulation projects possible future outcomes from the observed distribution.
10. Validation and command metadata
- argchecker.py likely validates CLI arguments and enforces consistency between flags such as is_test and future_bars.
- commands.json appears to store example or preset commands, supporting reproducible runs and user guidance.
11. Packaging and runtime
- The project is a pure Python analytics tool, not a web service.
- requirements.txt shows the key runtime dependencies: NumPy, Pandas, SciPy, scikit-learn, tqdm, and yfinance.
- Dockerfile indicates the tool can be run in a containerized environment, which is useful for reproducibility.
12. End-to-end data flow
- The high-level flow is: stock list → Yahoo Finance price history → return/covariance construction → strategy optimization → portfolio weights → backtest/simulation → benchmark comparison and output.
- There is no evidence of a database or long-lived state; the system appears to be file/input driven and compute-oriented.
13. Diagram boundary suggestion
- For an architecture diagram, the clearest split is:
- Inputs: stock list files, CLI flags, benchmark symbol
- Core engine: data loader + strategy manager + strategy implementations
- Evaluation: backtester + simulator
- Outputs: portfolio weights, performance plots/results, optional forward-test comparisons
Main entry points: portfolio_manager.py Portfolio manager · strategy_manager.py Strategy manager · strategy_helper_functions.py Strategy helpers · data_loader.py Data loader
Components 16
Inputs
User-controlled universe and run configuration.
- Stocks list · Primary ticker universe input file.
stocks/stocks.txt - All stocks · Broader ticker universe input file.
stocks/all_stocks.txt - Commands · Example or preset command configurations for reproducible execution.
commands.json
Core Engine
Data loading, orchestration, and strategy selection.
- Portfolio manager · Main command-line entrypoint that coordinates the end-to-end portfolio workflow.
portfolio_manager.py - Arg checker · Validates command-line arguments and run-mode consistency.
argchecker.py - Data loader · Fetches and prepares historical price data for the selected universe.
data_loader.py - Strategy manager · Selects and coordinates the portfolio construction strategies.
strategy_manager.py
Strategies
Portfolio construction algorithms and shared helpers.
- Eigen portfolio · Builds portfolios using eigenvector-based allocation logic.
strategies/eigen_portfolio_strategy.py - Min variance · Constructs low-volatility portfolios by minimizing variance.
strategies/minimum_variance_strategy.py - Max Sharpe · Optimizes portfolios for risk-adjusted return.
strategies/maximum_sharpe_ratio_strategy.py - Genetic algo · Searches for portfolio weights using evolutionary optimization.
strategies/genetic_algo_strategy.py - Strategy helpers · Shared preprocessing, return handling, and optimization helpers.
strategies/strategy_helper_functions.py
Evaluation
Backtesting and forward simulation of candidate portfolios.
- Backtester · Measures historical and out-of-sample portfolio performance against a benchmark.
backtester.py - Simulator · Generates forward-looking price or return scenarios from observed data.
simulator.py
Runtime
Packaging and execution metadata.
- Requirements · Python runtime dependencies used by the toolkit.
requirements.txt - Dockerfile · Container image definition for reproducible execution.
Dockerfile
Connections 21
- Stocks list → Portfolio manager: universe input · Ticker list selected for a run.
- All stocks → Portfolio manager: universe input · Alternative broader ticker list.
- Commands → Portfolio manager: preset commands · Provides runnable examples or presets.
- Portfolio manager → Arg checker: validate args · Checks CLI flags and run-mode constraints.
- Portfolio manager → Data loader: request data · Loads historical prices for the chosen universe.
- Data loader → Strategy manager: price/returns · Supplies prepared market data for strategy execution.
- Strategy manager → Eigen portfolio: dispatch · Runs eigen-portfolio construction.
- Strategy manager → Min variance: dispatch · Runs minimum-variance optimization.
- Strategy manager → Max Sharpe: dispatch · Runs maximum Sharpe optimization.
- Strategy manager → Genetic algo: dispatch · Runs genetic-algorithm optimization.
- Eigen portfolio → Strategy helpers: use helpers · Shares preprocessing and math utilities.
- Min variance → Strategy helpers: use helpers · Shares preprocessing and math utilities.
- Max Sharpe → Strategy helpers: use helpers · Shares preprocessing and math utilities.
- Genetic algo → Strategy helpers: use helpers · Shares preprocessing and math utilities.
- Strategy manager → Backtester: portfolio weights · Passes candidate portfolios for evaluation.
- Portfolio manager → Backtester: evaluate · Triggers historical and benchmark comparison.
- Portfolio manager → Simulator: simulate · Triggers forward outcome generation.
- Backtester → Simulator: shared stats · Uses historical return structure as analysis context.
- Requirements → Data loader: depends on · Lists runtime libraries used by ingestion and analytics.
- Requirements → Strategy manager: depends on · Lists runtime libraries used by optimization routines.
- Dockerfile → Portfolio manager: run target · Container entry environment for the CLI workflow.