How adithya916/reinforcement-learning-market-making-agent works
This is a compact PPO market-making research framework: a Gym environment simulates price movement and fills, agents choose quoting spreads, and the runner trains PPO before comparing it with an Avellaneda–Stoikov baseline. The graph separates environment dynamics, agent internals, and benchmark orchestration. The README documents the overall workflow; sampled source shows the concrete calls. Environment stages are conceptual responsibilities within one file, not separate runtime modules.
Main entry points: main_run.py Workflow Runner · ppo_agent.py PPO Agent · mm_env.py Market Environment
Components 14
Market Environment
- Market Environment
mm_env.py - Price and Volatility
mm_env.py - Order Execution
mm_env.py - PnL and Reward
mm_env.py
Trading Agents
- PPO Agent
ppo_agent.py - Actor-Critic Policy
ppo_agent.py - Transition Buffer
ppo_agent.py - AS Baseline
ppo_agent.py
Training and Benchmark
- Workflow Runner
main_run.py - PPO Training
main_run.py - Validation Rollouts
main_run.py - Performance Metrics
main_run.py - Comparison Report
main_run.py
Other components
- User
Connections 20
- User → Workflow Runner: runs
README.md - Workflow Runner → PPO Training: starts training
main_run.py - PPO Training → Market Environment: steps
main_run.py - PPO Training → PPO Agent: selects actions
main_run.py - PPO Training → PPO Agent: stores transitions
main_run.py - PPO Training → PPO Agent: updates policy
main_run.py - Workflow Runner → AS Baseline: creates
main_run.py - Workflow Runner → Validation Rollouts: runs validation
main_run.py - Validation Rollouts → Market Environment: steps
main_run.py - Validation Rollouts → PPO Agent: selects actions
main_run.py - Validation Rollouts → AS Baseline: gets actions
main_run.py - Validation Rollouts → Performance Metrics: provides PnL and inventory
main_run.py - Workflow Runner → Performance Metrics: calculates
main_run.py - Workflow Runner → Comparison Report: prints
main_run.py - Market Environment → Price and Volatility: updates
mm_env.py - Market Environment → Order Execution: handles fills
mm_env.py - Market Environment → PnL and Reward: computes reward
mm_env.py - PPO Agent → Actor-Critic Policy: samples and evaluates
ppo_agent.py - PPO Agent → Transition Buffer: stores transitions
ppo_agent.py - AS Baseline → AS Baseline: maps state to spread
ppo_agent.py