How ainomaly-ai/intelligent-rl-bot works
Project Type and Purpose
• This is a Python‑based reinforcement‑learning trading bot framework for the Solana ecosystem.
• It provides:
• A custom Gymnasium environment to simulate token trades (sol_env)
• A portfolio manager for SOL/token balance and P&L tracking (tokensim.py)
• Training scripts using Stable Baselines 3, SB3‑contrib, Ray RLlib, and Hyperopt for hyperparameter tuning
• Utility modules to feed market data, connect to Solana RPC, Jupiter DEX, SPL stream, and devnet tests
File Structure
Top‑level files:
• environment.yml, requirements.txt, pyproject.toml – dependency and build configs
• README.MD, LICENSE.MD – documentation and licensing
sol_trade/ (main package)
• sol_trade/sol_env/
– sol_env.py: Gym environment definition
– data.json: sample market data
– token_preprocess.py: data cleaning
• sol_trade/utils/
– feeder*.py: historical data and live stream feeders
– dashboard.py: plotting and TensorBoard dashboard utilities
– jupiter_connect.py: aggregation via Jupiter DEX
– sol_devnet_test.py: devnet integration tests
– tokensim*.py: portfolio simulation classes
• Training & tuning scripts at root of sol_trade/
– stable_baseline_test*.py, rayrlib_*.py, sb3_with_hyperopt.py
README for Insights
• Architecture is modular: trading env ←→ portfolio manager ←→ feeder modules
• RL agents (PPO, TRPO) interact with the environment, learn policies, and log to TensorBoard
• Mentions slippage handling, profit-transfer logic, and dynamic value updates
a. Main Components
1. CLI/Training Orchestrator
• Scripts: stable_baseline_test.py, rayrlib_train.py, sb3_with_hyperopt.py
2. RL Agent Layer
• Stable Baselines 3 / Ray RLlib / Hyperopt modules
3. Custom Gym Environment
• sol_trade.sol_env.TradeEnv
4. Portfolio Manager
• tokensim.Portfolio (balance, P&L, transfer methods)
5. Data Feeders & Connectors
• Historical feeder, SPL stream, Jupiter, Solana RPC devnet
6. Logging & Monitoring
• TensorBoard, dashboard.py, saved model checkpoints
b. Relationships and Data Flow
• Orchestrator invokes RL Agent
• Agent queries Environment for state →
• Environment uses Portfolio + Feeder modules to simulate trade results →
• Environment returns next state + reward →
• Agent updates policy →
• Logging module records metrics and checkpoints
• Hyperparameter tuner may spawn multiple Agent/Env instances in parallel
• External connectors pull market liquidity/prices (Jupiter, SPL stream, JSON)
c. Architectural Patterns
• Modular layering: clear separation of RL loop, environment logic, and I/O connectors
• Plugin pattern for feeders and connectors
• Experiment management via TensorBoard and Hyperopt
d. Key Technologies & Frameworks
• Python 3.x, Gymnasium, Stable‑Baselines3, SB3‑contrib, Ray RLlib, Hyperopt
• Solana RPC API, Jupiter aggregator SDK, SPL stream API
• pandas, matplotlib for data processing and plotting