How hasnocool/tradingview-script-downloader works
Main Components
a) Input Layer:
• Multiple fetch scripts (fetch.py, fetch_editors-picks.py, fetch_indicators.py, etc.)
• Configuration files (requirements.txt, docker-compose.yaml)
b) Processing Layer:
• Concurrent Download Manager
• Selenium WebDriver Controller
• Aiohttp Client Manager
• Data Processing Components (clean-data.py)
c) Storage Layer:
• File System Storage
• Backup System (backup-data.sh)
Component Relationships
[External Systems]
• v
[Input Layer] --> [Processing Layer] --> [Storage Layer]
Specific Implementation Details
a) External Systems (Top Layer):
• Both HTTP/HTTPS connections
b) Input Layer (Upper Middle):
• Configuration management
• Docker container boundaries (reference Dockerfile and docker-compose.yaml)
c) Processing Layer (Middle):
• Two parallel processing paths:
• Selenium WebDriver (Headless Firefox)
• Aiohttp Client Pool
• BeautifulSoup4 parsing component
d) Storage Layer (Bottom):
• File system storage component
• Backup system component
• Data flow for cleaned and raw data
Important Flows to Highlight
• Concurrent request handling
• Data cleaning pipeline
• Backup processes
• Docker containerization boundaries
Technical Details
Annotations for:
• Python version
• Key dependencies (Selenium, aiohttp, BeautifulSoup4)
• Concurrent processing limits
• File storage patterns