How sagunsh/sofifa-scraper works
Main Components
a) Input/Configuration Layer:
• CLI Arguments Handler
• Configuration Parameters (filename, max_pages, year)
b) Core Scraping Engine:
• Web Scraper Component
• Data Parser
• Rate Limiter/Request Manager
c) Output Layer:
• File Writer (CSV/JSON)
Component Relationships
• User input → CLI Arguments Handler
• CLI Handler → Scraper Configuration
• Scraper → External Service (sofifa.com)
• Scraper → Data Parser
• Parser → File Writer
• [User Input] → [CLI Arguments Parser]
• ↓
• [Configuration] → [Rate Limiter] → [Web Scraper] ← [sofifa.com]
• ↓
• [Data Parser]
• ↓
• [File Writer] → [Output File]
Additional Details
• Supported file formats (CSV/JSON) in the File Writer component
• HTTP requests flow to sofifa.com
• A note about rate limiting/responsible scraping
• Version compatibility (Python 3.6+)
Technical Specifications to Note
• Key Python dependencies from requirements.txt
• Command-line interface options
• Error handling paths
Important Notes
• A note about responsible usage
• Error handling paths
• Where rate limiting occurs