How preyanshz/stock_screener works
Main Components
• Python Application (app.py): This is the core of the scraper. It includes the logic for URL construction, data extraction, error handling, and writing outputs.
• Selenium Integration: Highlight that the script uses the Selenium library to manage web automation.
• External Browser Automation (ChromeDriver with Headless Chrome): Represent the chromedriver.exe that acts as the bridge between Selenium and Chrome running in headless mode.
• Target Website (screener.in): This external website is the source of company financial data.
• Output File (screener_data.txt): The final destination for the scraped data, representing data persistence/output.
Relationships
• The Python Application initiates the web scraping process.
• It calls Selenium’s API to control Chrome in headless mode via ChromeDriver.
• ChromeDriver launches the headless Chrome browser, which then opens and interacts with screener.in.
• Data flows back from the webpage through Selenium to the Python script.
• The extracted data is processed and finally stored in a text file and also printed to the console.
• There is a bidirectional feedback loop if error handling or debugging is needed (for instance, toggling headless mode).
Architectural Patterns & Key Principles
• Separation of Concerns: Even though the project is a script, think of it as having distinct stages – navigation, data extraction, data processing, and output management.
• External Dependencies: Clearly delineate the components that handle external interaction (i.e., Selenium, ChromeDriver, and the target website) from your core business logic.
• Modularity: Even if the entire functionality resides primarily in app.py, conceptually break down the code into “modules” or functions like get_stock_data, extract_section_data, and extract_table_data.
Relevant Technologies
• Python 3 – the main runtime environment.
• Selenium – for web automation.
• Google Chrome (in headless mode) and ChromeDriver – for browser automation.
• The target URL (screener.in) – the external data source.
• (Optional) OS-level components like file system access for writing screener_data.txt.
• Keep detailed annotations where needed—for example, you might note “Data Extraction” at the interface between Selenium and the webpage or “Error Handling” in the app.py layer.
Happy diagramming!