How lefterisloukas/edgar-crawler works
EDGAR-CRAWLER supports two user workflows: download filtered SEC filings and parse downloaded 10-K, 10-Q, and 8-K filings into structured JSON. The download script orchestrates index retrieval, filtering, filing metadata enrichment, and raw downloads; the extraction script reads configured inputs and extracts report sections using filing-specific item lists. The excerpts establish these entry points and core responsibilities, but omit some implementation details, so uncertain internal call links are left out.
Main entry points: extract_items.py Extraction workflow · download_filings.py Download workflow · datasets Raw filings · config.json Workflow configuration
Components 13
Filing download
- Download workflow
download_filings.py - Index retrieval and filtering
download_filings.py - Filing crawl and enrichment
download_filings.py - Raw filings
datasets - Filing metadata
datasets
Section extraction
- Extraction workflow
extract_items.py - Section extraction
extract_items.py - Filing item lists
item_lists.py - Structured JSON
datasets
Shared runtime
- Workflow configuration
config.json - Logging
logger.py
Other components
- Researcher
- SEC EDGAR
Connections 17
- Researcher → Download workflow: runs download
- Researcher → Extraction workflow: runs extraction
- Download workflow → Workflow configuration: reads settings
- Download workflow → Index retrieval and filtering: orchestrates
- Index retrieval and filtering → SEC EDGAR: requests indices
- Index retrieval and filtering → Filing crawl and enrichment: selects filings
- Filing crawl and enrichment → SEC EDGAR: requests filings
- Filing crawl and enrichment → Raw filings: writes filings
- Filing crawl and enrichment → Filing metadata: writes metadata
- Extraction workflow → Workflow configuration: reads settings
- Extraction workflow → Filing metadata: reads metadata
- Extraction workflow → Raw filings: reads filings
- Extraction workflow → Section extraction: invokes parser
- Section extraction → Filing item lists: uses item lists
- Section extraction → Structured JSON: writes JSON
- Download workflow → Logging: logs activity
- Extraction workflow → Logging: logs activity