How charity-kat/my-first-data-pipeline works
This is a small Python crypto-price data pipeline. A user runs the orchestrator, which invokes extraction from CoinGecko, transformation of the raw CSV, and loading of the clean CSV into SQLite. The transform adds an extraction timestamp and calculated USD/ZAR rate; loading also queries the table to verify its contents. The README documents no further capabilities, so this map follows the sampled runtime source.
Main entry points: load.py Database loading · pipeline.py Pipeline runner · extract.py Price extraction · transform.py Price transformation
Components 9
Pipeline runtime
- Pipeline runner
pipeline.py
Data stages
- Price extraction
extract.py - Price transformation
transform.py - Database loading
load.py
Data storage
- Raw prices CSV
data/raw_crypto_prices.csv - Clean prices CSV
data/clean_crypto_prices.csv - Crypto prices DB
data/crypto_pipeline.db
Other components
- User
- CoinGecko API
Connections 12
- User → Pipeline runner: runs
- Pipeline runner → Price extraction: calls first
- Pipeline runner → Price transformation: calls second
- Pipeline runner → Database loading: calls third
- Price extraction → CoinGecko API: requests prices
- Price extraction → Raw prices CSV: writes raw data
- Price transformation → Raw prices CSV: reads raw data
- Price transformation → Clean prices CSV: writes clean data
- Database loading → Clean prices CSV: reads clean data
- Database loading → Crypto prices DB: appends rows
- Database loading → Crypto prices DB: queries table
- Crypto prices DB → Database loading: returns rows