How bibijoulbi/prix-carburant works
Prix Carburants is a batch pipeline that downloads French public fuel-price data, extracts and parses its XML, cleans the records into a Pandas DataFrame, and stores station and price records in PostgreSQL. main.py coordinates the stages and commits or rolls back the database transaction. The download module also unpacks the ZIP; the parser and transformer are separate stages. Relationships are based on the README and the shown source excerpts; the SQL schema is present in the tree but was not sampled, so its details are not inferred.
Main entry points: main.py Pipeline runner · download.py Download and unpack · extraction.py XML parser · transformation.py Clean and transform
Components 9
Data ingestion
- Download and unpack
src/download.py
Data processing
- XML parser
src/extraction.py - Clean and transform
src/transformation.py
Database storage
- Database connection
src/database.py - Station insertion
src/database.py - Price insertion
src/database.py
Other components
- Public fuel data
- Pipeline runner
main.py - PostgreSQL
Connections 14
- Public fuel data → Download and unpack: provides data
src/download.py - Pipeline runner → Download and unpack: downloads data
main.py - Download and unpack → Pipeline runner: returns XML file
src/download.py - Pipeline runner → XML parser: parses file
main.py - XML parser → Pipeline runner: returns XML root
src/extraction.py - Pipeline runner → Clean and transform: transforms root
main.py - Clean and transform → Pipeline runner: returns DataFrame
src/transformation.py - Pipeline runner → Database connection: opens connection
main.py - Database connection → PostgreSQL: connects to
src/database.py - Pipeline runner → Station insertion: inserts stations
main.py - Pipeline runner → Price insertion: inserts prices
main.py - Station insertion → PostgreSQL: writes stations
src/database.py - Price insertion → PostgreSQL: writes prices
src/database.py - Pipeline runner → PostgreSQL: commits transaction
main.py