How muliroz/cinedata works
CineData is a Databricks data-engineering pipeline for preparing movie-industry data for analysis and GenAI use. Its principal workflow is CSV and PTAX ingestion into Bronze, cleaning and enrichment into Silver, then dimensional modeling into Gold. Gold includes movie dimensions, many-to-many bridges, a performance fact, and a Portuguese context document for LLM/RAG. The README documents these stages and outputs; the sampled source does not include notebook code, so the diagram reflects documented behavior rather than inspected implementation details.
Main entry points: Silver_to_Gold.ipynb Gold modeling · Landing_to_Bronze.ipynb Landing ingestion · Bronze_to_Silver.ipynb Silver curation · input_files Movie CSVs
Components 11
Landing to Bronze
- Landing ingestion
notebooks/Landing_to_Bronze.ipynb - Bronze Delta tables
notebooks/Landing_to_Bronze.ipynb
Bronze to Silver
- Silver curation
notebooks/Bronze_to_Silver.ipynb - Silver Delta tables
notebooks/Bronze_to_Silver.ipynb
Silver to Gold
- Gold modeling
notebooks/Silver_to_Gold.ipynb
Gold outputs
- Movie dimensions
notebooks/Silver_to_Gold.ipynb - Relationship bridges
notebooks/Silver_to_Gold.ipynb - Performance fact
notebooks/Silver_to_Gold.ipynb - GenAI context
notebooks/Silver_to_Gold.ipynb
Other components
- Movie CSVs
input_files - PTAX API
Connections 13
- Movie CSVs → Landing ingestion: reads CSVs
- PTAX API → Landing ingestion: fetches rates
- Landing ingestion → Bronze Delta tables: appends data
- Bronze Delta tables → Silver curation: reads Bronze
- Silver curation → Silver Delta tables: writes cleaned data
- Silver Delta tables → Gold modeling: reads Silver
- Gold modeling → Movie dimensions: writes dimensions
- Gold modeling → Relationship bridges: writes bridges
- Gold modeling → Performance fact: writes fact
- Movie dimensions → GenAI context: provides movie data
- Performance fact → GenAI context: provides metrics
- Relationship bridges → GenAI context: provides people links
- Gold modeling → GenAI context: generates document