How modelearth/profile works
Project type and purpose
• This is a monorepo for a static‐site–driven “Profile Tools” portal (model.earth/profile). It bundles:
• Multiple interactive visualization modules (charts/d3, charts/echarts, charts/sankey)
• A React/Vite “food-nutrition-app” under item/food-nutrition-app
• Data‐prep and ETL scripts (Python, Jupyter notebooks, DuckDB, Supabase) in prep/, impacts/, trade/, salesforce/, crm/, databricks/
• Hosting/deployment scripts targeting Azure static hosting (azure/azure.sh)
File structure
• Top‐level “azure/” contains deployment scripts and an HTML landing page for Azure Static Web Apps.
• “charts/” holds several self-contained D3/Chord, ECharts, and Sankey visualization demos.
• “item/food-nutrition-app” is a standalone React app (src/, public/, vite.config.js).
• “prep/”, “impacts/”, “trade/”, “salesforce/”, “crm/”, “databricks/” each contain data ingestion or transformation code (Python scripts, Jupyter notebooks, SQL loaders, DuckDB and Supabase loaders).
• Data files (CSV, JSON, Parquet) sit alongside their loader code or under impacts/json, impacts/parquet, trade/naics, etc.
• No monolithic backend—data pipelines produce static artifacts or load into lightweight DBs (DuckDB, Supabase).
README
• The only README at root simply brands this as “Profile Tools” and points to the online portal. All architectural details must be inferred from file tree.
a. Main components
1. External Data Sources
– Exiobase CSVs, NAICS spreadsheets, trade YAML, Salesforce directory data
2. ETL / Data Processing Layer
– Python scripts & Jupyter notebooks under prep/, impacts/, trade/, crm/, salesforce/
– Databricks notebooks & DuckDB loaders, Supabase loaders
3. Data Storage
– Local flat files (CSV, JSON, Parquet)
– DuckDB database files (.duckdb)
– Supabase managed PostgreSQL
4. Static Site Generator / Build
– “azure/azure.sh” deployment script
– Static HTML pages under each module (index.html, charts/*/*.html, etc.)
– React/Vite build for “food-nutrition-app”
5. Frontend Presentation Layer
– Static HTML/CSS/JS modules (D3, ECharts, Sankey)
– React app served from item/food-nutrition-app
6. Hosting and CDN
– Azure Static Web Apps
– CDN edge cache
7. External Services
– Supabase (optional runtime data queries)
– Salesforce API for directory updates
– Databricks compute environment (optional)
b. Relationships and data flows
• Hosting serves to end-user browsers. End-user browsers load frontends which may optionally call Supabase or DuckDB web endpoints.
• Salesforce directory update scripts feed into ETL and then potentially update Supabase or flat files.
c. Architectural patterns and design principles
• Micro-frontend style: each chart or demo lives in its own folder with its assets and data.
• “Jamstack” static‐first approach: precompute and serve artifacts; minimize runtime servers.
• Data pipelines are loosely coupled: ETL scripts produce files or DB tables that the site picks up at build time.
d. Key technologies and libraries
• Frontend: HTML/CSS/JS, D3.js, ECharts, Sankey, React + Vite
• Data Processing: Python (pandas, DuckDB), Jupyter, Databricks notebooks, Supabase Python clients
• Storage: flat files (CSV/JSON/Parquet), DuckDB, Supabase Postgres
• Deployment: Azure Static Web Apps, azure CLI (azure.sh)
Detail
• Directory’s role as a mini-component.
• How data formats (CSV, JSON, Parquet) feed into visualizations.
• Do not overcomplicate: treat each folder with its own HTML/JS as a self-contained micro-frontend.