How jhontejada95/cafe-index works
Project Type and Purpose
• CafuéIndex AI is a full-stack, data-driven application that:
• Indexes on-chain coffee prices (via SubQuery GraphQL)
• Cleans, processes and models the data with Python/pandas/scikit-learn
• Serves price predictions and AI explanations over a FastAPI endpoint
• Publishes price updates as a Polkadot/Substrate on-chain oracle
• Presents an interactive React frontend (with Polkadot.js wallet integration)
File-Structure Analysis
• Top-level directories/modules:
• data_indexing/ → SubQuery data fetch & storage (CSV/SQLite)
• data_processing/ → Preprocessor + Model Trainer → model.pkl
• prediction_service/ → FastAPI app.py (POST /predict)
• price_oracle/ → oracle.py + run_oracle.py (scheduled on-chain submissions)
• frontend/ → React + Vite + Tailwind + Polkadot.js wallet
• Supporting files: config.py, utils.py, main.py orchestrates index→process→train→oracle.
• Deployment: Dockerfile + docker-compose.yml orchestrate backend + frontend containers.
README Insights
• Confirms architecture: ETL → ML → API → UI → Blockchain oracle
• Highlights external dependencies: SubQuery GraphQL, DeepSeek AI, Polkadot Westend, Polkadot.js
Components
a. Data Sources & Storage
• External: SubQuery GraphQL endpoint
• Local: raw_coffee_prices.csv, processed_coffee_prices.csv, model.pkl, pipeline_status.json
b. ETL Pipeline (Python)
• Module: data_indexing/indexer.py
• Module: data_processing/preprocessor.py → cleans & feature-engineers
• Module: data_processing/model_trainer.py → trains & evaluates models
c. Model Artifacts
• model.pkl stored on disk
d. Prediction Service
• prediction_service/app.py → FastAPI server
• Serves /predict and health-check endpoints
• Optional DeepSeek integration for natural-language explanations
e. Frontend
• React (TypeScript) components & services
• PolkadotConnector.tsx → wallet connect, submitPrice via src/services/polkadot.ts
• AIQuerySection, PriceChart for visualization & queries
f. On-Chain Oracle
• price_oracle/run_oracle.py scheduled (cron or Docker)
• oracle.py constructs & signs transactions to Westend
g. Deployment
• Docker Compose network: backend container (FastAPI + ETL), frontend container (React/Vite)
• Shared environment / network settings via .env files & config.py
Relationships & Data Flows
• SubQuery → data_indexing → raw CSV/DB
• raw CSV → preprocessor → processed CSV
• processed CSV → model_trainer → model.pkl
• model.pkl → prediction_service (loaded on startup)
• Frontend → HTTP POST /predict → prediction_service → returns predictions + explanations
• main.py orchestrates ETL+train, can also invoke price_oracle.run
• run_oracle → Westend RPC → on-chain coffee price submissions
• Frontend wallet integration → user-driven on-chain submissions via Polkadot.js
Architectural Patterns & Principles
• Separation of Concerns: clear ETL vs ML vs API vs UI vs Blockchain layers
• Modularization: Python modules per function (indexing, processing, training, oracle)
• Microservice-style deployment: independently containerized frontend & backend
• Config-driven: central config.py, .env for environment overrides
• Reproducibility: raw vs processed data versioned on disk
• Extensibility: pluggable ML models, optional DeepSeek, swap SubQuery endpoint