# jhontejada95/cafe-index architecture

> Architecture diagram of the GitHub repository jhontejada95/cafe-index, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jhontejada95/cafe-index
- Repository: https://github.com/jhontejada95/cafe-index

## Overview

##### Project Type and Purpose
- CafuéIndex AI is a full-stack, data-driven application that:
- Indexes on-chain coffee prices (via SubQuery GraphQL)
- Cleans, processes and models the data with Python/​pandas/​scikit-learn
- Serves price predictions and AI explanations over a FastAPI endpoint
- Publishes price updates as a Polkadot/Substrate on-chain oracle
- Presents an interactive React frontend (with Polkadot.js wallet integration)

##### File-Structure Analysis
- Top-level directories/modules:
- data_indexing/ → SubQuery data fetch & storage (CSV/SQLite)
- data_processing/ → Preprocessor + Model Trainer → model.pkl
- prediction_service/ → FastAPI app.py (POST /predict)
- price_oracle/ → oracle.py + run_oracle.py (scheduled on-chain submissions)
- frontend/ → React + Vite + Tailwind + Polkadot.js wallet
- Supporting files: config.py, utils.py, main.py orchestrates index→process→train→oracle.
- Deployment: Dockerfile + docker-compose.yml orchestrate backend + frontend containers.

##### README Insights
- Confirms architecture: ETL → ML → API → UI → Blockchain oracle
- Highlights external dependencies: SubQuery GraphQL, DeepSeek AI, Polkadot Westend, Polkadot.js

##### Components
a. Data Sources & Storage
- External: SubQuery GraphQL endpoint
- Local: raw_coffee_prices.csv, processed_coffee_prices.csv, model.pkl, pipeline_status.json
b. ETL Pipeline (Python)
- Module: data_indexing/indexer.py
- Module: data_processing/preprocessor.py → cleans & feature-engineers
- Module: data_processing/model_trainer.py → trains & evaluates models
c. Model Artifacts
- model.pkl stored on disk
d. Prediction Service
- prediction_service/app.py → FastAPI server
- Serves /predict and health-check endpoints
- Optional DeepSeek integration for natural-language explanations
e. Frontend
- React (TypeScript) components & services
- PolkadotConnector.tsx → wallet connect, submitPrice via src/services/polkadot.ts
- AIQuerySection, PriceChart for visualization & queries
f. On-Chain Oracle
- price_oracle/run_oracle.py scheduled (cron or Docker)
- oracle.py constructs & signs transactions to Westend
g. Deployment
- Docker Compose network: backend container (FastAPI + ETL), frontend container (React/Vite)
- Shared environment / network settings via .env files & config.py

##### Relationships & Data Flows
- SubQuery → data_indexing → raw CSV/DB
- raw CSV → preprocessor → processed CSV
- processed CSV → model_trainer → model.pkl
- model.pkl → prediction_service (loaded on startup)
- Frontend → HTTP POST /predict → prediction_service → returns predictions + explanations
- main.py orchestrates ETL+train, can also invoke price_oracle.run
- run_oracle → Westend RPC → on-chain coffee price submissions
- Frontend wallet integration → user-driven on-chain submissions via Polkadot.js

##### Architectural Patterns & Principles
- Separation of Concerns: clear ETL vs ML vs API vs UI vs Blockchain layers
- Modularization: Python modules per function (indexing, processing, training, oracle)
- Microservice-style deployment: independently containerized frontend & backend
- Config-driven: central config.py, .env for environment overrides
- Reproducibility: raw vs processed data versioned on disk
- Extensibility: pluggable ML models, optional DeepSeek, swap SubQuery endpoint

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Services
    subgraph "External Services"
        SUBQ["SubQuery GraphQL"]:::external
        DEEP["DeepSeek AI"]:::external
        WESTEND["Polkadot Westend RPC"]:::external
    end

    %% Data Ingestion & Processing
    subgraph "Data Ingestion & Processing"
        idx["Indexer"]:::ingest
        pre["Preprocessor"]:::ingest
        raw["raw_coffee_prices.csv"]:::data
        proc_csv["processed_coffee_prices.csv"]:::data
    end

    SUBQ --> idx
    idx --> raw
    raw --> pre
    pre --> proc_csv

    %% ML Training
    subgraph "ML Training"
        trainer["Model Trainer"]:::ml
        model["model.pkl"]:::data
    end

    proc_csv --> trainer
    trainer --> model

    %% Prediction Service
    subgraph "Prediction Service"
        api["FastAPI App (/predict)"]:::api
    end

    model --> api

    %% Frontend
    subgraph "Frontend (React)"
        app["App.tsx"]:::frontend
        mainf["main.tsx"]:::frontend
        conn["PolkadotConnector.tsx"]:::frontend
        aiq["AIQuerySection.tsx"]:::frontend
        chart["PriceChart.tsx"]:::frontend
        pdot["polkadot.ts"]:::frontend
        api_svc["api.ts"]:::frontend
    end

    api -->|HTTP| api_svc
    api_svc --> app
    app --> mainf
    app --> conn
    app --> aiq
    app --> chart
    app --> pdot

    %% On-Chain Oracle
    subgraph "On-Chain Oracle"
        runner["run_oracle.py"]:::oracle
        oracle["oracle.py"]:::oracle
    end

    runner --> oracle
    oracle --> WESTEND

    conn -->|Wallet Tx| oracle

    %% Orchestration & Config
    subgraph "Orchestration & Configuration"
        main["main.py"]:::orchestration
        cfg["config.py"]:::orchestration
        util["utils.py"]:::orchestration
    end

    main --> idx
    main --> pre
    main --> trainer
    main --> runner
    main --> cfg
    main --> util

    %% Deployment
    subgraph "Deployment"
        dfb["Dockerfile"]:::deploy
        dcf["docker-compose.yml"]:::deploy
        fdb["frontend/Dockerfile"]:::deploy
        env[".env.example"]:::deploy
        fenv["frontend/.env.example"]:::deploy
    end

    dcf --> dfb
    dcf --> fdb
    dfb --> cfg
    fdb --> mainf
    dcf --> env
    dcf --> fenv

    %% Click Events
    click idx "https://github.com/jhontejada95/cafe-index/blob/master/data_indexing/indexer.py"
    click pre "https://github.com/jhontejada95/cafe-index/blob/master/data_processing/preprocessor.py"
    click raw "https://github.com/jhontejada95/cafe-index/blob/master/data/raw_coffee_prices.csv"
    click proc_csv "https://github.com/jhontejada95/cafe-index/blob/master/data/processed_coffee_prices.csv"
    click trainer "https://github.com/jhontejada95/cafe-index/blob/master/data_processing/model_trainer.py"
    click model "https://github.com/jhontejada95/cafe-index/blob/master/data/model.pkl"
    click api "https://github.com/jhontejada95/cafe-index/blob/master/prediction_service/app.py"
    click app "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/App.tsx"
    click mainf "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/main.tsx"
    click conn "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/components/PolkadotConnector.tsx"
    click aiq "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/components/AIQuerySection.tsx"
    click chart "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/components/PriceChart.tsx"
    click pdot "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/services/polkadot.ts"
    click api_svc "https://github.com/jhontejada95/cafe-index/blob/master/frontend/src/services/api.ts"
    click runner "https://github.com/jhontejada95/cafe-index/blob/master/price_oracle/run_oracle.py"
    click oracle "https://github.com/jhontejada95/cafe-index/blob/master/price_oracle/oracle.py"
    click main "https://github.com/jhontejada95/cafe-index/blob/master/main.py"
    click cfg "https://github.com/jhontejada95/cafe-index/blob/master/config.py"
    click util "https://github.com/jhontejada95/cafe-index/blob/master/utils.py"
    click dfb "https://github.com/jhontejada95/cafe-index/tree/master/Dockerfile"
    click dcf "https://github.com/jhontejada95/cafe-index/blob/master/docker-compose.yml"
    click fdb "https://github.com/jhontejada95/cafe-index/tree/master/frontend/Dockerfile"
    click env "https://github.com/jhontejada95/cafe-index/blob/master/.env.example"
    click fenv "https://github.com/jhontejada95/cafe-index/blob/master/frontend/.env.example"

    %% Styles
    classDef ingest fill:#D0E8FF,stroke:#004C99
    classDef ml fill:#DFF5D8,stroke:#228B22
    classDef api fill:#FFE5B4,stroke:#FF8C00
    classDef frontend fill:#E8D0FF,stroke:#800080
    classDef oracle fill:#FFE5B4,stroke:#FF8C00
    classDef orchestration fill:#F0F0F0,stroke:#666666
    classDef deploy fill:#F0F0F0,stroke:#666666
    classDef data fill:#CCCCCC,stroke:#333333,shape:cylinder
    classDef external stroke-dasharray: 5 5,fill:#F0F0F0,stroke:#666666
```
