# dcrebbin/meta-vision-api architecture

> Architecture diagram of the GitHub repository dcrebbin/meta-vision-api, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dcrebbin/meta-vision-api
- Repository: https://github.com/dcrebbin/meta-vision-api

## Overview

##### Project type and purpose
- This is a lightweight full-stack “hacky” integration:
- A browser-side bookmarklet (client) that watches Facebook Messenger for new photo messages and forwards their URLs
- A small REST API server (backend) built in TypeScript (running on Bun) that relays image URLs to the OpenAI GPT-4 Vision API, stores the results, and exposes them as static JSON

##### File structure
- Top-level files:
- `package.json`, `bun.lockb`, `tsconfig.json`, `.env.example` → Node/Bun project config
- `src/index.ts` → server entry point (defines REST endpoint `/api/gpt-4-vision`)
- `src/bookmarklet.js` → client-side JS to be injected into messenger.com via a bookmark
- `public/` → static serving directory (contains `data.json` after the server writes results)
- `.gitignore`, `README.md` → meta/config

##### README for insights
- Explains prerequisites (Meta Ray-Ban Smart Glasses + Messenger account + OpenAI key)
- Describes workflow: voice commands → Messenger chat on alternative account → bookmarklet observes new images → POST to local server → GPT-4 Vision call → write to `public/data.json` → read and display on glasses

##### Components and interactions
a. Main components
1. User & Meta Ray-Ban Smart Glasses (voice input / display)
2. Meta View App (on glasses)
3. Browser (messenger.com) with injected Bookmarklet code
4. Bookmarklet Observer (in-page JS)
5. Local REST API Server (Bun + TypeScript)
6. OpenAI GPT-4 Vision API (external)
7. Static Data Store (`public/data.json`)
b. Relationships & data flow
- User “says” voice command → Meta View App determines recipient
- Meta View App sends Messenger photo message to alt account
- In browser (messenger.com), BookmarkletObserver listens for new `<img>` nodes
- On new image: Observer ⟶ HTTP POST `/api/gpt-4-vision` (JSON with `imageUrl`)
- Server receives POST ⟶ forwards image URL to OpenAI GPT-4 Vision API
- On success: server writes the JSON result to `public/data.json`
- Glasses / Meta View App periodically fetches or is notified to fetch `data.json` to render the analysis on the display
c. Architectural patterns & principles
- Client-side injection (bookmarklet) as a lightweight “micro-frontend”
- Simple REST API façade pattern: server proxies to external AI service
- Static file serving for results → no database required
d. Key technologies
- Bookmarklet (vanilla JS + DOMNodeInserted event)
- Bun runtime, TypeScript, Express-style routing (in `index.ts`)
- OpenAI GPT-4 Vision API
- Static file hosting (via Bun’s static middleware or equivalent)

##### Elements
- Voice command → Meta View App → Messenger → Browser → Bookmarklet → Server → OpenAI → Server → JSON store → Glasses
- External services (OpenAI GPT-4 Vision, Meta services) → grayed out

##### Detailed capture of architectural elements
- User & Glasses vs. Meta View App vs. Messenger web UI vs. Bookmarklet observer
- REST API vs. OpenAI API vs. file store
- Use notes or annotations for limitations (DOMNodeInserted hack, CORS mode=no-cors, environment variables)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client-side components
    User["User"]:::external
    Glasses["Meta Ray-Ban Smart Glasses\n(voice I/O)"]:::client
    MetaApp["Meta View App\n(on Glasses)"]:::external
    Messenger["Messenger Web UI\n(messenger.com)"]:::client
    Bookmarklet["Bookmarklet Observer\n(DOMNodeInserted)"]:::client

    %% Server-side components
    Server["REST API Server\n(Bun + TypeScript)"]:::backend

    %% External services and data store
    OpenAI["OpenAI GPT-4 Vision API"]:::external
    DataStore["Static Data JSON\n(public/data.json)"]:::datastore

    %% File level config
    subgraph "Project Config & Docs"
        Pkg["package.json"]:::config
        Lock["bun.lockb"]:::config
        TSConfig["tsconfig.json"]:::config
        Env[".env.example"]:::config
        Readme["README.md"]:::config
        GitIgnore[".gitignore"]:::config
    end

    %% Data flow
    User -->|"voice command"| Glasses
    Glasses -->|"process command"| MetaApp
    MetaApp -->|"sends photo via Messenger"| Messenger
    Messenger -->|"new <img> detected"| Bookmarklet
    Bookmarklet -->|POST /api/gpt-4-vision| Server
    Server -->|"forward image URL"| OpenAI
    OpenAI -->|"analysis result"| Server
    Server -->|"writes JSON"| DataStore
    Glasses -->|"fetch data.json"| DataStore

    %% Click events
    click Bookmarklet "https://github.com/dcrebbin/meta-vision-api/blob/main/src/bookmarklet.js"
    click Server "https://github.com/dcrebbin/meta-vision-api/blob/main/src/index.ts"
    click DataStore "https://github.com/dcrebbin/meta-vision-api/tree/main/public/"
    click Pkg "https://github.com/dcrebbin/meta-vision-api/blob/main/package.json"
    click Lock "https://github.com/dcrebbin/meta-vision-api/blob/main/bun.lockb"
    click TSConfig "https://github.com/dcrebbin/meta-vision-api/blob/main/tsconfig.json"
    click Env "https://github.com/dcrebbin/meta-vision-api/blob/main/.env.example"
    click Readme "https://github.com/dcrebbin/meta-vision-api/blob/main/README.md"
    click GitIgnore "https://github.com/dcrebbin/meta-vision-api/blob/main/.gitignore"

    %% Styling
    classDef client fill:#D6EFFF,stroke:#0366d6,color:#0366d6
    classDef backend fill:#E4FFD6,stroke:#2f7f00,color:#2f7f00
    classDef external fill:#F0F0F0,stroke:#888888,color:#888888
    classDef datastore fill:#FFF0D6,stroke:#d38700,color:#d38700
    classDef config fill:#F5E6FF,stroke:#7a3ec7,color:#7a3ec7
```
