How yeldarby/nycerebro works
Project type and purpose
• This is a full‑stack, search‑driven monitoring app (“NYCerebro”) built with a Next.js frontend, serverless API routes, a Python/Node.js back‑end image‐processing pipeline, and external services (Roboflow, Supabase, Mapbox).
• Its goal is to let users enter a text query (e.g. “busy”), convert it to a CLIP embedding via Roboflow, compare it to live NYC camera embeddings in Supabase, and show the best live feed plus a heatmap.
Main components
a. Frontend (Next.js “v0-site”)
• Pages: home (app/page.tsx), search result (app/search/page.tsx + loading).
• UI components under components/ui and specialized components (HeatMap, LiveWebcam).
• Uses Mapbox JS for heatmap, tailwind for styling.
b. Serverless API (Next.js App Router)
• /api/search: accepts q, calls Roboflow, queries Supabase for top 25 embeddings.
• /api/getResults: returns best camera info and dynamic image URL.
• /api/proxy-image: streams/refetches raw camera frames to avoid CORS.
c. Image‐processing pipeline (resources folder)
• A Python script (example_supabase_update_workflow_block.py) scheduled or run in Roboflow custom block to upsert new camera embeddings.
• A Node.js helper or SQL scripts (convert-to-sql.js, update_clip_sql.py) to prepare or update Supabase schema/data.
d. External services
• Roboflow Workflow: accepts text (with dummy PNG), outputs CLIP embedding.
• Supabase: Postgres headless DB storing camera metadata, CLIP vectors, timestamps.
• Mapbox: client‐side heatmap visualization.
• NYC public camera streams: roughly 900 HLS/HTTP endpoints.
Relationships & data flows
• User → Frontend UI → Serverless /api/search?q=
→ (1) POST text to Roboflow Workflow → receives CLIP vector
→ (2) Query Supabase for nearest cameras by vector similarity
→ Returns metadata + camera IDs to frontend
→ Frontend renders LiveWebcam component by polling /api/proxy-image?cameraId= every 2 s.
→ Frontend renders Mapbox heatmap overlay using the geo‐coordinates from Supabase results.
• Separately: Python/Node.js pipeline (cron or Roboflow custom block)
→ Fetch latest frames from all NYC camera URLs
→ Send each to Roboflow for embedding (via InferenceHTTPClient)
→ Upsert camera ID, embedding vector, timestamp into Supabase (via Supabase client).
Architectural patterns & design principles
• Serverless microservices: Next.js API routes handle search logic.
• Event‐driven ingestion: scheduled or dataflow pipeline feeds Supabase.
• Separation of concerns: UI, API logic, and embedding‐generation pipeline are distinct.
• Vector search in Postgres (Supabase) using embeddings for nearest‐neighbor queries.
• CDN/edge deploy (Vercel) for Next.js, serverless functions, and static assets.
What
• Clear textual labels on each component
• Notes on timing: e.g., “frontend polls every 2 s”