How wasan1alharbi/commercial-awareness-simulator works
1. High-level shape
- This is a two-tier application: a React/Vite frontend in src/ and a Convex backend in convex/.
- Convex is not just the database here; it is also the game state store, simulation engine, realtime sync layer, vector search store, and background job runner.
2. Frontend boundary
- The UI is built with React + TypeScript + Vite, with rendering-heavy parts in PixiJS.
- src/components/Game.tsx, PixiGame.tsx, PixiViewport.tsx, and related components handle the town view, character rendering, movement feedback, messages, and interactive controls.
- The frontend is mostly a thin client: it subscribes to Convex data and sends player inputs back to the backend.
3. Client-to-backend data flow
- User actions flow through hooks like src/hooks/sendInput.ts, useWorldHeartbeat.ts, and serverGame.ts.
- The client reads live state from Convex queries and mutations, then renders the world from that reactive state.
- There is also historical/time-based UI support via useHistoricalTime.ts and useHistoricalValue.ts, which aligns with the backend’s historical state model.
4. Core world model
- The main simulation domain lives under convex/aiTown/.
- This subsystem defines the town’s world, map, locations, movement, players, agents, conversations, memberships, and input handling.
- game.ts, world.ts, movement.ts, player.ts, conversation.ts, and inputHandler.ts are the key modules that determine how the town evolves over time.
5. Engine and temporal state
- convex/engine/ provides the underlying simulation machinery, especially around historical objects and schema support.
- The presence of historicalObject.ts suggests the backend tracks state over time rather than only storing the latest snapshot, which is important for rewindable or time-indexed simulation behavior.
- This is the architectural basis for replayable world state and time-aware queries.
6. Agent cognition and memory
- convex/agent/ contains the memory/conversation layer used by AI characters.
- memory.ts, conversation.ts, and embeddingsCache.ts indicate that character interactions are stored, embedded, and searched semantically.
- This is where the project’s “generative agents” behavior is grounded: the agents do not just chat; they persist memories and retrieve relevant context from vector search.
7. LLM integration boundary
- convex/util/llm.ts is the provider abstraction for chat and embeddings.
- The repo is designed to work with Ollama locally by default, but can also use cloud LLMs such as OpenAI or Together.ai, or any OpenAI-compatible API.
- This is a major architectural boundary because conversation generation and memory embedding depend on the chosen model and embedding dimension.
8. Simulation orchestration
- convex/simulator/ contains extra simulation utilities such as agentWorldContext.ts, gateAgent.ts, and wikipedia.ts.
- This suggests the backend can enrich agent behavior with contextual knowledge and control which agents are allowed to act.
- The simulator layer sits above the world model and below the LLM calls, coordinating autonomous behavior.
9. Initialization and lifecycle
- convex/init.ts seeds the world, characters, and initial map state.
- convex/crons.ts manages background lifecycle tasks, including things like generating music and stopping inactive worlds.
- convex/testing.ts exposes operational controls such as wipe, resume, archive, and kick, which are important for running the simulation reliably during development and deployment.
10. Messaging and media
- convex/messages.ts and convex/music.ts handle the town’s chat and ambient music subsystems.
- Background music generation is an external side effect, likely tied to a webhook/action flow through Convex HTTP endpoints in convex/http.ts.
- This makes the app more than a text simulation: it also has generated audio and chat artifacts tied to world events.
11. Data, assets, and content pipeline
- Static art and map content live in data/ and public/assets/.
- data/characters.ts defines character identities and sprite metadata, while data/convertMap.js converts Tiled exports into the runtime map format.
- The src/editor/ directory is a separate content-authoring toolchain for maps, tiles, and sprites, which feeds the runtime assets rather than the main app loop.
12. Runtime/deployment boundaries
- The repo supports multiple deployment modes: Convex cloud, self-hosted Docker Compose, and Fly.io, with the frontend also deployable to Vercel.
- Dockerfile, docker-compose.yml, and the fly/ directory are operational infrastructure, not core app logic, but they affect how the frontend, backend, and dashboard are split in production.
- The key architectural dependency remains: frontend talks to Convex; Convex talks to LLM/music providers and stores all world state.
13. What to draw in an architecture diagram
- Draw the center as Convex backend with four internal zones: world/simulation, agents/memory, messages/music, and operations/init/crons.
- Show the React/Pixi client reading reactive state and sending inputs to Convex.
- Show external services hanging off the backend: LLM provider/Ollama, embedding/vector search, Replicate for music, and optional Clerk auth.
- Show data/ and src/editor/ as the content pipeline feeding maps, sprites, and character definitions into the runtime.
Main entry points: game.ts Game Core · Game.tsx Game UI · characters.ts Data Assets · world.ts World State
Components 23
Frontend
React/Vite client with Pixi rendering and live Convex subscriptions
- App Shell · Top-level React app bootstrap and page composition
src/App.tsx - Game UI · Main town view and UI orchestration
src/components/Game.tsx - Pixi Renderer · Canvas-based character, map, and viewport rendering
src/components/PixiGame.tsx - Client Hooks · Input sending, heartbeats, and historical time/value access
src/hooks/sendInput.ts - Chat UI · Message input and message list components
src/components/Messages.tsx - Simulator Shell · Client-side controls for simulation/debug state
src/components/SimulatorShell.tsx
Convex Backend
Simulation, agents, messaging, and operational lifecycle
- Backend Entry · Convex function entry and domain wiring
convex/aiTown/main.ts - Game Core · Top-level world update loop and orchestration
convex/aiTown/game.ts - World State · Town state, map, locations, and global simulation data
convex/aiTown/world.ts - Movement · Pathing and movement resolution
convex/aiTown/movement.ts - Input Handler · Validates and applies player inputs
convex/aiTown/inputHandler.ts - Players & Agents · Player and agent records plus descriptions and inputs
convex/aiTown/agent.ts - Conversations · Conversation state and membership tracking
convex/aiTown/conversation.ts - Historical Engine · Temporal object model and schema support for replayable state
convex/engine/historicalObject.ts - Agent Memory · Memories, conversation context, and embedding cache
convex/agent/memory.ts - Simulator Layer · Agent context, gating, and knowledge enrichment
convex/simulator/index.ts - Ops & Lifecycle · Initialization, cron jobs, testing controls, HTTP hooks
convex/init.ts - Messages & Music · Chat artifacts and ambient music generation
convex/messages.ts
Content Pipeline
Static data and editor tools that feed runtime assets
- Data Assets · Characters, sprites, map exports, and animations
data/characters.ts - Map Editor · Browser-based content authoring for maps and sprites
src/editor/index.html
External Services
Model, music, and auth providers used by the backend
- LLM Adapter · Provider abstraction for chat and embeddings
convex/util/llm.ts - Replicate · External music generation service
- Clerk Auth · Optional authentication boundary used by the app
Connections 26
- App Shell → Game UI: renders · App composes the main game experience
- Game UI → Pixi Renderer: draws · UI delegates world rendering to Pixi
- Game UI → Chat UI: shows chat · Chat and player interactions are rendered alongside the world
- Game UI → Simulator Shell: controls · Simulator/debug controls are exposed in the client
- Client Hooks → Backend Entry: sends input · Client submits player actions and heartbeats to Convex
- Client Hooks → World State: subscribes · Client reads live and historical state from Convex queries
- Client Hooks → Historical Engine: reads time · Historical hooks align the UI with time-indexed state
- Backend Entry → Game Core: orchestrates · Backend entry routes work into the simulation core
- Game Core → World State: updates · Simulation mutates the town world state
- Game Core → Movement: moves · Core loop resolves navigation and positional changes
- Game Core → Input Handler: applies · Incoming player input is validated and applied
- Game Core → Players & Agents: advances · Players and AI agents are progressed as actors
- Game Core → Conversations: tracks · Social state is maintained through conversations and memberships
- World State → Historical Engine: uses temporal state · World state is backed by historical objects and replayable schema
- Players & Agents → Agent Memory: remembers · Agents persist memories and retrieve relevant context
- Agent Memory → LLM Adapter: embeds · Memory retrieval depends on model embeddings
- Simulator Layer → Players & Agents: guides · Simulator coordinates agent behavior and context
- Simulator Layer → LLM Adapter: queries model · Autonomous behavior depends on LLM calls
- Ops & Lifecycle → Game Core: boots · Initialization seeds and lifecycle tasks manage simulation state
- Ops & Lifecycle → Messages & Music: schedules · Background jobs drive messages and music lifecycles
- Messages & Music → Replicate: generates audio · Ambient music generation is delegated externally
- Ops & Lifecycle → Clerk Auth: guards · Authentication boundary may be provided externally
- Data Assets → Map Editor: authored by · Editor tools produce and maintain runtime assets
- Data Assets → World State: feeds map data · Runtime world and map state are built from content assets
- Data Assets → Pixi Renderer: feeds sprites · Client rendering consumes sprite sheets and animations
- Map Editor → Data Assets: exports · Authoring pipeline emits maps, tiles, and animation data