How iamademar/glamli works
1. Repository shape
GLAMLI is a monorepo for a guided machine-learning research workflow. The production runtime is split into a React frontend, a Python API/agent backend, a background model-running worker, and two imported WEKA services. The top-level plans/ folder describes the intended build slices and architecture sequence.
2. User-facing application
apps/web is the production frontend: React + TypeScript + Vite. It is organized by workflow stage and feature slice rather than by generic UI layers. The main user journey covers upload, domain understanding, setup, run, explore, and prediction, with assistant chat embedded throughout. State is split across feature hooks and stores for workflow, session, assistant, and UI state.
3. Backend application boundary
apps/api is a FastAPI service that owns the application domain and persistence boundary. It exposes REST endpoints for auth, projects, datasets, workflow, events, experiment plans, model runs, predictions, study sessions, and assistant-related actions. It also contains the business services that transform uploaded data, manage study state, and coordinate model/LLM interactions.
4. Agent and LLM subsystem
Inside apps/api/app/agents, the backend has a dedicated LLM/agent layer with prompts, role definitions, tone/satisfaction logic, model registry, telemetry, and MCP tool wiring. This is the “single assistant” described in the README: it uses prompt templates per task, calls tools, and records telemetry about assistant behavior and user interactions.
5. Workflow orchestration
The backend has explicit workflow and graph modules (app/graph, app/workflow) plus a workflow state machine. This suggests the product is not just CRUD; it is a staged, stateful experience where the allowed actions and assistant behavior depend on the participant’s current step. The frontend mirrors this with workflow stores and stage-transition hooks.
6. Data and experiment pipeline
The application domain centers on datasets, domain specifications, column descriptions/groups, EDA checklists, experiment plans, model runs, and predictions. The API services perform normalization, filtering, feature explanation, experiment planning, confidence probing, and result explanation. These pieces form the path from uploaded data to a constrained WEKA experiment and then to prediction/testing.
7. Background execution
workers/model-runner is the asynchronous execution boundary. Its job is to run heavier model-experiment work outside the request/response cycle, then feed results back to the API’s persistent state. The presence of recovery services and model-run telemetry indicates the worker must support retries/resume and failure handling, not just one-shot batch jobs.
8. WEKA integration layer
services/weka-api is an imported Java/Javalin REST service backed by WEKA 3.9.6. It owns the low-level ML operations: training, evaluation, prediction, transforms, EDA, filtering, clustering, associations, and diagnostics. services/weka-mcp is a TypeScript MCP server that wraps that WEKA API and exposes a tool surface for the agent layer. In practice, the Python backend mediates user-facing workflow while the WEKA stack performs the actual ML computation.
9. Persistence and migrations
apps/api/app/db plus Alembic migrations show a relational database-backed backend. The migrations define the core study schema: users/auth sessions, projects, datasets and dataset versions, domain specs, experiment plans, model runs, predictions, telemetry, and related recovery/agent tables. The DB is the system of record for the guided study state.
10. Shared contracts and telemetry
packages/contracts contains shared API/agent contracts and generated types, bridging Python and TypeScript. packages/study-events defines the canonical research telemetry schema in both TS and Python, so frontend, backend, and analysis tooling can emit the same event envelope. This is important because study logging is a first-class product requirement, not an afterthought.
11. Fixtures, reference UI, and provenance
packages/test-fixtures provides fixed datasets and model artifacts for deterministic testing. apps/web-prototype is a non-production visual reference for the earlier “glamli_v3” UI and should not be treated as runtime code. docs/weka-service-provenance.md documents the imported WEKA service lineage, which matters for understanding the external boundary and maintenance ownership.
12. Operational model
The repo is built to run locally via Docker Compose first, with Azure deployment deferred. CI is present in .github/workflows, while compose.yaml is the main integration point for standing up the database, WEKA services, backend, worker, and eventually the frontend. The architecture is therefore service-oriented but intentionally development-local and study-centric.
Main entry points: compose.yaml Compose stack · services Domain services · api REST routes · weka-api WEKA API
Components 21
Frontend
React workflow UI and client-side state
- Web app · Production frontend for the guided research workflow
apps/web - Workflow stages · Upload, domain, setup, run, explore, and predict screens
apps/web/src/features - Assistant UI · Embedded chat, checklist prompts, and stage-aware replies
apps/web/src/features/assistant - Client state · Workflow, session, assistant, and UI state management
apps/web/src/store - API client · Client wrappers and query keys for backend calls
apps/web/src/api
Backend
FastAPI domain service and orchestration
- API service · Owns application domain, persistence, and public REST surface
apps/api - REST routes · Auth, projects, datasets, workflow, runs, predictions, and telemetry endpoints
apps/api/app/api - Domain services · Data normalization, study state, planning, explanations, and run coordination
apps/api/app/services - Workflow engine · Stage definitions and routing for allowed participant actions
apps/api/app/workflow - Graph logic · Workflow routing and state representation for the staged experience
apps/api/app/graph - Agent layer · Prompting, tool wiring, telemetry, roles, tone, and satisfaction logic
apps/api/app/agents - Database · System of record for studies, users, datasets, runs, predictions, and telemetry
apps/api/app/db - Migrations · Alembic revisions defining the core study schema
apps/api/app/alembic/versions - Model worker · Background execution boundary for heavier model-run work and recovery
apps/api/app/worker.py
ML Services
Imported WEKA runtime and MCP bridge
Shared
Contracts, telemetry, fixtures, and repo-level ops
- Contracts · Cross-runtime API and agent contracts for Python and TypeScript
packages/contracts - Study events · Canonical event envelope shared by frontend, backend, and analysis tooling
packages/study-events - Fixtures · Deterministic datasets and model artifacts for repeatable tests
packages/test-fixtures - Compose stack · Main local integration entrypoint for database and services
compose.yaml - Plans · Top-level build slices and intended implementation sequence
plans
Connections 31
- Web app → Workflow stages: renders · The app is organized around workflow-stage screens and feature slices
- Web app → Assistant UI: contains · Assistant chat is embedded throughout the user journey
- Web app → Client state: manages · Client state is split across dedicated stores and hooks
- Web app → API client: calls · Frontend data access goes through typed API wrappers
- Workflow stages → REST routes: reads/writes · Stage UI depends on backend workflow and domain endpoints
- Assistant UI → Agent layer: streams · Chat UI is backed by the assistant and prompt subsystem
- Client state → Study events: emits · Client telemetry uses the shared study-event envelope
- API client → REST routes: requests · Client API module is the frontend boundary to the backend
- API service → REST routes: exposes · FastAPI mounts the public REST surface
- REST routes → Domain services: delegates · Routes hand off to domain services for execution
- Domain services → Database: persists · Domain state and study artifacts are stored in the relational database
- Domain services → Model worker: enqueues · Heavy or recoverable work is pushed to the background worker
- Domain services → WEKA API: invokes · Core ML computation is performed by the WEKA service
- Model worker → WEKA API: runs · Worker executes model-run jobs through the WEKA boundary
- Model worker → Database: updates · Worker feeds results and recovery state back into persistence
- Agent layer → WEKA MCP: tools · Agent tool usage is mediated through the MCP wrapper
- WEKA MCP → WEKA API: proxies · MCP server forwards tool calls to the Java WEKA REST service
- Agent layer → Database: records · Assistant telemetry and outputs are stored for the study
- Workflow engine → Graph logic: defines · Workflow stages and graph routing describe allowed transitions
- REST routes → Workflow engine: enforces · Workflow endpoints respect the staged participant state
- Migrations → Database: migrates · Alembic revisions define the database schema and evolution
- Contracts → API service: types · Shared contracts bridge Python backend models and APIs
- Contracts → API client: types · Shared contracts also bridge frontend request/response types
- Study events → Domain services: schemas · Backend emits canonical study telemetry through shared schemas
- Study events → Client state: schemas · Frontend telemetry follows the same event definitions
- Fixtures → WEKA API: tests · Reference datasets and models support deterministic ML tests
- Compose stack → API service: orchestrates · Compose is the local runtime entrypoint for backend services
- Compose stack → Model worker: orchestrates · Compose also brings up the background worker
- Compose stack → WEKA API: orchestrates · Compose includes the imported WEKA API service
- Compose stack → WEKA MCP: orchestrates · Compose includes the MCP bridge service
- Plans → Compose stack: guides · Implementation plans describe the intended local stack and build slices