How chai-student/crk-model works
1. What this repo is
- A legacy/reference FastAPI model service for an AI smart vending machine stack.
- It is designed to run on a Jetson Orin Nano 4GB with TensorRT .engine inference.
- The repo is the direct Python runtime path; the README notes a newer Go-based service exists elsewhere for fresh deployments.
2. Process boundary and startup
- The process starts at services/model/model_service/main.py.
- api/manager.py builds the FastAPI app and handles lifespan/bootstrap concerns.
- core/config.py loads .env-based runtime settings into app.state.settings, which is now the central configuration boundary.
3. HTTP/API surface
- api/routes/health.py exposes health endpoints for liveness/readiness and runtime diagnostics.
- api/routes/trigger.py is the main ingest path for camera-triggered work.
- api/routes/multi_zone.py exposes judgment/summary endpoints for multi-zone evaluation.
- The route layer is thin; it delegates almost all work to services.
4. Primary data flow: trigger to judgment
- The camera system sends an AVI path or trigger payload into /trigger.
- The service does not infer directly on the AVI blob; it splits AVI into frames, runs inference per frame, and aggregates results.
- Trigger handling is orchestrated by service/trigger_service.py, which coordinates video processing, vision inference, and session updates.
5. Video ingestion and frame-level inference
- video/video_processor.py, frame_extractor.py, and frame_trace.py form the video processing boundary.
- The model service works at image-frame granularity after frame extraction.
- camera_roles.py and freezer_candidate_policy.py suggest the service distinguishes camera roles and selects candidate frames intelligently.
- voting_ensemble.py indicates per-frame predictions are combined before producing a final decision.
6. Vision/inference subsystem
- vision/yolo_wrapper.py is the model adapter around YOLO/TensorRT execution.
- Supporting modules such as top5_extractor.py, hand_filter.py, and hand_path_tracker.py refine raw detections into usable product signals.
- The model path is configured via settings and defaults to models/0204_morning.engine per the README.
- This is the main dependency on Jetson CUDA/TensorRT runtime availability.
7. Decision and weight logic
- engine/decision_engine.py and engine/models.py represent the higher-level business decision layer.
- weight/count_calculator.py and weight/strict_weight_matcher.py handle loadcell-to-product matching logic.
- The README notes the system sums two physical loadcell channels per vending zone, so weight math is zone-total based, not averaged.
- This layer turns detection outputs plus weight signals into final judgments.
8. Session and persistence boundaries
- session/door_session.py, door_session_store.py, global_door_session.py, and session_store.py manage active and completed evaluation sessions.
- session/product_aggregator.py, active_product_store.py, and freezer_close_aggregate.py hold interim product evidence during a session.
- yaml_persistence.py writes completed session artifacts under services/model/model_service/data/sessions/....
- This is the repo’s durable state boundary; runtime evaluation is session-centric rather than stateless.
9. Operational tracing and observability
- video/frame_trace.py records frame-split and inference tracing.
- The README notes persistent trace files under services/model/logs/frame_split_*.jsonl, plus optional sample frame export.
- Logging setup and terminal handling live in core/logging_config.py and core/terminal.py.
- This repo emphasizes post-run traceability because Jetson runtime behavior is only fully authoritative on-device.
10. Runtime, environment, and platform coupling
- core/runtime_env.py and core/model_validation.py guard Jetson-specific runtime assumptions.
- The service depends materially on FastAPI, Uvicorn, NumPy, Ultralytics, Torch/CUDA, and TensorRT.
- scripts/ contains Jetson setup, model conversion, and preview utilities, reinforcing that deployment and validation are tightly coupled to the device environment.
- The repo is explicitly sensitive to Jetson package selection and CUDA/TensorRT availability.
11. External systems and hardware interfaces
- Primary external inputs are the camera feed/AVI files and loadcell signals coming from the vending hardware stack.
- The docs tree references protocol contracts for camera, I/O board, node, and payment, indicating this model service sits in a larger device ecosystem even though the code here mainly consumes camera and weight data.
- The service’s outputs are decisions/judgments consumed by the rest of the vending-machine control system.
12. Testing and verification boundary
- services/model/tests/ contains unit and contract tests for the API, decision engine, session store, trigger pipeline, weight matching, and YOLO geometry.
- The README stresses that local tests validate imports and logic, but Jetson runtime checks are authoritative for camera decoding, TensorRT loading, and trigger behavior.
- That means the architecture has a clear split between host-side correctness and device-side operational readiness.
13. How to draw the architecture
- Model the service as a pipeline with four main layers:
1. HTTP/API ingress
2. Trigger orchestration and session management
3. Frame extraction + YOLO/vision inference
4. Decision/weight aggregation + persistence
- Put Jetson CUDA/TensorRT and camera/loadcell hardware outside the service boundary as runtime dependencies.
- Put YAML session artifacts and JSONL frame traces as durable outputs on the storage side.
Main entry points: manager.py App Manager · trigger_service.py Trigger Service · video_processor.py Video Processor · decision_engine.py Decision Engine
Components 27
API Boundary
FastAPI startup and HTTP routes
- Entrypoint · Starts the model service process and hands off to the app manager.
services/model/model_service/main.py - App Manager · Builds the FastAPI application and manages lifespan/bootstrap.
services/model/model_service/api/manager.py - Health Routes · Liveness, readiness, and runtime diagnostics endpoints.
services/model/model_service/api/routes/health.py - Trigger Routes · Main ingest surface for camera-triggered work.
services/model/model_service/api/routes/trigger.py - Multi-zone Routes · Judgment and summary endpoints for multi-zone evaluation.
services/model/model_service/api/routes/multi_zone.py
Core Runtime
Settings, logging, validation, and startup support
- Settings · Loads env-based runtime settings into application state.
services/model/model_service/core/config.py - Runtime Env · Checks device/runtime assumptions for the Jetson deployment.
services/model/model_service/core/runtime_env.py - Model Validation · Validates model/runtime readiness before inference use.
services/model/model_service/core/model_validation.py - Logging · Configures service logging and trace-friendly output.
services/model/model_service/core/logging_config.py
Trigger Pipeline
Orchestrates camera-triggered processing and judgments
- Trigger Service · Coordinates AVI/frame processing, inference, and session updates.
services/model/model_service/service/trigger_service.py - Judgment Service · Produces the final judgment from aggregated evidence.
services/model/model_service/service/judgment_service.py
Video + Vision
Frame extraction and TensorRT/YOLO inference
- Video Processor · Splits AVI input into frames and drives frame-level processing.
services/model/model_service/video/video_processor.py - Frame Extractor · Extracts usable frames from video input.
services/model/model_service/video/frame_extractor.py - Frame Trace · Records split and inference trace artifacts for post-run analysis.
services/model/model_service/video/frame_trace.py - Voting Ensemble · Combines per-frame predictions into a final result.
services/model/model_service/video/voting_ensemble.py - YOLO Wrapper · Wraps YOLO/TensorRT execution for vision inference.
services/model/model_service/vision/yolo_wrapper.py - Vision Filters · Refines detections into product signals and tracks hands.
services/model/model_service/vision/hand_filter.py - Top-5 Extractor · Extracts ranked product candidates from detections.
services/model/model_service/vision/top5_extractor.py
Session State
Active evaluation state and persistence
- Door Session · Manages session lifecycle and door-scoped evaluation state.
services/model/model_service/service/door_session_service.py - Session Store · Stores active and completed session state.
services/model/model_service/session/session_store.py - Door Store · Keeps door-scoped session records and transitions.
services/model/model_service/session/door_session_store.py - Global Session · Holds cross-session state for active evaluations.
services/model/model_service/session/global_door_session.py - Product Aggregator · Accumulates interim product evidence during a session.
services/model/model_service/session/product_aggregator.py - YAML Persistence · Writes completed session artifacts to disk.
services/model/model_service/session/yaml_persistence.py
Decision Logic
Weight matching and final business decisions
- Weight Logic · Matches loadcell totals to product evidence.
services/model/model_service/weight/count_calculator.py - Strict Matcher · Applies strict weight-to-product matching rules.
services/model/model_service/weight/strict_weight_matcher.py - Decision Engine · Turns detections and weight signals into final decisions.
services/model/model_service/engine/decision_engine.py
Connections 29
- Entrypoint → App Manager: boot · Process entry loads the FastAPI app manager.
- App Manager → Settings: loads · Bootstrap resolves runtime settings into app state.
- App Manager → Logging: configures · App startup configures logging and terminal behavior.
- App Manager → Runtime Env: guards · Startup checks Jetson/runtime assumptions.
- App Manager → Model Validation: validates · Startup validates model and runtime readiness.
- App Manager → Health Routes: mounts · Registers health and diagnostics routes.
- App Manager → Trigger Routes: mounts · Registers the main trigger ingest route.
- App Manager → Multi-zone Routes: mounts · Registers multi-zone judgment and summary routes.
- Trigger Routes → Trigger Service: delegates · HTTP ingest hands off to trigger orchestration.
- Multi-zone Routes → Judgment Service: delegates · Summary endpoints delegate to judgment logic.
- Trigger Service → Video Processor: orchestrates · Trigger handling starts video-to-frame processing.
- Trigger Service → Door Session: updates · Trigger flow updates session lifecycle state.
- Trigger Service → Decision Engine: evaluates · Trigger flow produces final decisions through the engine.
- Video Processor → Frame Extractor: splits · Video input is decomposed into frames.
- Video Processor → Frame Trace: traces · Frame splitting and inference are recorded for observability.
- Video Processor → YOLO Wrapper: infers · Frames are sent to the TensorRT-backed detector.
- YOLO Wrapper → Vision Filters: refines · Raw detections are filtered into usable signals.
- Vision Filters → Top-5 Extractor: ranks · Filtered detections feed product ranking.
- Video Processor → Voting Ensemble: aggregates · Per-frame outputs are combined into a final video result.
- Decision Engine → Weight Logic: combines · Decision logic consumes loadcell-derived product matching.
- Weight Logic → Strict Matcher: matches · Weight totals are matched to product candidates.
- Decision Engine → Product Aggregator: uses · Business decisions draw on accumulated product evidence.
- Product Aggregator → Session Store: stores · Product evidence is kept in session state.
- Door Session → Door Store: persists · Door-scoped sessions are stored and retrieved.
- Door Session → Global Session: shares · Session service coordinates cross-session state.
- Session Store → YAML Persistence: writes · Completed sessions are serialized as YAML artifacts.
- Trigger Service → Session Store: reads/writes · Trigger orchestration updates the active session boundary.
- Judgment Service → Decision Engine: evaluates · Judgment generation depends on the decision engine.
- Judgment Service → YAML Persistence: archives · Final judgments are captured in durable session output.