How xiaoyeshenger/java_pytorch_onnx_tensorrt_yolov8_vue works
Project type and purpose
• This is a full-stack AI algorithm “middle platform” (中台) system combining:
• A Java Spring Boot backend exposing REST APIs
• A Vue.js frontend single-page app
• Python (and optional C++) inference modules for YOLOv8 on GPU/CPU
• Shell scripts to glue platform components
• Docker for containerized deployment
• Purpose: let Java developers upload/manage deep-learning models, schedule video inference tasks, perform real-time inference (via ONNX or TensorRT), push inference frames/results to a media server and notify downstream systems.
File structure
• Top-level:
• data/… – AI model files, platform docs, front-end code, OS-specific inference scripts, SQL schema, Dockerfile
• src/ – Java Spring Boot monolith organized by layers (controller, service, dao, mapper, etc.), AOP, annotation, code-generation templates
• img/ – architecture and UI screenshots
• pom.xml, mvnw – Maven build
• Front-end under data/front_code (Vue)
• Linux and Windows folders under data/linux and data/windows host separate script sets and Python/C++ inference code
• Resources: application.properties, FreeMarker templates (codeGen), MyBatis configs
• Deployment: Dockerfile under data/docker, shell run scripts under data/linux/... and data/windows/…
README for additional insights
• Lists external dependencies: MySQL, Redis, MongoDB, RocketMQ, Minio, ZLM media server, FFmpeg, Nginx.
Main components
a. Frontend
• Vue.js app (npm)
• Served by Nginx (or local dev server)
• Connects to backend via REST (HTTP/JSON), RTMP/FLV/HLS for video streams
b. Backend (Java Spring Boot)
• Controllers → Services → DAOs (MyBatis + Mongo + Redis)
• AOP/annotations for logging, encryption, validation
• Code-Gen module (FreeMarker)
• Message producers/consumers to RocketMQ
• File storage client to Minio
• ShellInvoker: triggers inference .sh/.bat scripts
c. Inference Workers
• Python modules: ONNXRuntime, PyTorch, Ultralytics Yolov8, OpenCV, FFmpeg
• Optional C++ code (TensorRT, DeepStream)
• Shell scripts launch inference; workers read video (local path or stream), run inference, generate frames + metadata
• Push inference images/results to Minio + RocketMQ
d. Media Server & Streaming
• FFmpeg (inside Python worker) pushes to ZLMediaKit over RTMP/FLV
• ZLMediaKit streams to web front-end
e. Data Stores & Middleware
• MySQL – system metadata, user, task, model tables
• MongoDB – optional document storage
• Redis – cache, locks
• RocketMQ – event bus for task start, inference results, failure retries
• Minio – object store for model binaries, inference snapshots
f. Deployment Infrastructure
• Docker container (Ubuntu20.04 + Java, Python, CUDA, cuDNN, TensorRT)
• GPU node (e.g. Nvidia T4)
• Kubernetes or plain Docker run commands (with --gpus)
• Nginx container serves frontend
Relationships and interactions
1. User → Vue.js UI
2. UI → HTTP → Spring Boot REST controllers
3. Controllers delegate to services → persist to MySQL/Mongo; produce task messages to RocketMQ
4. Service triggers inference shell script (with modelKey, shellKey)
5. Shell launches Python worker (or C++ executable)
6. Worker reads video stream/file, runs inference →
• FFmpeg pushes processed frames to ZLMediaKit
• Worker uploads snapshot images + metadata to Minio
• Worker publishes inference result to RocketMQ
7. Spring Boot consumers (RocketMQ listeners) consume inference results → store alarm records, logs, push HTTP notifications to customers
8. UI requests video streams from ZLMediaKit and reads alarm/task data from backend
Architectural patterns and design principles
• Layered architecture (Presentation → API → Business → Persistence)
• Aspect-Oriented Programming for cross-cutting concerns (logging, encryption)
• Template-based code generation (FreeMarker)
• Event-driven integration via RocketMQ
• Polyglot runtimes: Java, Python, Shell, optional C++
• OS abstraction: folder structure under data/linux vs data/windows
• Containerization and immutable infrastructure
Detailed elements
• Frontend: Vue.js → Nginx
• API Gateway: SpringBoot REST
• Business logic: Service layer, AOP modules
• Persistence: MySQL, MongoDB, Redis
• Event Bus: RocketMQ producers & consumers
• Object Store: Minio
• Inference: Shell scripts invoking Python ONNXRuntime & TensorRT modules
• Streaming: FFmpeg → ZLMediaKit → browser
• Deployment: Docker host with GPU, volumes, GPU flags