How imannil/pet-act-vision works
PetActVision classifies pet activities in short videos. Its primary user workflow is upload in the Next.js interface, submit to FastAPI, decode and normalize a frame sequence, run the CNN-plus-temporal classifier, and return a label, probabilities, explanation, and preview frames. Offline preparation, training, and evaluation form a separate model-development workflow. The README documents dataset downloading and Docker support, but sampled code does not establish their runtime wiring; the graph therefore omits uncertain links and deployment machinery.
Main entry points: predict.py Inference orchestration · main.py FastAPI endpoints · evaluate_model.py Model evaluation · preprocess_video.py Video preprocessing
Components 15
Web experience
- Upload and results UI
frontend/src/app/page.tsx
Prediction service
- FastAPI endpoints
src/api/main.py - Inference orchestration
src/inference/predict.py
Model pipeline
- Frame extraction
src/features/extract_frames.py - CNN temporal model
src/models/video_model.py - Model checkpoint
models - Model training
src/models/train_model.py - Model evaluation
src/models/evaluate_model.py - Evaluation metrics
models
Dataset preparation
- Labeled raw videos
data/raw - Video preprocessing
src/data/preprocess_video.py - Sequences and manifest
data/processed - Dataset loading
src/data/dataset.py - Dataset acquisition
src/data/download_dataset.py
Other components
- Video uploader
Connections 21
- Video uploader → Upload and results UI: uploads video
- Upload and results UI → FastAPI endpoints: POST /predict
- FastAPI endpoints → Inference orchestration: predicts video
- FastAPI endpoints → Inference orchestration: extracts previews
- Inference orchestration → Model checkpoint: loads weights
- Inference orchestration → Frame extraction: extracts and normalizes
- Inference orchestration → CNN temporal model: runs prediction
- FastAPI endpoints → Upload and results UI: returns results
- Labeled raw videos → Video preprocessing: reads clips
- Video preprocessing → Frame extraction: extracts sequences
- Video preprocessing → Sequences and manifest: writes sequences and manifest
- Video preprocessing → Sequences and manifest: writes label map
- Sequences and manifest → Dataset loading: reads manifest and samples
- Dataset loading → Model training: provides batches
- Model training → CNN temporal model: trains model
- Model training → Model checkpoint: saves best weights
- Sequences and manifest → Model evaluation: reads manifest and samples
- Model evaluation → CNN temporal model: evaluates model
- Model evaluation → Model checkpoint: loads weights
- Model evaluation → Evaluation metrics: writes confusion matrix
- Dataset acquisition → Labeled raw videos: acquires data