How heiliaavola/tehisintellekti_rakendmise_projekt_tooteliin works
1. Repository purpose and overall shape
This repo is a stepwise computer-vision pipeline for packaged meat inspection on a production line. It evolves from simple RTSP frame capture to motion-triggered snapshotting, barcode reading, product/area cropping, OCR for date reading, and later evaluation of real-time feasibility. The code is organized by milestone folders (seminar2–seminar7, ettevalmistus3–ettevalmistus6), each representing a distinct stage of the pipeline.
2. Primary runtime boundary: RTSP video ingestion
The upstream data source is an RTSP camera stream, referenced in the README as rtsp://172.17.37.81:8554/<toode>. Early scripts in seminar2, ettevalmistus3, and ettevalmistus4 handle reading frames from the stream and deciding when to save a frame based on motion or visual markers. This is the entry boundary for the whole system.
3. Stage 1: motion and trigger detection
ettevalmistus3/RTSP_liikumine.py and related seminar2/RTSP_* scripts implement the first control logic: detect movement, determine when a product has stopped, and choose a usable frame. This stage reduces a continuous stream into discrete candidate images for downstream processing. It is the first major filtering point in the architecture.
4. Stage 2: barcode reading and product metadata mapping
seminar3/yl1_triipkoodi_lugemine_run.py, seminar3/yl2_toote_info_run.py, and supporting files like barcode_tools.py and barcode_data.json form the barcode subsystem. The barcode reader is a concrete external dependency here: Dynamsoft barcode reading is used to decode codes from saved images. The decoded barcode is then mapped to product metadata such as product name and shelf-life information.
5. Integrated snapshot pipeline
ettevalmistus4/RTSP_liikumine_es4.py combines movement-triggered capture with barcode extraction into a more complete “tick-based” workflow. Each trigger produces a time-stamped product observation containing barcode and product metadata. This stage bridges raw video acquisition and structured inspection output.
6. Stage 3: geometric cropping of product regions
seminar4/yl1_toodete_cut.py, seminar4/yl2_alade_cut.py, seminar4/alade_cut.py, and seminar4/toodete_cut_lahendus.py implement the slicing subsystem. Here, full-frame product images are normalized: the product is isolated, rotated/oriented, resized, and then smaller regions are cut out for later tasks such as date OCR or label analysis. This is a major boundary: the pipeline changes from “whole-frame inspection” to “region-of-interest processing.”
7. RTSP slicing as a reusable pipeline
ettevalmistus5/yl1_rtsp_slicing.py moves the cropping logic back into the live RTSP flow. It supports offline replay from saved data and debugging of crop quality. The ettevalmistus5 folder contains the intermediate artifacts and helper code needed to make this stage reproducible, including templates and barcode metadata. This is where the system becomes a reusable end-to-end capture-and-crop service.
8. OCR experimentation and model comparison
seminar5/yl1_OCR.py, seminar5/yl2_OCR.py, and especially seminar5/ocr_compare.py compare different OCR approaches for reading dates from the cropped regions. The repo explicitly treats OCR as a pluggable subsystem: local models and cloud-backed OpenRouter workflows are compared for accuracy, speed, and cost. This stage consumes the cropped date/label images produced earlier, not raw video.
9. Real-time date recognition pipeline
ettevalmistus6/RTSP_with_date.py is the integration point for live date detection. It combines RTSP capture, barcode identification, crop extraction, and OCR into one real-time pipeline, with attention to throughput and alert logic. The ettevalmistus6 data and answer files suggest this stage is also used for reporting and evaluation against a target takt time.
10. Evaluation and experiment outputs
seminar6 adds a measurement layer around detection quality. The results/ directory stores per-product JSON metrics for DINO and MAE-based experiments, along with histogram PDFs. This makes the architecture not only a production pipeline but also an experiment harness: model outputs are logged and compared across products and label types.
11. Latest model-based classification experiments
seminar7 appears to be a later experimental branch focused on threaded RTSP processing and learned classification. Files such as RTSP_threadded.py, sildid_DINO.py, logistic_regression.py, and toodete_tuvastamine.py suggest a model pipeline using DINO embeddings plus logistic regression to classify product/label regions. The folder structure with product_area samples per product indicates a dataset boundary for supervised learning experiments, separate from the earlier rule-based OCR flow.
12. Data, templates, and artifacts are first-class inputs
Several folders contain JSON templates, barcode metadata, sample crops, generated PDFs, and result files. These are not just outputs; they are part of the system’s architecture because each stage depends on specific intermediate artifacts from the previous stage. The key boundary is that each milestone folder is self-contained, with its own sample data and evaluation outputs, rather than one global shared dataset.
13. Environment and external dependencies
The project is designed around a Conda environment (dependencies/environment.yml, Python 3.11). The architecture depends materially on OpenCV for image/video handling, Dynamsoft for barcode decoding, Pillow/numpy/matplotlib for image processing and analysis, and optionally EasyOCR / Torch / Strhub / PaddleX for OCR comparisons. OpenRouter is an optional external service used in OCR evaluation, controlled by OPENROUTER_API_KEY.
14. What an architecture diagram should emphasize
The clearest system view is a left-to-right pipeline: RTSP stream → motion/trigger detection → snapshot capture → barcode decoding and product metadata lookup → product normalization/cropping → OCR or model-based inspection → metrics/reporting. The folders map naturally onto stages, and the main architectural boundaries are between live video ingestion, intermediate image artifacts, and evaluation/modeling experiments.
Main entry points: RTSP_with_date.py Live date · RTSP_liikumine_es4.py Tick pipeline · yl1_triipkoodi_lugemine_run.py Decode code · yl1_toodete_cut.py Product cut
Components 24
Live capture
RTSP ingestion and trigger detection stages.
- RTSP stream · Production-line video source referenced by the pipeline.
- Frame read · Early RTSP frame capture and visual trigger experimentation.
seminar2/RTSP_lugemine.py - Motion trigger · Detects movement and selects a usable frame once product motion settles.
ettevalmistus3/RTSP_liikumine.py - Tick pipeline · Combines movement-triggered capture with barcode extraction.
ettevalmistus4/RTSP_liikumine_es4.py
Barcode stage
Decode barcodes and map them to product metadata.
- Barcode tools · Shared barcode parsing and lookup helpers.
seminar3/barcode_tools.py - Decode code · Runs barcode recognition on captured images.
seminar3/yl1_triipkoodi_lugemine_run.py - Map product · Maps decoded barcodes to product name and shelf-life data.
seminar3/yl2_toote_info_run.py
Cropping stage
Normalize products and extract regions of interest.
- Product cut · Normalizes product images before downstream inspection.
seminar4/yl1_toodete_cut.py - Area cut · Extracts smaller inspection regions from the product image.
seminar4/yl2_alade_cut.py - Cut solution · Alternative or improved product/area cutting implementation.
seminar4/toodete_cut_lahendus.py - RTSP slicing · Brings cropping logic into the live RTSP flow.
ettevalmistus5/yl1_rtsp_slicing.py
OCR stage
Date reading and OCR comparison pipelines.
- OCR compare · Compares OCR approaches for reading dates from cropped regions.
seminar5/ocr_compare.py - Local OCR · Runs the local OCR approach on cropped date images.
seminar5/yl1_OCR.py - Cloud OCR · Runs the alternative OCR workflow for comparison.
seminar5/yl2_OCR.py - Live date · End-to-end live date recognition over RTSP.
seminar6/RTSP_with_date.py
Experimentation
Evaluation, metrics, and learned classification experiments.
- DINO eval · Evaluates DINO-style product/label experiments and writes metrics.
seminar6/sildid_DINO.py - MAE eval · Evaluates MAE-style experiments for the same inspection targets.
seminar6/sildid_MAE.py - Threaded RTSP · Threaded RTSP processing for later-stage experimentation.
seminar7/RTSP_threadded.py - Classifier · Logistic-regression classifier over learned features.
seminar7/logistic_regression.py - DINO classify · Uses DINO embeddings for product/label classification experiments.
seminar7/sildid_DINO.py - Product detect · Combines detection and classification for product-area samples.
seminar7/toodete_tuvastamine.py
Artifacts
Shared templates, barcode maps, and result outputs.
- Barcode map · Product barcode metadata used across inspection stages.
seminar3/barcode_data.json - Results · Stored metrics, histograms, and experiment outputs.
seminar6/results
Environment
Runtime and dependency specification.
- Env spec · Conda environment and package requirements for the pipeline.
dependencies/environment.yml
Connections 25
- RTSP stream → Frame read: frames · Initial live video ingestion.
- Frame read → Motion trigger: trigger · Motion logic filters continuous frames into candidates.
- Motion trigger → Tick pipeline: snapshot · Captured frames feed the integrated tick-based pipeline.
- Tick pipeline → Decode code: decode · Snapshots are sent to barcode reading.
- Decode code → Barcode tools: helpers · Uses shared barcode utilities.
- Decode code → Map product: code · Decoded barcode is mapped to product metadata.
- Map product → Barcode map: lookup · Reads the shared barcode reference table.
- Tick pipeline → Product cut: crop · Inspection frames are normalized into product regions.
- Product cut → Area cut: roi · Product regions are further sliced into areas of interest.
- Area cut → Cut solution: refine · Alternative crop logic improves region extraction.
- Product cut → RTSP slicing: live flow · Crop logic is reused in the RTSP-based slicing pipeline.
- RTSP slicing → OCR compare: crops · Cropped date regions are benchmarked by OCR approaches.
- OCR compare → Local OCR: compare · One branch uses the local OCR implementation.
- OCR compare → Cloud OCR: compare · One branch uses the cloud-backed OCR implementation.
- Local OCR → Live date: integrate · Local OCR feeds the real-time date pipeline.
- Cloud OCR → Live date: integrate · Cloud OCR can also feed the real-time date pipeline.
- Live date → Results: measure · Real-time pipeline outputs are evaluated and stored.
- DINO eval → Results: write · DINO experiments generate metrics artifacts.
- MAE eval → Results: write · MAE experiments generate metrics artifacts.
- Threaded RTSP → DINO classify: features · Threaded live capture supports embedding-based classification.
- DINO classify → Classifier: predict · Learned classifier consumes embeddings.
- Product detect → Threaded RTSP: stream · Later product detection experiments reuse threaded capture.
- Product detect → DINO classify: learned vision · Classification experiments operate on product-area samples.
- Env spec → Frame read: runtime · Common Python environment supports the pipeline stages.
- Env spec → Live date: runtime · Common Python environment supports the live OCR pipeline.