How giclab/flexvit works
FlexViT is an FPGA accelerator integration for TensorFlow Lite vision-transformer inference, built as a SECDA-TFLite delegate. The main workflow selects supported model nodes, prepares and tiles their inputs and weights, dispatches DMA transfers to the accelerator, and returns quantized outputs to the TFLite graph. README also documents simulation benchmarks and inference-difference checks, hardware assets, and artifact evaluation with several INT8 models. Edges reflect sampled delegate and driver behavior plus README-described runtime capabilities; simulation and board execution orchestration belongs to SECDA-TFLite, not this repository.
Main entry points: models INT8 models · vit_delegate.cc ViT delegate · scheduler.sc.h Data scheduler · acc_container.h Accelerator data container
Components 23
TFLite integration
- ViT delegate
src/src/secda_delegates/vit_delegate/v9/vit_delegate.cc - Delegate provider
src/src/secda_delegates/vit_delegate/v9/vit_delegate_provider.cc - External delegate adaptor
src/src/secda_delegates/vit_delegate/v9/vit_delegate_adaptor.cc
Data preparation
- Tensor preparation
src/src/secda_delegates/vit_delegate/v9/util_prep.h - Operator utilities
src/src/secda_delegates/vit_delegate/v9/util.h - Accelerator data container
src/src/secda_delegates/vit_delegate/v9/accelerator/driver/acc_container.h - Tiled DMA driver
src/src/secda_delegates/vit_delegate/v9/accelerator/driver/vit_driver.h
FPGA accelerator
- SystemC binding
src/src/secda_delegates/vit_delegate/v9/accelerator/driver/systemc_binding.h - Accelerator module
src/src/secda_delegates/vit_delegate/v9/accelerator/acc.sc.h - Data scheduler
src/src/secda_delegates/vit_delegate/v9/accelerator/scheduler.sc.h - Processing elements
src/src/secda_delegates/vit_delegate/v9/accelerator/PE.sc.h - Post-processing unit
src/src/secda_delegates/vit_delegate/v9/accelerator/PPU.sc.h
Models and hardware
- INT8 models
src/data/models - FPGA bitstreams
src/hardware_automation/generated - Accelerator configuration
src/hardware_automation/configs/vit_9_0.json
Evaluation workflows
- Simulation workflows
src/src/tensorflow - Layer benchmark
src/src/tensorflow/launch.json - Inference difference
src/src/tensorflow/launch.json - Artifact evaluation
ARTIFACT_EVALUATION.md
Other components
- Model developer
- TensorFlow Lite runtime
- SECDA-TFLite flows
- FPGA board
Connections 27
- Model developer → INT8 models: selects model
- INT8 models → TensorFlow Lite runtime: loaded by
- Model developer → Delegate provider: enables delegate
- Model developer → External delegate adaptor: configures options
- Delegate provider → ViT delegate: creates
- External delegate adaptor → ViT delegate: creates
- TensorFlow Lite runtime → ViT delegate: delegates nodes
- ViT delegate → Tensor preparation: prepares operators
- ViT delegate → Accelerator data container: sets layer data
- Tensor preparation → Accelerator data container: formats tensors
- Accelerator data container → Tiled DMA driver: provides buffers
- Tiled DMA driver → SystemC binding: sends DMA data
- SystemC binding → Accelerator module: connects signals
- Accelerator module → Data scheduler: runs
- Data scheduler → Processing elements: schedules compute
- Data scheduler → Post-processing unit: triggers postprocess
- Processing elements → Post-processing unit: provides accumulations
- Post-processing unit → Tiled DMA driver: returns output
- Accelerator configuration → SECDA-TFLite flows: configures hardware
- FPGA bitstreams → FPGA board: programs
- SECDA-TFLite flows → FPGA board: runs inference
- Simulation workflows → Layer benchmark: offers
- Simulation workflows → Inference difference: offers
- INT8 models → Layer benchmark: benchmarked by
- INT8 models → Inference difference: checked by
- Artifact evaluation → INT8 models: documents evaluation
- Artifact evaluation → SECDA-TFLite flows: describes workflow