How google-ai-edge/ai-edge-torch works
Project type and purpose
• This is an open-source Python library (“AI Edge Torch”) that converts PyTorch models into TensorFlow Lite flatbuffers, and provides a native Generative API for authoring and optimizing transformer-based LLMs for on-device inference.
File structure
• Top-level folders:
• build and CI configs (Bazel, GitHub workflows, WORKSPACE)
• docs (pytorch_converter walkthroughs, generative doc)
• ai_edge_torch/ – the main Python package, subdivided into:
– _convert/ : core conversion pipeline, FX-based graph transforms
– generative/ : transformer layer implementations, quantization recipes, example conversion tools
– odml_torch/ : StableHLO/MLIR builders, TF-integration, JAX bridge
– quantize/ : quantization utilities, pt2e quantizer
– debug/, lowertools/, hlfb/ : helper and fallback modules
– testing/ : test harness, model coverage
– examples/ : Python and C++ usage guides, end-to-end tests
• Recognize plugin-style FX passes, builder pattern, clear separation of API vs. transformation vs. backends.
README for additional insights
• Two main features:
• PyTorch Converter (Beta): uses torch.export() → FX passes → MLIR/StableHLO → TFLite exporter
• Generative API (Alpha): implements efficient transformer layers, supports Lora, KV cache, quant recipes, E2E examples
• External integrations: PyTorch, tf-nightly/TensorFlow Lite, MediaPipe LLM Inference API.