How huggingface/swift-coreml-transformers works
Project Type & Purpose
• This is a Swift‐based, on‑device machine‑learning demo suite (iOS apps) showcasing Core ML implementations of Transformer models (BERT, DistilBERT, GPT‑2, DistilGPT‑2).
• It contains two demo targets (CoreMLBert and CoreMLGPT2), a shared Swift Sources folder that implements tokenizers and model wrappers, static Resources (.mlmodel files and vocabularies), and an offline model_generation Python toolchain for converting PyTorch/ONNX models into Core ML models.
File Structure Analysis
• Top level:
• CoreMLBert.xcodeproj / CoreMLGPT2 (each an Xcode app target)
• Sources/ (shared Swift code: tokenizers, model wrappers, utilities)
• Resources/ (pretrained .mlmodel files + vocab & merge files + sample JSON)
• model_generation/ (Python scripts & requirements for converting/exporting models)
• CoreMLBertTests / CoreMLGPT2Tests (unit tests for Swift implementations)
• Patterns & Config:
• Each app target follows MVC: ViewController + LoaderView + AppDelegate/SceneDelegate + storyboard assets.
• Sources layer implements core Transformer logic (tokenization, MLMultiArray utils, decoding strategies).
• Resources layer holds static ML models consumed by the app.
• model_generation acts as an offline “backend” for model conversion.
README Insights
• Explains purpose: Swift Core ML Transformers, demo apps for QA (BERT/SQuAD) and text generation (GPT‑2).
• Lists features: tokenizers, model conversion scripts, demos.
• Shows that both on‑device inference and offline conversion pipelines exist.
Diagramming the Architecture
a. Main Components
• iOS App Frontend (UIKit/View layer) – two targets: CoreMLBert & CoreMLGPT2
• Application Logic (Controller layer: ViewController, LoaderView)
• Transformer Core Library (Sources: tokenizers, model wrappers, utils)
• Core ML Models (.mlmodel files in Resources)
• Offline Model Generator (Python scripts in model_generation/)
• Test Suites (Swift unit tests)
• External Services (none at runtime; only Xcode & Git LFS at build time)
b. Relationships & Data Flow
1. User interacts with UI → triggers question‑answer or text‑generation request
2. Controller calls into Transformer Core Library → uses tokenizer to encode input
3. Encoded tokens → MLMultiArray → sent to Core ML model
4. Core ML model inference → MLMultiArray output → decoded by Transformer Library
5. Controller updates UI with decoded text
6. Offline: Python scripts read PyTorch/ONNX model → convert to .mlmodel → place in Resources
c. Architectural Patterns & Principles
• Layered architecture: UI layer → Controller → Core Logic → ML Model
• Single Responsibility in Sources (tokenization vs tensor utilities vs decoding)
• Offline vs Runtime separation (model_generation is not part of app bundle)
• MVC (or MVVM‐light) pattern in each iOS demo target
d. Key Technologies & Frameworks
• Swift, UIKit, Xcode (iOS app framework)
• Core ML 3 (on‑device inference)
• Python + ONNX/PyTorch + coremltools (model_generation)
• MLMultiArray utilities (Swift extensions)
• git‑lfs for large model storage
Capturing Essential Details
• Both demo targets side by side, converging on the shared Core Library and Resources.
• Explicitly annotate the model_generation directory as an offline build‐time component.
• Test targets linking into the Sources to validate tokenization and model wrappers.
• Call out the use of git‑lfs for model file storage as a build prerequisite.