knights-analytics/hugot

1. What this repo is

Hugot is a Go-first inference and training library for Hugging Face-style transformer models exported to ONNX. The repository is structured as a reusable library, with backend selection and pipeline behavior controlled at build time and at session creation time.

2. Public entry point and session lifecycle

The root hugot.go plus backend-specific variants (hugot_go.go, hugot_ort.go, hugot_xla.go, and disabled stubs) form the main API surface. A caller creates a Session, attaches one or more pipelines to it, and later destroys the session to release model/runtime resources.

3. Backend boundary is the key architecture seam

The project supports three execution paths:

- pure Go / GoMLX,

- ONNX Runtime,

- OpenXLA.

These are separated with build tags and paired *_disabled.go files so unsupported backends compile cleanly when omitted. This boundary affects everything from model loading to tokenizer choice and whether training is available.

4. Pipeline layer is where user tasks are modeled

The pipelines/ package contains task-specific orchestration for:

- text classification,

- feature extraction,

- token classification,

- question answering,

- zero-shot classification,

- cross-encoder,

- text generation,

- image classification,

- object detection,

- tabular prediction.

Each pipeline is responsible for shaping inputs, invoking the backend model, and transforming raw outputs into task-specific result structs.

5. Execution flow: preprocess → model → postprocess

The common inference flow is: input data enters a pipeline, the tokenizer converts text to model-ready tensors, the backend model executes ONNX inference, and pipeline-specific postprocessing converts scores/logits/boxes/tokens into domain outputs. backends/pipeline.go, backends/tokenizer.go, and backends/model.go define the shared abstractions behind that flow.

6. Tokenizer and model implementations are backend-specific

Tokenization is split between Go and Rust-backed implementations (tokenizer_go.go, tokenizer_rust.go) because ORT/XLA require the Rust tokenizer artifact, while the Go backend can stay cgo-free. Model execution is split across model_ort.go, model_gomlx.go, and their disabled counterparts. This is an important dependency boundary because tokenizer/runtime choice is coupled to the selected backend.

7. Generative and training paths are specialized

Generative inference is handled separately in backends/generative.go and is currently ORT-focused, specifically for text generation. Training/fine-tuning is isolated in hugot_training*.go and backends/model_gomlx.go, with XLA/GoMLX used for converting and training models before serializing back to ONNX. In practice, training support is narrower than inference support and is limited to feature extraction/semantic similarity workflows.

8. Data and model acquisition

downloader.go plus the filesystem abstraction in util/fileutil manage loading model artifacts from local disk or external filesystems. The README notes support for the viant/afs abstraction, which means the model source is not hardwired to POSIX files; object stores like S3 can be plugged in through filesystem adapters.

9. Supporting utilities are small but domain-specific

The util/ subpackages handle common data transformations that multiple pipelines need:

- imageutil for image loading/normalization,

- vectorutil for embedding/vector operations,

- safeconv for defensive numeric conversions,

- fileutil for path/file handling.

These utilities sit below pipelines and above the backend layer.

10. Datasets and test fixtures support training and evaluation

datasets/ contains abstractions for dataset loading and XLA-compatible dataset logic. testcases/ holds embedded JSONL and vector fixtures for semantic similarity, token classification, and classification outputs. These fixtures mirror the repo’s core workflows and are used to validate end-to-end pipeline behavior.

11. Test matrix reflects backend fragmentation

The tests/ tree is split by backend (go, ort, xla, training), which mirrors the architecture rather than the package layout. This indicates the main system boundary is not by feature alone, but by runtime/backend capability.

12. Build, packaging, and runtime dependencies are part of the architecture

The repository includes multiple Dockerfiles, compose files, and GitHub Actions workflows for building, testing, and releasing across backend combinations. External runtime dependencies materially affect the architecture: ONNX Runtime shared libraries, GoMLX/OpenXLA, Rust tokenizers, and CUDA-capable accelerators are all first-class deployment concerns, not optional extras.

13. Architectural takeaway

The repo is best diagrammed as a layered system: API/session entry at the top, task pipelines in the middle, shared preprocess/model/postprocess abstractions underneath, and pluggable execution backends plus filesystem/model-loading infrastructure at the bottom. The most important boundaries are backend selection, tokenizer implementation, and the separation between inference pipelines and the narrower training/generative flows.