How onnx/sklearn-onnx works
Project type and purpose
• This is an open-source Python library/tool (skl2onnx) that converts scikit-learn pipelines and models into ONNX format.
• Its main use case is: take a trained scikit-learn estimator, translate it operator by operator into ONNX, and output a .onnx file for high-performance inference (e.g., with ONNX Runtime).
File structure
• Top level:
• pyproject.toml, requirements*.txt → build/dependency descriptors
• benchmarks/ → performance tests and plots
• docs/ → user documentation, tutorials, Sphinx configuration
• skl2onnx/ → core package
• tests/ & tests_onnxmltools/ → extensive unit and integration tests
• Inside skl2onnx/:
• __main__.py → CLI entry point (python -m skl2onnx)
• convert.py & _parse.py → front-end API for model conversion
• operator_converters/ → one file per scikit-learn operator converter
• algebra/ & common/ → internal graph-manipulation, topology, and ONNX operator factories
• helpers/ → utilities for ONNX graph inspection and extension
• sklapi/ → convenience wrappers around scikit-learn types
• tutorial/ → embedded demo scripts
• CI: .github/workflows (Linting, testing, release)
README for additional insights
• Entry function: to_onnx()
• External dependencies: onnx, onnxruntime
• Extensibility: supports registering custom converters and external libraries
a. Main components
1. User API / CLI
2. Converter Front-End (convert.py, to_onnx)
3. Parser (_parse.py)
4. Operator Registry (mapping sklearn operator → converter)
5. Operator Converters (operator_converters/)
7. Graph Builder & Topology (common/, algebra/)
8. ONNX Model Output
9. External Runtime (onnxruntime)
10. Tests & Benchmarks & Docs (supporting modules)
b. Relationships & data flow
• User imports to_onnx or calls CLI → feeds sklearn model + sample input
• Parser inspects model metadata, options → builds an internal topology
• For each node in sklearn pipeline:
• Lookup converter in Operator Registry
• Operator Converter emits ONNX operator nodes
• Algebra/Common modules assemble full ONNX graph
• Final graph serialized to .onnx
• (Optional) Benchmarks use onnxruntime to measure performance
c. Architectural patterns
• Plugin/Registry pattern for operator converters
• Modular separation: parsing, conversion, graph construction, serialization
• Layered design: API → conversion engine → graph builder → output
d. Key technologies
• Python 3
• ONNX protobuf model specification
• scikit-learn introspection
• onnxruntime for testing/benchmarks
• Sphinx for documentation
Be detailed but focused
• Every major sub-directory as a component
• Plugin registry and modular converter files
• Don’t show every single operator file, but illustrate “operator_converters/*” as one component with many plugins