ITADN
xadupre/onnx-light
代码 动态 Issues69 合并请求68 版本
xadupre/onnx-light · 文件 下载 ZIP
文件最后提交记录最后更新时间
README.md

onnx-light

onnx-light logo

core build-reduced Build Release Wheels Build C++ Release Artifacts asan-ubsan fuzz hardening Documentation Doxygen Style clang-format clang-tidy Typing SBOM Spelling pixi INT ir-py codecov Ruff Code style: black

Documentation

See also the ONNX roadmap for the upstream project's direction and priorities.

Note: onnx-light started from the upstream ONNX pull request onnx/onnx#7208, which is the initial code base from which this project diverged.

onnx without protobuf

  • ONNX Files larger than 2 GB (protobuf is limited to 2Gb)
  • Parallel loading and saving: significantly faster compared to the single-threaded path
  • Zero-copy parsing – creates the ModelProto without any tensor copy
  • Aligned external tensor offsets – external tensor data can be written with explicit offset alignment
  • No serialize/parse round-trip for C++ tools – the Python ModelProto is the C++ ModelProto
  • Supports protobuf (onnx) and flatbuffers (onnxruntime) format.

Modular C++ libraries

The C++ code is split into several small libraries so a downstream project can link only what it needs:

  • onnx_light::lib_onnx_proto – protobuf-compatible message types, parser / serializer, external data, optional encrypted save / load (AES-256-CBC or ChaCha20-Poly1305).
  • onnx_light::lib_onnx_core – implements all the generic functionalities (runtime value types and execution engine, the LightOpSchema data structures, the symbolic expression engine and the kernel / shape-inference dispatch tables) but ships no concrete operators. The dispatch tables start empty and are filled by the extension libraries below.
  • onnx_light::lib_onnx_op – lightweight LightOpSchema registrations for ONNX operator domains, with no shape inference.
  • onnx_light::lib_onnx_lib – full ONNX-compatible schemas (with history), checker, inliner, shape inference and version converter.
  • onnx_light::lib_onnx_shape – shape-inference dispatch table, expression engine and graph optimization helpers.
  • onnx_light::lib_onnx_kernels – C++ kernels used to generate the beckend
  • onnx_light::lib_onnx_backend_test – C++ backend test infrastructure and reference operator kernels.

onnx_core only implements the mechanisms: the actual operator schemas, kernels, shape-inference and peak-memory functions are registered into the shared dispatch tables owned by onnx_core by the extension libraries (onnx_op, onnx_shapes, onnx_kernels, ...) through their Register*Functions() entry points. This keeps the extensions independent from each other while sharing the same core engine.

Kernels and Backend Tests

  • Each operator has a corresponding runtime implementation in C++, it is used to generated the C++ output of the backend tests.
  • Fully written in C++, it can be used in any language.
  • Outputs are always generated with a C++ kernel.
  • The kernels can be used without the backend tests.

Software Bill of Materials (SBOM)

A CycloneDX 1.7 Software Bill of Materials is shipped at the root of the repository as sbom.cdx.json and is also included in the source distribution. It lists the third-party components bundled into the built artifacts (currently only nanobind, used to expose the C++ extension to Python). The file is validated against the CycloneDX 1.7 schema by the SBOM GitHub Actions workflow.

Getting started

Install the package in editable mode:

pip install -e .[dev] -v

or

python setup.py build_ext --inplace

setup.py build_ext configures CMake with -DCMAKE_BUILD_TYPE=Release by default (unless CMAKE_ARGS already sets CMAKE_BUILD_TYPE). --cpp-tests can be used to build the C++ unit tests and run them with ctest.

To speed up compilation with multiple threads, pass --parallel (or -j) with the number of jobs:

python setup.py build_ext --inplace --parallel 8

By default, python setup.py build_ext now auto-enables parallel builds (--parallel <cpu_count>) unless CMAKE_BUILD_PARALLEL_LEVEL is already set.

Alternatively, when installing with pip, you can control parallel builds using the CMAKE_BUILD_PARALLEL_LEVEL environment variable:

CMAKE_BUILD_PARALLEL_LEVEL=8 pip install -e .[dev] -v

Run a quick check:

python -c "import onnx_light; print(onnx_light.__version__)"

Build and run the C++ unit tests from the editable build:

With pip install:

pip install -C build-dir=build -C cmake.build-type=Debug -C cmake.define.ONNX_LIGHT_BUILD_TESTS=ON -e .[dev] -v
ctest --test-dir build --output-on-failure

With setup.py, --cpp-tests builds the C++ unit tests and runs them with ctest in one step:

python setup.py build_ext --inplace --build-temp build --cpp-tests

The Python package is built and installed inplace before the C++ unit tests are built and run, so the editable install is always available even if a C++ test fails to build or run.

On multi-config generators such as Visual Studio, add the matching configuration to ctest: use -C Debug when the build was configured with cmake.build-type=Debug, and -C Release after python setup.py build_ext --cpp-tests.

Load a model with parallel tensor parsing:

import onnx_light.onnx

model = onnx_light.onnx.load("model.onnx", num_threads=4)
print(model.ir_version)

Using onnx_light as a C++ library

Installing the C++ library

Build and install the static library and headers to a local prefix (Python extension not required):

cmake -S . -B build-install
  -DCMAKE_BUILD_TYPE=Release \
  -DONNX_LIGHT_BUILD_PYTHON=OFF \
  -DCMAKE_INSTALL_PREFIX=/usr/local
cmake --build  build-install
cmake --install build-install

This installs:

  • liblib_onnx_proto.a, liblib_onnx_op.a, and liblib_onnx_lib.a (the static libraries) into <prefix>/lib
  • All public C++ headers into <prefix>/include/onnx_light
  • CMake package config files into <prefix>/lib/cmake/onnx_light

Using find_package(onnx_light) in your project

Once installed, any CMake project can locate and link the library with:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_lib)

If the code only needs protobuf-compatible message parsing/serialization and does not need operator schemas, checker, or shape inference, it can link against the lighter onnx_light::lib_onnx_proto target instead:

find_package(onnx_light REQUIRED)
target_link_libraries(my_target PRIVATE onnx_light::lib_onnx_proto)

If the code needs lightweight math operator schemas without shape inference, it can link against onnx_light::lib_onnx_op and query onnx_op::math::GetAllOnnxOpMathSchemasWithHistory().

Pass -DCMAKE_PREFIX_PATH=<prefix> when configuring your project if the library was installed to a non-standard prefix.