.. _l-design-differences:

Detailed Differences between ``onnx`` and ``onnx_light``
========================================================

This section explains the internal design of *onnx-light* and how it differs from
the reference :epkg:`onnx` package.
Both packages share the same on-disk format (binary protobuf-encoded ``.onnx``
files) and expose very similar Python APIs, so ``onnx_light`` can act as a
near-drop-in replacement for common model loading and inspection tasks.
The key differences lie in how the serialization layer is implemented.

No protobuf dependency
-----------------------

``onnx`` wraps the official :epkg:`Protocol Buffers`
(protobuf) runtime.  Every message class is
auto-generated by the ``protoc`` compiler from ``.proto`` schema files, and
the resulting Python objects delegate all parsing and serialization to the
``libprotobuf`` C++ library.
``onnx_light`` ships its own hand-written parser and serializer implemented
entirely in C++ (see ``onnx_light/onnx_proto/``).  There is **no
dependency on protobuf at compile time or at runtime**.

* The C++ shared object built by ``onnx_light`` is smaller
  because it does not statically link any portion of ``libprotobuf``.
* All parsing and serialization code lives in a single self-contained library
  that can be consumed by other C++ projects without installing protobuf
  (see :epkg:`C++ onnx-light examples`).
* C++ projects that only need protobuf-compatible message types can link
  ``onnx_light::lib_onnx_proto`` directly; ``onnx_light::lib_onnx_lib`` is only
  needed when features tied to operator notions are required (checker, schema
  lookup, shape inference, version conversion, ...).
* The wire format produced by ``onnx_light`` is 100 % compatible with the
  official ONNX binary format, so models can be freely exchanged between the
  two libraries.

----

No serialize / parse round-trip when calling C++ tools
------------------------------------------------------

In the official :epkg:`onnx` package the Python message classes
(:class:`~onnx_light.onnx_lib.ModelProto`, :class:`~onnx_light.onnx_lib.GraphProto`, :class:`~onnx_light.onnx_lib.NodeProto`, :class:`~onnx_light.onnx_lib.TensorProto`, ...) are
generated by ``protoc`` and live entirely in Python (or in the pure-Python
protobuf runtime when the C++ accelerator is not available).  They are
**not** the same objects as the ``onnx::ModelProto`` C++ instances used by
the bundled native tools (checker, shape inference, inliner, version
converter, ...).

Every call from Python into one of those C++ helpers therefore needs a
full serialize / parse round-trip:

.. code-block:: python

    # onnx/inliner.py – simplified
    result = C.inline_local_functions(model.SerializeToString(), convert_version)
    inlined_model = ModelProto()
    inlined_model.ParseFromString(result)

The same pattern shows up in ``onnx.checker``
(``C.check_model(model.SerializeToString(), ...)``),
``onnx.shape_inference`` (``model.SerializeToString()`` then
:meth:`ParseFromString` on the result), and the version converter.  For large
models this serialize/parse pair dominates the wall-clock time of the
call even when the C++ work itself is cheap.

``onnx_light`` does not have this split: the Python :class:`~onnx_light.onnx_lib.ModelProto` exposed
by ``onnx_light.onnx`` *is* the C++ :class:`~onnx_light.onnx_lib.ModelProto`, bound through
``nanobind`` (see ``onnx_light/onnx_py/_onnxpy_lib.cc``).  The bundled
tools take the object by reference and operate on it in place:

.. code-block:: cpp

    // onnx_light/onnx_py/_onnxpy_lib.cc – inliner binding
    inliner_mod.def(
        "inline_local_functions",
        [](const ModelProto &model, bool convert_version) {
          inliner::CheckFunctionCallCycles(model);
          ModelProto copy = model;
          inliner::InlineLocalFunctions(copy, convert_version);
          return copy;
        });

No bytes are produced and no parser runs.  The inliner still makes an
internal C++ copy of the :class:`~onnx_light.onnx_lib.ModelProto` so the caller's model is left
unchanged, but that copy is a structural deep-copy of C++ objects, not a
serialize + parse round-trip — it is orders of magnitude cheaper than the
protobuf path and no temporary ``bytes`` object is materialised on the
Python side.  The same direct-call pattern is used by
:mod:`onnx_light.onnx.checker`, :mod:`onnx_light.onnx.shape_inference`, and the
version converter, so invoking any of these helpers on a model already
loaded by ``onnx_light`` is essentially free apart from the work of the
algorithm itself.

----

Files larger than 2 GB
-----------------------

The protobuf C++ runtime enforces a hard **2 GB message-size limit** at
parsing time.  Loading or saving a model larger than that threshold with the
standard ``onnx`` package raises a ``DecodeError``.
``onnx_light`` imposes no such limit.  Internally it tracks byte counts with
64-bit unsigned integers throughout the parsing and serialization path, so
models of arbitrary size are supported.

----

Buffered file I/O
-----------------

For file-based loading, ``onnx_light`` uses :cpp:class:`~onnx_light::utils::FileStream`
(``stream.h`` / ``stream.cc``), a buffered binary reader that opens the
file with ``std::ifstream`` and reads ahead in 4096-byte chunks.  On
POSIX platforms a second file descriptor is opened for parallel block reads
via ``pread``.
The ``onnx`` package reads the whole file into a Python ``bytes`` object first
and then passes it to protobuf, which copies it again internally.

----

Parallel tensor loading and saving
----------------------------------

Large ONNX models contain hundreds or thousands of initializers (tensor
weights).  Parsing or serializing these sequentially is the dominant cost
when loading or saving a model.
``onnx_light`` exposes a ``num_threads`` option on both
:func:`onnx_light.onnx.load` and :func:`onnx_light.onnx.save` that
distributes the initializer parsing / writing across a thread pool:

.. code-block:: python

    import onnx_light.onnx as onnxl

    model = onnxl.load("model.onnx", num_threads=4)
    onnxl.save(model, "model.onnx", num_threads=4)

On the C++ side the thread pool is implemented in ``thread_pool.h`` /
``thread_pool.cc``.  Each worker independently parses (or writes) a slice
of the initializer list, so wall-clock load and save time scales with the
number of hardware threads available.  In practice loading or saving a
large model is significantly faster than with the
single-threaded path (see
:ref:`l-example-plot-threads-load-save` for a detailed benchmark).
The standard ``onnx`` package is single-threaded; it offers no built-in
parallel loading or saving mechanism.

----

Zero-copy parsing
-----------------

When the full model bytes are already in memory (e.g. downloaded into a
``bytes`` object), ``onnx_light`` can skip the ``malloc + memcpy`` that would
normally be used to copy each tensor's raw data into an owned buffer:

.. code-block:: python

    import onnx_light.onnx as onnxl

    serialized = open("model.onnx", "rb").read()   # keep alive!
    model = onnxl.load(serialized, no_copy=True)
    # tensor.raw_data now points directly into 'serialized'

Internally, each :class:`~onnx_light.onnx_lib.TensorProto` stores a non-owning :cpp:class:`~onnx_light::utils::ByteSpan` (from
``simple_span.h``) that borrows the bytes from the source buffer.  The
borrowed span's ``is_borrowed()`` predicate can be used to check whether raw
data is owned or borrowed.

.. warning::
    The source ``bytes`` object **must** remain alive for as long as the model
    is in use.  Freeing it while ``raw_data`` fields still point into it
    causes undefined behavior.  This constraint does not exist in the standard
    ``onnx`` package.

----

C++ class generation via macros
---------------------------------

The ``onnx`` package generates Python message classes from ``.proto`` schema
files using ``protoc``.  ``onnx_light`` takes a different approach: message
classes are generated **at compile time** from a small set of C++ macros
defined in ``stream_class.h``:

* ``BEGIN_PROTO(cls, doc)`` / ``END_PROTO()`` — open/close a message class.
* ``FIELD(type, name, order, doc)`` — declare a scalar field with typed
  accessors ``ref_<name>()``, ``has_<name>()``, ``set_<name>()``.
* ``FIELD_STR(name, order, doc)`` — shorthand for
  :cpp:class:`~onnx_light::utils::OptionalString` fields (a nullable string that
  distinguishes "unset" from "empty"); also accepts ``std::string``,
  ``const char *``, and :cpp:class:`~onnx_light::utils::RefString`. The
  ``str_<name>()`` accessor returns a ``const std::string &`` (a shared empty
  string when unset, never throwing).
* ``FIELD_REPEATED_STR(type, name, order, doc)`` — declare a repeated string
  field backed by :cpp:class:`~onnx_light::utils::RepeatedStringField`
  (a list of :cpp:class:`~onnx_light::utils::String` values).
* ``FIELD_REPEATED(type, name, order, doc)`` — declare a repeated (list)
  field.
* ``SERIALIZATION_METHOD()`` — inject :meth:`ParseFromString`,
  :meth:`SerializeToString`, :meth:`ParseFromStream`, and :meth:`SerializeToStream`
  declarations.

The resulting classes in ``onnx.h`` closely mirror the protobuf-generated
classes so that code originally written for ``onnx`` can be adapted with
minimal changes.

----

String types in proto messages
---------------------------------

Proto string fields in ``onnx_light`` use one of three C++ types, each with a
distinct role:

:cpp:class:`~onnx_light::utils::RefString`
    A **non-owning view** into existing bytes (a pointer + length pair).  The
    parser uses it to point at substrings inside the parse buffer without
    copying.  ``RefString`` is valid only as long as the source buffer lives;
    it is produced by the zero-copy parser path and consumed immediately to
    populate an owning field.

:cpp:class:`~onnx_light::utils::String`
    A thin ``std::string`` subclass used for **repeated string fields** (for
    example ``NodeProto::input``, ``NodeProto::output``).  It behaves like
    ``std::string`` in all respects and is always either empty or non-empty —
    there is no additional "null" state.

:cpp:class:`~onnx_light::utils::OptionalString`
    A thin ``std::optional<std::string>`` subclass used for **singular
    (optional) string fields** (for example ``NodeProto::name``,
    ``NodeProto::doc_string``).  It adds a ``null()`` predicate that returns
    ``true`` when the field is absent (``std::nullopt``), which is distinct
    from the field being present but empty.  The serializer emits a string
    field only when ``!field.null()``, reproducing the proto3 rule that
    default-valued fields are omitted from the wire format.

The mapping between C++ types and proto field categories is:

.. list-table::
   :header-rows: 1
   :widths: 30 20 50

   * - Proto field kind
     - C++ type
     - Notes
   * - Singular ``string`` (``FIELD_STR``)
     - ``OptionalString``
     - Absent (null) vs. present-but-empty are distinguished; absent
       fields are not written to the wire.
   * - Repeated ``string`` (``FIELD_REPEATED_STR``)
     - ``RepeatedStringField`` of ``String``
     - Ordinary list; each element is always a valid, owned string.
   * - Parser temporary / zero-copy view
     - ``RefString``
     - Points into the parse buffer; ownership does not transfer.

----

External-data / multi-file models
----------------------------------

Large ONNX models can be split across two files: a small ``.onnx`` file that
holds the graph structure and a separate binary blob (the *external data file*)
that holds the raw tensor weights.  This layout allows the structural metadata
to be inspected quickly without loading the weights and makes it possible to
memory-map only the weight region.

Saving with external data
~~~~~~~~~~~~~~~~~~~~~~~~~

Pass a ``location`` argument to :func:`onnx_light.onnx.save` to route tensor weights to a
separate file:

.. code-block:: python

    import onnx_light.onnx as onnxl

    # model.onnx – graph structure only
    # model.onnx.data – all tensor weights
    onnxl.save(model, "model.onnx", location="model.onnx.data")

Serializing to two files does **not** mutate the in-memory :class:`~onnx_light.onnx_lib.ModelProto`.
``onnx_light`` applies external-data metadata on a temporary copy while writing
and keeps the original model unchanged.
The ``location`` value stored inside the ``.onnx`` metadata is automatically
reduced to a *relative* path (just the file name) when an absolute path is
provided, so the two files can be moved together without breaking the
reference.

Loading with external data
~~~~~~~~~~~~~~~~~~~~~~~~~~

When the ``.onnx`` file already references an external data file through its
tensor metadata, :func:`onnx_light.onnx.load` can discover and load the weights
automatically:

.. code-block:: python

    import onnx_light.onnx as onnxl

    model = onnxl.load("model.onnx", load_external_data=True)

To override the data-file location (for example when the file has been moved),
pass ``location`` explicitly:

.. code-block:: python

    model = onnxl.load("model.onnx", location="/data/weights.bin",
                        load_external_data=True)

When ``no_copy=True`` is combined with external data, ``onnx_light`` reads
each external weights file once into a shared model-owned buffer and every
tensor points into that shared storage. This avoids one allocation and copy
per tensor while still handling split external-data files transparently.

Splitting external data across multiple files
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For very large models it can be useful to cap the size of each external weight
file.  Set ``max_external_file_size`` (in bytes) and :func:`onnx_light.onnx.save` will
automatically open a new file once the limit is reached, appending ``.1``,
``.2``, … suffixes to the base name:

.. code-block:: python

    import onnx_light.onnx as onnxl

    # Produces: model.onnx, model.onnx.data, model.onnx.data.1, …
    onnxl.save(
        model,
        "model.onnx",
        location="model.onnx.data",
        max_external_file_size=2 * 1024 ** 3,  # 2 GB per file
    )

When loading, only the primary location (``model.onnx.data``) needs to be
specified; the loader automatically opens ``model.onnx.data.1``,
``model.onnx.data.2``, … as required.
All I/O is performed in C++ via :cpp:class:`~onnx_light::utils::TwoFilesWriteStream` /
:cpp:class:`~onnx_light::utils::TwoFilesStream`, so no Python overhead is incurred per tensor.

----

Encrypted model save / load
-----------------------------

``onnx_light`` optionally supports saving and loading models in an
**AES-256-CBC encrypted** binary format (extension ``.onnxc``).  The
standard ``onnx`` package offers no equivalent functionality.
The feature is available only when ``onnx_light`` is built with OpenSSL
(``-DONNX_LIGHT_HAS_OPENSSL``); when OpenSSL is absent the helpers raise
``NotImplementedError`` with a clear message.

File format
~~~~~~~~~~~

The encrypted file is a compact, self-contained binary:

.. code-block:: text

    Offset  Size  Field
    ------  ----  -----
         0     8  Magic: "ONNXCRY1"
         8    16  Random PBKDF2 salt
        24    16  Random AES-CBC initialisation vector
        40     N  AES-256-CBC ciphertext (PKCS#7-padded protobuf payload)

Key derivation uses **PBKDF2-HMAC-SHA256** with 100 000 iterations, which
makes brute-force attacks on the passphrase computationally expensive.

Python API (file-based)
~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

    import onnx_light.onnx as onnxl

    # Save an encrypted model to a file
    onnxl.save_encrypted(model, "model.onnxc", key="my_passphrase")

    # Load and decrypt from a file
    model = onnxl.load_encrypted("model.onnxc", key="my_passphrase")

    # A wrong key raises RuntimeError
    model = onnxl.load_encrypted("model.onnxc", key="wrong")  # RuntimeError

Python API (in-memory / bytes)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

When no file I/O is desired, the model can be encrypted to a ``bytes``
object and decrypted back directly:

.. code-block:: python

    import onnx_light.onnx as onnxl

    # Encrypt to bytes (no file written)
    blob: bytes = onnxl.save_encrypted_string(model, key="my_passphrase")

    # Decrypt from bytes
    model = onnxl.load_encrypted_string(blob, key="my_passphrase")

The ``bytes`` object produced by :func:`onnx_light.onnx.save_encrypted_string` is in the
same ``ONNXCRY1`` format as the file produced by :func:`onnx_light.onnx.save_encrypted`,
so the two forms are interchangeable.

C++ API
~~~~~~~

.. code-block:: cpp

    #include "onnx_crypt.h"

    // File-based
    ONNX_LIGHT_NAMESPACE::SaveEncryptedModel(model, "model.onnxc", "passphrase");
    ONNX_LIGHT_NAMESPACE::LoadEncryptedModel(model, "model.onnxc", "passphrase");

    // In-memory
    std::string blob = ONNX_LIGHT_NAMESPACE::SaveEncryptedModelToString(model, "passphrase");
    ONNX_LIGHT_NAMESPACE::LoadEncryptedModelFromString(model, blob, "passphrase");

See :ref:`l-api-onnx-onnx-proto-onnx-crypt` for the full C++ API reference.

----

No checked-in markdown files for operators
-------------------------------------------

The official :epkg:`onnx` repository tracks operator metadata in a set of
markdown files (``docs/Operators.md``, ``docs/Operators-ml.md``,
``docs/Changelog.md``, ``docs/Changelog-ml.md``,
``docs/TestCoverage.md``, ``docs/TestCoverage-ml.md``).  These files are
**generated** from the C++ schemas and the Python test suite, and the
``onnx`` CI fails if a change to an operator schema is not accompanied by
a regeneration of the corresponding markdown files.

``onnx_light`` does **not** ship any equivalent set of generated markdown
files, and adding a new operator (or modifying an existing schema, opset
history, kernel, shape-inference rule, backend test case, …) **never**
requires editing or regenerating any ``.md`` file in this repository.

All operator documentation is produced on the fly by the Sphinx build
from the live ``LightOpSchema`` objects (see
:ref:`l-design-test-coverage`), so the single source of truth is the
C++ code itself.  Contributors should therefore:

* never commit a regenerated ``Operators.md`` / ``Changelog.md`` /
  ``TestCoverage.md`` (or any ``-ml`` variant);
* never add a new ``.md`` file under ``docs/`` to describe a new
  operator — the operator pages are generated automatically.

----

API compatibility
-----------------

``onnx_light`` aims to be a functional subset of the ``onnx`` Python API for
the most common operations:

.. list-table::
   :header-rows: 1
   :widths: 40 30 30

   * - Operation
     - ``onnx``
     - ``onnx_light``
   * - Load from file
     - ``onnx.load(path)``
     - ``onnxl.load(path)``
   * - Load from bytes
     - ``onnx.load_from_string(b)``
     - ``onnxl.load(b)``
   * - Save to file
     - ``onnx.save(model, path)``
     - ``onnxl.save(model, path)``
   * - Save with external data
     - ``onnx.save_model(model, path, save_as_external_data=True, location=loc)``
     - ``onnxl.save(model, path, location=loc)``
   * - Save external data with aligned tensor offsets
     - not supported
     - ``opts = onnxl.SerializeOptions(); opts.alignment = 4096; model.SerializeToFile(path, opts, loc)``
   * - Load with external data
     - ``onnx.load(path, load_external_data=True)``
     - ``onnxl.load(path, load_external_data=True)``
   * - Load external data with shared no-copy buffers
     - not supported
     - ``onnxl.load(path, load_external_data=True, no_copy=True)``
   * - Split external data
     - not supported
     - ``onnxl.save(model, path, location=loc, max_external_file_size=N)``
   * - Save encrypted to file
     - not supported
     - ``onnxl.save_encrypted(model, path, key=k)``
   * - Load encrypted from file
     - not supported
     - ``onnxl.load_encrypted(path, key=k)``
   * - Save encrypted to bytes
     - not supported
     - ``onnxl.save_encrypted_string(model, key=k)``
   * - Load encrypted from bytes
     - not supported
     - ``onnxl.load_encrypted_string(blob, key=k)``
   * - Parse a message
     - ``msg.ParseFromString(b)``
     - ``msg.ParseFromString(b)``
   * - Serialize a message
     - ``msg.SerializeToString()``
     - ``msg.SerializeToString()``
   * - Parallel load
     - not supported
     - ``onnxl.load(path, num_threads=N)``
   * - Zero-copy parse
     - not supported
     - ``onnxl.load(b, no_copy=True)``
   * - File size limit
     - 2 GB (protobuf)
     - unlimited

Some helper utilities present in ``onnx`` (shape inference, model checker,
etc.) are not yet implemented in ``onnx_light``, which focuses on fast,
dependency-free loading and saving.

----

Summary
-------

.. list-table::
   :header-rows: 1
   :widths: 35 30 35

   * - Aspect
     - ``onnx``
     - ``onnx_light``
   * - Serialization runtime
     - Google protobuf
     - Custom C++ (no protobuf)
   * - Max model size
     - 2 GB
     - Unlimited
   * - File I/O
     - Read-into-bytes
     - Memory-mapped (mmap)
   * - Tensor loading
     - Single-threaded
     - Optional parallel (thread pool)
   * - Raw-data copying
     - Always copied
     - Zero-copy option (``no_copy=True``)
   * - External data (2-file)
     - Yes (``save_model`` / ``load``)
     - Yes (``save`` / ``load``)
   * - External data no-copy shared buffers
     - No
     - Yes (``load(..., no_copy=True)``)
   * - Split external data (N files)
     - No
     - Yes (``max_external_file_size``)
   * - Tensor offset alignment in external files
     - No
     - Yes (``SerializeOptions.alignment``)
   * - Standalone C++ library
     - Yes
     - Yes (``onnx_light::lib_onnx_proto`` for proto-only code,
       ``onnx_light::lib_onnx_lib`` when operator-aware APIs are needed)
   * - Wire format
     - ONNX binary protobuf
     - ONNX binary protobuf (identical)
   * - Encrypted save / load
     - No
     - Yes (AES-256-CBC, requires OpenSSL)
   * - Python ↔ C++ tool calls (checker, inliner, shape inference, ...)
     - Serialize + parse round-trip per call
     - Direct call (Python object is the C++ object)
