PEP: 774
Title: Removing the LLVM requirement for JIT builds
Author: Savannah Ostrowski <savannah@python.org>
Discussions-To: https://discuss.python.org/t/78548/
Status: Deferred
Type: Standards Track
Created: 27-Jan-2025
Python-Version: 3.14
Post-History: `27-Jan-2025 <https://discuss.python.org/t/78548/>`__
Resolution: `14-Mar-2025 <https://discuss.python.org/t/78548/34>`__

Abstract
========

Since Python 3.13, CPython has been able to be configured and built with an
experimental just-in-time (JIT) compiler via the ``--enable-experimental-jit``
flag on Linux and Mac and ``--experimental-jit`` on Windows. To build CPython with
the JIT enabled, users are required to have LLVM installed on their machine
(initially, with LLVM 16 but more recently, with LLVM 19). LLVM is responsible
for generating stencils that are essential to our copy-and-patch JIT (see :pep:`744`).
These stencils are predefined, architecture-specific templates that are used
to generate machine code at runtime.

This PEP proposes removing the LLVM build-time dependency for JIT-enabled builds
by hosting the generated stencils in the CPython repository. This approach
allows us to leverage the checked-in stencils for supported platforms at build
time, simplifying the contributor experience and address concerns raised at the
Python Core Developer Sprint in September 2024. That said, there is a clear
tradeoff to consider, as improved developer experience does come at the cost of
increased repository size.

It is important to note that this PEP is not a proposal to accept or reject the
JIT itself but rather to determine whether the build-time dependency on LLVM is
acceptable for JIT builds moving forward. If this PEP is rejected, we will
proceed with the status quo, retaining the LLVM build-time requirement. While
this dependency has served the JIT development process effectively thus far, it
introduces setup complexity and additional challenges that this PEP seeks to
alleviate.

Motivation
==========

At the Python Core Developer Sprint that took place in September 2024, there was
discussion about the next steps for the JIT - a related discussion also took
place on `GitHub <https://github.com/python/cpython/issues/115869>`__. As part
of that discussion, there was also a clear appetite for removing the LLVM
requirement for JIT builds in preparation for shipping the JIT off by default in
3.14. The consensus at the sprint was that it would be sufficient to provide
pre-generated stencils for non-debug builds for Tier 1 platforms and that
checking these files into the CPython repo would be adequate for the limited
number of platforms (though more options have been explored; see `Rejected
Ideas`_).

Currently, building CPython with `the JIT requires LLVM
<https://github.com/python/cpython/tree/main/Tools/jit#installing-llvm>`__ as a
build-time dependency. Despite not being exposed to end users, this dependency
is suboptimal. Requiring LLVM adds a setup burden for developers and those who
wish to build CPython with the JIT enabled. Depending on the operating system,
the version of LLVM shipped with the OS may differ from that required by our JIT
builds, which introduces additional complexity to troubleshoot and resolve. With
few core developers currently contributing to and maintaining the JIT, we also
want to make sure that the friction to work on JIT-related code is minimized as
much as possible.

With the proposed approach, hosting pre-compiled stencils for supported
architectures can be generated in advance, stored in a central location, and
automatically used during builds. This approach ensures reproducible builds,
making the JIT a more stable and sustainable part of CPython's future.

Rationale
=========

This PEP proposes checking JIT stencils directly into the CPython repo as the
best path forward for eliminating our build-time dependency on LLVM.

This approach:

* Provides the best end-to-end experience for those looking to build CPython
  with the JIT
* Lessens the barrier to entry for those looking to contribute to the JIT
* Ensures builds remain reproducible and consistent across platforms without
  relying on external infrastructure or download mechanisms
* Eliminates variability introduced by network conditions or potential
  discrepancies between hosted files and the CPython repository state, and
* Subjects stencils to the same review processes we have for all other JIT-related
  code

However, this approach does result in a slight increase in overall
repository size. Comparing repo growth on commits over the past 90 days, the
difference between the actual commits and the same commits with stencils added
amounts to a difference of 0.03 MB per stencil file. This is a small increase in
the context of the overall repository size, which has grown by 2.55 MB in the
same time period. For six stencil files, this amounts to an upper bound of 0.18 MB.
The current total size of the stencil files for all six platforms is 7.2 MB. [#stencil_calc]_

These stencils could become larger in the future with changes to register
allocation, which would introduce 5-6 variants per instruction in each stencil
file (5-6x larger). However, if we ended up going this route, there are
additional modifications we could make to stencil files that could help
counteract this size increase (e.g., stripping comments, minimizing the
stencils).

Specification
=============

This specification outlines the proposed changes to remove the build-time
dependency on LLVM and the contributor experience if this PEP is accepted.

Repository changes
------------------

The CPython repository would now host the pre-compiled JIT stencils in a new
subdirectory in ``Tools/jit`` called ``stencils/``. At present, the JIT is tested
and built for six platforms, so to start, we'd check in six stencil files. In
the future, we may check in additional stencil files if support for additional
platforms is desired or relevant.

.. code-block:: text

    cpython/
        Tools/
            jit/
                stencils/
                    aarch64-apple-darwin.h
                    aarch64-unknown-linux-gnu.h
                    i686-pc-windows-msvc.h
                    x86_64-apple-darwin.h
                    x86_64-pc-windows-msvc.h
                    x86_64-pc-linux-gnu.h

Workflow
--------

The workflow changes can be split into two parts, namely building CPython with
the JIT enabled and working on the JIT's implementation.

Building CPython with the JIT
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Precompiled JIT stencil files will be stored in the ``Tools/jit/stencils``
directory, with each file name corresponding to its target triple as outlined
above. At build time, we determine whether to use the checked in stencils or to
generate a new stencil for the user's platform. Specifically, for contributors
with LLVM installed, the ``build.py`` script in ``Tools/jit/stencils`` will allow
them to regenerate the stencil for their platform. Those without LLVM can rely
on the precompiled stencil files directly from the repository.

Working on the JIT's implementation (or touching JIT files)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In continuous integration (CI), stencil files will be automatically validated and updated when changes
are made to JIT-related files. When a pull request is opened that touches these
files, the ``jit.yml`` workflow, which builds and tests our builds, will run as
usual.

However, as part of this, we will introduce a new step that diffs the current
stencils in the repo against those generated in CI. If there is a diff for a
platform's stencil file, a patch file for the updated stencil is generated and
the step will fail. Each patch is uploaded to GitHub Actions. After CI is
finished running across all platforms, the patches are aggregated into a single
patch file for convenience. You can download this aggregated patch, apply it
locally, and commit the updated stencils back to your branch. Then, the
subsequent CI run will pass.

Reference Implementation
========================

Key parts of the `reference implementation <https://github.com/python/cpython/pull/129331>`__ include:

- |CI|_: The CI workflow responsible for generating stencil patches.

- |jit_stencils|_: The directory where stencils are stored.

- |targets|_: The code to compile and parse the templates at build time.

.. |CI| replace:: ``.github/workflows/jit.yml``
.. _CI: https://github.com/python/cpython/blob/main/.github/workflows/jit.yml

.. |jit_stencils| replace:: ``Tools/jit/stencils``
.. _jit_stencils: https://github.com/python/cpython/blob/main/Tools/jit/stencils

.. |targets| replace:: ``Tools/jit/_targets``
.. _targets: https://github.com/python/cpython/blob/main/Tools/jit/_targets.py

Ignoring the stencils themselves and any necessary JIT README changes, the
changes to the source code to support reproducible stencil generation and
hosting are minimal (around 150 lines of changes).

Rejected Ideas
==============

Several alternative approaches were considered as part of the research and
exploration for this PEP. However, the ideas below either involve
infrastructural cost, maintenance burden, or a worse overall developer
experience.

Using Git submodules
--------------------

Git submodules are a poor developer experience for hosting stencils because they
create a different kind of undesirable friction. For instance, any
updates to the JIT would necessitate regenerating the stencils and committing
them to a separate repository. This introduces a convoluted process: you must
update the stencils in the submodule repository, commit those changes, and then
update the submodule reference in the main CPython repository. This disconnect
adds unnecessary complexity and overhead, making the process brittle and
error-prone for contributors and maintainers.

Using Git subtrees
------------------

When using subtrees, the embedded repository becomes part of the main
repository, similar to what's being proposed in this PEP. However, subtrees
require additional tooling and steps for maintenance, which adds unnecessary
complexity to workflows.

Hosting in a separate repository
--------------------------------

While splitting JIT stencils into a separate repository avoids the storage
overhead associated with hosting the stencils, it adds complexity to the build
process. Additional tooling would be required to fetch the stencils and
potentially create additional and unnecessary failure points in the workflow.
This separation also makes it harder to ensure consistency between the stencils
and the CPython source tree, as updates must be coordinated across the
repositories.

Hosting in cloud storage
------------------------

Hosting stencils in cloud storage like S3 buckets or GitHub raw storage
introduces external dependencies, complicating offline development
workflows. Also, depending on the provider, this type of hosting comes with
additional cost, which we'd like to avoid.

Using Git LFS
-------------

Git Large File Storage (LFS) adds a tool dependency for contributors,
complicating the development workflow, especially for those who may not already
use Git LFS. Git LFS does not work well with offline workflows since files
managed by LFS require an internet connection to fetch when checking out
specific commits, which is disruptive for even basic Git workflows. Git LFS has
some free quota but there are `additional
costs <https://docs.github.com/en/billing/managing-billing-for-your-products/managing-billing-for-git-large-file-storage/about-billing-for-git-large-file-storage>`__
for exceeding that quota which are also undesirable.

Maintain the status quo with LLVM as a build-time dependency
------------------------------------------------------------

Retaining LLVM as a build-time dependency upholds the existing barriers to
adoption and contribution. Ultimately, this option fails to address the core
challenges of accessibility and simplicity, and fails to eliminate the
dependency which was deemed undesirable at the Python Core Developer Sprint in
the fall (the impetus for this PEP), making it a poor long-term solution.

Footnotes
=========

.. [#stencil_calc] Calculated using this `Gist
   <https://gist.github.com/savannahostrowski/c66ac62267e626d82aafa27939468e30>`__.
   This script replays commits for roughly the past 90 days, generates the
   stencil file for the platform for each commit, and then commits the stencil
   file into a copy of the repository if they change. The calculation compares
   the before and after of the repository after running ``git gc --aggressive``,
   which is used to pack the repo (similar to what GitHub does on repo clone).

Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
