PEP: 771
Title: Default Extras for Python Software Packages
Author: Thomas Robitaille <thomas.robitaille@gmail.com>, Jonathan Dekhtiar <jonathan@dekhtiar.com>
Sponsor: Pradyun Gedam <pradyunsg@gmail.com>
Discussions-To: https://discuss.python.org/t/94905/
Status: Draft
Type: Standards Track
Topic: Packaging
Created: 13-Jan-2025
Post-History:
  `15-Jan-2025 <https://discuss.python.org/t/77892/>`__,
  `06-Feb-2025 <https://discuss.python.org/t/79706/>`__,
  `09-Jun-2025 <https://discuss.python.org/t/94905/>`__,

Abstract
========

:pep:`508` specifies a mini-language for
declaring package dependencies. One feature of this language is the ability to
specify *extras*, which are optional components of a distribution that, when
used, install additional dependencies. This PEP proposes a mechanism to allow
one or more extras to be installed by default if none are provided explicitly.

Motivation
==========

Various use cases for default extras and possible solutions in this PEP were discussed
extensively on `this DPO thread <https://discuss.python.org/t/adding-a-default-extra-require-environment/4898>`__.
These fall into two broad cases that provide the
motivation for the present PEP.

Recommended but not required dependencies
-----------------------------------------

Package maintainers often use extras to declare optional dependencies that
extend the functionality or performance of a package. In some cases, it can be
difficult to determine which dependencies should be required and which should be
categorized as extras. A balance must be struck between the needs of typical
users, who may prefer most features to be available by default, and users who
want minimal installations without large, optional dependencies. One solution
with existing Python packaging infrastructure is for package maintainers to
define an extra called, for example, ``recommended``, which
includes all non-essential but suggested dependencies. Users are then instructed to
install the package using ``package[recommended]``, while those who prefer more
control can use ``package``. However, in practice, many users are unaware
of the ``[recommended]`` syntax, placing the burden on them to know this for a
typical installation. Having a way to have recommended dependencies be installed
by default while providing a way for users to request a more minimal installation
would satisfy this use case, and this PEP describes a solution to this.

Examples of packages that demonstrate this pattern by encouraging users to
include extra dependencies by default include:

* `astropy <https://www.astropy.org/>`_: ``astropy[recommended]``
* `sunpy <https://www.sunpy.org/>`_: ``sunpy[all]``
* `fastapi <https://fastapi.tiangolo.com/>`_: ``fastapi[standard]``

Packages supporting multiple backends or frontends
--------------------------------------------------

Another common use case for using extras is to define different backends or
frontends and dependencies that need to be installed for each backend or
frontend. A package might need at least one backend or frontend to be installed
in order to be functional, but may be flexible on which backend or frontend this
is. Concrete examples of such frontends or backends include:

* The Qt frontend library, which can be provided by `PyQt5 <https://pypi.org/project/PyQt5/>`_, `PyQt6
  <https://pypi.org/project/PyQt6/>`_, `PySide2
  <https://pypi.org/project/PySide2/>`_, or `PySide6
  <https://pypi.org/project/PySide6/>`_
* Backend databases, such as `MySQL <https://www.mysql.com/>`_, `PostgreSQL
  <https://www.postgresql.org/>`_, `SQLite <https://www.sqlite.org/>`_

With current packaging standards, maintainers have to either
require one of the backends or frontends or require users
to always specify extras, e.g., ``package[backend]``, and therefore risk users
having an unusable installation if they only install ``package``. Having a
way to specify one or more default backend or frontend and providing a way to
override these defaults would provide a much better experience for users, and
the approach described in this PEP will allow this.

Note that this PEP does not aim to address the issue of disallowing conflicting
or incompatible extras, for example if a package requires exactly one frontend
or backend package. There is currently no mechanism in Python packaging
infrastructure to disallow conflicting or incompatible extras to be installed,
and this PEP does not change that.

Examples of packages that require at least one backend or frontend to work and
recommend a default extra to install a backend or frontend include:

* `kivy <https://kivy.org/>`_: ``kivy[base]``

* `napari <https://napari.org>`_: ``napari[all]``

* `glueviz <https://www.glueviz.org>`_: ``glueviz[qt]``

In all three cases, installing the package without any extras results in a
broken installation, and this is a commonly reported support issue for some of these
packages.

Rationale
=========

A number of possible solutions have been discussed extensively by
the community for several years, including in `this DPO thread
<https://discuss.python.org/t/adding-a-default-extra-require-environment/4898>`__
as well as in numerous issues and pull requests. The solution that is
presented below:

* is an opt-in solution which means that package maintainers can choose whether or not to use it
* is flexible enough to accommodate both of the major use cases described in `Motivation`_
* re-uses the syntax from :pep:`508`

It is the only solution, out of all those discussed, that meets all three criteria.

Specification
=============

``Default-Extra`` Metadata Field
---------------------------------

A new multiple-use metadata field, ``Default-Extra``, will be added to the `core package
metadata <https://packaging.python.org/en/latest/specifications/core-metadata/#core-metadata>`_.
For this field, each entry must be a string specifying an extra that will be
automatically included when the package is installed without any extras being specified explicitly.

Only entries already specified in a `Provides-Extra
<https://packaging.python.org/en/latest/specifications/core-metadata/#provides-extra-multiple-use>`_
entry can be used in a ``Default-Extra`` entry.

Examples::

    Default-Extra: recommended
    Default-Extra: backend1
    Default-Extra: backend2
    Default-Extra: backend3

Since this introduces a new field in the core package metadata, this will require
`Metadata-Version <https://packaging.python.org/en/latest/specifications/core-metadata/#metadata-version>`_
to be bumped to the next minor version (2.5 at the time of writing).

New key in ``[project]`` metadata table
---------------------------------------

A new key will be added to the ``[project]`` table in project metadata as
originally defined in :pep:`621` and now defined in the `PyPA specifications
<https://packaging.python.org/en/latest/specifications/pyproject-toml/>`_. This key will be named
``default-optional-dependency-keys`` with the following description:

* `TOML <https://toml.io/>`_ type: Array of strings
* Corresponding core metadata field: ``Default-Extra``

Each string in ``default-optional-dependency-keys`` must be the name of an extra
defined in `optional-dependencies
<https://packaging.python.org/en/latest/specifications/pyproject-toml/#dependencies-optional-dependencies>`_,
and each extra in this array will be converted to a matching ``Default-Extra``
entry in the core package metadata. Examples of valid usage which would
produce the example ``Default-Extra`` entries presented in the previous section are:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "recommended",
    ]

and:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "backend1",
        "backend2",
        "backend3"
    ]

Overriding default extras
-------------------------

If extras are explicitly given in a dependency specification, the default
extras are ignored. Otherwise, the default extras are installed.

For example, if a package defines an ``extra1`` default extra as well as a
non-default ``extra2`` extra, then if a user were to install the package with:

.. code-block:: console

    $ pip install package

the default ``extra1`` dependency would be included. If the user instead
installs the package with:

.. code-block:: console

    $ pip install package[extra2]

then the ``extra2`` extra would be installed but the default ``extra1`` extra
would be ignored.

If the same package is specified multiple times in an installation command or
dependency tree, the default extras must be installed if any of the instances of
the package are specified without extras.  For instance, if one installs a
package ``spam`` where ``package`` appears several times in the dependency
tree::

    spam
    ├── tomato
    │   ├── package[extra2]
    └── egg
        └── package

then the default extra should be installed because ``package`` appears at least
once with no extras specified.

An empty set of extras, such as ``package[]`` should be interpreted as meaning
that the package should be installed *without* any default extras (unless
``package`` appears elsewhere in the dependency tree, in which case, the default
extra *would* be installed as mentioned above). This
would provide a universal way of obtaining a minimal installation of a package.

We also note that some tools such as `pip`_ currently ignore unrecognized
extras, and emit a warning to the user to indicate that the extra has not been
recognized, e.g:

.. code-block:: console

    $ pip install package[non-existent-extra]
    WARNING: package 3.0.0 does not provide the extra 'non-existent-extra'
    ...

For tools that behave like this (rather than raising an error), if an extra is
recognized as invalid in a dependency specification, it should be ignored, and
if all specified extras are invalid, then this should be considered equivalent
to ``package[]`` (rather than ``package``) and *not* install any default extras.

Finally, we note (as also discussed in `Relying on tooling to deselect
any default extras`_) that package installers are allowed to implement their
own options to control the above behavior, for example implementing an option
that disables default extras for some or all packages regardless of where these
packages appear in the dependency tree. If such tool-specific options are
implemented, tool developers should make these opt-in,
and users should experience the above PEP 771 behavior as default.

Examples
--------

In this section we take a look at the use cases described in the `Motivation`_
section and how these can now be addressed by using the specification outlined
above.

Recommended dependencies and minimal installations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

First, we consider the case of packages that want recommended but not strictly
required dependencies installed by default, while also providing a way to only
install the required dependencies.

In order to do this, a package maintainer would define an extra called
``recommended`` containing the recommended but not required dependencies, and
would choose to have this be included as a default extra:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "recommended"
    ]

    [project.optional-dependencies]
    recommended = [
        "package1",
        "package2"
    ]

If this package was called ``package``, users installing ``package`` would
then get the equivalent of ``package[recommended]``. Users could alternatively
install ``package[]`` which would install the package without the default extras.

To take a one of the concrete examples of package from the `Motivation`_
section, the `astropy`_ package defines a ``recommended`` extra that users are
currently instructed to install in the default installation instructions.
With this PEP, the ``recommended`` extra could be declared as being a default
extra:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "recommended"
    ]

    [project.optional-dependencies]
    recommended = [
        "scipy",
        "..."
    ]

meaning that installing:

.. code-block:: console

    $ pip install astropy

would then install optional but recommended dependencies such as
`scipy <https://www.scipy.org>`_. Advanced users who want a minimal install
could then use:

.. code-block:: console

    $ pip install astropy[]

Packages requiring at least one backend or frontend
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

As described in `Motivation`_, some packages may support multiple backends
and/or frontends, and in some cases it may be desirable to ensure that there
is always at least one backend or frontend package installed, as the package
would be unusable otherwise. Concrete examples of this might include a GUI
application that needs a GUI library to be present to be usable but is able
to support different ones, or a package that can rely on different computational
backends but needs at least one to be installed.

In this case, package maintainers could make the choice to define an extra
for each backend or frontend, and provide a default, e.g.:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "backend1"
    ]

    [project.optional-dependencies]
    backend1 = [
        "package1",
        "package2"
    ]
    backend2 = [
        "package3"
    ]

If packages can support e.g. multiple backends at the same time, and some of
the backends should always be installed, then the dependencies for these must be given
as required dependencies rather than using the default extras mechanism.

To take one of the concrete examples mentioned in `Motivation`_, the `napari`_ package
can make use of one of `PyQt5`_, `PyQt6`_, `PySide2`_, or `PySide6`_, and users currently
need to explicitly specify ``napari[all]`` in order to have one of these be installed,
or e.g., ``napari[pyqt5]`` to explicitly specify one of the frontend packages. Installing
``napari`` with no extras results in a non-functional package. With this PEP, ``napari``
could define the following configuration:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "pyqt5"
    ]

    [project.optional-dependencies]
    pyqt5 = [
        "PyQt5",
        "..."
    ]
    pyside2 = [
        "PySide2",
        "..."
    ]
    pyqt6 = [
        "PyQt6",
        "..."
    ]
    pyside6 = [
        "PySide6",
        "..."
    ]

meaning that:

.. code-block:: console

    $ pip install napari

would work out-of-the-box, but there would still be a mechanism for users to
explicitly specify a frontend, e.g.:

.. code-block:: console

    $ pip install napari[pyside6]

Supporting extras that should not remove default extras
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

An additional case we consider here is where a package maintainer wants to
support the ability for users to opt-in to non-default extras, without removing
default extras. Essentially, they might want:

* ``package[]`` to give an installation without any extras
* ``package`` to install recommended dependencies (in a ``recommended`` extra)
* ``package[alternative]`` to not install default extras, but to install an alternative set of optional dependencies (in an ``alternative`` extra)
* ``package[additional]`` to install both recommended and additional dependencies (in an ``additional`` extra)

This could be achieved with e.g:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "recommended"
    ]

    [project.optional-dependencies]
    recommended = [
        "package1",
        "package2"
    ]
    alternative = [
        "package3"
    ]
    additional = [
        "package[recommended]",
        "package4"
    ]

The ability for a package to reference itself in the extras is supported by
existing Python packaging tools.

Once again considering a concrete example, `astropy`_ could with this PEP define a
``recommended`` extra (as described in `Recommended dependencies and minimal
installations`_). However, it also defines other extras, including for example
``jupyter``, which adds packages that
enhance the user experience inside `Jupyter <https://jupyter.org/>`_-based
environments. It is possible that users opting in to this extra would still want
recommended dependencies to be installed. In this case, the following
configuration would solve this case:

.. code-block:: toml

    [project]
    default-optional-dependency-keys = [
        "recommended"
    ]

    [project.optional-dependencies]
    recommended = [
        "scipy",
        "..."
    ]
    jupyter = [
        "astropy[recommended]",
        "ipywidgets",
        "..."
    ]

Users installing:

.. code-block:: console

    $ pip install astropy[jupyter]

would then get the same as:

.. code-block:: console

    $ pip install astropy[recommended, jupyter]

Packages with multiple kinds of defaults
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In some cases, it may be that packages need multiple kinds of defaults. As an example,
in `Packages requiring at least one backend or frontend`_, we considered the case of packages
that have *either* backends or frontends, but in some cases, packages may have to support
backends *and* frontends, and want to specify one or more default frontend and one or
more default backend.

Ideally, one may want the following behavior:

.. code-block:: console

    $ pip install package  # installs default backend and frontend
    $ pip install package[]  # installs no backends or frontends
    $ pip install package[backend1]  # installs backend1 and default frontend
    $ pip install package[frontend2]  # installs frontend2 and default backend
    $ pip install package[backend1, frontend2]  # installs backend1 and frontend2

However, this PEP chooses not to provide a mechanism for making it so that e.g., if
``backend1`` is specified, the default backend would be disabled, but the
default frontend would be enabled, since this adds complexity.

Maintainers should instead for now document that if a backend or frontend is
explicitly specified, both backend and frontend need to be specified.
Discoverability for users who want to do this should not be an issue however since users
need to read the documentation in any case to find out what backends or frontends are
available, so they can be shown at the same time how to properly use the extras for
backends and frontends.

One option to increase user friendliness is that maintainers can create extras
called for example ``defaultbackend`` and ``defaultfrontend`` which do install
the default backend and frontend. They can then recommend that users do:

.. code-block:: console

    $ pip install package  # installs default backend and frontend
    $ pip install package[]  # installs no backends or frontends
    $ pip install package[backend1, defaultfrontend]  # installs backend1 and default frontend
    $ pip install package[defaultbackend, frontend2]  # installs frontend2 and default backend
    $ pip install package[backend1, frontend2]  # installs backend1 and frontend2

This would allow (if desired) users to then get whatever the recommended backend
is, even if that default changes in time.

If there is a desire to implement a better solution in future, we believe this
PEP should not preclude this. For example, one could imagine in future adding
the ability for an extra to specify *which* default extras it disables, and if
this is not specified then explicitly specified extras would disable all default
extras (consistent with the present PEP).

Backward Compatibility
======================

Packages not using default extras
---------------------------------

Once support for this PEP is added to tools in the packaging ecosystem, packages
that do not make use of default extras will continue to work as-is and there
should be no break in compatibility.

Packages using default extras
-----------------------------

Once packages start defining default extras, those defaults will only be honored
with recent versions of packaging tools which implement this PEP, but those
packages will remain installable with older packaging tools -- with the main
difference being that the default extras will not be installed automatically
when older packaging tools are used.

As described in `How to teach this`_,
package authors need to carefully evaluate when and how they adopt
the default extra feature depending on their user base, as some actions (such as
moving a required dependency to a default extra) will likely result in breakage
for users if a significant fraction of them are still using older package
installers that do not support default extras. In this sense, package authors
should be aware that this feature, if used in certain ways, can cause
backward-compatibility issues for users, and they are thus responsible for
ensuring that they minimize the impact to users.

Packaging-related tools
-----------------------

The most significant backward-compatibility aspect is related to assumptions
packaging tools make about extras -- specifically, this PEP changes the
assumption that extras are no longer exclusively additive in terms of adding
dependencies to the dependency tree, and specifying some extras can result in
fewer dependencies being installed.

A specific example of change in behavior can be seen with `pip`_: consider a
package ``package`` which has a required dependency of ``numpy``, and a default
extra called ``recommended`` which includes ``scipy``. If a user installs
``package[]``, only ``package`` and ``numpy`` will be installed. If a user then
does:

.. code-block:: console

    $ pip freeze > requirements.txt

then ``requirements.txt`` will contain e.g.::

    package==1.0.2
    numpy==2.1.0

However, if the user then installs the requirements from this file using:

.. code-block:: console

    $ pip install -r requirements.txt

then pip will install ``package`` (which will include the default extra) as well
as ``numpy``, so the final environment will contain ``scipy``. A solution in this
specific case is for the user to pass ``--no-deps`` to ``pip install`` to avoid
resolving the dependency tree, but the point here is to illustrate that there
may be changes in behavior in packaging tools due to the change in the
assumption about what impact an extra can have.

It is worth noting that the recently-accepted :pep:`751` defines a new file
format which is intended to replace alternatives such as the ``pip freeze``
output and other tools in future. The new file format is designed so that the
packages in the file are installed *without* resolving dependencies, which means
that it will be fully compatible with default extras as specified in this PEP,
and will avoid the issue with ``pip freeze``/``pip install -r`` mentioned above.

Security Implications
=====================

There are no known security implications for this PEP.

How to teach this
=================

This section outlines information that should be made available to people in
different groups in the community in relation to the implementation of this PEP.
Some aspects described below will be relevant even before the PEP is fully
implemented in packaging tools as there are some preparations that can be done
in advance of this implementation to facilitate any potential transition later
on. The groups covered below are:

- `Package end users`_
- `Package authors`_
- `Packaging repository maintainers`_

Package end users
-----------------

Package users should be provided with clear installation instructions that show
what extras are available for packages and how they behave, for example
explaining that by default some recommended dependencies or a given frontend or
backend will be installed, and how to opt out of this or override defaults,
depending what is available.

Package authors
---------------

While the mechanism used to define extras and the associated rule about when to
use it are clear, package authors need to carefully consider several points
before adopting this capability in their packages, to avoid inadvertently breaking
backward-compatibility.

Supporting older versions of package installers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Package installers such as `pip`_ or `uv <https://docs.astral.sh/uv/>`_ will not
necessarily implement support for default extras at the same time, and once they
do it is likely that package authors will want to keep supporting users who do
not have the most recent version of a package installer. In this case, the
following recommendations would apply:

* Moving a package from being a required dependency to a default extra would be
  a breaking change, because older versions of package installers would not
  recognise the concept of default extras, and would then install the package
  with fewer dependencies, which could affect users that would have been relying
  on these. Therefore, changing dependencies from required to a default extra in
  an established package should only be done in future once the developers only
  want to support users with installers that implement this PEP.

* Making an existing extra become a default should be safer, such as making
  ``recommended`` in `astropy`_ be a default extra, but in order to support users
  with older versions of package installers, the documentation should still mention
  the extra explicitly as long as possible (until it is clear that most/all users
  are using package installers that implement this PEP). There is no downside to
  keeping the extra be explicitly mentioned, but this will ensure that users with
  modern tooling who do not read documentation (which may be a non-negligible
  fraction of the user community) will start getting the recommended
  dependencies by default.

* Since prior to this PEP, ``package[]`` was equivalent to ``package``,
  authors will be able to document ``package[]`` as a backward-compatible
  universal way of getting a minimal installation. For packages that define
  default extras, installing ``package[]`` will always give a minimal
  installation even with older versions of packaging tools such as `pip`_, and
  releases of this package that pre-date the introduction of default extras for
  a specific package will also be installable with ``package[]`` (although in
  these cases this will be equivalent to ``package``). For packages that do not
  define default extras, ``package[]`` will continue to be equivalent to
  ``package``.

Avoiding the addition of many default dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

One temptation for authors might be to include many dependencies by default since
they can provide a way to opt out from these. We recommend however that authors
carefully consider what is included by default to avoid unnecessarily bloating
installations and complicating dependency trees. Using default extras does not
mean that all extras need to be defaults, and there is still scope for users to
explicitly opt in to non-default extras.

Default extras should generally be treated with the same "weight" as required
dependencies. When a package is widely used, introducing a default extra will
result in that extra's dependencies being transitively included -- unless all
downstream packages are updated to explicitly opt out using minimal installation
specifications.

As an example, the `pytest <https://docs.pytest.org/>`_ package currently has nearly 1,500 plugins that depend on it. If pytest were to add a default extra and those plugins were not updated accordingly, installing a plugin would include the default extras' dependencies. This doesn’t preclude the use of default extras, but addition of default extras requires careful evaluation of its downstream effects.

Inheriting from default extras
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If package authors choose to make an extra be installed by default, it is important
that they are aware that if users explicitly specify another extra, the default may
not be installed, unless they use the approach described in `Supporting extras
that should not remove default extras`_.

There are cases, such as the interchangeable backends,
where ignoring the default if an extra is explicitly specified is the right
thing to do. However, for other cases, such as using default extras to include
recommended dependencies while still providing a way to do minimal installs, it
may be that many of the other extras *should* explicitly 'inherit' the default
extra(s), so package authors should carefully consider in which cases they want
the default extras to be installed.

Incompatible extras
^^^^^^^^^^^^^^^^^^^

In some cases, it may be that packages have extras that are mutually
incompatible. In this case, we recommend against using the default extra
feature for any extra that contains a dependency that could be incompatible with
another.

Consider a package that has extras ``package[A]`` and ``package[B]``. Users
could already currently try and install ``package[A]`` and ``package[B]`` or
``package[A,B]`` which would result in a broken installation, however it would
at least be explicit that both extras were being installed. Making ``A`` into a
default extra however could lead to unintuitive issues. A user could do:

.. code-block:: console

    $ pip install package  # this installs package[A]
    $ pip install package[B]

and end up with a broken installation, even though A and B were never explicitly
both installed. For this reason, we recommend against using default extras
for dependencies where this is likely to be an issue.

Circular dependencies
^^^^^^^^^^^^^^^^^^^^^

Authors need to take special care when circular dependencies are present. For instance,
consider the following dependency tree::

    package1
    └── package2
        └── package1

If ``package1`` has a default extra named ``recommended`` then:

.. code-block:: console

    $ pip install package1[]

will still result in the ``recommended`` extra being installed if ``package2``
continues to depend on ``package1`` (with no extras specified). This could be
solved by changing the dependency tree to instead be::

    package1
    └── package2
        └── package1[]

assuming that indeed ``package2`` does not depend on any features provided by
the extra dependencies of ``package1``. Authors therefore need to carefully
consider a migration plan, coordinating with the authors of ``package2``.

Documenting packages with default extras
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Regardless of how default extras are used, package authors should aim to ensure
that their package's documentation makes it clear how extras are to be
used. 'Best practices' documentation should mention:

* that installing ``package`` will be equivalent to ``package[<default extras>]``
* that installing ``package[]`` will include only minimal/required dependencies,
  but that this will not guarantee that optional dependencies do not get installed
  if ``package`` appears anywhere else in the dependency tree
* what other optional extras are available, and whether or not they disable the
  default extras (since this can be controlled as described in `Supporting
  extras that should not remove default extras`_)
* any instructions specific to packages that might have e.g. default backends
  *and* frontends (as described in `Packages with multiple kinds of defaults`_)

Packaging repository maintainers
--------------------------------

The impact on individuals who repackage Python libraries for different
distributions, such as `conda <https://docs.conda.io>`_, `Homebrew
<https://brew.sh/>`_, linux package installers (such as ``apt`` and ``yum``) and
so on, needs to be considered. Not all package distributions have mechanisms
that would line up with the approach described. In fact, some distributions such
as conda, do not even have the concept of extras.

There are two cases to consider here:

* In cases where the repackaging is done by hand, such as for a number of conda-forge
  recipes, and especially where there is no equivalent to extras, the
  introduction of default extras should not have a large impact since manual
  decisions already have to be made as to which dependencies to include (for
  example, the conda-forge recipe for the `astropy`_ package mentioned in the
  `Motivation`_ includes all the ``recommended`` dependencies by default since
  there is no way for users to explicitly request them otherwise).

* In cases where the repackaging is done in an automated, way, distribution maintainers
  will need to carefully consider how to treat default extras, and this may
  imply a non-negligible amount of work and discussion.

It is impossible for a PEP such as this to exhaustively consider each of the
different package distributions. However, ultimately, default extras should be
understood as how package authors would like their package to be installed for
the majority of users, and this should inform decisions about how default extras
should be handled, whether manually or automatically.

Reference Implementation
========================

The following repository contains a fully functional demo package
that makes use of default extras:

https://github.com/wheel-next/pep_771

This makes use of modified branches of several packages, and the following
links are to these branches:

* `Setuptools <https://github.com/wheel-next/setuptools/tree/pep_771>`_
* `pip <https://github.com/wheel-next/pip/tree/pep_771>`_
* `importlib_metadata <https://github.com/wheel-next/importlib_metadata/tree/pep_771>`_

In addition, `this branch <https://github.com/astrofrog/flit/tree/default-extras-pep>`_
contains a modified version of the `Flit
<https://flit.pypa.io/en/stable/>`_ package.

The implementations above are proofs-of-concept at this time and the existing changes have
not yet been reviewed by the relevant maintainers. Nevertheless, they are
functional enough to allow for interested maintainers to try these out.

Rejected Ideas
==============

Using a meta-package for recommended installations
--------------------------------------------------

Using existing packaging tools and infrastructure, package maintainers who want
to provide a minimal installation for some users and a default non-minimal
installation for regular users (e.g. with recommended dependencies or a default
backend) can technically already achieve this if they are willing to distribute
two packages instead of one -- for example ``package-core`` which would be the main package
with minimal dependencies, and ``package`` which would be a metapackage that
would depend on ``package-core`` with optional dependencies enabled.

Taking once again a concrete example from the `Motivation`_
section, the `astropy`_ package defines a ``recommended`` extra that users are
currently instructed to install in the default installation instructions.
In principle, one could rename the existing ``astropy`` package to e.g. ``astropy-core``
and then create a new ``astropy`` package which would be a metapackage that would
contain the following dependencies section:

.. code-block:: toml

    dependencies = [
        "astropy-core[recommended]"
    ]

Since users may want to pin or place version constraints on the ``astropy``
meta-package (e.g. ``astropy>5.0``), the metapackage would need to follow
the same versions as the core package, and would need to use strict pinning
in the dependency section, e.g.:

.. code-block:: toml

    version = "7.1.0"
    dependencies = [
        "astropy-core[recommended]==7.1.0"
    ]

This idea may seem appealing because it is technically already feasible. However, in
practice, many projects have opted not to do this, for a number of reasons, which
we now take a look at. Some of these may not be applicable to future new projects,
but some of them apply to all projects, old and new.

Mismatch between package and module name
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In terms of naming, there are two main options for a package that wants to use the metapackage
approach:

* The first option is for the existing package to be kept as-is, which means
  that ``package`` would provide the minimal installation, and to then create a
  new metapackage with a different name, such as ``package-all``. However, this
  suffers from one of the problems that motivated this PEP in the first place -
  users are often not aware that they can do e.g. ``package[recommended]``, so
  in the same way, they might not realise that ``package-all`` exists. This once
  again places the burden on the average user to discover this, rather then
  shifting some of the burden to more advanced users.

* The second option is for the existing package to be renamed to e.g. ``package-core``, and
  for the new meta-package to be called ``package``. This is a better option
  than the first one, but is not ideal, as it then introduces a non-intuitive
  mismatch between the package name and module name, in that ``package-core`` provides
  the ``package`` module, and ``package`` does not provide any module. An example of why
  this would lead to confusion is that an average user might think that uninstalling
  the ``package`` module would be done with e.g.:

  .. code-block:: shell

      $ pip uninstall package

  but this would not be the case (the ``package`` module would still work), and
  it may not be obvious to this user that the ``package-core`` package even
  exists.

Multiple repositories or monorepos
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This approach requires either maintaining two repositories instead of one, or
switching to using a monorepo which would contain both packages. Neither option
is ideal:

* Splitting into two repositories places a long-term additional burden on
  maintainers, who have to make sure that these stay in sync (in terms of
  version but also other aspects such as extras, as will be discussed in
  `Synchronizing metadata`_). In addition, the naming issue mentioned in
  `Mismatch between package and module name`_ has additional complications here
  – either the names of the repositories match the packages, in which case any
  user who has a checkout of the previous ``package`` repository will need to
  update their remote URLs or any git clone URLs to point to the
  ``package-core`` repository. The alternative is to preserve the ``package``
  repository to contain the ``package-core`` package, and have a different name
  for the meta-package, but this could lead to confusion.

* Switching to a monorepo may be a significant change for some projects,
  and it is not uncommon for tools to assume by default that a single repository
  corresponds to a single package - while these can often be configured to then
  work with a monorepo, it is an additional burden on the maintainers. In
  addition, if the main package is moved to a sub-directory in the monorepo, any
  user that is e.g. pip installing the package from the repository URL will need
  to adjust this to install from a sub-directory (adding ``subdirectory=`` to
  the repo URL), and any existing workflows that clone the repository and assume
  the previous layout would break.

Depending on the minimal package
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Packages that need to depend on package versions that are older than the first
version where the split was done will not easily be able to depend on the
minimal package. Whereas with the main proposal in this PEP, downstream users
will be able to depend on e.g. ``package[]>version`` where ``version`` pre-dates
the introduction of default extras, with the splitting approach it will not be
possible for downstream users to depend on e.g. ``package-core>version``, since
``package-core`` did not previously exist.

A possible solution to this is for developers to release no-op metadata packages
for all old versions of a package, but this is a significant additional burden
on the maintainers.

Uninstallation
^^^^^^^^^^^^^^

As alluded to when referring to naming issues in `Mismatch between package and
module name`_, uninstalling packages will no longer work the way users expect. A
user doing:

.. code-block:: shell

    $ pip uninstall package

will still be left with ``package-core``, but may not realise it. This is
not just confusing, but is in effect a breaking change that may impact a number
of existing workflows.

Package distributions
^^^^^^^^^^^^^^^^^^^^^

Having two packages instead of one would increase the long-term maintenance cost
of package distributions simply by virtue of the fact that two packages would
have to be released instead of one, and in some cases this would introduce extra
manual work at each release.

Synchronizing metadata
^^^^^^^^^^^^^^^^^^^^^^

The main metadata that would be important to keep synchronized between the main
package and the metapackage is the version. Anytime a new release of the core
package is done, the metapackage would need to have its version updated as well
as the version pinning for the core package in the dependencies.

In addition, all extras defined in the core package would need to be redefined
and kept in sync in the metapackage. For example, if ``package`` defines a
``additional`` extra, users should still be able to install
``package[additional]``, but users installing the ``package-core`` package should
also have the option of doing ``package-core[additional]``.

Other metadata that would need to be kept in sync includes for example author
information and project URLs.

Summary
^^^^^^^

Overall, this solution would imply a significantly higher maintenance burden,
not just in terms of initial set-up and transition (which could already be
prohibitive for large established projects), but also in terms of long-term
maintenance. This also has the potential for breaking user workflows (in
particular around the issue of repositories, and e.g. uninstallation). For all
these reasons, we do not consider it a compelling alternative to the present PEP.

Syntax for deselecting extras
-----------------------------

One of the main competing approaches was as follows: instead of having defaults
be unselected if any extras were explicitly provided, default extras would need
to be explicitly unselected.

In this picture, a new syntax for unselecting extras would be introduced as an
extension of the mini-language defined in :pep:`508`. If a package defined
default extras, users could opt out of these defaults by using a minus sign
(``-``) before the extra name. The proposed syntax update would have been as follows::

    extras_list   = (-)?identifier (wsp* ',' wsp* (-)?identifier)*

Valid examples of this new syntax would have included, e.g.:

* ``package[-recommended]``
* ``package[-backend1, backend2]``
* ``package[pdf, -svg]``

However, there are two main issues with this approach:

* One would need to define a number of rules for how to interpret corner cases
  such as if an extra and its negated version were both present in the same
  dependency specification (e.g. ``package[pdf, -pdf]``) or if a dependency
  tree included both ``package[pdf]`` and ``package[-pdf]``, and the rules would
  not be intuitive to users.

* More critically, this would introduce new syntax into dependency specification,
  which means that if any package defined a dependency using the new syntax, it
  and any other package depending on it would no longer be installable by existing
  packaging tools, so this would be a major backward compatibility break.

For these reasons, this alternative was not included in the final proposal.

Adding a special entry in ``extras_require``
--------------------------------------------

A potential solution that has been explored as an alternative to introducing the
new ``Default-Extra`` metadata field would be to make use of an extra with a
'special' name.

One example would be to use an empty string::

    Provides-Extra:
    Requires-Dist: numpy ; extra == ''

The idea would be that dependencies installed as part of the 'empty' extras
would only get installed if another extra was not specified. An implementation
of this was proposed in https://github.com/pypa/setuptools/pull/1503, but it
was found that there would be no way to make this work without breaking
compatibility with existing usage. For example, packages using Setuptools via
a ``setup.py`` file can do::

    setup(
        ...
        extras_require={'': ['package_a']},
    )

which is valid and equivalent to having ``package_a`` being defined in
``install_requires``, so changing the meaning of the empty string would
break compatibility.

In addition, no other string (such as ``'default'``) can be used as a special
string since all strings that would be a backward-compatible valid extras name
may already be used in existing packages.

There have been suggestions of using the special ``None`` Python variable, but
again this is not possible, because even though one can use ``None`` in a ``setup.py`` file,
this is not possible in declarative files such as ``setup.cfg`` or
``pyproject.toml``, and furthermore ultimately extras names have to be converted
to strings in the package metadata. Having::

    Provides-Extra: None

would be indistinguishable from the string 'None' which may already be used as
an extra name in a Python package. If we were to modify the core metadata
syntax to allow non-string 'special' extras names, then we would be back to
modifying the core metadata specification, which is no better than
introducing ``Default-Extra``.

Relying on tooling to deselect any default extras
-------------------------------------------------

Another option to unselect extras would be to implement this at the
level of packaging tools. For instance, pip could include an option such as:

.. code-block:: console

    $ pip install package --no-default-extras

This option could apply to all or specific packages, similar to
the ``--no-binary`` option, e.g.,:

.. code-block:: console

    $ pip install package --no-default-extras :all:

The advantage of this approach is that tools supporting default extras could
also support unselecting them. This approach would be similar to the ``--no-install-recommends``
option for the ``apt`` tool.

However, this solution is not ideal on its own because it would not allow packages to
specify themselves that they do not need some of the default extras of a
dependency. It would also carry risks for users who might disable all default
extras in a big dependency tree, potentially breaking packages in the tree that
rely on default extras at any point.

Nevertheless, this PEP does not disallow this approach and it is up to the
maintainers of different packaging tools to decide if they want to support this
kind of option. It is a flag that could at the very least be useful for package
maintainers who want to identify places in dependency trees where default extras
are being relied on. However, if it is supported, it should be made clear that
using this flag does not guarantee a functional environment.

Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.
