.. Copyright (c) 2024 Lawrence Livermore National Security, LLC and other
   HYPRE Project Developers. See the top-level COPYRIGHT file for details.

   SPDX-License-Identifier: (MIT)

.. _PythonInterface:

Python Interface
================

The Python interface provides thin bindings to the `hypredrive` C library. It accepts solver
options as Python dictionaries, YAML strings, or YAML files. The interface assembles sparse
matrices from compressed sparse row (CSR) data or SciPy CSR matrices. It runs the solve
lifecycle and returns the local solution as a NumPy array.

libHYPREDRV and HYPRE perform the solver operations. The Python layer handles input
normalization, data type checks, YAML conversion, and result extraction.

Prerequisites
-------------

- Python 3.9 or newer.
- NumPy.
- Cython 3.0 or newer when building from source.
- An MPI implementation with compiler wrappers available at build time.
- ``mpi4py`` for distributed solves from Python.
- SciPy when passing ``scipy.sparse.csr_matrix`` objects directly.

Installation
------------

There are three install modes:

- GitHub Actions wheel artifacts for host-only installation with a compatible MPI run time.
- Source installation with an installed or build-tree ``libHYPREDRV``.
- In-tree CMake builds for developers and continuous integration (CI).

Source builds keep Python packaging independent from ordinary C builds while still
linking against libHYPREDRV.

Source install:

.. code-block:: bash

   python -m pip install ./interfaces/python

When run from a full hypredrive checkout, this automatically builds and bundles
the in-tree HYPREDRV/HYPRE libraries. When run from a standalone source
distribution, CMake falls back to finding an installed MPI-enabled
HYPREDRV/HYPRE stack. Binary MPI wheels are currently GitHub Actions artifacts,
not PyPI or TestPyPI packages.

Against an installed hypredrive:

.. code-block:: bash

   cmake --install build --prefix $HOME/opt/hypredrive
   pip install ./interfaces/python \
     --config-settings=cmake.define.HYPREDRV_PYTHON_BUNDLE_CORE=OFF \
     --config-settings=cmake.define.CMAKE_PREFIX_PATH=$HOME/opt/hypredrive

Use Python source distributions for downstream packages and developer
environments where CMake can find ``HYPREDRV`` and ``HYPRE``. These distributions
require the C library stack.

Against an in-tree development build:

.. code-block:: bash

   cmake -S . -B build -DBUILD_SHARED_LIBS=ON -DHYPREDRV_ENABLE_TESTING=OFF
   cmake --build build --parallel
   pip install -e ./interfaces/python \
     --config-settings=cmake.define.HYPREDRV_DIR=$PWD/build

The top-level CMake build can also build the Python extension directly:

.. code-block:: bash

   cmake -S . -B build -DHYPREDRV_ENABLE_PYTHON=ON
   cmake --build build --target _core --parallel

Use this mode for developer and CI builds. It requires Python, NumPy, and Cython
during configuration and build. Thus, CMake disables ``HYPREDRV_ENABLE_PYTHON``
by default.

Experimental wheel artifacts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The project CI can build experimental Python wheel artifacts for Linux and macOS.
These wheels bundle host-only ``libHYPREDRV`` and ``libHYPRE`` inside the Python
package, but they do not bundle MPI.

On pull requests, the wheel workflow runs only when the PR has the
``Run Python Wheels`` label. It can also be started manually with
``workflow_dispatch``.

1. Download a wheel artifact from a GitHub Actions ``Python Wheels`` workflow run.

2. Extract the wheel from the zip file.

3. Install the wheel in a virtual environment that has a compatible MPI run time:

.. code-block:: bash

   python -m venv .venv
   source .venv/bin/activate

   unzip hypredrive-wheels-*.zip -d wheelhouse
   python -m pip install wheelhouse/hypredrive-*.whl

CI builds wheel artifacts for each MPI flavor. Use an ``mpich`` wheel with an
MPICH-compatible runtime and an ``openmpi`` wheel with an OpenMPI-compatible
runtime.

Use a source install instead for custom HYPRE builds, GPU support,
BIGINT/MIXEDINT, vendor MPI stacks, or downstream-packager control over shared
libraries.

At runtime, the package records how it was built:

.. code-block:: python

   import hypredrive as hd
   print(hd.BUILD_INFO)

Install optional dependencies through package extras:

.. code-block:: bash

   pip install ./interfaces/python[mpi]
   pip install ./interfaces/python[scipy]
   pip install -e ./interfaces/python[test]

``mpi4py`` is optional for package import and serial solves, but it is the
supported MPI boundary for distributed Python use. HypreDrive accepts
``mpi4py.MPI.Comm`` objects and forwards the underlying communicator to the C
library instead of exposing its own Python MPI wrapper.

Quick start
-----------

One-shot solve
~~~~~~~~~~~~~~

.. code-block:: python

   import numpy as np
   import scipy.sparse as sp
   import hypredrive as hd

   n = 64
   diag_main = 2.0 * np.ones(n)
   diag_off = -np.ones(n - 1)
   A = sp.diags([diag_off, diag_main, diag_off], [-1, 0, 1], format="csr")
   b = np.ones(n)

   options = hd.configure(
       solver="pcg",
       preconditioner="amg",
       pcg={"max_iter": 100, "relative_tol": 1.0e-8},
       amg={"print_level": 0},
   )

   result = hd.solve(A, b, options=options)

   print("solution norm:", result.solution_norm)
   print("first entries:", result.x[:5])

Reusable driver
~~~~~~~~~~~~~~~

Use ``HypreDrive`` when one process needs to solve multiple related systems and reuse the
driver lifecycle.

.. code-block:: python

   import hypredrive as hd

   with hd.HypreDrive(options="my_config.yaml") as drv:
       for step in range(num_steps):
           drv.set_matrix_from_csr(build_matrix(step))
           drv.set_rhs(build_rhs(step))
           drv.solve()
           x = drv.get_solution()

Both ``HypreDrive`` and ``hd.solve`` accept an ``input_args`` keyword with CLI override
tokens. These tokens override ``options``. For example, use
``input_args=["--solver:pcg:max_iter", "100"]``. The bundled examples accept these tokens
after ``-a`` or ``--args``.

Distributed solve with MPI
~~~~~~~~~~~~~~~~~~~~~~~~~~

With ``mpi4py``, each rank provides its local CSR slab. Row bounds are global and
inclusive. Column indices are global.

.. code-block:: python

   from mpi4py import MPI
   import hypredrive as hd

   comm = MPI.COMM_WORLD
   rank = comm.Get_rank()
   size = comm.Get_size()

   indptr, col_indices, data, rhs, row_start, row_end = build_local_slab(rank, size)

   with hd.HypreDrive(options=opts, comm=comm) as drv:
       drv.set_matrix_from_csr(
           indptr,
           col_indices,
           data,
           row_start=row_start,
           row_end=row_end,
       )
       drv.set_rhs(rhs)
       drv.solve()
       x_local = drv.get_solution()

CSR input details
-----------------

``set_matrix_from_csr`` accepts a SciPy CSR matrix or raw
``(indptr, col_indices, data)`` arrays. Raw arrays require ``row_start`` and
``row_end``. These values define the inclusive global rows that the rank owns.
``indptr`` has the length ``nrows + 1``. Here,
``nrows = row_end - row_start + 1``.

``col_indices`` contains global column indices.
HYPRE copies the data during IJ assembly. Thus, you can release all input buffers after
the call returns.

A SciPy sparse matrix without an explicit row range selects single-rank, full-local
assembly. The interface uses CSR input directly. It first converts other SciPy sparse
formats to CSR. With ``row_start`` and ``row_end``, the matrix is the local rank slab.
Set its row count to ``row_end - row_start + 1``.

The CSR array API does not support empty local row ranges. Each participating
MPI rank owns at least one row.

Configuration input
-------------------

For the ``options`` argument, pass one of these values:

.. list-table::
   :header-rows: 1

   * - Input
     - Behavior
   * - ``dict``
     - Converted to YAML in memory and parsed by hypredrive.
   * - ``str`` containing a newline
     - Treated as a YAML document.
   * - ``str`` or ``pathlib.Path`` naming an existing file
     - The binding loads and parses the file contents.
   * - ``None``
     - Uses the Python binding's minimal default YAML.

The Python interface and CLI accept the same YAML keys and solver options. See
:ref:`InputFileStructure`.

Testing
-------

After installing the package in editable mode with test dependencies:

.. code-block:: bash

   pip install -e ./interfaces/python[test]
   python -m pytest interfaces/python/tests/test_solve_serial.py -v

Run MPI tests under an MPI process manager:

.. code-block:: bash

   mpirun -np 2 python -m pytest \
     interfaces/python/tests/test_solve_mpi.py \
     interfaces/python/tests/laplacian/test_example_mpi.py -v

Tests that require the native extension or ``mpi4py`` skip when those optional runtime
components are unavailable.

Current limitations
-------------------

- The Python interface currently targets real-valued solves.
- The binding copies solution data back to host NumPy arrays.
- GPU/device execution is not exposed as a Python-native data path.
- The one-shot API exposes only the solution array and solution norm.
- Distributed Python solves require ``mpi4py`` and use ``Comm.py2f()`` plus the
  C-side ``MPI_Comm_f2c`` bridge.
- Python examples are in ``interfaces/python/examples``. ``laplacian/laplacian.py`` is the
  MPI-capable 3D example. ``laplacian/laplacian2d_seq.py`` is the serial 2D example.
  ``darcy/darcy_mixed.py`` is the mixed Darcy example.
