# CUDA.jl

*CUDA programming in Julia*

[![][doi-img]][doi-url] [![][docs-stable-img]][docs-stable-url] [![][docs-dev-img]][docs-dev-url] [![][buildkite-img]][buildkite-url] [![][codecov-img]][codecov-url] [![][benchmark-img]][benchmark-url]

[doi-img]: https://zenodo.org/badge/doi/10.1109/TPDS.2018.2872064.svg
[doi-url]: https://ieeexplore.ieee.org/abstract/document/8471188

[docs-stable-img]: https://img.shields.io/badge/docs-stable-blue.svg
[docs-stable-url]: https://cuda.juliagpu.org/stable/

[docs-dev-img]: https://img.shields.io/badge/docs-dev-blue.svg
[docs-dev-url]: https://cuda.juliagpu.org/dev/

[buildkite-img]: https://badge.buildkite.com/27aaeb352a9420297ed2d30cb055ac383a399ea8f121599912.svg?branch=main
[buildkite-url]: https://buildkite.com/julialang/cuda-dot-jl

[codecov-img]: https://codecov.io/gh/JuliaGPU/CUDA.jl/branch/main/graph/badge.svg
[codecov-url]: https://codecov.io/gh/JuliaGPU/CUDA.jl

[benchmark-img]: https://img.shields.io/badge/benchmarks-Chart-yellowgreen
[benchmark-url]: https://cuda.juliagpu.org/bench/

The CUDA.jl package is the main programming interface for working with NVIDIA CUDA GPUs
using Julia. It features a user-friendly array abstraction, a compiler for writing CUDA
kernels in Julia, and wrappers for various CUDA libraries.


## Quick start

Before all, make sure you have a recent NVIDIA driver. On Windows, also make sure you have
the [Visual C++ redistributable](https://aka.ms/vs/16/release/vc_redist.x64.exe) installed.
You do not need to install the CUDA Toolkit.

CUDA.jl can be installed with the Julia package manager. From the Julia REPL, type `]` to
enter the Pkg REPL mode and run:

```
pkg> add CUDA
```

Or, equivalently, via the `Pkg` API:

```julia
julia> import Pkg; Pkg.add("CUDA")
```

For an overview of the CUDA toolchain in use, you can run the following command after
importing the package:

```julia
julia> using CUDA

julia> CUDA.versioninfo()
```

This may take a while, as it will precompile the package and download a suitable version of
the CUDA toolkit. If your GPU is not fully supported, the above command (or any other
command that initializes the toolkit) will issue a warning.

For quick testing, you can also use [the `juliagpu/cuda.jl` container
image](https://github.com/JuliaGPU/CUDA.jl/pkgs/container/cuda.jl/versions) from the GitHub
Container Registry, which provides Julia, a precompiled version of CUDA.jl, and a matching
CUDA toolkit. Tags are named after the CUDA major version (`cuda12` or `cuda13`); the
bare tag tracks the latest stable release, while `<version>-cuda<major>` pins to a
specific release and `dev-cuda<major>` tracks the latest `main` build:

```sh
# latest stable release, CUDA 13
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:cuda13

# a specific release, CUDA 12
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:v6.1.0-cuda12

# latest main build, CUDA 13
docker run -it --rm --gpus=all ghcr.io/juliagpu/cuda.jl:dev-cuda13
```

For more usage instructions and other information, please refer to [the
documentation](https://juliagpu.github.io/CUDA.jl/stable/).


## Organization

From version 6.0 on, CUDA.jl is a meta-package the imports the following
registered subpackages:
- CUDACore: the core package with the main CUDA API wrappers and the compiler for writing CUDA
  kernels in Julia;
- CUDATools: developer tools including the profiler, CUPTI, NVML, and code reflection
  utilities;
- cuBLAS: wrappers for the cuBLAS library;
- cuFFT: wrappers for the cuFFT library;
- cuRAND: wrappers for the cuRAND library;
- cuSPARSE: wrappers for the cuSPARSE library;
- cuSOLVER: wrappers for the cuSOLVER library.

Other registered subpackages that are not imported by default include:
- cuTENSOR: wrappers for the cuTENSOR library;
- cuStateVec: wrappers for the cuStateVec library;
- cuTensorNet: wrappers for the cuTensorNet library;
- CUDNN: wrappers for the cuDNN library.

While importing CUDA.jl will remain the recommended way to use these packages,
you can also import them separately in order to optimize precompilation or load
times, or (in the future) reduce the size of the required artifacts.

CUDA.jl and its subpackages are are always released together with version
numbers that move in lockstep.


## Requirements

The latest development version of CUDA.jl requires **Julia 1.10** or higher. If you are
using an older version of Julia, you need to use a previous version of CUDA.jl. This will
happen automatically when you install the package using Julia's package manager.

Note that CUDA.jl may not work with a custom build of Julia; it is recommended that you
install Julia using the [official binaries](https://julialang.org/downloads/) or
[juliaup](https://github.com/JuliaLang/juliaup).

The latest version of CUDA.jl also has certain requirements that cannot be enforced by the
package manager:

- Host platform: only 64-bit Linux and Windows are supported;
- Device hardware: only NVIDIA GPUs with **compute capability 5.0** (Maxwell) or higher are
  supported;
- NVIDIA driver: a driver for **CUDA 12** or newer is required;
- CUDA toolkit (in case you need to use your own): only **CUDA toolkit 12.0** or newer are
  supported.

If you cannot meet these requirements, you may need to install an older version of CUDA.jl:

* CUDA.jl v5.8 is the last version with support for CUDA 11, and consequently Kepler GPUs (removed in v5.9)
* CUDA.jl v5.3 is the last version with support for PowerPC (removed in v5.4)
* CUDA.jl v4.4 is the last version with support for CUDA 11.0-11.3 (deprecated in v5.0)
* CUDA.jl v4.0 is the last version to work with CUDA 10.2 (removed in v4.1)
* CUDA.jl v3.8 is the last version to work with CUDA 10.1 (removed in v3.9)
* CUDA.jl v1.3 is the last version to work with CUDA 9-10.0 (removed in v2.0)


## Selecting a CUDA Toolkit

CUDA.jl will automatically download and use a CUDA Toolkit that's supported
by your NVIDIA driver as well as most of the devices in your system. There are
two cases where you may want to select a different version:

- **You have a GPU that is not supported by the latest CUDA Toolkit**.
  Although CUDA.jl tries to select a runtime that supports most of your GPUs,
  specific devices may end up being unsupported by the active toolkit.
  In that case, call e.g. `CUDA.set_runtime_version!(v"12.9")` to
  use a specific toolkit which still supports the device in question.
- **You want to use a CUDA Toolkit that is already installed on your system**.
  This is not recommended, but if you want to do it anyway, call
  `CUDA.set_runtime_version!(local_toolkit=true)`.

Both these options will be remembered across sessions via a local preference,
which can also be set directly (see `LocalPreferences.toml`).


## Supporting and Citing

Much of the software in this ecosystem was developed as part of academic research. If you
would like to help support it, please star the repository as such metrics may help us secure
funding in the future. If you use our software as part of your research, teaching, or other
activities, we would be grateful if you could cite our work. The
[CITATION.bib](https://github.com/JuliaGPU/CUDA.jl/blob/main/CITATION.bib) file in the
root of this repository lists the relevant papers.


## Project Status

The package is tested against, and being developed for, Julia 1.10 and above. Main
development and testing happens on x86 Linux, but the package is expected to work on
Windows and ARM and as well.


## Questions and Contributions

Usage questions can be posted on the [Julia Discourse
forum](https://discourse.julialang.org/c/domain/gpu) under the GPU domain and/or in the #gpu
channel of the [Julia Slack](https://julialang.org/community/).

Contributions are very welcome, as are feature requests and suggestions. Please open an
[issue](https://github.com/JuliaGPU/CUDA.jl/issues) if you encounter any problems.

### Note for contributors about CI usage

You can pass special flags to Buildkite to avoid overburdening the GPU CI agents, which
are a shared resource. The flag should be added to your top level commit message (**not**
the extended message).

Available flags are:
  - `[only julia]`: test only on the various Julia versions CUDA.jl supports, skipping testing subpackages and the various CUDA toolkits
  - `[only tests]`: don't run the benchmark or documentation steps
  - `[only subpackages]`: skip all the Julia and CUDA toolkit version runs, only test `cuTENSOR`, `cuStateVec`, `cuTensorNet`, and `CUDNN`
  - `[only docs]`: only build the documentation, skip all tests and benchmarks
  - `[only special]`: only test the special cases of multi-GPU, the various default memory locations, compute sanitizer, and the GPU-less environment
  - `[only downstream]`: test only the "downstream" packages which depend on CUDA.jl and its libraries, such as Enzyme.jl
  - `[skip downstream]`: don't the "downstream" packages
  - `[skip special]`: don't run the above described special tests
  - `[skip tests]`: skip all tests, only run benchmarks and documentation
  - `[skip benchmarks]`: don't run the benchmarks

If you use these flags, make sure you don't exclude tests your changes could actually impact.
