Some recent cuda commit broke the enzyme.jl tests?
bugenzyme
Not sure what since CI has been red (and seemingly ignored)?
@maleadt @vchuravy
```
Test (Worker) │ time (s) │ GC (s) │ Alloc (MB) │ RSS (MB) │ GC (s) │ GC % │ Alloc (MB) │ RSS (MB) │
--
| extensions/enzyme (1) │ 851.38 │ 0.01 │ 0.00 │ 116.00 │ 18.68 │ 2.2 │ 106925.54 │ 4345.29 │
|
| Test Summary: \| Pass Error Total Time
| Overall \| 17 3 20 14m41.5s
| extensions/enzyme \| 17 3 20 14m11.4s
| compiler_job_from_backend \| 1 1 0.8s
| Make_zero \| 2 2 24.8s
| Forward Kernel \| 1 1 1m40.3s
| Reverse Kernel \| 3 3 6m39.7s
| Forward Fill! \| 2 2 21.7s
| Reverse Fill! \| 3 3 40.5s
| Forward allocate \| 1 1 1.9s
| Reverse allocate \| 1 1 3.8s
| Forward broadcast \| 0 0.2s
| Forward sum \| 1 1 36.2s
| Reverse sum \| 1 1 28.8s
| Forward setadd \| 1 1 1m00.4s
| setadd sum \| 2 2 44.0s
| Reverse sum abs2 \| 1 1 44.6s
| FAILURE
|
| Error in testset Forward Kernel:
| Error During Test at /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:32
| Got exception outside of a @test
| MethodError: no method matching forward(::EnzymeCore.EnzymeRules.FwdConfigWidth{1, true, false, false, false}, ::EnzymeCore.Const{typeof(cufunction)}, ::Type{EnzymeCore.Const{CUDACore.HostKernel{typeof(Main.var"##extensions/enzyme#241".square_kernel!), Tuple{CuDeviceVector{Float32, 1}}}}}, ::EnzymeCore.Const{typeof(Main.var"##extensions/enzyme#241".square_kernel!)}, ::EnzymeCore.Const{Type{Tuple{CuDeviceVector{Float32, 1}}}})
| The function `forward` exists, but no method is defined for this combination of argument types.
|
| Closest candidates are:
| forward(::EnzymeCore.EnzymeRules.FwdConfig, ::EnzymeCore.Const{typeof(hypot)}, ::Any, ::EnzymeCore.Annotation, ::EnzymeCore.Annotation, ::EnzymeCore.Annotation, ::EnzymeCore.Annotation...) where N
| @ Enzyme ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/internal_rules/math.jl:6
| forward(::Any, ::EnzymeCore.Const{typeof(finalizer)}, ::Any, ::EnzymeCore.Const, ::Any)
| @ Enzyme ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/internal_rules/core.jl:408
| forward(::Any, ::EnzymeCore.Const{typeof(cufunction)}, ::Type{<:EnzymeCore.Duplicated}, ::EnzymeCore.Const{F}, ::EnzymeCore.Const{TT}; kwargs...) where {F, TT}
| @ EnzymeCoreExt /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/ext/EnzymeCoreExt.jl:39
| ...
|
| Stacktrace:
| [1] custom_rule_method_error
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/rules/customrules.jl:976 [inlined]
| [2] kernel_compile
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/compiler/execution.jl:60 [inlined]
| [3] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/compiler/execution.jl:183 [inlined]
| [4] square!
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:28 [inlined]
| [5] fwddiffejulia_square__58054wrap
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:0
| [6] macro expansion
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6856 [inlined]
| [7] enzyme_call
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6326 [inlined]
| [8] ForwardModeThunk
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6226 [inlined]
| [9] autodiff
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/Enzyme.jl:688 [inlined]
| [10] autodiff
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/Enzyme.jl:577 [inlined]
| [11] autodiff(mode::EnzymeCore.ForwardMode{false, EnzymeCore.FFIABI, false, false, false}, f::typeof(Main.var"##extensions/enzyme#241".square!), args::EnzymeCore.Duplicated{CuArray{Float32, 1, CUDACore.DeviceMemory}})
| @ Enzyme ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/Enzyme.jl:549
| [12] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:37 [inlined]
| [13] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [14] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:33
| [15] include(mod::Module, _path::String)
| @ Base ./Base.jl:562
| [16] include
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/packages/ParallelTestRunner/kZFbj/src/ParallelTestRunner.jl:420 [inlined]
| [17] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [18] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [19] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [20] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [21] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:104 [inlined]
| [22] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/utilities.jl:35 [inlined]
| [23] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/memory.jl:906 [inlined]
| [24] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:103
| Error in testset Forward allocate:
| Error During Test at /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:89
| Got exception outside of a @test
| UndefVarError: `Mem` not defined in `CUDA`
| Suggestion: check for spelling errors or missing imports.
| Stacktrace:
| [1] alloc
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:87 [inlined]
| [2] fwddiffejulia_alloc_202532wrap
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:0
| [3] macro expansion
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6856 [inlined]
| [4] enzyme_call
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6326 [inlined]
| [5] ForwardModeThunk
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6226 [inlined]
| [6] autodiff
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/Enzyme.jl:688 [inlined]
| [7] autodiff(mode::EnzymeCore.ForwardMode{true, EnzymeCore.FFIABI, false, false, false}, f::typeof(Main.var"##extensions/enzyme#241".alloc), ::Type{EnzymeCore.Duplicated}, args::EnzymeCore.Const{Int64})
| @ Enzyme ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/Enzyme.jl:557
| [8] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:90 [inlined]
| [9] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [10] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:90
| [11] include(mod::Module, _path::String)
| @ Base ./Base.jl:562
| [12] include
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/packages/ParallelTestRunner/kZFbj/src/ParallelTestRunner.jl:420 [inlined]
| [13] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [14] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [15] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [16] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [17] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:104 [inlined]
| [18] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/utilities.jl:35 [inlined]
| [19] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/memory.jl:906 [inlined]
| [20] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:103
| Error in testset Reverse allocate:
| Error During Test at /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:97
| Got exception outside of a @test
| UndefVarError: `Mem` not defined in `CUDA`
| Suggestion: check for spelling errors or missing imports.
| Stacktrace:
| [1] alloc
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:87 [inlined]
| [2] augmented_julia_alloc_204053wrap
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:0
| [3] macro expansion
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6856 [inlined]
| [4] enzyme_call
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6326 [inlined]
| [5] (::Enzyme.Compiler.AugmentedForwardThunk{Ptr{Nothing}, EnzymeCore.Const{typeof(Main.var"##extensions/enzyme#241".alloc)}, EnzymeCore.Duplicated{CuArray{Float32, 1}}, Tuple{EnzymeCore.Const{Int64}}, 1, true, @NamedTuple{1::Core.LLVMPtr{Tuple{}, 0}, 2::Core.LLVMPtr{Tuple{}, 0}, 3::Core.LLVMPtr{Tuple{}, 0}, 4, 5, 6, 7, 8, 9, 10::Core.LLVMPtr{Tuple{}, 0}, 11}})(fn::EnzymeCore.Const{typeof(Main.var"##extensions/enzyme#241".alloc)}, args::EnzymeCore.Const{Int64})
| @ Enzyme.Compiler ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/dev/Enzyme/src/compiler.jl:6274
| [6] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:99 [inlined]
| [7] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [8] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/extensions/enzyme.jl:98
| [9] include(mod::Module, _path::String)
| @ Base ./Base.jl:562
| [10] include
| @ ~/.cache/julia-buildkite-plugin/depots/3cc01fab-3357-4a7a-9294-cde2d3115a97/packages/ParallelTestRunner/kZFbj/src/ParallelTestRunner.jl:420 [inlined]
| [11] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [12] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [13] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:105 [inlined]
| [14] macro expansion
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Test/src/Test.jl:1709 [inlined]
| [15] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:104 [inlined]
| [16] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/utilities.jl:35 [inlined]
| [17] macro expansion
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/CUDACore/src/memory.jl:906 [inlined]
| [18] top-level scope
| @ /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/setup.jl:103
| ERROR: LoadError: Test run finished with errors
| in expression starting at /var/lib/buildkite-agent/builds/gpuci-1/julialang/cuda-dot-jl/test/runtests.jl:223
| ERROR: Package CUDA errored during testing
| Stacktrace:
| [1] pkgerror(msg::String)
| @ Pkg.Types ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/Types.jl:68
| [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool)
| @ Pkg.Operations ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2128
| [3] test
| @ ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/Operations.jl:2011 [inlined]
| [4] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::@Kwargs{io::IOContext{IO}})
| @ Pkg.API ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/API.jl:481
| [5] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{IO}, kwargs::@Kwargs{coverage::Bool, test_args::Cmd})
| @ Pkg.API ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/API.jl:159
| [6] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::@Kwargs{coverage::Bool, test_args::Cmd})
| @ Pkg.API ~/.cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.11/julia-1.11-latest-linux-x86_64/share/julia/stdlib/v1.11/Pkg/src/API.jl:174
| [7] top-level scope
| @ none:27
| 🚨 Error: The command exited with status 1
```
7 条评论