KubeRay presubmits can test a stale raymini image after a Ray version bump

**What happened**: The raymini image tag does not move when the Ray version it contains does, so a Ray bump is tested against the previously published image and the resulting failures land on unrelated PRs a day later. `RAY_VERSION` is read off the `FROM` line of `hack/testing/ray/Dockerfile`, but the tag raymini is published under is a separate constant: https://github.com/kubernetes-sigs/kueue/blob/6431b80617c4f5a4efee9c51e286ca749f9a2586/Makefile#L62-L63 The raymini build takes the first as a build arg, feeds it to `hack/testing/ray-mini/Dockerfile`, and publishes under the second: https://github.com/kubernetes-sigs/kueue/blob/6431b80617c4f5a4efee9c51e286ca749f9a2586/Makefile#L576-L586 and the e2e setup only builds locally when that tag is absent from the registry: https://github.com/kubernetes-sigs/kueue/blob/6431b80617c4f5a4efee9c51e286ca749f9a2586/hack/testing/e2e-common.sh#L674-L681 raymini is not published by `image-pushing-postsubmit`. It is published by `image-pushing-periodic`, which rebuilds from main and writes the same two tags again: https://github.com/kubernetes-sigs/kueue/blob/6431b80617c4f5a4efee9c51e286ca749f9a2586/Makefile#L264-L270 The registry shows the tag being reused. `ray-project-mini:0.0.4` currently resolves to `sha256:fd32a471aa130e6200a33abb45787eff924d146cbe185b0197984d79b5f89281`, uploaded 2026-08-15 11:16 UTC, and its layer history is: ``` ARG RAY_VERSION=2.57.0 RUN ... pip3 install --no-cache-dir ray[default,serve]==${RAY_VERSION} ... ``` The index published 24 hours earlier, `sha256:6ad299992df9036aa...` built 2026-08-14T11:09:21Z, carries `RAY_VERSION=2.56.1` in the same layer. Same tag, different Ray. #14463 changed the Dockerfile from Ray 2.56.1 to 2.57.0 and merged at 2026-08-14 19:16 UTC. Its own KubeRay presubmits were green, which is what the code above predicts: `0.0.4` already existed, so the run pulled the 2.56.1 image rather than building the 2.57.0 one the PR was introducing. After the periodic rebuilt the tag on 08-15 at 11:16 UTC, the same seven KubeRay jobs started failing on unrelated PRs. On #14421 they were green at 08-14 16:53 UTC and failed at 08-15 15:40 UTC with no change to the KubeRay path. #14495, #14505 and #13846 show the same seven, and the three failing specs are the same in each: `Should run a RayService if admitted` on shard 0, `Should run a rayservice with InTreeAutoscaling` on shard 2, and `Should run a RayService on worker if admitted` on the MultiKueue shard. None of those four PRs touches the KubeRay path. **What you expected to happen**: A PR that changes the Ray version builds and tests raymini for that version. A published raymini tag keeps the contents it was published with, so a KubeRay failure is attributed to the change that introduced it. **How to reproduce it (as minimally and precisely as possible)**: ``` $ crane manifest us-central1-docker.pkg.dev/k8s-staging-images/kueue/ray-project-mini:0.0.4 ``` Read the Ray version out of the config blob's layer history, then compare against `hack/testing/ray/Dockerfile` at the commit under test. They are independent. **Anything else we need to know?**: One shape that would fix it is to put the Ray version in the tag and keep a separate revision for changes to the recipe itself: ```make RAYMINI_BUILD_VERSION ?= 0.0.4 RAYMINI_VERSION ?= $(RAY_VERSION)-$(RAYMINI_BUILD_VERSION) ``` A Ray bump then names a tag that does not exist yet, the existing fallback in `e2e-common.sh` builds it locally, and the bump PR tests what it is actually changing. The periodic publishes the version-specific tag afterwards instead of overwriting one that older Ray versions are still being tested against. `RAYMINI_VERSION` stays overridable. I have not sent a patch for this. I would rather agree the shape first, since the tag is also referenced from the periodic and from `hack/make/test.mk`, and someone closer to the image plumbing may prefer a digest pin or a per-commit tag instead. This is separate from whatever Ray 2.57.0 does at runtime. I have not looked at the RayService failures themselves yet, and with version-specific tags that question would at least be reproducible on the PR that raises the version. **Environment**: - Kubernetes version (use `kubectl version`): n/a, this is about CI image provenance - Kueue version (use `git describe --tags --dirty --always`): main at 6431b80 - Cloud provider or hardware configuration: n/a - OS (e.g: `cat /etc/os-release`): n/a - Kernel (e.g. `uname -a`): n/a - Install tools: n/a - Others: n/a --- This issue was written in part with the assistance of generative AI. I verified the source references and the registry timestamps myself.