Rootless Quadlet network cleanup fails during systemd user manager shutdown
kind/bugnetworktriaged
### Issue Description
During an orderly reboot, the generated service for a rootless Quadlet container attached to a custom bridge network exits with status 125. The systemd user manager stops Podman's pasta and pause scopes while the generated service is still cleaning up. Teardown sees that pasta has stopped and tries to recreate its scope, which systemd rejects because the exit transaction is already queued. Netavark then tries to start Aardvark DNS, but that scope is also rejected and cleanup fails. Explicitly running `systemctl --user stop repro.service` succeeds with status zero, and the service recovers on the next boot.
### Steps to reproduce the issue
1. On a disposable systemd host with persistent journald storage, install Podman 6.0.2 with netavark 2.0.0, Aardvark DNS 2.0.0, and pasta. Create a rootless user with subordinate UID and GID ranges, then enable lingering for it:
```console
sudo loginctl enable-linger podman-repro
```
2. In a normal login session as `podman-repro`, pull the image and create the Quadlet directory:
```console
podman pull quay.io/libpod/alpine@sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f
mkdir -p ~/.config/containers/systemd
```
3. Create `~/.config/containers/systemd/repro.network`:
```systemd
[Network]
NetworkName=quadlet-shutdown-repro
Driver=bridge
GlobalArgs=--log-level=debug
```
4. Create `~/.config/containers/systemd/repro.container`:
```systemd
[Unit]
Description=Rootless Quadlet shutdown reproducer
[Container]
ContainerName=quadlet-shutdown-repro
Image=quay.io/libpod/alpine@sha256:fa93b01658e3a5a1686dc3ae55f170d8de487006fb53a28efcd12ab0710a2e5f
Network=repro.network
Exec=/bin/sh -c 'trap "exit 0" TERM INT; while :; do sleep 1; done'
Pull=never
GlobalArgs=--log-level=debug
[Install]
WantedBy=default.target
```
5. Reload the user manager, start the generated service, and confirm that it is active with an empty `PartOf` property:
```console
systemctl --user daemon-reload
systemctl --user start repro.service
systemctl --user show repro.service --property=ActiveState --property=PartOf
```
6. As a control, stop the generated service explicitly, check its result, and start it again:
```console
systemctl --user stop repro.service
systemctl --user show repro.service --property=ActiveState --property=Result --property=ExecMainStatus
systemctl --user start repro.service
```
7. Reboot without first stopping `repro.service`:
```console
sudo systemctl reboot
```
8. After the next boot, inspect the previous boot's journal for `podman-repro`:
```console
sudo journalctl --boot=-1 "_UID=$(id -u podman-repro)" --output=short-monotonic --no-hostname --no-pager
```
### Describe the results you received
systemd rejects both transient-scope starts. The Aardvark DNS failure makes network cleanup fail and `repro.service` exit with status 125:
<details>
<summary>Sanitized journal excerpt</summary>
```text
[ 559.255428] systemd[1126]: Activating special unit Exit the Session...
[ 559.256608] systemd[1126]: Stopping podman-pause-36b10a9e.scope...
[ 559.256754] systemd[1126]: Stopping rootless-netns-7cc3b5ec.scope...
[ 559.258856] systemd[1126]: Stopping Rootless Quadlet shutdown reproducer...
[ 560.609834] repro[2146]: time="2026-08-06T14:12:13Z" level=warning msg="rootless netns program no longer running, trying to start it again"
[ 560.620161] systemd[1126]: Requested transaction contradicts existing jobs: Transaction for rootless-netns-5f7e98d9.scope/start is destructive (shutdown.target has 'start' job queued, but 'stop' is included in transaction).
[ 560.621346] repro[2146]: time="2026-08-06T14:12:13Z" level=error msg="failed to move the rootless netns pasta process to the systemd user.slice: Transaction for rootless-netns-5f7e98d9.scope/start is destructive (shutdown.target has 'start' job queued, but 'stop' is included in transaction)."
[ 560.770411] repro[2146]: time="2026-08-06T14:12:14Z" level=debug msg="Successfully cleaned up container a890aa29574a58421b00dba2f56fed91b6023ac0e73d8ef0418370a2fcbc1845"
[ 560.778402] repro[2146]: time="2026-08-06T14:12:14Z" level=debug msg="Failed to remove container a890aa29574a58421b00dba2f56fed91b6023ac0e73d8ef0418370a2fcbc1845: cleaning up container a890aa29574a58421b00dba2f56fed91b6023ac0e73d8ef0418370a2fcbc1845: removing container a890aa29574a58421b00dba2f56fed91b6023ac0e73d8ef0418370a2fcbc1845 network: netavark (exit code 1): remove aardvark entries: aardvark-dns failed to start: Failed to start transient scope unit: Transaction for run-p2197-i2198.scope/start is destructive (shutdown.target has 'start' job queued, but 'stop' is included in transaction)."
[ 560.812869] systemd[1126]: repro.service: Control process exited, code=exited, status=125/n/a
[ 560.902664] systemd[1126]: repro.service: Failed with result 'exit-code'.
```
</details>
### Describe the results you expected
An orderly reboot should let the Quadlet container finish network cleanup before its rootless helper scopes stop. The service should stop successfully without trying to recreate helpers after the user manager starts its exit transaction.
### podman info output
```yaml
host:
arch: arm64
buildahVersion: 1.44.1
cdiSpecDirs:
- /etc/cdi
- /var/run/cdi
cgroupControllers:
- cpu
- io
- memory
- pids
cgroupManager: systemd
cgroupVersion: v2
conmon:
package: Unknown
path: /nix/store/aqc0pah4g2mx0152yxc3d1a1rkyj9a3k-podman-helper-binary-wrapper/bin/conmon
version: 'conmon version 2.2.1, commit: c8cc2c4db27531bd4e084ce7857f73cd21ee639d'
cpuUtilization:
idlePercent: 98.86
systemPercent: 0.67
userPercent: 0.47
cpus: 6
databaseBackend: sqlite
distribution:
codename: yarara
distribution: nixos
version: "26.05"
eventLogger: journald
freeLocks: 2047
hostname: <redacted>
idMappings:
gidmap:
- container_id: 0
host_id: 100
size: 1
- container_id: 1
host_id: 165536
size: 65536
uidmap:
- container_id: 0
host_id: 2001
size: 1
- container_id: 1
host_id: 165536
size: 65536
kernel: 6.18.40
linkmode: dynamic
logDriver: journald
memFree: 9855471616
memTotal: 10384941056
networkBackend: netavark
networkBackendInfo:
backend: netavark
defaultNetwork: podman
dns:
package: Unknown
path: /nix/store/430z0gdarqmpjwafwdx5hhaygj96vdn7-podman-6.0.2/libexec/podman/aardvark-dns
version: aardvark-dns 2.0.0
package: Unknown
path: /nix/store/430z0gdarqmpjwafwdx5hhaygj96vdn7-podman-6.0.2/libexec/podman/netavark
version: netavark 2.0.0
ociRuntime:
name: crun
package: Unknown
path: /nix/store/aqc0pah4g2mx0152yxc3d1a1rkyj9a3k-podman-helper-binary-wrapper/bin/crun
version: |-
crun version 1.27.1
commit: 3ec076b3b6714ec2f1a10533cf18d5605a6de637
rundir: /run/user/2001/crun
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +YAJL
os: linux
pasta:
executable: /nix/store/430z0gdarqmpjwafwdx5hhaygj96vdn7-podman-6.0.2/libexec/podman/pasta
package: Unknown
version: |
pasta 2026_07_16.090d739
Copyright Red Hat
GNU General Public License, version 2 or later
<https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
remoteSocket:
exists: true
path: /run/user/2001/podman/podman.sock
rootlessNetworkCmd: pasta
rootlessPortForwarder: rootlessport
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: true
seccompEnabled: true
seccompProfilePath: ""
selinuxEnabled: false
serviceIsRemote: false
swapFree: 5192019968
swapTotal: 5192019968
uptime: 0h 1m 2.00s
variant: v8
plugins:
authorization: null
log:
- k8s-file
- none
- passthrough
- journald
network:
- bridge
- macvlan
- ipvlan
volume:
- local
registries:
search:
- docker.io
store:
containerStore:
number: 1
paused: 0
running: 1
stopped: 0
graphDriverName: overlay
graphOptions: {}
graphRoot: /home/podman-repro/.local/share/containers/storage
graphRootAllocated: 66787553280
graphRootUsed: 40321662976
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "true"
Supports d_type: "true"
Supports shifting: "false"
Supports volatile: "true"
Using metacopy: "false"
imageCopyTmpDir: /var/tmp
imageStore:
number: 1
runRoot: /run/user/2001/containers
transientStore: false
volumePath: /home/podman-repro/.local/share/containers/storage/volumes
version:
APIVersion: 6.0.2
Built: 315532800
BuiltTime: Tue Jan 1 00:00:00 1980
GitCommit: ""
GoVersion: go1.26.5
Os: linux
OsArch: linux/arm64
Version: 6.0.2
```
### Podman in a container
No
### Privileged Or Rootless
Rootless
### Upstream Latest Release
Yes
### Additional environment details
NixOS 26.05 with Linux 6.18.40 and systemd 260.2, running in an aarch64 virtual machine.
### Additional information
Podman issue #22451 shows a similar shutdown transaction, but its shutdown discussion had application stop-signal and restart-policy confounders. It was closed as a duplicate of a boot-time network-readiness issue and is locked. This reproducer has neither confounder. The sequence also reproduced with Podman 5.8.2, and the recovery path was still present in container-libs `main` at `84b9d5bbc7d67f8de5234fcd1916f8cbffba45e4` on 2026-08-06. Netavark 2.1.0 and Aardvark DNS 2.1.0 were published shortly before this run but were not part of the tested Podman environment. Their changes do not affect the Aardvark notification or systemd-scope restart path involved here.
1 条评论