Pi 5 macb: eth0 wedges with constant high RX error rate after co-boot with managed switch — only `modprobe -r macb && modprobe macb` recovers
### Describe the bug
On Pi 5 with the onboard Cadence GEM MAC + Broadcom BCM54213PE PHY in `rgmii-id` mode, if the Pi powers on at the same time as its upstream managed switch (in our case a Meraki Z4), `eth0` lands in a wedged state where the kernel reports `Link is Up - 1Gbps/Full` and `networkctl` reports `degraded (configuring) / online`, but every inbound frame is corrupted at the MAC layer. `ip -s link show eth0` shows RX errors incrementing at ~17,800/sec while valid RX packets stay at 3 for the entire stuck window. TX is clean — outbound frames reach the switch and the switch replies, but every reply is dropped as a CRC/framing error before the stack ever sees it. ARP for the default gateway never resolves (`ip neigh` reports `FAILED`), DHCP never completes, and the state persists indefinitely (10+ minutes observed) across all softer interventions. Only a full `macb` driver reload (`modprobe -r macb && sleep 5 && modprobe macb`) clears it; after reload the link comes back at 1 Gbps in 3 seconds and traffic flows normally with zero RX errors.
I'm filing this here because the failure mode is Pi-side hardware behavior; we're running Ubuntu 24.04 LTS for Pi (`6.8.0-1056-raspi`), not Pi OS, but the macb driver, BCM54213PE PHY init, and device tree are the same components.
### Steps to reproduce the behaviour
Deterministic when both conditions hold:
1. Pi and an upstream managed switch (Meraki Z4 in our case) are both power-cycled, with overlapping boot windows — e.g., a site power outage where both come back at the same time.
2. The Pi's `eth0` brings up carrier *before* the switch's port is fully ready to forward traffic, so MAC + PHY + link-partner autoneg simultaneously.
If the switch is already up and idle when the Pi boots, the Pi comes up clean every time. The race is specifically at simultaneous link-partner bringup.
Other devices on the same Z4 LAN (PDU, displays — non-Pi hardware) come back fine through the same switch in the same window, so the Z4 itself is forwarding correctly. The issue is specific to the Pi 5's macb + BCM54213PE path.
### Device (s)
Raspberry Pi 5
### System
`raspinfo` is not packaged on Ubuntu for Pi, and `vcgencmd` does not work without `/dev/vcio` (Ubuntu does not create it by default). Answering the individual questions:
- **OS:** Ubuntu 24.04.1 LTS (`PRETTY_NAME="Ubuntu 24.04.1 LTS"` from `/etc/os-release`), not Raspberry Pi OS.
- **Firmware version (vcgencmd):** unable to retrieve — `/dev/vcio` not present, `vcgencmd version` fails with `Can't open device file: /dev/vcio`. Hardware is Raspberry Pi 5 Model B Rev 1.1 (`/proc/device-tree/model`).
- **Kernel version:** `Linux 3pm-lebanon-murfreesboro-detection 6.8.0-1056-raspi #60-Ubuntu SMP PREEMPT_DYNAMIC Thu May 7 21:58:14 UTC 2026 aarch64 aarch64 aarch64 GNU/Linux`
- **Driver:** `macb`, version `6.8.0-1056-raspi`, bus `1f00100000.ethernet`, no firmware-version reported (`ethtool -i eth0`).
- **PHY:** `Broadcom BCM54213PE` per kernel boot log.
- **PHY mode:** `rgmii-id` per kernel boot log (`configuring for phy/rgmii-id link mode`).
### Logs
### Kernel macb / PHY events across a failed boot and the subsequent driver-reload recovery
```
# === failed initial bringup ===
macb 1f00100000.ethernet eth0: PHY [1f00100000.ethernet-ffffffff:01] driver [Broadcom BCM54213PE] (irq=POLL)
macb 1f00100000.ethernet eth0: configuring for phy/rgmii-id link mode
macb 1f00100000.ethernet: gem-ptp-timer ptp clock registered.
macb 1f00100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
macb 1f00100000.ethernet eth0: Link is Down # switch reconfiguring port (~29s after first up)
macb 1f00100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off # link back, but PHY now wedged
# (~2 minutes of stuck state with zero useful traffic; see RX error table below)
# === modprobe -r macb / modprobe macb (the recovery) ===
macb 1f00100000.ethernet eth0: Link is Down
macb 1f00100000.ethernet: gem-ptp-timer ptp clock unregistered.
macb 1f00100000.ethernet eth0: Cadence GEM rev 0x00070109 at 0x1f00100000 irq 109 (2c:cf:67:c2:64:76)
macb 1f00100000.ethernet eth0: PHY [1f00100000.ethernet-ffffffff:01] driver [Broadcom BCM54213PE] (irq=POLL)
macb 1f00100000.ethernet eth0: configuring for phy/rgmii-id link mode
macb 1f00100000.ethernet: gem-ptp-timer ptp clock registered.
macb 1f00100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off # 3s after modprobe macb
```
### RX error counter — `ip -s link show eth0` snapshots every 30s during the stuck window
| t (s) | RX bytes | RX packets | **RX errors** | TX bytes | TX packets | TX errors |
| ----: | -------: | ---------: | -------------: | -------: | ---------: | --------: |
| 0 | 480 | 3 | **1,587,323** | 2,152 | 12 | 0 |
| 33 | 480 | 3 | **2,983,608** | 2,152 | 12 | 0 |
| 193 | 480 | 3 | **5,671,686** | 2,152 | 12 | 0 |
| 385 | 480 | 3 | **8,896,373** | 2,152 | 12 | 0 |
| 577 | 480 | 3 | **12,120,833** | 2,152 | 12 | 0 |
- RX errors increment at **~17,800/sec** (~1.2% of 1 Gbps line rate, consistent with broadcast/ARP traffic from the LAN being received and corrupted).
- Valid RX packets stay at **3** — those arrived in the first ms after carrier-up, before the wedge set in.
- TX counters unchanged throughout — outbound DHCP/ARP requests went out early; with no responses ever delivered, retry queues quiet.
- `ip neigh` for the gateway: `10.110.10.1 dev eth0 FAILED` for the entire window.
After `modprobe -r macb && modprobe macb`, taken ~90s into the recovered state:
| | RX bytes | RX packets | RX errors | TX bytes | TX packets |
| ----------- | --------: | ---------: | --------: | -------: | ---------: |
| post-reload | 2,046,944 | 1,993 | **0** | 264,196 | 1,468 |
RX errors flat at 0; valid traffic flowing.
Kernel `ifindex` for `eth0` increments across the reload (e.g. `2 → 4`), confirming a full device tear-down and re-probe rather than a state-only
change.
### Userspace layers all report healthy during the stuck window
```
ethtool eth0: Speed: 1000Mb/s, Duplex: Full, Auto-negotiation: on, Link detected: yes
networkctl status: State: degraded (configuring), Online state: online
kernel: macb 1f00100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
```
### What does NOT clear the wedged state
- `networkctl reconfigure eth0` — state persists.
- `systemctl restart systemd-networkd` (full networkd restart) — state persists; RX errors continue accumulating across the new networkd instance.
- `ethtool -r eth0` — `Cannot restart autonegotiation: Operation not supported` (macb does not implement `ETHTOOL_NWAY_RST`).
- `ethtool --reset eth0 phy` — `Cannot issue ETHTOOL_RESET: Operation not supported` (macb does not implement `ETHTOOL_RESET`).
- Natural carrier flaps during the stuck window (`Lost carrier` → `Gained carrier` from the switch settling) do **not** clear it — counters continue
incrementing across the flap.
I did not test `ip link set eth0 down/up` in isolation because we escalated directly to driver reload after the `ethtool` mechanisms were unsupported.
### EEE state
`ethtool --show-eee eth0` returns `netlink error: Operation not supported`, so EEE is not exposed by this driver/kernel — this issue is unlikely to be
EEE-related.
## Additional context
### Hypothesis (offered with appropriate uncertainty)
The PHY's RGMII RX delay configuration appears to land in a wrong state when the MAC, the PHY, and the link partner all autoneg simultaneously during
co-boot. TX timing is sourced from the MAC's own clock and is unaffected (which is why outbound frames reach the switch and the switch replies). RX
timing depends on the PHY presenting received data correctly relative to its recovered clock from the line; if the RX delay register is mis-programmed,
every inbound symbol is sampled at the wrong edge → CRC fail → MAC counts as error → frame dropped.
Consistent with this:
- TX clean, RX 100% errored
- All userspace layers see a healthy link (autoneg succeeded, speed/duplex correct)
- Persistent across every intervention that doesn't re-run the PHY's `config_init` callback
- Cleared by driver reload, which on re-probe does call `phy_init_hw()` → `bcm54xx_config_init` in `drivers/net/phy/broadcom.c` where delay registers
are programmed
What I have NOT verified:
- That the specific failing register is the RX delay (vs. some other PHY config). Would need MDIO register dump during the failure window; `mdio-tools`
is not packaged on Ubuntu by default.
- Whether `rgmii-rxid` or `rgmii-txid` would race the same way.
- Whether a newer kernel has a fix already; we haven't tried 6.9 or 6.10 base.
### Workaround currently deployed
A small systemd oneshot at boot that pings the default gateway 30s after `multi-user.target`, and if unreachable, runs `modprobe -r macb && sleep 5 &&
modprobe macb`, retrying up to 9 times with 60s between attempts (~10-minute total budget). Total auto-recovery time observed: ~3 minutes from kernel
boot to operational network with no human intervention.
### Related issues (reviewed and ruled out as distinct)
- [#6420](https://github.com/raspberrypi/linux/issues/6420) — "Pi 5 sometimes has no LAN connection after booting." Their Pi gets DHCP and is reachable
on the LAN; only internet egress fails. Routing/upstream-class issue, opposite of ours.
- [siderolabs/sbc-raspberrypi#91](https://github.com/siderolabs/sbc-raspberrypi/issues/91) — "Silent network death on Talos." Explicitly notes RX error
counters stay at **0** during their failure — silent at the MAC layer. Ours is the loud failure mode. Their suspected causes (EEE AutogrEEEn LPI-wake
race, MAC TX-ring hang during TSO/GSO) don't match the all-RX-frames-corrupted signature.
- [#6855](https://github.com/raspberrypi/linux/issues/6855) — Generic LAN packet loss; not co-boot-specific, no RX-error analysis.
### Additional logs available on request
- Full `journalctl -t eth0-diag` showing per-30s `ip -s link`, `ip neigh`, `ip addr`, `ethtool`, `networkctl status`, ping result across the 10-minute
stuck window
- Full kernel macb/PHY events for both the failed boot and the successful recovery boot
- Full `systemd-networkd` events
---
### Additional context
_No response_
1 条评论