Active/active source failover: a standby publisher's route is never presented to the relay serving the active source (mesh), so a dead source is not failed over — even with `moq-lite-06` cost routing

## Why this matters (the requirement) We are evaluating MoQ as a primary-distribution transport for linear broadcast, where the baseline expectation is **no visible failure during contracted content**. The standard broadcast pattern is **1+1 active/active**: two independent source→publisher chains run hot, and if the active one dies the receiver keeps playing with no (or a bounded, hitless) artefact. Today broadcasters achieve this with ST 2022-7 at the IRD. We want to understand the intended MoQ answer, because MoQ already ships most of the machinery, and we would like to help land whatever is missing rather than route around it. This is a **requirements + reproduction + "which direction do you want"** issue, not a bug report. ## What already works (confirmed by source + drills) - **The origin's multi-source route table exists and does the right thing.** An origin holds every route it knows for a broadcast path, ranks them (`route_order` = announced-before-offline → cumulative cost → hop length → stable hash), parks losers as silent standbys, and on active-source death `reselect` promotes the winner with tracks re-splicing at the next group boundary — covered by `test_route_failover` (`rs/moq-net/src/model/origin.rs`). **The splice/failover primitive is not the gap.** - **Cost-based ranking landed** (#2424: cumulative route cost ranks above hop count on lite-06 announcements) and **announce ids** (#2160), both in `moq-lite-06-wip`. - **Transport reconnection and fan-out work.** A publisher survives a relay restart and re-announces; N subscribers get byte-identical output. ## The gap (reproduced) The origin's failover primitive is **never fed a second live route** for the same broadcast at the node that needs it. 1. **Two publishers, one relay (same broadcast).** The second `active` announce makes the path `unroutable` and **both** publishers are torn down — there is no "standby publisher for this path" concept on lite-05. 2. **Two publishers, two-relay mesh** (`pubA→relayA`, `pubB→relayB`, meshed). The pair coexists, but when `pubA` is killed **the output freezes permanently — no failover.** Relay A's session log shows it only ever held its **own** local route for the broadcast (its connections were: 2 subscribers + 1 publisher + the cluster peer); **relay B never advertised its standby `pubB` across the cluster link.** So relay A had nothing to reselect. ### `moq-lite-06` cost routing was tested and does **not** by itself close this Because `moq-lite-06-wip` is deliberately excluded from the default advertised set/ALPN list (`rs/moq-net/src/version.rs`) and negotiates only when both peers opt in, we re-ran (2) with lite-06 forced **end-to-end** — both relays via `--server-version` *and* `--client-version moq-lite-06-wip`, and both clients via `--client-version` (cluster log confirms `connected version=moq-lite-06-wip`). The result was **identical**: coexist, then freeze on `pubA` death. Cost routing had nothing to rank, because the standby route is never *advertised* to relay A. **Pricing routes cannot help choose a route that is never propagated.** So cost routing is necessary but not sufficient for this requirement. We believe the propagation gap is: a relay announces only its **one best route per path**, and the split-horizon loop filter (`exclude_hop`) suppresses the return announcement, so a *standby* publisher that is not currently being carried is invisible to the relay serving the active source. (Chicken-and-egg: relay A won't pull `pubB` until it fails over, and it can't fail over because it never learned `pubB` exists.) ## What we're asking **Which direction do you intend for active/active source failover, so we can contribute to it?** Two candidate mechanisms, both building on parts you already have: - **A. Relay-mesh standby-route propagation.** Let a relay advertise a *standby* route for a broadcast (its local publisher) across the cluster mesh — at a high `route.cost` (lite-06) so it never wins while the active route is healthy — so that the relay serving the active source holds the standby in its origin table and `reselect` promotes it on failure. This makes a **single-homed** subscriber ride out an active-source death with no downstream merge. It needs (a) propagation of not-currently-carried standby routes past announce coalescing / split-horizon, plus (b) the lite-06 cost ranking that already exists. - **B. Subscriber multi-homing into one origin.** Let a subscriber (or its local origin) connect to **two relays** and register both as routes for the same broadcast, so the *existing* `best_route`/`reselect` splice does the active/active failover at the subscriber's origin. This reuses the shipped primitive and mostly needs a client able to hold two upstream sessions feeding one origin (cf. #2241, "migrate subscriptions transparently across connections"). Our read is that **B is the more tractable near-term win** (the origin splice already works; it just needs two routes in front of it), while **A is the more complete answer** for single-homed receivers. We are happy to prototype either — we would like your steer on the preferred design before we do, and whether an existing issue/PR already tracks this. ## Acceptance test we would validate against Two publishers on the same broadcast across two meshed relays (or two relays behind one multi-homed subscriber). Kill the active publisher. **The subscriber's output continues with at most a single group-boundary artefact and no permanent stall.** We have this drill scripted (`cluster_failover.sh`) and can share it / turn it into an integration test. ## References - `rs/moq-net/src/model/origin.rs` — `best_route` / `reselect` / `test_route_failover` - `rs/moq-net/src/lite/announce.rs`, `rs/moq-net/src/model/broadcast.rs` — route cost - `rs/moq-net/src/version.rs`, `rs/moq-net/src/lite/version.rs` — lite-06 opt-in gating - #2424 (route cost, merged), #2160 (announce ids, merged), #2241 (subscription migration across connections)