Lagging workers can return incomplete data because they don't wait to catch up to the given token

### Background *Spawning from https://github.com/element-hq/synapse/pull/19644 which has roots in https://github.com/element-hq/synapse/pull/19558#discussion_r2977673208,* When using workers and multiple writers for a stream, each worker has it's own view of the 'current' world. They notify each other of updates via replication and catch up over time. One request can be routed to a worker and respond with a given `next_batch` token, then the next request can be routed to a different worker which may be lagging behind in the stream. When this happens, if the worker just naively serves the request and fetches data, it could be missing data in the gap between where it's lagging and the `next_batch` token. This results in flawed responses (not seamless). The way to solve this is to have the worker wait until it catches up to the given `next_batch` token. This way the worker knows it at-least has all of the data that the previous worker did; and can fetch whatever new data it has is between the `next_batch` and wherever it's current position happens to be. ### Do we wait anywhere? We currently only wait for the `/sync` endpoints 😵: https://github.com/element-hq/synapse/blob/62f23fed27808d11fdd3e3089838bec7f188d787/synapse/handlers/sync.py#L415-L423 https://github.com/element-hq/synapse/blob/62f23fed27808d11fdd3e3089838bec7f188d787/synapse/handlers/sliding_sync/__init__.py#L148-L155 ### We need to wait everywhere Given I had to create a whole new function [`wait_for_multi_writer_stream_token(...)`](https://github.com/element-hq/synapse/pull/19644) in order for us to wait properly for some new endpoint, it's a clear sign that we're not properly waiting anywhere else. Because we're not waiting, all of other endpoints that fetch data with tokens are subject to flawed pagination and data fetching. Also, as a best practice, we should probably be using the full [`StreamToken`](https://github.com/element-hq/synapse/blob/62f23fed27808d11fdd3e3089838bec7f188d787/synapse/types/__init__.py#L1063-L1081) for every endpoint even though some endpoints may only fetch data in a single category. To explain the contrary, if for example, an endpoint was only dealing with receipts, we could maybe get away with using the single receipts token. But if the endpoint starts querying anything else like events, it will end up giving flawed data as the requests are passed from worker to worker. And this problem is hard to spot as the code evolves over time. Best to just use a token that snapshots the entire position of what that worker knows about (`StreamToken`). ### Dev notes Related docs: - [`docs/development/synapse_architecture/streams.md`](https://github.com/element-hq/synapse/blob/62f23fed27808d11fdd3e3089838bec7f188d787/docs/development/synapse_architecture/streams.md) ### Todo - [ ] List out and audit all endpoints to see whether they wait (including admin endpoints) ### Definition of done - [ ] Ensure all endpoints wait until they've caught up to the given token