HTTP 500 while calling /sync

### Description When opening Element, I received an "unable to connect to homeserver" error. I cleared my client cache, assuming that was the problem, but immediately after had a pop up saying "Unable to connect to Homeserver. Retrying…" on the initial sync screen. Upon checking Firefox network logs, it turns out every /sync request is being met with a HTTP 500 Internal server error. Even better yet, it's a *cached* HTTP 500! ### Steps to reproduce 1. Hit /_matrix/client/v3/sync with any parameters. Bug showed with both `since` and no `since`. ### Homeserver starstruck.systems ### Synapse Version 0.151.0 ### Installation Method Other (please mention below) ### Database mono, clean ### Workers Single process ### Platform Ubuntu LXC, master process and worker processes controlled by systemd, same-metal postgres in another lxc, redis on the same lxc hosting synapse. Patchset: https://git.nexy7574.co.uk/star/starnapse/commits/branch/starnapse ### Configuration Many experimental features enabled, presence disabled, synapse-http-antispam module enabled - full config at https://starstruck.systems/~star/synapse.yaml ### Relevant log output ```shell Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: 2026-04-24 22:51:07,415 - synapse.util.caches.response_cache - 388 - INFO - GET-294024 - [sync]: using completed cached result for [(@nex:starstruck.systems, 0, None, '0', False, 'RUWAODMJKX', 129335, True)] Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: 2026-04-24 22:51:07,415 - synapse.http.server - 147 - ERROR - GET-294024 - Failed handle request via 'SyncRestServlet': <XForwardedForRequest at 0x7e205103ef90 method='GET' uri='/_matrix/client/v3/sync?filter=0&timeout=0&org.matrix.msc4222.use_state_after=true&set_presence=online&_cacheBuster=1777067467276' clientproto='HTTP/1.1' site='8101'> Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: Traceback (most recent call last): Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/http/server.py", line 335, in _async_render_wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: callback_return = await self._async_render(request) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/http/server.py", line 576, in _async_render Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: callback_return = await raw_callback_return Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/rest/client/sync.py", line 271, in on_GET Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: sync_result = await self.sync_handler.wait_for_sync_for_user( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 365, in wait_for_sync_for_user Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: res = await self.response_cache.wrap( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/util/caches/response_cache.py", line 402, in wrap Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await make_deferred_yieldable(result) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1195, in __iter__ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: result.raiseException() Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/python/failure.py", line 455, in raiseException Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: raise self.value.with_traceback(self.tb) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: result = context.run( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return g.throw(self.value.with_traceback(self.tb)) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/util/caches/response_cache.py", line 343, in cb Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await callback(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 470, in _wait_for_sync_for_user Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: result = await self.current_sync_for_user( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 538, in current_sync_for_user Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: sync_result = await self.generate_sync_result( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 1760, in generate_sync_result Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ) = await self._generate_sync_entry_for_rooms(sync_result_builder) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 2307, in _generate_sync_entry_for_rooms Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: await concurrently_execute(handle_room_entries, room_entries, 10) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/util/async_helpers.py", line 298, in concurrently_execute Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: await yieldable_gather_results( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/util/async_helpers.py", line 347, in yieldable_gather_results Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: raise dfe.subFailure.value from None Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: result = context.run( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return g.throw(self.value.with_traceback(self.tb)) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/util/async_helpers.py", line 284, in _concurrently_execute_inner Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: await maybe_awaitable(func(value)) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 2295, in handle_room_entries Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: await self._generate_room_entry( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 2831, in _generate_room_entry Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: state = await self.compute_state_delta( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/handlers/sync.py", line 1201, in compute_state_delta Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: state = await self.store.get_events(list(state_ids.values())) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 949, in _wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await func(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 624, in get_events Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: events = await self.get_events_as_list( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 949, in _wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await func(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 949, in _wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await func(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 680, in get_events_as_list Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: event_entry_map = await self.get_unredacted_events_from_cache_or_db( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 949, in _wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await func(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 917, in get_unredacted_events_from_cache_or_db Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: missing_events: dict[str, EventCacheEntry] = await delay_cancellation( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1187, in __iter__ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: yield self Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/twisted/internet/defer.py", line 1857, in _inlineCallbacks Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: result = context.run(gen.send, result) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 903, in get_missing_events_from_cache_or_db Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: raise e Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 896, in get_missing_events_from_cache_or_db Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: db_missing_events = await self._get_events_from_db( Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/logging/opentracing.py", line 949, in _wrapper Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: return await func(*args, **kwargs) Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: File "/opt/synapse/.venv/lib/python3.12/site-packages/synapse/storage/databases/main/events_worker.py", line 1491, in _get_events_from_db Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: d["room_id"], Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: ~^^^^^^^^^^^ Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: KeyError: 'room_id' Apr 24 22:51:07 starstruck synapse-generic-worker-1[2686041]: 2026-04-24 22:51:07,420 - synapse.access.http.8101 - 643 - INFO - GET-294024 - 159.26.116.19 - 8101 - {@nex:starstruck.systems} Processed request: 0.004sec/0.002sec ru=(0.002sec, 0.000sec) db=(0.000sec/0.000sec/1) 55B 500 "GET /_matrix/client/v3/sync?filter=0&timeout=0&org.matrix.msc4222.use_state_after=true&set_presence=online&_cacheBuster=1777067467276 HTTP/1.1" "Mozilla/5.0 (X11; Linux x86_64; rv:149.0) Gecko/20100101 Firefox/149.0" [0 dbevts] ``` ### Anything else that would be useful to know? I am now unable to use the account affected