Erik Johnston dfd00a986f Fix sliding sync performance slow down for long lived connections. (#19206) ... Fixes https://github.com/element-hq/synapse/issues/19175 This PR moves tracking of what lazy loaded membership we've sent to each room out of the required state table. This avoids that table from continuously growing, which massively helps performance as we pull out all matching rows for the connection when we receive a request. The new table is only read when we have data in a room to send, so we end up reading a lot fewer rows from the DB. Though we now read from that table for every room we have events to return in, rather than once at the start of the request. For an explanation of how the new table works, see the [comment](https://github.com/element-hq/synapse/blob/erikj/sss_better_membership_storage2/synapse/storage/schema/main/delta/93/02_sliding_sync_members.sql#L15-L38) on the table schema. The table is designed so that we can later prune old entries if we wish, but that is not implemented in this PR. Reviewable commit-by-commit. --------- Co-authored-by: Eric Eastwood <erice@element.io>		2025-12-12 10:02:57 +00:00
..
databases	Port Clock functions to use Duration class (#19229)	2025-12-01 13:55:06 +00:00
util	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
__init__.py	Reference Matrix Home Server	2014-08-12 15:10:52 +01:00
test__base.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_account_data.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_appservice.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_background_update.py	Port Clock functions to use Duration class (#19229)	2025-12-01 13:55:06 +00:00
test_base.py	Remove MockClock() (#18992)	2025-09-30 11:27:29 -05:00
test_cleanup_extrems.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_client_ips.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_database.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_devices.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_directory.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_e2e_room_keys.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_end_to_end_keys.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_event_chain.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_event_federation.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_event_push_actions.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_events_bg_updates.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_events.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_id_generators.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_invite_rule.py	Implement MSC4380: Invite blocking (#19203)	2025-11-26 16:12:14 +00:00
test_main.py	Correctly mention previous copyright (#16820)	2024-01-23 11:26:48 +00:00
test_monthly_active_users.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_profile.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_purge.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_receipts.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_redaction.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_registration.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_relations.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_rollback_worker.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_room_search.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_room.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_roommember.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_sliding_sync_tables.py	Fix sliding sync performance slow down for long lived connections. (#19206)	2025-12-12 10:02:57 +00:00
test_state_deletion.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_state.py	Fix case where get_partial_current_state_deltas could return >100 rows (#18960)	2025-11-26 17:17:04 +00:00
test_stream.py	Use type hinting generics in standard collections (#19046)	2025-10-22 16:48:19 -05:00
test_thread_subscriptions.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_transactions.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_txn_limit.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00
test_unsafe_locale.py	Refactor Histogram metrics to be homeserver-scoped (#18724)	2025-07-29 15:35:38 -05:00
test_user_directory.py	Write union types as X | Y where possible (#19111)	2025-11-06 14:02:33 -06:00
test_user_filters.py	Introduce Clock.call_when_running(...) to include logcontext by default (#18944)	2025-09-22 10:27:59 -05:00