element-synapse/scripts-dev
Erik Johnston dfd00a986f
Fix sliding sync performance slow down for long lived connections. (#19206)
Fixes https://github.com/element-hq/synapse/issues/19175

This PR moves tracking of what lazy loaded membership we've sent to each
room out of the required state table. This avoids that table from
continuously growing, which massively helps performance as we pull out
all matching rows for the connection when we receive a request.

The new table is only read when we have data in a room to send, so we
end up reading a lot fewer rows from the DB. Though we now read from
that table for every room we have events to return in, rather than once
at the start of the request.

For an explanation of how the new table works, see the
[comment](https://github.com/element-hq/synapse/blob/erikj/sss_better_membership_storage2/synapse/storage/schema/main/delta/93/02_sliding_sync_members.sql#L15-L38)
on the table schema.

The table is designed so that we can later prune old entries if we wish,
but that is not implemented in this PR.

Reviewable commit-by-commit.

---------

Co-authored-by: Eric Eastwood <erice@element.io>
2025-12-12 10:02:57 +00:00
..
build_debian_packages.py Write union types as X | Y where possible (#19111) 2025-11-06 14:02:33 -06:00
check_line_terminators.sh Update license headers 2023-11-21 15:29:58 -05:00
check_locked_deps_have_sdists.py Use type hinting generics in standard collections (#19046) 2025-10-22 16:48:19 -05:00
check_schema_delta.py Fix sliding sync performance slow down for long lived connections. (#19206) 2025-12-12 10:02:57 +00:00
check-newsfragment.sh Update book location 2023-12-13 16:15:22 +00:00
complement.sh Export SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES from scripts-dev/complement.sh (#19208) 2025-11-21 19:01:43 -06:00
config-lint.sh Fix Shellcheck SC2006: Use $(...) notation 2021-10-22 23:08:55 +01:00
database-save.sh Drop unused table presence (#14825) 2023-01-13 14:17:03 +00:00
docker_update_debian_changelog.sh Update license headers 2023-11-21 15:29:58 -05:00
dump_macaroon.py Change the format of access tokens away from macaroons (#5588) 2021-05-12 15:04:51 +01:00
federation_client.py Write union types as X | Y where possible (#19111) 2025-11-06 14:02:33 -06:00
gen_config_documentation.py Write union types as X | Y where possible (#19111) 2025-11-06 14:02:33 -06:00
generate_sample_config.sh Add suffices to scripts in scripts-dev (#12137) 2022-03-02 18:00:26 +00:00
lint.sh Pydantic v2 (#19071) 2025-10-31 09:22:22 +00:00
make_full_schema.sh Make it easier to use DataGrip w/ Synapse's schema (#14982) 2023-02-15 13:51:37 +00:00
mypy_synapse_plugin.py Write union types as X | Y where possible (#19111) 2025-11-06 14:02:33 -06:00
next_github_number.sh Fix typo 2023-12-13 16:37:10 +00:00
release.py Remove "Updates to locked dependencies" section from changelog (#19254) 2025-12-02 16:45:41 +00:00
schema_versions.py Write union types as X | Y where possible (#19111) 2025-11-06 14:02:33 -06:00
sign_json.py Introduce Clock.call_when_running(...) to include logcontext by default (#18944) 2025-09-22 10:27:59 -05:00