Skip to content

Conversation

@Ratio2
Copy link
Contributor

@Ratio2 Ratio2 commented Oct 11, 2022

With a large number of encrypted folders and devices, they are subject to a large CPU load. Moreover, hashes are recalculated for different devices and for each connection.

Since there can be a limited number of hashes, it was decided to cache them in memory.

Testing

  1. Connecting to encrypted folders on other devices works correctly.
  2. Reconnection (pause/resume) occurs without significant delay, although before the change, the connection with 3 devices lasted about 30 seconds.

Documentation

Not required

Authorship

Your name and email will be added automatically to the AUTHORS file
based on the commit metadata.

)

With a large number of encrypted folders and devices, they are subject to a
large CPU load. Moreover, hashes are recalculated for different devices and
for each connection.

Since there can be a limited number of hashes, it was decided to cache them
in memory.
Copy link
Member

@calmh calmh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems reasonable in general.


import "sync"

type Cache[K comparable, V any] struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really just a worse sync.Map, an actual cache would have some max size or invalidation logic etc. Better to use a normal map with a mutex at the call site, or a sync.Map, so that the behavior is clear.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made some improvements

Copy link
Member

@calmh calmh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks to me like a simpler approach would be to make the existing folderKeyRegistry longer lived than currently (maybe declare one at the top level instead of constructing in NewConnection) and have it key on folder+password (to handle password changes). Having multiple levels of caching seems unnecessarily complicated.

calmh added a commit to calmh/syncthing that referenced this pull request Mar 10, 2023
This adds a cache to the expensive key generation operations. It's fixes
size LRU/MRU stuff to keep memory usage bounded under absurd conditions.

Also closes syncthing#8600.
@st-review
Copy link

🤖 beep boop

I'm going to close this pull request as it has been idle for more than 90 days.

This is not a rejection, merely a removal from the list of active pull requests that is periodically reviewed by humans. The pull request can be reopened when there is new activity, and merged once any remaining issues are resolved.

To permanently exempt this PR from bot nagging, add the slow-pr label.

@st-review st-review closed this Mar 10, 2023
calmh added a commit that referenced this pull request Mar 12, 2023
This adds a cache to the expensive key generation operations. It's fixes
size LRU/MRU stuff to keep memory usage bounded under absurd conditions.

Also closes #8600.
@st-review st-review added the frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion label Mar 9, 2024
@syncthing syncthing locked and limited conversation to collaborators Mar 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

frozen-due-to-age Issues closed and untouched for a long time, together with being locked for discussion

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants