Cache channel users to avoid redundant API calls and reduce bottleneck#646
Merged
rusq merged 1 commit intorusq:masterfrom Mar 19, 2026
Merged
Cache channel users to avoid redundant API calls and reduce bottleneck#646rusq merged 1 commit intorusq:masterfrom
rusq merged 1 commit intorusq:masterfrom
Conversation
When fetching threads with threadOnly=true, procChannelUsers was called for each thread in the same channel, making redundant GetUsersInConversation API calls. This adds a userCache similar to chanCache to store channel users and skip the API call when already fetched. Co-Authored-By: Claude <noreply@anthropic.com>
volker-fr
commented
Mar 17, 2026
volker-fr
commented
Mar 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When fetching threads with threadOnly=true, procChannelUsers was called for each thread in the same channel, making redundant GetUsersInConversation API calls. This adds a userCache similar to chanCache to store channel users and skip the API call when already fetched.
On
slackdump resume -threadsthis significantly increases the speed. In a simple test slack where only one channel was backed up withslackdump archive, this double the speed.This issue fixes one of the root causes of the slowness reported in #560
NOTE: At the same time it is unclear how beneficial this change really is. In #647 the bottleneck of
slackdump resume -threadsgot improved by not having to fetch users -- and therefore this caching. If #647 is merged, this leaves the usage of this caching probably to only 1-2 edge cases -- if et al. I didn't fully understand the other uses cases here and if it would benefit from caching.Speed testing
4.1.0 takes 24sec
% ./slackdump-4.1.0 \ resume \ -threads \ test-slackdump-dev ... 0.69user 0.47system 0:24.44elapsed 4%CPU (0avgtext+0avgdata 52144maxresident)k 0inputs+91440outputs (0major+10474minor)pagefaults 0swapswith caching it only takes around 8 seconds
% ./slackdump-this-pr \ resume \ -threads \ test-slackdump-dev ... 0.55user 0.34system 0:07.90elapsed 11%CPU (0avgtext+0avgdata 51716maxresident)k 0inputs+56112outputs (0major+9244minor)pagefaults 0swaps