-
-
Notifications
You must be signed in to change notification settings - Fork 773
[Bug]: AzureStorageProvider Performance is Slow on Large Associated Storage Accounts #5969
Description
Is there an existing issue for this?
- I have searched the existing issues
What happened?
When uploading, deleting, moving, or performing other actions using an AzureFolderProvider mapped file the performance of the operation degrades as the particular mapping grows and adds more files.,
Steps to reproduce?
- Configure a Folder Mapping with an AzureFolder (Idealy a clean setup)
- Upload a file, notice fast performance
- Update the folder in Azure/DNN to have many files > 1000 or so and it seems to be visible
- Upload a file againIdeally
Current Behavior
When completing an upload or other operation, you will find that the upload zooms to completed on the UI, but then it takes many seconds, up to 30-40 at times to refresh the listing and truly complete the operation.being completed on the UI, but then it takes many seconds, up to 30-40 at times,
Expected Behavior
This should be a quick operation and should not slow down with the addition of new files/folders.
Relevant log output
No response
Anything else?
In looking at this it seems to be directly related to the behaviors utilized with regards to cache management within the AzureFolderProvider itself.
After each operation you can see that it purges the cache -
Dnn.Platform/DNN Platform/Providers/FolderProviders/AzureFolderProvider/AzureFolderProvider.cs
Line 331 in 87bed09
| this.ClearCache(folderMapping.FolderMappingID); |
The action to re-fill the cache is here -
Dnn.Platform/DNN Platform/Providers/FolderProviders/AzureFolderProvider/AzureFolderProvider.cs
Line 241 in 87bed09
| protected override IList<IRemoteStorageItem> GetObjectList(FolderMappingInfo folderMapping) |
The issue is that since the mapping is per provider mapping, not per folder or otherwise, this requires a full enumeration of the Azure Blob storage account.
This has two net impacts to operators using Azure Storage Providers
- The performance issues as noted
- Additional billing in Azure as there is pricing for the number of reads etc. (Granted lower costs, but still a cost)
I think we need to look at better strategies for caching, or updating the cache for this provider.
Affected Versions
9.13.2 (latest release), 9.13.3 (alpha), 10.0.0 (alpha)
What browsers are you seeing the problem on?
Firefox, Chrome, Safari, Microsoft Edge
Code of Conduct
- I agree to follow this project's Code of Conduct