Skip to content

Sort during find results#947

Closed
shargon wants to merge 4 commits intoneo-project:masterfrom
shargon:sort-find-keys
Closed

Sort during find results#947
shargon wants to merge 4 commits intoneo-project:masterfrom
shargon:sort-find-keys

Conversation

@shargon
Copy link
Copy Markdown
Member

@shargon shargon commented Jul 23, 2019

Close #946

@codecov-io
Copy link
Copy Markdown

codecov-io commented Jul 23, 2019

Codecov Report

Merging #947 into master will increase coverage by 0.33%.
The diff coverage is 89.47%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #947      +/-   ##
==========================================
+ Coverage   45.39%   45.72%   +0.33%     
==========================================
  Files         178      178              
  Lines       12662    12732      +70     
==========================================
+ Hits         5748     5822      +74     
+ Misses       6914     6910       -4
Impacted Files Coverage Δ
neo/IO/Caching/DataCache.cs 73.66% <89.47%> (+8.34%) ⬆️
neo/IO/Caching/CloneCache.cs 60.86% <0%> (+17.39%) ⬆️
neo/Ledger/StorageKey.cs 82.6% <0%> (+17.39%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d048805...aa0d92b. Read the comment docs.

{
lock (dictionary)
{
return FindUnsorted(key_prefix).OrderBy(u => u.Key, this);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This forces the entire set of search results to be loaded into memory for sorting, which can lead to attacks.

Copy link
Copy Markdown
Member

@vncoelho vncoelho Jul 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still nowadays I have dificulties with C#.

Looking at it I would think that it finds and returns the set, then, it sorts the results.

In this sense, maybe it needs to be split into two steps, right, @erikzhang?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to do some optimization. Because the records read from the underlying storage (leveldb) is already sorted.

Copy link
Copy Markdown
Member

@vncoelho vncoelho Jul 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, master.
We are going to reflect on that and think about an efficient way.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will propose another way

@shargon shargon requested a review from erikzhang July 23, 2019 12:59

// We need to return a, b and cached results between them

if (a.Key == null && a.Value == null)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReferenceEquals(a,n) doesn't work :S

@erikzhang
Copy link
Copy Markdown
Member

@shargon I implemented a more optimized fix, please take a look. #950

shargon added a commit to shargon/neo that referenced this pull request Jul 24, 2019
shargon added a commit that referenced this pull request Jul 24, 2019
@erikzhang erikzhang mentioned this pull request Jul 24, 2019
@shargon shargon deleted the sort-find-keys branch July 30, 2019 07:53
Thacryba pushed a commit to simplitech/neo that referenced this pull request Feb 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Storage.Find returns items out of order

4 participants