Skip to content

Cache nonexsistent key in datacache#1599

Closed
Qiao-Jin wants to merge 10 commits intoneo-project:masterfrom
Qiao-Jin:cache_nonexistent_key_in_datacache
Closed

Cache nonexsistent key in datacache#1599
Qiao-Jin wants to merge 10 commits intoneo-project:masterfrom
Qiao-Jin:cache_nonexistent_key_in_datacache

Conversation

@Qiao-Jin
Copy link
Copy Markdown
Contributor

@Qiao-Jin Qiao-Jin commented Apr 23, 2020

Close #1600

}

private readonly Dictionary<TKey, Trackable> dictionary = new Dictionary<TKey, Trackable>();
private readonly HashSet<TKey> nonexistentKeySet = new HashSet<TKey>();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should have a maximum number of entries.

Copy link
Copy Markdown
Contributor Author

@Qiao-Jin Qiao-Jin Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I also think so. I think we should have maximum number of entries both for this hashset and dictionary above.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nonexistentKeySet is a bloom filter.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it could be a bloom filter.

Copy link
Copy Markdown
Member

@erikzhang erikzhang Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But there is a bloom filter in leveldb, why do we need another?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll test the leveldb bloom filter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After testing it seems situation is much better after a 20-bit bloom filter added to levelDB startup. But still it will take somehow longer time for persisting than this commit, maybe due to more calling stacks

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Tommo-L
Copy link
Copy Markdown
Contributor

Tommo-L commented May 29, 2020

Replaced by neo-project/neo-modules#253

@cloud8little
Copy link
Copy Markdown
Contributor

cloud8little commented May 29, 2020

Test Env:
OS: Windows 10 X64
CPU: 4 Core, Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz,1498 Mhz
RAM: 16GB

Test Condition:

Transaction send speed: 2000 tx/s
Consensus node number: 1
Transaction sender node number: 1
Block time: 15s
Functional Smoke Test Passed:

send neo/gas
deploy contracts.
single CN consensus
TPS Test Result:
pr1507 TPS: 850 Total TX: 678,937
pr1507+pr1666 TPS: 826 Total TX: 754,246
pr1507+pr1666 + pr1599 TPS: 898 Total TX: 999,783

a74197f948ce6f7efa154d14f8c008e

b35f7a0474ff5b3436717abbbe17f1d

@erikzhang
Copy link
Copy Markdown
Member

This can be closed?

@Qiao-Jin
Copy link
Copy Markdown
Contributor Author

Qiao-Jin commented Jun 5, 2020

Closed due to Add BloomFilter to levelDB

@Qiao-Jin Qiao-Jin closed this Jun 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cache nonexsistent key in datacache

5 participants