Skip to content

Introduce proxy_server and new-mock-engine-store#120

Merged
CalvinNeo merged 23 commits intopingcap:raftstore-proxy-6.2from
CalvinNeo:try-add-new-mock-engine-store-in-6.2
Jul 14, 2022
Merged

Introduce proxy_server and new-mock-engine-store#120
CalvinNeo merged 23 commits intopingcap:raftstore-proxy-6.2from
CalvinNeo:try-add-new-mock-engine-store-in-6.2

Conversation

@CalvinNeo
Copy link
Member

@CalvinNeo CalvinNeo commented Jul 11, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: ref pingcap/tiflash#5170

Problem Summary:

  1. Introduce crate proxy_server for a decouple of component/server, it will be used to start a proxy service.
  2. Introduce crate new-mock-engine-store as a new mock-engine-store plus a test_raftstore implementation for proxy. This will enable us to run mix-mode tests. Tests in tests/failpoints and tests/integrations are obsolete tests and will be removed once the refactor ends. New tests should be added in tests/proxy with support of new-mock-engine-store.
  3. Introduce crate engine_tiflash as a new KvEngine. It currently just works like former RocksEngine, unless we enable compat_new_proxy. Since some PRs have not been merged in TiKV, we can only enable this feature in some tests in tests/proxy. It can not be enabled in formal codes, since it can filter some writes.
  4. cherry pick newly merge TiKV PRs, aka on_empty_cmd.
  5. Fix some bugs.

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

CalvinNeo and others added 18 commits July 4, 2022 21:48
@CalvinNeo
f
0b7bf7f 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix
fea5a4a 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
add engine_tiflash with this version of engine_rocks
67c47da 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fmt
c96c0c3 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy-6.2' into try-…
a3d6722 
…add-new-mock-engine-store-in-6.2
@CalvinNeo
add engine_tiflash actually
e3a716b 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix
9bfec36 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
fix some tests
5afb804 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
0f38d9d 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
it runs if we do_write everytime
ed1708b 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
add ensure_no_common_unrecognized_keys to config checker
8ab1acb 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
reorg tests
5237c17 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo @ti-chi-bot
raftstore: Implement coprocessor observer on_empty_cmd (tikv#12851)
4d4c583 
ref tikv#12849

Support new observers on_empty_cmd.

Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
@CalvinNeo
fix tests
b147018 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
move proxy-related code in component/server into proxy_server
3d50bcc 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
Merge remote-tracking branch 'upstream/raftstore-proxy-6.2' into try-…
8833251 
…add-new-mock-engine-store-in-6.2
@CalvinNeo
move codes in component/server/src/setup.rs into proxy_server
f0ad93d 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
support new ffis in mock-engine-store and new-mock-engine-store
453eb87 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo CalvinNeo mentioned this pull request Jul 11, 2022
Closed
@CalvinNeo CalvinNeo changed the title Try add new mock engine store in 6.2 Introduce proxy_server and new-mock-engine-store Jul 11, 2022
@CalvinNeo
add disclaimer
573efc6 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo CalvinNeo mentioned this pull request Jul 11, 2022
Merged
12 tasks
@CalvinNeo
enlength some test time
af8b448 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo CalvinNeo requested a review from hehechen July 13, 2022 11:43
CalvinNeo added 3 commits July 14, 2022 12:35
@CalvinNeo
Remove pub fields of PdCluster, add feature to origin import
cb5bc33 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
f
2559388 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo
add
a4c3483 
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
@CalvinNeo CalvinNeo merged commit b33b1eb into pingcap:raftstore-proxy-6.2 Jul 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@hehechen hehechen hehechen approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@CalvinNeo @hehechen