Skip to content

Fix the incorrect value of the max thread size (#9881)#9882

Merged
ti-chi-bot[bot] merged 13 commits intopingcap:release-8.1from
ti-chi-bot:cherry-pick-9881-to-release-8.1
Feb 18, 2025
Merged

Fix the incorrect value of the max thread size (#9881)#9882
ti-chi-bot[bot] merged 13 commits intopingcap:release-8.1from
ti-chi-bot:cherry-pick-9881-to-release-8.1

Conversation

@ti-chi-bot
Copy link
Member

@ti-chi-bot ti-chi-bot commented Feb 18, 2025

This is an automated cherry-pick of #9881

What problem does this PR solve?

Issue Number: close #9880, ref #9745

Problem Summary:

See the issue. We make setAuto not static as a workaround. However, we have to handle the coupling of Proxy and Settings.h.

What is changed and how it works?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

@CalvinNeo @ti-chi-bot
This is an automated cherry-pick of pingcap#9881
4c5fce8 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-8.1 This PR is cherry-picked to release-8.1 from a source PR. labels Feb 18, 2025
@ti-chi-bot ti-chi-bot mentioned this pull request Feb 18, 2025
Merged
12 tasks
@ti-chi-bot
Copy link
Member Author

ti-chi-bot commented Feb 18, 2025

@CalvinNeo This PR has conflicts, I have hold it.
Please resolve them or ask others to resolve them, then comment /unhold to remove the hold label.

@ti-chi-bot ti-chi-bot bot added do-not-merge/cherry-pick-not-approved size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 18, 2025
@CalvinNeo
resolve
67f9a13 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
CalvinNeo
CalvinNeo approved these changes Feb 18, 2025
View reviewed changes
Copy link
Member

@CalvinNeo CalvinNeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved cherry-pick-approved Cherry pick PR approved by release team. and removed do-not-merge/cherry-pick-not-approved labels Feb 18, 2025
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025

/hold

@CalvinNeo
readapt
5e1f1a4 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Feb 18, 2025
@CalvinNeo
test
68ea6ee 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 18, 2025
@CalvinNeo
f
696de68 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025

/hold

@ti-chi-bot ti-chi-bot bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 18, 2025
@CalvinNeo
debug
a450de5 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025 

[2025/02/18 12:53:36.798 +08:00] [INFO] [Server.cpp:1117] ["wait for tiflash proxy initializing"] [thread_id=1]
[2025/02/18 12:53:37.198 +08:00] [INFO] [Server.cpp:1120] ["tiflash proxy is initialized"] [thread_id=1]
[2025/02/18 12:53:37.216 +08:00] [ERROR] [BaseDaemon.cpp:367] [########################################] [source=BaseDaemon] [thread_id=2]
[2025/02/18 12:53:37.216 +08:00] [ERROR] [BaseDaemon.cpp:368] ["(from thread 1) Received signal Segmentation fault(11)."] [source=BaseDaemon] [thread_id=2]
[2025/02/18 12:53:37.216 +08:00] [ERROR] [BaseDaemon.cpp:398] ["Address: 0x7ffe0056b720"] [source=BaseDaemon] [thread_id=2]
[2025/02/18 12:53:37.216 +08:00] [ERROR] [BaseDaemon.cpp:404] ["Access: read."] [source=BaseDaemon] [thread_id=2]
[2025/02/18 12:53:37.216 +08:00] [ERROR] [BaseDaemon.cpp:410] ["Attempted access has violated the permissions assigned to the memory area."] [source=BaseDaemon] [thread_id=2]
[2025/02/18 12:53:37.357 +08:00] [ERROR] [BaseDaemon.cpp:560] ["\n       0xc8bc34b\tfaultSignalHandler(int, siginfo_t*, void*) [tiflash+210486091]\n  0x7f86190e9630\t<unknown symbol> [libpthread.so.0+63024]\n  0x7ffe0056b720\t<unknown symbol>\n       0x441124c\tDB::Server::main(std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > > const&) [tiflash+71373388]\n       0xf9f5945\tPoco::Util::Application::run() [tiflash+262101317]\n       0xfa11915\tPoco::Util::ServerApplication::run() [tiflash+262215957]\n       0x440bd97\tDB::Server::run() [tiflash+71351703]\n       0xfa11af3\tPoco::Util::ServerApplication::run(int, char**) [tiflash+262216435]\n       0x441c438\tmainEntryClickHouseServer(int, char**) [tiflash+71418936]\n       0x42c1951\tmain [tiflash+69998929]\n  0x7f8618914555\t__libc_start_main [libc.so.6+140629]"] [source=BaseDaemon] [thread_id=2]

@CalvinNeo
d2
29d7e8a 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
fix
57d935e 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
fix
ccca77a 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
fix
6551eba 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
fix
1a68513 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025 

/home/jenkins/agent/workspace/tiflash-build-common/tiflash/dbms/src/Storages/DeltaMerge/tests/gtest_segment_read_task.cpp:468: Failure
Expected equality of these values:
  persist_set_wn->getColumnFileCount()
    Which is: 5
  4

@CalvinNeo
remove
eeb81d0 
Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
@CalvinNeo
Copy link
Member

CalvinNeo commented Feb 18, 2025

/cc @JaySon-Huang

@ti-chi-bot ti-chi-bot bot requested a review from JaySon-Huang February 18, 2025 10:14
@JaySon-Huang
Set app_type disagg right after global_context created
2b64eb4 
Signed-off-by: JaySon-Huang <tshent@qq.com>
JaySon-Huang
JaySon-Huang reviewed Feb 18, 2025
View reviewed changes
dbms/src/Server/Server.cpp
Comment on lines +1081 to +1083
global_context->setApplicationType(Context::ApplicationType::SERVER);
global_context->getSharedContextDisagg()->disaggregated_mode = disaggregated_mode;
global_context->getSharedContextDisagg()->use_autoscaler = use_autoscaler;
Copy link
Contributor

@JaySon-Huang JaySon-Huang Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also pick the change of #9886

JaySon-Huang
JaySon-Huang approved these changes Feb 18, 2025
View reviewed changes
Copy link
Contributor

@JaySon-Huang JaySon-Huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Feb 18, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Feb 18, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CalvinNeo, JaySon-Huang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [CalvinNeo,JaySon-Huang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Feb 18, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-02-18 03:01:29.303072687 +0000 UTC m=+930331.699294743: ☑️ agreed by CalvinNeo.
  • 2025-02-18 10:23:37.726874383 +0000 UTC m=+956860.123096445: ☑️ agreed by JaySon-Huang.

@JaySon-Huang
Copy link
Contributor

JaySon-Huang commented Feb 18, 2025

/unhold

@ti-chi-bot ti-chi-bot bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 18, 2025
@ti-chi-bot ti-chi-bot bot merged commit bdb33ae into pingcap:release-8.1 Feb 18, 2025
4 checks passed
JaySon-Huang added a commit to JaySon-Huang/tiflash that referenced this pull request Apr 25, 2025
@ti-chi-bot @CalvinNeo @JaySon-Huang
Fix the incorrect value of the max thread size (pingcap#9881) (pingca…
dbf3e4c 
…p#9882)

ref pingcap#9745, close pingcap#9880

Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>

Co-authored-by: Calvin Neo <calvinneo1995@gmail.com>
Co-authored-by: JaySon-Huang <tshent@qq.com>
@JaySon-Huang JaySon-Huang mentioned this pull request Apr 25, 2025
Merged
12 tasks
JaySon-Huang added a commit that referenced this pull request Apr 25, 2025
@JaySon-Huang @CalvinNeo @ti-chi-bot
[hotfix] Fix stale read can not eliminate the learner read on TiFlash (…
8dbb1ba 
…#10134)

Note:
* This change to the FFI interface for proxy adaptation. And the related
issue is currently suspected to potentially cause fluctuations in
LearnerRead latency in some cases.
#9841,
#9882
* This change to the FFI interface for proxy adaptation
#9893
* Fix stale read can not eliminate the learner read
#10132


### What problem does this PR solve?

Issue Number: ref #4982, close
#10046

Problem Summary:
 update proxy to raftstore-proxy-8.1
 Proxy PR: 
 Including:\nSubmodule contrib/tiflash-proxy caa7a0c1dd..96545a2632:
  > Fix safe ts not updated (#429) (#434)

The leader_safe_ts, self_safe_ts are always 0 in the TiFlash side,
making TiFlash can not eliminate the learner read on region without any
updates.

### What is changed and how it works?

For detailed reasons why safe_ts failed to be updated, checkout the
description on pingcap/tidb-engine-ext#429.

### Check List

Tests <!-- At least one of them must be included. -->

- [ ] Unit test
- [ ] Integration test
- [x] Manual test (add detailed scripts or steps below)
- [ ] No code

Side effects

- [ ] Performance regression: Consumes more CPU
- [ ] Performance regression: Consumes more Memory
- [ ] Breaking backward compatibility

Documentation

- [ ] Affects user behaviors
- [ ] Contains syntax changes
- [ ] Contains variable changes
- [ ] Contains experimental features
- [ ] Changes MySQL compatibility

### Release note

<!-- bugfix or new feature needs a release note -->

```release-note
Fix the issue that `SELECT ... AS OF TIMESTAMP` can not eliminate the learner read on TiFlash as expected
```

---------

Signed-off-by: Calvin Neo <calvinneo1995@gmail.com>
Signed-off-by: JaySon-Huang <tshent@qq.com>
Signed-off-by: CalvinNeo <calvinneo1995@gmail.com>
Co-authored-by: Calvin Neo <CalvinNeo@users.noreply.github.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: Calvin Neo <calvinneo1995@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JaySon-Huang JaySon-Huang JaySon-Huang approved these changes

@CalvinNeo CalvinNeo CalvinNeo approved these changes

Assignees

@CalvinNeo CalvinNeo

Labels

approved cherry-pick-approved Cherry pick PR approved by release team. lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. type/cherry-pick-for-release-8.1 This PR is cherry-picked to release-8.1 from a source PR.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ti-chi-bot @CalvinNeo @JaySon-Huang