Don't block the exclusion of stateless processes by the free capacity check #9468

johscheuer · 2023-02-24T10:39:51Z

Currently the exclude command will block exclusions if the (new) free capacity is below 10%. This also blocks exclusions for processes that don't serve a storage role e.g. stateless processes. I don't think it make sense to block the exclusion of a stateless process based on the free capacity. With those changes the free capacity check will be skipped if no process in the exclusion set is serving a storage role, if at least one process is serving a storage role the free capacity check will be done.

Code-Reviewer Section

The general pull request guidelines can be found here.

Please check each of the following things and check all boxes before accepting a PR.

The PR has a description, explaining both the problem and the solution.
The description mentions which forms of testing were done and the testing seems reasonable.
Every function/class/actor that was touched is reasonably well documented.

For Release-Branches

If this PR is made against a release-branch, please also check the following:

This change/bugfix is a cherry-pick from the next younger branch (younger release-branch or main if this is the youngest branch)
There is a good reason why this PR needs to go into a release branch and this reason is documented (either in the description above or in a linked GitHub issue)

fdb-windows-ci · 2023-02-24T10:45:15Z

Doxense CI Report for Windows 10

Commit ID: 8083f6e
Result: ❌ FAILED
Build Logs (available for 30 days)

foundationdb-ci · 2023-02-24T10:46:32Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 0de74d1
Duration 0:04:05
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:46:32Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 0de74d1
Duration 0:04:04
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:46:49Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 0de74d1
Duration 0:04:18
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:47:52Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 8083f6e
Duration 0:07:50
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:47:55Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 8083f6e
Duration 0:07:54
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:48:04Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 8083f6e
Duration 0:08:04
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:48:59Z

Result of foundationdb-pr-macos on macOS Monterey 12.x

Commit ID: 8083f6e
Duration 0:08:57
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:49:08Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 0de74d1
Duration 0:06:41
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-02-24T10:49:39Z

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Commit ID: 8083f6e
Duration 0:09:41
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-02-24T10:50:20Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 8083f6e
Duration 0:10:23
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

fdb-windows-ci · 2023-02-24T11:13:39Z

Doxense CI Report for Windows 10

Commit ID: 0de74d1
Result: ❌ FAILED
Build Logs (available for 30 days)

fdbclient/SpecialKeySpace.actor.cpp

johscheuer

I start working on this tomorrow and bring it into a better state.

… check

foundationdb-ci · 2023-03-08T07:24:10Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 1550f3c
Duration 0:04:03
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-08T07:24:20Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 1550f3c
Duration 0:04:14
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-08T07:24:20Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 1550f3c
Duration 0:04:15
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-08T07:24:28Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 1550f3c
Duration 0:04:19
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-03-08T07:26:30Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 5673062
Duration 0:08:31
Result: ❌ FAILED
Error: reference not found for primary source and source version 56730628b5e3c0d48b2044e35feb92b048c9c87e
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

foundationdb-ci · 2023-03-08T07:26:30Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 5673062
Duration 0:08:31
Result: ❌ FAILED
Error: reference not found for primary source and source version 56730628b5e3c0d48b2044e35feb92b048c9c87e
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

sbodagala · 2023-03-08T22:54:18Z

Also, you might want to run simulation tests over this change set (the default testing that all server side PRs do), by invoking "jsd" in octeto (it is a function defined in .bashrc).

foundationdb-ci · 2023-03-15T14:40:48Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: b317928
Duration 0:04:13
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-15T14:40:50Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: b317928
Duration 0:04:16
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-15T14:40:52Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: b317928
Duration 0:04:15
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-15T14:40:57Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: b317928
Duration 0:04:19
Result: ❌ FAILED
Error: Error while executing command: if [[ $(git diff --shortstat 2> /dev/null | tail -n1) == "" ]]; then echo "CODE FORMAT CLEAN"; else echo "CODE FORMAT NOT CLEAN"; echo; echo "THE FOLLOWING FILES NEED TO BE FORMATTED"; echo; git ls-files -m; echo; exit 1; fi. Reason: exit status 1
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

fdb-windows-ci · 2023-03-15T16:04:57Z

Doxense CI Report for Windows 10

Commit ID: b317928
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

jzhou77

LGTM. This PR needs to be reformatted.

jzhou77 · 2023-03-15T16:22:43Z

fdbclient/SpecialKeySpace.actor.cpp

+					// don't have to check any further since we don't case in this variable about the count of excluded
+					// storage servers but only about if we exclude any storage server with the provided addresses.


Please rephrase this comment.

Updated, I hope this sounds better.

foundationdb-ci · 2023-03-22T15:45:56Z

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Commit ID: 694263a
Duration 0:14:19
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

fdb-windows-ci · 2023-03-22T15:51:10Z

Doxense CI Report for Windows 10

Commit ID: 694263a
Result: ✔️ SUCCEEDED
Build Logs (available for 30 days)

foundationdb-ci · 2023-03-22T15:53:13Z

Result of foundationdb-pr-clang on Linux CentOS 7

Commit ID: 694263a
Duration 0:21:38
Result: ❌ FAILED
Error: Error while executing command: ctest -j ${NPROC} --no-compress-output -T test --output-on-failure. Reason: exit status 8
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-22T16:00:05Z

Result of foundationdb-pr-macos-m1 on macOS Ventura 13.x

Commit ID: 694263a
Duration 0:28:31
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-22T16:08:42Z

Result of foundationdb-pr-macos on macOS Ventura 13.x

Commit ID: 694263a
Duration 0:37:10
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-22T16:41:40Z

Result of foundationdb-pr on Linux CentOS 7

Commit ID: 694263a
Duration 1:10:05
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)

foundationdb-ci · 2023-03-22T16:55:09Z

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Commit ID: 694263a
Duration 1:23:35
Result: ✅ SUCCEEDED
Error: N/A
Build Log terminal output (available for 30 days)
Build Workspace zip file of the working directory (available for 30 days)
Cluster Test Logs zip file of the test logs (available for 30 days)

johscheuer requested review from jzhou77 and sbodagala February 24, 2023 10:39

sbodagala reviewed Feb 24, 2023

View reviewed changes

fdbclient/SpecialKeySpace.actor.cpp Outdated Show resolved Hide resolved

sbodagala reviewed Feb 24, 2023

View reviewed changes

fdbclient/SpecialKeySpace.actor.cpp Outdated Show resolved Hide resolved

sbodagala previously approved these changes Feb 24, 2023

View reviewed changes

johscheuer commented Mar 6, 2023

View reviewed changes

johscheuer dismissed sbodagala’s stale review via 5673062 March 8, 2023 07:17

johscheuer added 3 commits March 8, 2023 08:19

Don't block the exclusion of stateless processes by the free capacity…

db8c60c

… check

Fix syntax

bae627f

Make use of precomputed exclude check

1550f3c

johscheuer force-pushed the dont-block-exclusion-stateless-processes branch from 5673062 to 1550f3c Compare March 8, 2023 07:19

johscheuer marked this pull request as ready for review March 8, 2023 07:20

Only consider newly excluded processes

b317928

johscheuer requested a review from sbodagala March 15, 2023 14:36

jzhou77 reviewed Mar 15, 2023

View reviewed changes

sbodagala previously approved these changes Mar 15, 2023

View reviewed changes

Format code and update comment

694263a

johscheuer dismissed sbodagala’s stale review via 694263a March 22, 2023 15:31

johscheuer requested review from jzhou77 and sbodagala March 22, 2023 15:34

jzhou77 approved these changes Mar 22, 2023

View reviewed changes

sbodagala approved these changes Mar 22, 2023

View reviewed changes

jzhou77 merged commit 18a0fa0 into apple:main Mar 23, 2023

johscheuer mentioned this pull request Mar 23, 2023

Backport exclusion fix #9468 #9789

Merged

5 tasks

johscheuer deleted the dont-block-exclusion-stateless-processes branch March 23, 2023 16:24

johscheuer mentioned this pull request Apr 4, 2023

[Relase 7.2] Don't block the exclusion of stateless processes by the free capacity check #9892

Merged

5 tasks

		// don't have to check any further since we don't case in this variable about the count of excluded
		// storage servers but only about if we exclude any storage server with the provided addresses.

Don't block the exclusion of stateless processes by the free capacity check #9468

Don't block the exclusion of stateless processes by the free capacity check #9468

Uh oh!

Conversation

johscheuer commented Feb 24, 2023

Code-Reviewer Section

For Release-Branches

Uh oh!

fdb-windows-ci commented Feb 24, 2023

Doxense CI Report for Windows 10

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-macos on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-macos-m1 on macOS Monterey 12.x

Uh oh!

foundationdb-ci commented Feb 24, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

fdb-windows-ci commented Feb 24, 2023

Doxense CI Report for Windows 10

Uh oh!

Uh oh!

Uh oh!

johscheuer left a comment

Choose a reason for hiding this comment

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 8, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

sbodagala commented Mar 8, 2023

Uh oh!

foundationdb-ci commented Mar 15, 2023

Result of foundationdb-pr on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 15, 2023

Result of foundationdb-pr-clang-ide on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 15, 2023

Result of foundationdb-pr-clang on Linux CentOS 7

Uh oh!

foundationdb-ci commented Mar 15, 2023

Result of foundationdb-pr-cluster-tests on Linux CentOS 7

jzhou77 left a comment •

edited

Loading