Skip to content

Fix db_wal_test::TruncateLastLogAfterRecoverWithoutFlush failure#6437

Closed
guyuqi wants to merge 2 commits intofacebook:masterfrom
guyuqi:fix-db-wal-test
Closed

Fix db_wal_test::TruncateLastLogAfterRecoverWithoutFlush failure#6437
guyuqi wants to merge 2 commits intofacebook:masterfrom
guyuqi:fix-db-wal-test

Conversation

@guyuqi
Copy link
Copy Markdown
Contributor

@guyuqi guyuqi commented Feb 20, 2020

TruncateLastLogAfterRecoverWithoutFlush case depends on fallocate support
of underlying file system.

On a file system which lacks of this feature, like zfs, it will fail to allocate predefined file size as this test case intends to do;

So a check block is added to detect fallocate support and skip test if not.
The related work is done by @JunHe77. Thanks!

Signed-off-by: Yuqi Gu yuqi.gu@arm.com

TruncateLastLogAfterRecoverWithoutFlush case depends on fallocate support
of underlying file system. On a file system which lacks of this feature,
like zfs, it will fail to allocate predefined file size as this test case
intends to do;
So a check block is added to detect fallocate support and skip test if not.

Change-Id: I731f89ef39b062bf65e8e495a0018c32a4ecb76a
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
@adamretter
Copy link
Copy Markdown
Collaborator

adamretter commented Feb 20, 2020

@guyuqi So we have a compile time flag ROCKSDB_FALLOCATE_PRESENT which is calculated, and controls aspects of the code that depend on fallocate support. See - https://github.com/facebook/rocksdb/blob/master/build_tools/build_detect_platform#L248

The code that you patch is only enabled when that flag is true. So it would seem to me that your patch indicates a problem with detecting a working fallocate (or some feature of it - Is this some combination of fallocate and truncate?).

Either way, it would seem prudent to fix the detection (so that it can be used across the codebase), either:

  1. modifying the existing conditions for ROCKSDB_FALLOCATE_PRESENT, or
  2. if this is about fallocate and truncate working together, then adding an additional flag

What do you think?

@guyuqi
Copy link
Copy Markdown
Contributor Author

guyuqi commented Feb 21, 2020

@adamretter
The ROCKSDB_FALLOCATE_PRESENT is the flag for compile-time.
This PR is to check whether fallocate is supported or not for run-time.

So this patch is needed when binary code run on a node which doesn't support fallocate though the code was compiled properly on another node where ROCKSDB_FALLOCATE_PRESENT was detected as 'true'.

@adamretter
Copy link
Copy Markdown
Collaborator

@guyuqi I am a bit confused, as you said it is needed for #6436

But in that PR (i.e. for Travis) the code is compiled and run on the same platform.

@guyuqi
Copy link
Copy Markdown
Contributor Author

guyuqi commented Feb 25, 2020

the code is compiled and run on the same platform.

@adamretter
In https://github.com/facebook/rocksdb/blob/master/build_tools/build_detect_platform#L248-L261:
fallocate(fd, FALLOC_FL_KEEP_SIZE, 0, 1024); wouldn't be executed actually.
So It just includes the head file linux/falloc.h and enabled ROCKSDB_FALLOCATE_PRESENT and the CI test would failed.

@JunHe77
Copy link
Copy Markdown
Contributor

JunHe77 commented Feb 25, 2020

@adamretter as you mentioned in build_detect_platform, this flag is validated in compile time by checking if the test prog can be built or not, which means as long as <linux/falloc.h> is presented, this macro ROCKSDB_FALLOCATE_PRESENT will be defined.
But if underlying file system doesn't support fallocate feature, this test will fail to execute even on the very same machine where build is done.
Example:
Built passed but execution failed: https://travis-ci.org/JunHe77/rocksdb/jobs/654724257#L329-L367

@guyuqi
Copy link
Copy Markdown
Contributor Author

guyuqi commented Feb 25, 2020

@siying , could you please have a look into this PR?

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@siying has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

close(fd);
ASSERT_OK(options.env->DeleteFile(fname_test_fallocate));
if (alloc_status != 0) {
fprintf(stderr, " Skipped preallocated space check: %s\n", strerror(err_number));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be more comfortable if we only skip when err_number == ENOSYS || err_number == EOPNOTSUPP and otherwise fail the test on alloc_status != 0.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.
Thanks for your comments.

Change-Id: I95d7326e18939d5bdfbc0ed064f1b3d5127f5f66
Signed-off-by: Yuqi Gu <yuqi.gu@arm.com>
@facebook-github-bot
Copy link
Copy Markdown
Contributor

@guyuqi has updated the pull request. Re-import the pull request

Copy link
Copy Markdown
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdillinger has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Copy Markdown
Contributor

@pdillinger pdillinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! I think we now have adequate protection against the test accidentally, quietly being broadly skipped.

@pdillinger pdillinger self-assigned this Mar 5, 2020
@facebook-github-bot
Copy link
Copy Markdown
Contributor

@pdillinger merged this pull request in e171a21.

sthagen added a commit to sthagen/facebook-rocksdb that referenced this pull request Mar 6, 2020
Fix db_wal_test::TruncateLastLogAfterRecoverWithoutFlush failure (facebook#6437)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants