Skip to content

test/objectstore/store_test: kill ExcessiveFragmentation test case.#35318

Merged
tchaikov merged 1 commit intoceph:masterfrom
ifed01:wip-ifed-fix-ex-frag-test
Jun 11, 2020
Merged

test/objectstore/store_test: kill ExcessiveFragmentation test case.#35318
tchaikov merged 1 commit intoceph:masterfrom
ifed01:wip-ifed-fix-ex-frag-test

Conversation

@ifed01
Copy link
Contributor

@ifed01 ifed01 commented May 29, 2020 •
edited
Loading

Fixes: https://tracker.ceph.com/issues/45788

Signed-off-by: Igor Fedotov ifedotov@suse.com

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard backend
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@ifed01 ifed01 requested a review from neha-ojha May 29, 2020 19:31
@yuriw
Copy link
Contributor

yuriw commented May 29, 2020

wip-yuri2-testing-2020-05-29-2208-MASTER

@neha-ojha
Copy link
Member

neha-ojha commented Jun 1, 2020

@ifed01 Failing with

2020-05-30T09:46:12.015 INFO:teuthology.orchestra.run.smithi117.stdout:[ RUN      ] ObjectStore/StoreTestSpecificAUSize.ExcessiveFragmentation/2
2020-05-30T09:46:18.305 INFO:teuthology.orchestra.run.smithi117.stdout:written 104857600
2020-05-30T09:46:23.863 INFO:teuthology.orchestra.run.smithi117.stdout:written 209715200
2020-05-30T09:46:30.755 INFO:teuthology.orchestra.run.smithi117.stdout:written 314572800
2020-05-30T09:46:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:/build/ceph-16.0.0-2022-g13caff82147/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)' thread 7f195fce80c0 time 2020-05-30T09:46:37.208782+0000
2020-05-30T09:46:37.216 INFO:teuthology.orchestra.run.smithi117.stderr:/build/ceph-16.0.0-2022-g13caff82147/src/os/bluestore/BlueStore.cc: 12863: ceph_abort_msg("unexpected error")
2020-05-30T09:46:37.217 INFO:teuthology.orchestra.run.smithi117.stderr: ceph version 16.0.0-2022-g13caff82147 (13caff821474978bc0100ccc82c519b90575e1dd) pacific (dev)
2020-05-30T09:46:37.217 INFO:teuthology.orchestra.run.smithi117.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe1) [0x7f1955b81cd8]
2020-05-30T09:46:37.217 INFO:teuthology.orchestra.run.smithi117.stderr: 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0xc47) [0x56210db02287]
2020-05-30T09:46:37.217 INFO:teuthology.orchestra.run.smithi117.stderr: 3: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x2c3) [0x56210db04be3]
2020-05-30T09:46:37.218 INFO:teuthology.orchestra.run.smithi117.stderr: 4: (ObjectStore::queue_transaction(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ceph::os::Transaction&&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x82) [0x56210d9700c2]
2020-05-30T09:46:37.218 INFO:teuthology.orchestra.run.smithi117.stderr: 5: (int queue_transaction<boost::scoped_ptr<ObjectStore> >(boost::scoped_ptr<ObjectStore>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>, ceph::os::Transaction&&)+0x20d) [0x56210d974a1d]
2020-05-30T09:46:37.218 INFO:teuthology.orchestra.run.smithi117.stderr: 6: (StoreTestSpecificAUSize_ExcessiveFragmentation_Test::TestBody()+0x5d3) [0x56210d901053]
2020-05-30T09:46:37.218 INFO:teuthology.orchestra.run.smithi117.stderr: 7: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x77) [0x56210dc16b07]
2020-05-30T09:46:37.219 INFO:teuthology.orchestra.run.smithi117.stderr: 8: (testing::Test::Run()+0xc3) [0x56210dc0cdc3]
2020-05-30T09:46:37.219 INFO:teuthology.orchestra.run.smithi117.stderr: 9: (testing::TestInfo::Run()+0x135) [0x56210dc0cf25]
2020-05-30T09:46:37.219 INFO:teuthology.orchestra.run.smithi117.stderr: 10: (testing::TestSuite::Run()+0xb5) [0x56210dc0d005]
2020-05-30T09:46:37.219 INFO:teuthology.orchestra.run.smithi117.stderr: 11: (testing::internal::UnitTestImpl::RunAllTests()+0x488) [0x56210dc0d528]
2020-05-30T09:46:37.220 INFO:teuthology.orchestra.run.smithi117.stderr: 12: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x77) [0x56210dc17017]
2020-05-30T09:46:37.220 INFO:teuthology.orchestra.run.smithi117.stderr: 13: (testing::UnitTest::Run()+0x98) [0x56210dc0d738]
2020-05-30T09:46:37.220 INFO:teuthology.orchestra.run.smithi117.stderr: 14: (main()+0x6fc) [0x56210d8a8fcc]
2020-05-30T09:46:37.220 INFO:teuthology.orchestra.run.smithi117.stderr: 15: (__libc_start_main()+0xe7) [0x7f1954736b97]
2020-05-30T09:46:37.221 INFO:teuthology.orchestra.run.smithi117.stderr: 16: (_start()+0x2a) [0x56210d8b07ba]

http://pulpito.ceph.com/yuriw-2020-05-30_02:19:25-rados-wip-yuri2-testing-2020-05-29-2208-MASTER-distro-basic-smithi/5104626/

@ifed01
Copy link
Contributor Author

ifed01 commented Jun 1, 2020

@ifed01 Failing with

This needs Adam's fix to prevent DB/WAL growth:
#35277

@neha-ojha
Copy link
Member

neha-ojha commented Jun 3, 2020

@ifed01 Failing with

This needs Adam's fix to prevent DB/WAL growth:
#35277

@yuriw could you please batch this PR with #35277 and rerun your tests?

@yuriw
Copy link
Contributor

yuriw commented Jun 3, 2020

wip-yuri2-testing-2020-06-03-2341-MASTER

@neha-ojha
Copy link
Member

neha-ojha commented Jun 5, 2020

@ifed01 Failing with

This needs Adam's fix to prevent DB/WAL growth:
#35277

Testing this PR along with #35277 results in a different failure.
http://pulpito.ceph.com/yuriw-2020-06-04_18:03:48-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5118037/

2020-06-05T00:13:43.622 INFO:teuthology.orchestra.run.smithi173.stdout:[ RUN      ] ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixNoCsum/2
.
.
2020-06-05T00:24:59.488 INFO:teuthology.orchestra.run.smithi173.stderr:/build/ceph-16.0.0-2214-g64a1a64/src/test/objectstore/store_test.cc: In function 'virtual void SyntheticWorkloadState::C_SyntheticOnReadable::finish(int)' thread 7fc5db5fe700 time 2020-06-05T00:24:58.993144+0000
2020-06-05T00:24:59.488 INFO:teuthology.orchestra.run.smithi173.stderr:/build/ceph-16.0.0-2214-g64a1a64/src/test/objectstore/store_test.cc: 3813: FAILED ceph_assert(bl_eq(state->contents[hoid].data, r2))
2020-06-05T00:24:59.488 INFO:teuthology.orchestra.run.smithi173.stderr: ceph version 16.0.0-2214-g64a1a64 (64a1a6455982e19dbbfd3edd4f271c0432507975) pacific (dev)
2020-06-05T00:24:59.488 INFO:teuthology.orchestra.run.smithi173.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x154) [0x7fc5ea43955a]
2020-06-05T00:24:59.488 INFO:teuthology.orchestra.run.smithi173.stderr: 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7fc5ea439735]
2020-06-05T00:24:59.489 INFO:teuthology.orchestra.run.smithi173.stderr: 3: (SyntheticWorkloadState::C_SyntheticOnReadable::finish(int)+0x37d) [0x55abcd6bee3d]
2020-06-05T00:24:59.489 INFO:teuthology.orchestra.run.smithi173.stderr: 4: (Context::complete(int)+0x9) [0x55abcd678f39]
2020-06-05T00:24:59.489 INFO:teuthology.orchestra.run.smithi173.stderr: 5: (Finisher::finisher_thread_entry()+0x195) [0x7fc5ea496695]
2020-06-05T00:24:59.489 INFO:teuthology.orchestra.run.smithi173.stderr: 6: (()+0x76db) [0x7fc5f41906db]
2020-06-05T00:24:59.489 INFO:teuthology.orchestra.run.smithi173.stderr: 7: (clone()+0x3f) [0x7fc5e90eb88f]
2020-06-05T00:24:59.490 INFO:teuthology.orchestra.run.smithi173.stderr:*** Caught signal (Aborted) **
2020-06-05T00:24:59.490 INFO:teuthology.orchestra.run.smithi173.stderr: in thread 7fc5db5fe700 thread_name:cfin
2020-06-05T00:24:59.491 INFO:teuthology.orchestra.run.smithi173.stderr: ceph version 16.0.0-2214-g64a1a64 (64a1a6455982e19dbbfd3edd4f271c0432507975) pacific (dev)
2020-06-05T00:24:59.491 INFO:teuthology.orchestra.run.smithi173.stderr: 1: (()+0x12890) [0x7fc5f419b890]
2020-06-05T00:24:59.491 INFO:teuthology.orchestra.run.smithi173.stderr: 2: (gsignal()+0xc7) [0x7fc5e9008e97]
2020-06-05T00:24:59.492 INFO:teuthology.orchestra.run.smithi173.stderr: 3: (abort()+0x141) [0x7fc5e900a801]
2020-06-05T00:24:59.492 INFO:teuthology.orchestra.run.smithi173.stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a5) [0x7fc5ea4395ab]
2020-06-05T00:24:59.492 INFO:teuthology.orchestra.run.smithi173.stderr: 5: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x7fc5ea439735]
2020-06-05T00:24:59.492 INFO:teuthology.orchestra.run.smithi173.stderr: 6: (SyntheticWorkloadState::C_SyntheticOnReadable::finish(int)+0x37d) [0x55abcd6bee3d]
2020-06-05T00:24:59.492 INFO:teuthology.orchestra.run.smithi173.stderr: 7: (Context::complete(int)+0x9) [0x55abcd678f39]
2020-06-05T00:24:59.493 INFO:teuthology.orchestra.run.smithi173.stderr: 8: (Finisher::finisher_thread_entry()+0x195) [0x7fc5ea496695]
2020-06-05T00:24:59.493 INFO:teuthology.orchestra.run.smithi173.stderr: 9: (()+0x76db) [0x7fc5f41906db]
2020-06-05T00:24:59.493 INFO:teuthology.orchestra.run.smithi173.stderr: 10: (clone()+0x3f) [0x7fc5e90eb88f]

@ifed01
Copy link
Contributor Author

ifed01 commented Jun 5, 2020

@neha-ojha - this is a different test case which is failing this time - ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixNoCsum/2

And patch from this PR couldn't cause it for sure as it modifies specific test case only. There are some chances Adam's fix can cause this but IMO that's doubtful too. So I bet it's a different issue and it's appearance is just a coincidence...

@ifed01
Copy link
Contributor Author

ifed01 commented Jun 5, 2020

@neha-ojha @yuriw - may be another run for both PRs and then one more for Adam's PR exclusively if failure is still observed?

@neha-ojha
Copy link
Member

neha-ojha commented Jun 5, 2020

@neha-ojha @yuriw - may be another run for both PRs and then one more for Adam's PR exclusively if failure is still observed?

@ifed01 makes sense, let's see how the rerun of the test with both PRs does http://pulpito.ceph.com/yuriw-2020-06-05_16:39:00-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5120440/

@neha-ojha
Copy link
Member

neha-ojha commented Jun 5, 2020

@neha-ojha @yuriw - may be another run for both PRs and then one more for Adam's PR exclusively if failure is still observed?

@ifed01 makes sense, let's see how the rerun of the test with both PRs does http://pulpito.ceph.com/yuriw-2020-06-05_16:39:00-rados-wip-yuri2-testing-2020-06-03-2341-MASTER-distro-basic-smithi/5120440/

@ifed01 rerun failed in the ExcessiveFragmentation test case

2020-06-05T21:22:00.330 INFO:teuthology.orchestra.run.smithi081.stdout:[ RUN      ] ObjectStore/StoreTestSpecificAUSize.ExcessiveFragmentation/2
2020-06-05T21:22:06.589 INFO:teuthology.orchestra.run.smithi081.stdout:written 104857600
2020-06-05T21:22:11.823 INFO:teuthology.orchestra.run.smithi081.stdout:written 209715200
2020-06-05T21:22:18.803 INFO:teuthology.orchestra.run.smithi081.stdout:written 314572800
2020-06-05T21:22:27.671 INFO:teuthology.orchestra.run.smithi081.stderr:/build/ceph-16.0.0-2214-g64a1a64/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)' thread 7fd5562910c0 time 2020-06-05T21:22:27.665056+0000
2020-06-05T21:22:27.671 INFO:teuthology.orchestra.run.smithi081.stderr:/build/ceph-16.0.0-2214-g64a1a64/src/os/bluestore/BlueStore.cc: 12863: ceph_abort_msg("unexpected error")
2020-06-05T21:22:27.672 INFO:teuthology.orchestra.run.smithi081.stderr: ceph version 16.0.0-2214-g64a1a64 (64a1a6455982e19dbbfd3edd4f271c0432507975) pacific (dev)
2020-06-05T21:22:27.672 INFO:teuthology.orchestra.run.smithi081.stderr: 1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xe1) [0x7fd54c11bc08]
2020-06-05T21:22:27.672 INFO:teuthology.orchestra.run.smithi081.stderr: 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ceph::os::Transaction*)+0xc47) [0x557b66201277]
2020-06-05T21:22:27.672 INFO:teuthology.orchestra.run.smithi081.stderr: 3: (BlueStore::queue_transactions(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, std::vector<ceph::os::Transaction, std::allocator<ceph::os::Transaction> >&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x2c3) [0x557b66203bd3]
2020-06-05T21:22:27.672 INFO:teuthology.orchestra.run.smithi081.stderr: 4: (ObjectStore::queue_transaction(boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ceph::os::Transaction&&, boost::intrusive_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x82) [0x557b6606f0b2]
2020-06-05T21:22:27.673 INFO:teuthology.orchestra.run.smithi081.stderr: 5: (int queue_transaction<boost::scoped_ptr<ObjectStore> >(boost::scoped_ptr<ObjectStore>&, boost::intrusive_ptr<ObjectStore::CollectionImpl>, ceph::os::Transaction&&)+0x5b) [0x557b6607385b]
2020-06-05T21:22:27.673 INFO:teuthology.orchestra.run.smithi081.stderr: 6: (StoreTestSpecificAUSize_ExcessiveFragmentation_Test::TestBody()+0x5d3) [0x557b66000043]
2020-06-05T21:22:27.673 INFO:teuthology.orchestra.run.smithi081.stderr: 7: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x77) [0x557b66315af7]
2020-06-05T21:22:27.673 INFO:teuthology.orchestra.run.smithi081.stderr: 8: (testing::Test::Run()+0xc3) [0x557b6630bdb3]
2020-06-05T21:22:27.674 INFO:teuthology.orchestra.run.smithi081.stderr: 9: (testing::TestInfo::Run()+0x135) [0x557b6630bf15]
2020-06-05T21:22:27.674 INFO:teuthology.orchestra.run.smithi081.stderr: 10: (testing::TestSuite::Run()+0xb5) [0x557b6630bff5]
2020-06-05T21:22:27.674 INFO:teuthology.orchestra.run.smithi081.stderr: 11: (testing::internal::UnitTestImpl::RunAllTests()+0x488) [0x557b6630c518]
2020-06-05T21:22:27.674 INFO:teuthology.orchestra.run.smithi081.stderr: 12: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x77) [0x557b66316007]
2020-06-05T21:22:27.674 INFO:teuthology.orchestra.run.smithi081.stderr: 13: (testing::UnitTest::Run()+0x98) [0x557b6630c728]
2020-06-05T21:22:27.675 INFO:teuthology.orchestra.run.smithi081.stderr: 14: (main()+0x6fc) [0x557b65fa7fbc]
2020-06-05T21:22:27.675 INFO:teuthology.orchestra.run.smithi081.stderr: 15: (__libc_start_main()+0xe7) [0x7fd54accdb97]
2020-06-05T21:22:27.675 INFO:teuthology.orchestra.run.smithi081.stderr: 16: (_start()+0x2a) [0x557b65faf7aa]

@ifed01
test/objectstore/store_test: kill ExcessiveFragmentation test case.
b852703 
This test case was introduced by ceph#18494
to verify allocation failure handling while gifting during bluefs rebalance
Not it looks outdated as there is no periodic gifting any more.

Fixes: https://tracker.ceph.com/issues/45788

Signed-off-by: Igor Fedotov <ifedotov@suse.com>
@ifed01 ifed01 force-pushed the wip-ifed-fix-ex-frag-test branch from 75a9a63 to b852703 Compare June 9, 2020 15:38
@ifed01
Copy link
Contributor Author

ifed01 commented Jun 9, 2020

@neha-ojha - going another way and killing this case - looks like it's pretty outdated for now.

@ifed01 ifed01 changed the title test/objectstore/store_test: fix ExcessiveFragmentation test case. test/objectstore/store_test: kill ExcessiveFragmentation test case. Jun 9, 2020
@tchaikov
Copy link
Contributor

tchaikov commented Jun 11, 2020

failures tracked by

@tchaikov tchaikov merged commit ae37135 into ceph:master Jun 11, 2020
@smithfarm smithfarm mentioned this pull request Jul 11, 2020
Merged
@smithfarm smithfarm mentioned this pull request Oct 26, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@neha-ojha neha-ojha neha-ojha approved these changes

Assignees

No one assigned

Labels

bug-fix needs-qa tests wip-kefu-testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ifed01 @yuriw @neha-ojha @tchaikov