Skip to content

Fix some more issues found with cppcheck#48

Closed
dalgaaf wants to merge 11 commits intoceph:masterfrom
dalgaaf:wip-da-sca-cppcheck-performance
Closed

Fix some more issues found with cppcheck#48
dalgaaf wants to merge 11 commits intoceph:masterfrom
dalgaaf:wip-da-sca-cppcheck-performance

Conversation

@dalgaaf
Copy link
Contributor

@dalgaaf dalgaaf commented Feb 12, 2013

Fix some performance issues found by cppcheck.

While fixing some performance issues in cppcheck added error handling to:
qa/workunits/direct_io/test_short_dio_read.c
qa/workunits/direct_io/test_sync_io.c

Fix cppcheck performance issues (12 issues like this):

[fusetrace/fusetrace_ll.cc:343] -> [fusetrace/fusetrace_ll.cc:347]:
  (performance) Variable 'in' is reassigned a value before the old
  one has been used.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Add error handling for open(), posix_memalign() and malloc().
Reuse code for read_* and write_* functions.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Add error handling for open() calls.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/ceph_authtool.cc:124]: (performance) Possible inefficient
  checking for 'caps' emptiness.
[src/ceph_authtool.cc:237]: (performance) Possible inefficient
  checking for 'caps' emptiness.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/mds/CDentry.h:234]: (performance) Possible inefficient
  checking for 'projected' emptiness.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/mds/MDSMap.h:448]: (performance) Possible inefficient
  checking for 'up' emptiness.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Fix cppcheck message:

[src/client/Client.cc:3903]: (performance) Passing the result
  of c_str() to a function that takes std::string as argument
  no. 2 is slow and redundant.

Instead using extra string variable and passing std::string::c_str()
to _lookup() use path[i].c_str().

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/client/Client.cc:3649]: (performance) Possible inefficient
  checking for 'mds_sessions' emptiness.
[src/client/Client.cc:7489]: (performance) Possible inefficient
  checking for 'osds' emptiness.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/client/SyntheticClient.cc:2706]: (performance) Possible
  inefficient checking for 'subdirs' emptiness

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
Use empty() since it should be prefered as it has, following the
standard, a constant time complexity regardless of the containter
type. The same is not guaranteed for size().

warning from cppchecker was:
[src/cls/lock/cls_lock.cc:209]: (performance) Possible inefficient
  checking for 'lockers' emptiness.

Signed-off-by: Danny Al-Gaaf <danny.al-gaaf@bisect.de>
@gregsfortytwo
Copy link
Member

As I said on the list, can you:

  1. Fix up 23abfa9, which is maintaining the call to c_str() that was reported in the first place. ;)
  2. Separate the new-code error handling from the cppcheck fixes and do separate pull requests?
  3. Evaluate the cppcheck variable initialization fixes given the comments I made on Fix more performance issues found by cppcheck #51? :)

ghost pushed a commit that referenced this pull request Feb 12, 2014
blkid -s TYPE /dev/fd0 has been verified to hang forever on a
H8DMR-82 supermicro motherboard running

3.8.0-33-generic #48~precise1-Ubuntu SMP Thu Oct 24 16:28:06 UTC 2013
x86_64

It is unlikely that ceph will ever be used on floppy disks, they
can be blacklisted.

http://tracker.ceph.com/issues/6827 fixes: #6827

Signed-off-by: Loic Dachary <loic@dachary.org>
(cherry picked from commit 6570197)
chamdoo pushed a commit to chamdoo/ceph that referenced this pull request Nov 13, 2015
XinzeChi pushed a commit to XinzeChi/ceph that referenced this pull request Jan 29, 2016
os/FileStore: For getxattr, enlarge the value size avoid try again.
ddiss pushed a commit to ddiss/ceph that referenced this pull request Oct 25, 2016
rpm: Fix creation of mount.ceph symbolic link for SUSE distros
p-se pushed a commit to p-se/ceph that referenced this pull request Jan 31, 2018
…2-cluster-status

mgr/dashboard_v2: Added status route to cluster
cbodley added a commit to cbodley/ceph that referenced this pull request Mar 21, 2018
5538352 Merge pull request ceph#49 from cbodley/wip-remove-by-ptr
2dbd66d PriorityQueueBase uses RequestRef&& for remove_by callbacks
729d8c0 Merge pull request ceph#48 from ivancich/wip-add-license
3f2e0b4 Added LGPL2.1 license to all files.

git-subtree-dir: src/dmclock
git-subtree-split: 5538352
votdev referenced this pull request in votdev/ceph Apr 9, 2018
5538352 Merge pull request #49 from cbodley/wip-remove-by-ptr
2dbd66d PriorityQueueBase uses RequestRef&& for remove_by callbacks
729d8c0 Merge pull request #48 from ivancich/wip-add-license
3f2e0b4 Added LGPL2.1 license to all files.

git-subtree-dir: src/dmclock
git-subtree-split: 5538352
votdev referenced this pull request in votdev/ceph Apr 9, 2018
5538352 Merge pull request #49 from cbodley/wip-remove-by-ptr
2dbd66d PriorityQueueBase uses RequestRef&& for remove_by callbacks
729d8c0 Merge pull request #48 from ivancich/wip-add-license
3f2e0b4 Added LGPL2.1 license to all files.

git-subtree-dir: src/dmclock
git-subtree-split: 5538352
ivancich added a commit to ivancich/ceph-fork that referenced this pull request Apr 18, 2018
511dca623a Allow the dmclock library to handle the "cost" of an operation/request. This is in contrast to just measuring ops. If the cost is set at the value 1, then it's equivalent to the old code.
607fc04548 Update .gitignore to deal with cscope files.
729d8c0 Merge pull request ceph#48 from ivancich/wip-add-license
3f2e0b4 Added LGPL2.1 license to all files.

git-subtree-dir: src/dmclock
git-subtree-split: 511dca623aae83aeb3431529e108060f96c6f9bd
dotnwat added a commit that referenced this pull request Jul 29, 2018
sdpeters added a commit to sdpeters/ceph that referenced this pull request May 29, 2019
…base.

All in pursuit of issue ceph#48 after the latest rebase.

The aio_comp() operations in comments there cause an assert in init or open now.

FLUSH_SOURCE_WRITEBACK requests complete synchonously now, like they did before the rebase.

None of this fixes the test assert in issue ceph#48.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
sdpeters added a commit to sdpeters/ceph that referenced this pull request May 29, 2019
for Ceph RBD.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Resorting to pruning out tests that fail with RWL.

Don't add RWL on images with the MIGRATING feature bit

They src and dest may have the same image name.

Also don't create image writeback at all if there is no image cache
configured.

Make deep copy test use different image names for src and dest

Fix tests for added async completions, used whether RWL is enabled or not.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

This block guard implementation isn't used anymore

Handle zero length requests

Tighten up block guard release. Remove assert for what wasn't a bug.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Skip these tests when RWL is enabled for now.

It's unclear whether they should work with the cache enabled or not.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Do a flush() before shut_down() to complete all in-flight writes and append a final sync point.

Flush to RADOS on ::flush()

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make DiscardCopyup test not simultaneously open the same image > once

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Replaced ineffective pmem_bl copies. TestInternal passes now with RWL enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

TestFUA is incompatible with RWL (multiple open of same image)

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Internal flush may be called before init succeeds

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make RWL perf logging configurable

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Require !RWL for tests that open images > once

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Log image name and ID in key ops.

Use async completion to avoid a deadlock seen during testing

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Flush/invalidate image cache during Resize

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable more tests that open images > once. DeepCopy tests shoudl work on RWL, but don't.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make invalidate after flush configurable

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

When RWL is enabled, set environment variable to skip actual PMDK persist work

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

All migration tests skipped when RWL is enabled.

Migration probably needs to disable ImageCache first.

One more DeepCopy test that dosn't work with RWL.

Tweak tests to not open images > once

Tests can close all images before fixture teardown. Avoids reopens that fil with write-back cache.

Allow saved image_ctxs to be closed by name

Close the image before re-opening it

Skip tests that reopen images when RWL is enabled

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable slow PMDK persistence in tests

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Consolidate constructors

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Force init, shutdown, and flush() completions to op_work_queue

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Didn't need a special case for image cache after all

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix apparent copy-and-paste error from KernelDevice.cc

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert gratuitous whitespace changes

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Adds IMAGE_CACHE feature bit. Moves image cache init logic to ImageState

This is in prep. for initializing ImageCache during exclusive lock acquire.

Add types for image cache config

Can fetch image cache state during open

Record ImageCache state on image close

Init & shut down image cache in exclusive lock acquire/release

Disable checks for dirty.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Lock tests expect specific feature tests

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Remove deprecated io prio calls

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Include image-cache feature options

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make mock tests work with IMAGE_CACHE feature on

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make more tests work with IMAGE_CACHE feature enabled

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

More adaptation to the feature bit universe

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Reworked ImageRequestWQ<>::flush() to use ::aio_flush() again, but with a FlushSource arg.

Avoids use of flush_image(), which I think was getting blocked with the user requests in the
DeepCopy test, which was failing.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Flush to journal for internal or user flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Undo previous.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add IMAGE_CACHE cases to RBD unit test

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix apparent copy-and-paste error

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't build PMDK or RWL on ARM while PMDK on ARM64 is still experimental

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Skip test_librbd when features includes IMAGE_CACHE, but WITH_RWL is not defined.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make read_only flag const, since that's how it's used

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix message

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't init image cache when opening R/O unless there's a dirty one

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add evn vars to inhibit flush and retire on close for RWL debugging

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add ImageCache action to build

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add env vars for testing image cache rbd tool actions

There's a regression here, but I'm committing anyway so I can migrate
to gitlab now.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix lockdep issue in shutdown when not flushing/retiring.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Use FLUSH_SOURCE_SHUTDOWN to identify shutown flushes.

In general, a whoile lot of plumbing to get the ability to suppress RWL flushing
on close, so we can test removing and discarding the image-cache with the rbd tool.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix rbd cli test output for new image-cache comand and --no-flush option to bench

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Clean up logging from previous test

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix expected calls of block_writes() which now include the flush source.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Always init exclusive lock when the image cache feature is enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Remove bogus assert

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Checkpoint rbd image-cache command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert "Always init exclusive lock when the image cache feature is enabled."

This reverts commit 314c82f.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix message

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Track image-cache option changes

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixed some things clang didn't like, which may have broken rtags

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Split WITH_PMEM into WITH_PMEM_DEVICE and WITH_PMDK

WITH_PMDK causes the PMDK (/NVML) to be built.

WITH_PMEM_DEVICE causes the PMEM OSD to be built.

WITH_PMEM_DEVICE, or WITH_RWL requires WITH_PMDK

Primordial support for WITH_PMDK_PKG enables use of PMDK
installed on platform. We'd probably like finer grained
control over this (build most of Ceph without PMDK installed,
and make PMDK an install dependency for only a few selected
things).

We can now enable RWL without requiring the PMEM OSD to build
and pass tests.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't assert if exclusive lock release failed.

On failure, these things may not have been done.
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Also make etags file

ctags -e, into src/TAGS.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert "Don't assert if exclusive lock release failed."

This reverts commit 184ba6f5e97357380d30f8012f28c34c38e6818c

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable almost all deep copy tests when RWL is enabled

These used to pass, and made interesting test cases, but it
probably doesn't make sense to do this with the cache enabled
anyway.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Ignore etags file TAGS

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Adds ReplicatedWriteLogInternal.h

Hopefully trims enough off of ReplicatedWriteLog.cc that gitlab will
show the diffs in a merge request.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Shift more things from RWL.cc to RWLInternal.h

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Finish moving lines

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

One more try for gitlab-friendly file sizes.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

To fix IO hange problem caused by aio_flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add IO test for RWL

It does IO test with RWL, and tries to cover the test case when
RWL cache is full. But it is not guranteed.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix the bug 'First IO hangs if RWL can't be initialized'.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Update default and minimum value of rwl_size.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixes the image-cache show command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Update RWL full case to use default value.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add pool name in the name of rwl cache file.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixed rebase issues.

Look closer at whether FLUSH_SOURCE_WRITEBACK should be a no-op in
RWL, or an assert.

Look closer at discard granularity passed to lower cache. This replaces
skip_partial. We were always passing skip_partial = false, so the RWL
and the image got the same discard results (regions which read as zero,
vs. whatever was there before).

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix expected usage output

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Probably rebase noise

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Repeat compare buffer as necessary to match length of read in compare-and-write

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable some more migration tests when IMAGE_CACHE feature is enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

fix image-cache disable feature command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Start rwl threads after rwl_init.

Signed-off-by: Li Xiaoyan <xiaoyan.li@intel.com>

change ENOENT to EPERM when open image cache failed;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

fixed issue ceph#35: If the image metadata says there is no cache, any existing pool file with the same name should be deleted.

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

only image cache state.present=0 and has pool file, then del the pool file with the same name;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

do this in rwl_int() function;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

Pass through everything to writeback interface

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

fix ceph#41:IO hang when rwl_size = 10G or 20G and without rate_iops limit, While there's no IO problem when rwl_size=1G

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

add high/low water log entries threshold and trigger it to retire entries when there's no enough log entries;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

clean indent issue;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

modify indentation based on Scott's feedback;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

Fix the issue of flushing rwl cache when showing image cache.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Fix image-cache show issue.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

image-cache discard-dirty CLI command

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

pmemobj_close won't fail.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Rebase repairs. Mostly trivial except async op handling in flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make imageFlushRequest::send_request() more like it was before the rebase.

All in pursuit of issue ceph#48 after the latest rebase.

The aio_comp() operations in comments there cause an assert in init or open now.

FLUSH_SOURCE_WRITEBACK requests complete synchonously now, like they did before the rebase.

None of this fixes the test assert in issue ceph#48.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Replace namespace declaration that is present in master but not our branch

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Supplied initial values for things not initialized by all constructors

I saw compiler warnings about these fields being used uninitialized and thought it
might be related to the rebase regression (it probably isn't).

It does look like these could be left uninitialized if the ReplicatedWriteLogSpec
constructor with no argumets was used.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Bad merge

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix unit test problem.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
sdpeters added a commit to sdpeters/ceph that referenced this pull request May 29, 2019
…base.

All in pursuit of issue ceph#48 after the latest rebase.

The aio_comp() operations in comments there cause an assert in init or open now.

FLUSH_SOURCE_WRITEBACK requests complete synchonously now, like they did before the rebase.

None of this fixes the test assert in issue ceph#48.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>
sdpeters added a commit to sdpeters/ceph that referenced this pull request May 31, 2019
for Ceph RBD.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Resorting to pruning out tests that fail with RWL.

Don't add RWL on images with the MIGRATING feature bit

They src and dest may have the same image name.

Also don't create image writeback at all if there is no image cache
configured.

Make deep copy test use different image names for src and dest

Fix tests for added async completions, used whether RWL is enabled or not.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

This block guard implementation isn't used anymore

Handle zero length requests

Tighten up block guard release. Remove assert for what wasn't a bug.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Skip these tests when RWL is enabled for now.

It's unclear whether they should work with the cache enabled or not.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Do a flush() before shut_down() to complete all in-flight writes and append a final sync point.

Flush to RADOS on ::flush()

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make DiscardCopyup test not simultaneously open the same image > once

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Replaced ineffective pmem_bl copies. TestInternal passes now with RWL enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

TestFUA is incompatible with RWL (multiple open of same image)

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Internal flush may be called before init succeeds

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make RWL perf logging configurable

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Require !RWL for tests that open images > once

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Log image name and ID in key ops.

Use async completion to avoid a deadlock seen during testing

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Flush/invalidate image cache during Resize

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable more tests that open images > once. DeepCopy tests shoudl work on RWL, but don't.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make invalidate after flush configurable

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

When RWL is enabled, set environment variable to skip actual PMDK persist work

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

All migration tests skipped when RWL is enabled.

Migration probably needs to disable ImageCache first.

One more DeepCopy test that dosn't work with RWL.

Tweak tests to not open images > once

Tests can close all images before fixture teardown. Avoids reopens that fil with write-back cache.

Allow saved image_ctxs to be closed by name

Close the image before re-opening it

Skip tests that reopen images when RWL is enabled

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable slow PMDK persistence in tests

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Consolidate constructors

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Force init, shutdown, and flush() completions to op_work_queue

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Didn't need a special case for image cache after all

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix apparent copy-and-paste error from KernelDevice.cc

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert gratuitous whitespace changes

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Adds IMAGE_CACHE feature bit. Moves image cache init logic to ImageState

This is in prep. for initializing ImageCache during exclusive lock acquire.

Add types for image cache config

Can fetch image cache state during open

Record ImageCache state on image close

Init & shut down image cache in exclusive lock acquire/release

Disable checks for dirty.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Lock tests expect specific feature tests

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Remove deprecated io prio calls

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Include image-cache feature options

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make mock tests work with IMAGE_CACHE feature on

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make more tests work with IMAGE_CACHE feature enabled

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

More adaptation to the feature bit universe

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Reworked ImageRequestWQ<>::flush() to use ::aio_flush() again, but with a FlushSource arg.

Avoids use of flush_image(), which I think was getting blocked with the user requests in the
DeepCopy test, which was failing.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Flush to journal for internal or user flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Undo previous.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add IMAGE_CACHE cases to RBD unit test

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix apparent copy-and-paste error

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't build PMDK or RWL on ARM while PMDK on ARM64 is still experimental

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Skip test_librbd when features includes IMAGE_CACHE, but WITH_RWL is not defined.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make read_only flag const, since that's how it's used

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix message

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't init image cache when opening R/O unless there's a dirty one

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add evn vars to inhibit flush and retire on close for RWL debugging

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add ImageCache action to build

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add env vars for testing image cache rbd tool actions

There's a regression here, but I'm committing anyway so I can migrate
to gitlab now.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix lockdep issue in shutdown when not flushing/retiring.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Use FLUSH_SOURCE_SHUTDOWN to identify shutown flushes.

In general, a whoile lot of plumbing to get the ability to suppress RWL flushing
on close, so we can test removing and discarding the image-cache with the rbd tool.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix rbd cli test output for new image-cache comand and --no-flush option to bench

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Clean up logging from previous test

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix expected calls of block_writes() which now include the flush source.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Always init exclusive lock when the image cache feature is enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Remove bogus assert

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Checkpoint rbd image-cache command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert "Always init exclusive lock when the image cache feature is enabled."

This reverts commit 314c82f.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix message

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Track image-cache option changes

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixed some things clang didn't like, which may have broken rtags

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Split WITH_PMEM into WITH_PMEM_DEVICE and WITH_PMDK

WITH_PMDK causes the PMDK (/NVML) to be built.

WITH_PMEM_DEVICE causes the PMEM OSD to be built.

WITH_PMEM_DEVICE, or WITH_RWL requires WITH_PMDK

Primordial support for WITH_PMDK_PKG enables use of PMDK
installed on platform. We'd probably like finer grained
control over this (build most of Ceph without PMDK installed,
and make PMDK an install dependency for only a few selected
things).

We can now enable RWL without requiring the PMEM OSD to build
and pass tests.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Don't assert if exclusive lock release failed.

On failure, these things may not have been done.
Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Also make etags file

ctags -e, into src/TAGS.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Revert "Don't assert if exclusive lock release failed."

This reverts commit 184ba6f5e97357380d30f8012f28c34c38e6818c

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable almost all deep copy tests when RWL is enabled

These used to pass, and made interesting test cases, but it
probably doesn't make sense to do this with the cache enabled
anyway.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Ignore etags file TAGS

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Adds ReplicatedWriteLogInternal.h

Hopefully trims enough off of ReplicatedWriteLog.cc that gitlab will
show the diffs in a merge request.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Shift more things from RWL.cc to RWLInternal.h

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Finish moving lines

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

One more try for gitlab-friendly file sizes.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

To fix IO hange problem caused by aio_flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add IO test for RWL

It does IO test with RWL, and tries to cover the test case when
RWL cache is full. But it is not guranteed.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix the bug 'First IO hangs if RWL can't be initialized'.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Update default and minimum value of rwl_size.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixes the image-cache show command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Update RWL full case to use default value.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Add pool name in the name of rwl cache file.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fixed rebase issues.

Look closer at whether FLUSH_SOURCE_WRITEBACK should be a no-op in
RWL, or an assert.

Look closer at discard granularity passed to lower cache. This replaces
skip_partial. We were always passing skip_partial = false, so the RWL
and the image got the same discard results (regions which read as zero,
vs. whatever was there before).

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix expected usage output

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Probably rebase noise

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Repeat compare buffer as necessary to match length of read in compare-and-write

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Disable some more migration tests when IMAGE_CACHE feature is enabled.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

fix image-cache disable feature command

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Start rwl threads after rwl_init.

Signed-off-by: Li Xiaoyan <xiaoyan.li@intel.com>

change ENOENT to EPERM when open image cache failed;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

fixed issue ceph#35: If the image metadata says there is no cache, any existing pool file with the same name should be deleted.

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

only image cache state.present=0 and has pool file, then del the pool file with the same name;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

do this in rwl_int() function;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

Pass through everything to writeback interface

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

fix ceph#41:IO hang when rwl_size = 10G or 20G and without rate_iops limit, While there's no IO problem when rwl_size=1G

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

add high/low water log entries threshold and trigger it to retire entries when there's no enough log entries;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

clean indent issue;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

modify indentation based on Scott's feedback;

Signed-off-by: Yuan Lu <yuan.y.lu@intel.com>

Fix the issue of flushing rwl cache when showing image cache.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Fix image-cache show issue.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

image-cache discard-dirty CLI command

Signed-off-by: Mahati Chamarthy <mahati.chamarthy@intel.com>

pmemobj_close won't fail.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>

Rebase repairs. Mostly trivial except async op handling in flush.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Make imageFlushRequest::send_request() more like it was before the rebase.

All in pursuit of issue ceph#48 after the latest rebase.

The aio_comp() operations in comments there cause an assert in init or open now.

FLUSH_SOURCE_WRITEBACK requests complete synchonously now, like they did before the rebase.

None of this fixes the test assert in issue ceph#48.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Replace namespace declaration that is present in master but not our branch

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Supplied initial values for things not initialized by all constructors

I saw compiler warnings about these fields being used uninitialized and thought it
might be related to the rebase regression (it probably isn't).

It does look like these could be left uninitialized if the ReplicatedWriteLogSpec
constructor with no argumets was used.

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Bad merge

Signed-off-by: Scott Peterson <scott.d.peterson@intel.com>

Fix unit test problem.

Signed-off-by: Xiaoyan Li <xiaoyan.li@intel.com>
m-ildefons pushed a commit to m-ildefons/ceph that referenced this pull request Oct 6, 2022
rgw/sfs: Lists only the buckets owned by the user.
ivoalmeida pushed a commit to ivoalmeida/ceph that referenced this pull request Feb 27, 2024
Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
yuvalif pushed a commit to yuvalif/ceph that referenced this pull request Mar 13, 2024
Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
yuvalif pushed a commit to yuvalif/ceph that referenced this pull request Aug 21, 2024
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
yuvalif pushed a commit to yuvalif/ceph that referenced this pull request Sep 16, 2024
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 514fc29)
Moisi pushed a commit to Moisi/ceph that referenced this pull request Oct 14, 2024
…patch for put-bucket-lifecycle-configuration.

Merge in STORAGEENG/ceph from OBJGEN1-512-add-storequery-api to aka_version_18.2.1

* commit '4190955fa2f0b6494a9d533e48d2b1c4c828718f':
  Fix bad interaction of StoreQuery and anon authz.
  Updates from upstream Authenticator.
  FMT_STRING-ify storequery source.
  Significant integration changes to storequery.
  Fix stupid logic bug in op handler.
  Apply validate-only to put-bucket-lifecycle-configuration
  Wire StoreQuery into REST mgrs and handlers.
  Always log 'Akamai StoreQuery' at level 0.
  Namespace-ify rgw_rest_storequery.{cc,h}.
  Manual port of storequery from v17 step 1.
mattbenjamin pushed a commit to linuxbox2/ceph that referenced this pull request Oct 22, 2024
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 514fc29)
athanatos pushed a commit to athanatos/ceph that referenced this pull request Feb 14, 2025
Add LGPL2.1 license to all source files
andrewschoen referenced this pull request in ibmstorage/ceph Feb 25, 2025
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 514fc29)
rishabh-d-dave pushed a commit to rishabh-d-dave/ceph that referenced this pull request Mar 19, 2025
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 514fc29)
hasan4791 pushed a commit to hasan4791/ceph that referenced this pull request Jan 6, 2026
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 9786c32e2f98840a249405f64639e43606d09b87)

When cherry-picking to the 9.0 branch, this conflict occurred:

Conflicts:
    src/pybind/mgr/tox.ini

[testenv:test]
setenv = {[testenv]setenv}
deps = {[testenv]deps}
commands = {[testenv]commands}

<<<<<<< HEAD
=======
[testenv:fix]
basepython = python3
deps =
    autopep8
modules =
    alerts \
    balancer \
    call_home_agent \
    cephadm \
    cli_api \
    crash \
    devicehealth \
    diskprediction_local \
    insights \
    iostat \
    nfs \
    orchestrator \
    prometheus \
    rgw \
    status \
    telemetry
commands =
    python --version
    autopep8 {[autopep8]addopts} \
      {posargs:{[testenv:fix]modules}}
>>>>>>> 9786c32e2f9 (mgr/call_home_agent: IBM Call Home Agent module)

[testenv:pylint]

We ewsolved it by removing the original changes.
phlogistonjohn pushed a commit to phlogistonjohn/ceph that referenced this pull request Mar 6, 2026
This is a combination of 18 commits to ease maintenance.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit c9deac1)

Resolves: rhbz#2235256
(cherry picked from commit eb3b298)
(cherry picked from commit de6cbfb)

Conflicts:
    ceph.spec.in

    A new line was missing between the block of "%files node-proxy"
    and that of "%files mgr-callhome".

Please note that the changes in de6cbfb
to src/cephadm/cephadm.py were reset in this commit, since they were
extracted and cherry-picked to a separate call-home-cephadm branch.

pybind/mgr: add call_home_agent to CMakeLists.txt and tox.ini

This fixes the previous commit
(c9deac1) and could be squashed down in
future rebases.

Related: rhbz#2235256

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit e74a8c4)
(cherry picked from commit a58aff2)

mgr/call_home_agent: fix reports frequency

Inventory reports frequency should be daily (60*60*24).
Status reports frequency should be every 30 minutes (60*30).

Move content of options.py into module.py, no reason for separate files.

Resolves: rhbz#2241825

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27da98d)
(cherry picked from commit 0bd94e2)

mgr/call_home_agent: add "request_time" to all events payload

Storage Insights requires that a "request_time" key will be included in
the "payload" section of all events. Its value is unix timestamp
milliseconds.

Resolves: rhbz#2248640
https://bugzilla.redhat.com/show_bug.cgi?id=2248640

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 981b0ea)
(cherry picked from commit 0b79454)

mgr/ccha: Fix decoding issue

The encoded JWT token inside password must be managed as raw string

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e7e618e7f064a75434e3822c4ef9499ddab43fd1)
(cherry picked from commit 9acaa16)

mgr/ccha: Fix help for ceph callhome show command

Resolves: rhbz#2243795
          https://bugzilla.redhat.com/show_bug.cgi?id=2243795

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit cd4356588a615d83d9ae3cd7806008bd8b4cdae1)
(cherry picked from commit 14bb3e9)

mgr/ccha: Fix ceph callhome get user output

Call home manager module options names and fields shown in the
callhome get user command are now the same

Resolves: rhbz#2243796
          https://bugzilla.redhat.com/show_bug.cgi?id=2243796

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit d99ca03f04dac20fa7efd22fba6a548a3c7d10df)
(cherry picked from commit ccf42ac)

mgr/callhome: Add hardware status to inventory reports (ceph#48)

Hardware status will be fetched daily from Node Proxy, and will be added to the
inventory reports.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 9be771d7e63e21b185a65f1d0fecb4c959a3c058)

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434
(cherry picked from commit 209527a)

mgr/callhome: Send alerts to Call Home (ceph#47)

Add the functionality of sending Prometheus alerts to IBM Call Home.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit aed81302e05451fb2c0d271d199a099623b205d9)

Resolves: rhbz#2264432
          https://bugzilla.redhat.com/show_bug.cgi?id=2264432
(cherry picked from commit e5a1462)

mgr/callhome: Upload diagnostics

Implementation of the upload diagnostics functionality in Call Home Agent

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 2830d82ef60f93ee156e196f7fe8959071deabfa)

Resolves: rhbz#2264444
          https://bugzilla.redhat.com/show_bug.cgi?id=2264444
(cherry picked from commit 6a3d65e)

mgr/ccha: Fix sos report file corrupted in EcuRep

The sos report file upload to EcuRep cannot be unpacked

    This commit can be safely squashed along with:
    6a3d65e
    mgr/callhome: Upload diagnostics

Resolves: rhbz#2266236
          https://bugzilla.redhat.com/show_bug.cgi?id=2266236

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit a56fe4624845d248fe534f87f145abc984f281fb)
(cherry picked from commit 89a4ea1)

mgr/ccha: Fix inventory report (ceph#54)

The inventory report cannot be generated

This commit can be safely squashed along with:
    209527a
    mgr/callhome: Add hardware status to inventory reports
in future releases

Resolves: rhbz#2264434
          https://bugzilla.redhat.com/show_bug.cgi?id=2264434

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit 0320c857e0a7151f4846f2187c48c1fc2a3c0c98)
(cherry picked from commit 6e4b152)

mgr/ccha: Upload diagnostics level 1 report error (ceph#55)

Resolves: rhbz#2268399
          https://bugzilla.redhat.com/show_bug.cgi?id=2268399

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit bcfcabf3a2f965c84db868f75f2b9fbdbfd2c2d7)
(cherry picked from commit efde76b)

cephadm: make cephadm sos cmd more robust (ceph#56)

This commit is extracted from bac75bb
in order to include only the call home module changes.

See:
https://gitlab.cee.redhat.com/ceph/ceph/-/commit/bac75bb1b9052982fd9b4ebc3ff5116b67081c54?merge_request_iid=520

This commit was created with:

  $ git format-patch -1 bac75bb -- src/pybind/mgr/call_home_agent/module.py
  0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  $
  $ git apply -3 0001-mgr-ccha-make-cephadm-sos-cmd-more-robust-56.patch
  Applied patch to 'src/cephadm/cephadm.py' cleanly.

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>

mgr/ccha: Remove jti error message when no credentials (ceph#61)

Avoid the annoying error message if not credentials present
Fix error if registry credentials are set using ceph cephadm registry-reg_credentials
Changed default regex for registry urls

Resolves: rhbz#2231489
          https://bugzilla.redhat.com/show_bug.cgi?id=2231489

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit aac9d3a359af145ec6cae92836e387d65978602e)
(cherry picked from commit 9dd5647)

mgr/callhome: use static Transfer ID

ECuRep requires Transfer ID credentials (user ID and password). In this fix we
are adding the option to load them from the encrypted keys file instead of
asking the user to populate them. The keys from the files are the default. As a
workaround, we are leaving the option to manually populate the module options,
in case we ever need it.

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 27eddcb76bd2f67eee13b7539bb006d286ba560d)
(cherry picked from commit 55e45c7)

mgr/ccha: Increase default value for cooling window

Increased upload_snap_cooling_window_seconds option value to 1 day

Resolves: rhbz#2273565
          https://bugzilla.redhat.com/show_bug.cgi?id=2273565

Signed-off-by: Juan Miguel Olmo Martínez <jolmomar@ibm.com>
(cherry picked from commit e6787dbdca28e2a007d405775f145dbad94a68c9)
(cherry picked from commit 7181f16)

BF-2271537: mgr/callhome: pick up SI event ID (ceph#65)

Storage Insights event ID was not picked up correctly which prevented Ceph from
listening to SI triggered requests, and thus not fulfilling them and updating
on their status.
Status for operations updated to match SI expectations

Resolves: rhbz#2271537

Signed-off-by: Yaarit Hatuka <yhatuka@ibm.com>
Co-authored-by: Yaarit Hatuka <yhatuka@ibm.com>
(cherry picked from commit 7d87f5cd50a21ff16675c69ff6dac55235cdbc22)
(cherry picked from commit 5290a81)
(cherry picked from commit 9786c32e2f98840a249405f64639e43606d09b87)

When cherry-picking to the 9.0 branch, this conflict occurred:

Conflicts:
    src/pybind/mgr/tox.ini

[testenv:test]
setenv = {[testenv]setenv}
deps = {[testenv]deps}
commands = {[testenv]commands}

<<<<<<< HEAD
=======
[testenv:fix]
basepython = python3
deps =
    autopep8
modules =
    alerts \
    balancer \
    call_home_agent \
    cephadm \
    cli_api \
    crash \
    devicehealth \
    diskprediction_local \
    insights \
    iostat \
    nfs \
    orchestrator \
    prometheus \
    rgw \
    status \
    telemetry
commands =
    python --version
    autopep8 {[autopep8]addopts} \
      {posargs:{[testenv:fix]modules}}
>>>>>>> 9786c32e2f9 (mgr/call_home_agent: IBM Call Home Agent module)

[testenv:pylint]

We ewsolved it by removing the original changes.

(cherry picked from commit 213fde6)

When cherry-picking to the 9.1 branch, this conflict occurred:

Conflicts:

<<<<<<< HEAD
cheroot~=10.0
=======
pyjwt
>>>>>>> 213fde6 (mgr/call_home_agent: IBM Call Home Agent module)

We resolved it by keeping both requirements.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants