rgw, common, build: drop NSS support by rzarzynski · Pull Request #27834 · ceph/ceph

rzarzynski · 2019-04-28T20:40:11Z

These commits bring:

switch from NSS to OpenSSL for AWS-like crypto services in RadosGW (see rgw: switch AES_256_{CBC, ECB_encrypt} from NSS to OpenSSL #27659),
initialization code for old versions of OpenSSL,
removal of support for PKI/PKIz token validation at RadosGW's side,
eradication of our dependency on NSS.

TODO:

update doc.

CMakeLists.txt

src/common/ceph_crypto.cc

cbodley · 2019-04-30T13:48:30Z

cc @theanalyst, who worked on some of this openssl locking stuff for rgw in src/rgw/rgw_http_client_curl.cc

src/common/ceph_crypto.cc

rzarzynski · 2019-05-07T10:30:09Z

src/common/ceph_mutex.h

    void unlock() {}
  };

+  struct dummy_shared_mutex : dummy_mutex {


Added this for crimson (WITH_SEASTAR builds`).

tchaikov

the changes under common and test LGTM.

cbodley

passing unittest_rgw_crypto and encryption s3-tests 👍

tchaikov · 2019-05-19T21:52:01Z

@rzarzynski needs rebase

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rzarzynski · 2019-06-17T21:07:47Z

jenkins retest this please (mgr-dashboard-frontend-unittests on x86; exhausted memory on arm64)

rzarzynski · 2019-06-20T22:25:54Z

Thanks, @tchaikov! Just pushed the amended commit. No other changes apart the CONF_modules_free().

tchaikov · 2019-06-22T17:54:31Z

valgrind still complained:

rzarzynski · 2019-06-24T13:19:25Z

WIP. Building a fix candidate (will be used to resurrect the @tchaikov's run).

rzarzynski · 2019-06-24T17:58:12Z

wget -O - http://qa-proxy.ceph.com/teuthology/rzarzynski-2019-06-24_15:36:48-rados-wip-nss-drop-cms-distro-basic-smithi/4064797/remote/smithi061/log/valgrind/mon.b.log.gz | gunzip | less

Much less but there is still a bunch of traces related to ERR_get_state(). Fixing.

rzarzynski · 2019-06-26T05:21:25Z

The resurrected @tchaikov's run is finally green: http://pulpito.front.sepia.ceph.com/rzarzynski-2019-06-25_23:22:46-rados-wip-nss-drop-cms-distro-basic-smithi/.
I will squash the fixup after review.

src/common/ceph_crypto.cc

This is based on Civetweb's. Modern OpenSSLs don't require the extra calls as they are self-initializing [1][2]. [1]: https://wiki.openssl.org/index.php/Library_Initialization [2]: https://www.openssl.org/docs/faq.html#PROG1 Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

The rgw::keystone::TokenCache::RevokeThread is being dropped as a part the effort for removing support for PKI/PKIz token types RadosGW. Both of them are not available since OS Ocata release in 2016. Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

tchaikov · 2019-07-04T14:11:07Z

rzarzynski added rgw common build/ops labels Apr 28, 2019

rzarzynski requested review from cbodley and tchaikov April 28, 2019 20:40

tchaikov reviewed Apr 29, 2019

View reviewed changes

CMakeLists.txt Show resolved Hide resolved

CMakeLists.txt Show resolved Hide resolved

src/common/ceph_crypto.cc Outdated Show resolved Hide resolved

rzarzynski force-pushed the wip-nss-drop-cms branch 2 times, most recently from 307f794 to bd1b8fe Compare April 30, 2019 05:36

tchaikov reviewed Apr 30, 2019

View reviewed changes

src/common/ceph_crypto.cc Outdated Show resolved Hide resolved

cbodley requested a review from theanalyst April 30, 2019 13:49

theanalyst reviewed Apr 30, 2019

View reviewed changes

src/common/ceph_crypto.cc Outdated Show resolved Hide resolved

src/common/ceph_crypto.cc Outdated Show resolved Hide resolved

rzarzynski force-pushed the wip-nss-drop-cms branch from 8fdadab to 7ce2f89 Compare May 7, 2019 09:12

rzarzynski commented May 7, 2019

View reviewed changes

tchaikov approved these changes May 7, 2019

View reviewed changes

cbodley approved these changes May 7, 2019

View reviewed changes

rzarzynski mentioned this pull request May 7, 2019

rgw: switch AES_256_{CBC, ECB_encrypt} from NSS to OpenSSL #27659

Closed

mattbenjamin requested a review from kalebskeithley May 17, 2019 12:20

tchaikov added the needs-rebase label May 19, 2019

tchaikov assigned rzarzynski May 19, 2019

rzarzynski added 4 commits June 17, 2019 14:45

rgw: switch AES_256_CBC::cbc_transform() from NSS to OpenSSL.

5b5862f

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: generalize the symmetric crypto with evp_sym_transform().

16c5c46

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: switch AES_256_ECB_encrypt() from NSS to OpenSSL.

efb6c3a

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: drop the unused, NSS-based AES_256_CTR.

2d5e48f

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rzarzynski force-pushed the wip-nss-drop-cms branch from 9997fc6 to a20829c Compare June 17, 2019 14:36

rzarzynski removed the needs-rebase label Jun 17, 2019

tchaikov added the needs-qa label Jun 18, 2019

cbodley added the wip-cbodley-testing label Jun 18, 2019

rzarzynski force-pushed the wip-nss-drop-cms branch from a20829c to adc0c76 Compare June 20, 2019 22:24

tchaikov added needs-qa wip-kefu-testing labels Jun 21, 2019

tchaikov removed needs-qa wip-kefu-testing labels Jun 23, 2019

rzarzynski added DNM and removed DNM labels Jun 24, 2019

tchaikov approved these changes Jun 26, 2019

View reviewed changes

tchaikov reviewed Jun 26, 2019

View reviewed changes

src/common/ceph_crypto.cc Show resolved Hide resolved

rzarzynski added 11 commits June 26, 2019 22:09

rgw: drop the local validation of PKI/PKIz tokens.

b8afbbe

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: drop the local validation of PKI/PKIz tokens, part 2.

f2b49b3

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rgw: drop the helpers for CMS handling.

7f5f989

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common, rgw: drop the CMS decoder.

ca545e1

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common, crypto, build: get rid of NSS entirely.

79edb93

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: drop the cct param from ceph::crypto::init().

b966fa4

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

build, common, crypto, rgw: drop USE_OPENSSL as OpenSSL is obligatory.

b998637

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

common: OpenSSL's init code uses std::vector now.

2b77c3f

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

doc: drop the information about PKI tokens.

16f2506

Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>

rzarzynski force-pushed the wip-nss-drop-cms branch from 0711cfe to 16f2506 Compare June 26, 2019 21:56

tchaikov added the wip-kefu-testing label Jul 3, 2019

tchaikov merged commit a4ddc4b into ceph:master Jul 5, 2019

rzarzynski mentioned this pull request Jul 17, 2019

common: give lockdep's group name to OpenSSL's mutexes #28987

Merged

Conversation

rzarzynski commented Apr 28, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cbodley commented Apr 30, 2019

Uh oh!

Uh oh!

Uh oh!

rzarzynski May 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tchaikov left a comment

Choose a reason for hiding this comment

Uh oh!

cbodley left a comment

Choose a reason for hiding this comment

Uh oh!

tchaikov commented May 19, 2019

Uh oh!

rzarzynski commented Jun 17, 2019

Uh oh!

rzarzynski commented Jun 20, 2019

Uh oh!

tchaikov commented Jun 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rzarzynski commented Jun 24, 2019

Uh oh!

rzarzynski commented Jun 24, 2019

Uh oh!

rzarzynski commented Jun 26, 2019

Uh oh!

Uh oh!

tchaikov commented Jul 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

rzarzynski May 7, 2019 •

edited

Loading

tchaikov commented Jun 22, 2019 •

edited

Loading

tchaikov commented Jul 4, 2019 •

edited

Loading