Skip to content

Add FIPS support#1233

Merged
fviard merged 2 commits intos3tools:masterfrom
maroth96:maroth96-add-fips-support
Oct 4, 2022
Merged

Add FIPS support#1233
fviard merged 2 commits intos3tools:masterfrom
maroth96:maroth96-add-fips-support

Conversation

@maroth96
Copy link
Copy Markdown
Contributor

@maroth96 maroth96 commented Feb 2, 2022

When FIPS mode is enabled, s3cmd raises a ValueError exception with the message 'digital envelope routines: EVP_DigestInit_ex] disabled for FIPS' (see below). This change will make s3cmd compatible with FIPS mode.

$ fips-mode-setup --check
FIPS mode is enabled.

$ s3cmd put ...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  ...
Problem: <class 'ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
S3cmd:   2.2.0
python:   3.6.8 (default, Jan 19 2022, 23:28:49) 
[GCC 8.5.0 20210514 (Red Hat 8.5.0-7)]
environment LANG=en_US.UTF-8
Traceback (most recent call last):
  File "./s3cmd/s3cmd", line 3209, in <module>
    rc = main()
  File "./s3cmd/s3cmd", line 3106, in main
    rc = cmd_func(args)
  File "./s3cmd/s3cmd", line 392, in cmd_object_put
    local_list, single_file_local, exclude_list, total_size_local = fetch_local_list(args, is_src = True)
  File "/home/michaelroth/s3cmd/S3/FileLists.py", line 367, in fetch_local_list
    total_size = _fetch_local_list_info(local_list)
  File "/home/michaelroth/s3cmd/S3/FileLists.py", line 238, in _fetch_local_list_info
    md5 = loc_list.get_md5(relative_file) # this does the file I/O
  File "/home/michaelroth/s3cmd/S3/FileDict.py", line 48, in get_md5
    md5 = Utils.hash_file_md5(self[relative_file]['full_name'])
  File "/home/michaelroth/s3cmd/S3/Utils.py", line 105, in hash_file_md5
    h = md5()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

@maroth96
Copy link
Copy Markdown
Contributor Author

maroth96 commented Feb 2, 2022

This should fix this issue: #1005

@fviard fviard added this to the 2.3.0 milestone Mar 2, 2022
@fviard
Copy link
Copy Markdown
Contributor

fviard commented Mar 2, 2022 •
edited
Loading

Thank you for your PR, I never heard about this parameter for hashlib but it is awesome to unblock people in such a case.

I notice here (https://code.djangoproject.com/ticket/28401) that it is only available since python 3.9 or on Fedora/redhat distributions. But still better than nothing.

I would suggest the following improvement to your PR:
As the md5 will be used a lot of times and also in multiple places in the code, you could do something like:

  • Do the try/except just once inside BaseUtils.py. Something like:
from functools import partial
from hashlib import md5_real
try:
   md5 = md5_real()
except Exception as exc:
  # Comment to explain that it is for fips
  try:
    md5_real(usedforsecurity=False)
    md5 = partial(md5_real, usedforsecurity=False)
  except:
      # Raise original issue
      raise exc
  • import md5 from BaseUtils.py and use it directly in the other places of s3cmd

@maroth96
Copy link
Copy Markdown
Contributor Author

maroth96 commented Mar 4, 2022

Done. md5 is moved to BaseUtils.

@maroth96
Copy link
Copy Markdown
Contributor Author

maroth96 commented Apr 4, 2022 •
edited
Loading

@fviard Is this change OK for submission?

@fviard fviard modified the milestones: 2.3.0, 2.4.0 Oct 2, 2022
@fviard fviard merged commit 7977018 into s3tools:master Oct 4, 2022
@fviard
Copy link
Copy Markdown
Contributor

fviard commented Oct 4, 2022

Merged, thank you very much and sorry for the delay. I think that I did not notice that you pushed the requested changes.

Btw, I will modify further the code in another commit, because as the "usedforsecurity" will be available upstream, it will be good to have that as a default case.

@uipatherthangaraje
Copy link
Copy Markdown

uipatherthangaraje commented Dec 7, 2022

When is the plan to release it? how can I consume this fix? thanks

@rajivml
Copy link
Copy Markdown

rajivml commented Dec 7, 2022

@fviard glad this is fixed but I couldn't find a release post october 3rd where this fix is available https://github.com/s3tools/s3cmd/releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

feature-request

Projects

None yet

Milestone

2.4.0

Development

Successfully merging this pull request may close these issues.

4 participants

@maroth96 @fviard @uipatherthangaraje @rajivml