Checklist

• I have verified that the issue exists against the master branch of Celery.
• This has already been asked to the discussions forum first.
• I have read the relevant section in the
  contribution guide
  on reporting bugs.
• I have checked the issues list
  for similar or identical bug reports.
• I have checked the pull requests list
  for existing proposed fixes.
• I have checked the commit log
  to find out if the bug was already fixed in the master branch.
• I have included all related issues and possible duplicate issues
  in this issue (If there are none, check this box anyway).

Mandatory Debugging Information

• I have included the output of celery -A proj report in the issue.
  (if you are not able to do this, then at least specify the Celery
  version affected).
• I have verified that the issue exists against the master branch of Celery.
• I have included the contents of pip freeze in the issue.
• I have included all the versions of all the external dependencies required
  to reproduce this bug.

Optional Debugging Information

• I have tried reproducing the issue on more than one Python version
  and/or implementation.
• I have tried reproducing the issue on more than one message broker and/or
  result backend.
• I have tried reproducing the issue on more than one version of the message
  broker and/or result backend.
• I have tried reproducing the issue on more than one operating system.
• I have tried reproducing the issue on more than one workers pool.
• I have tried reproducing the issue with autoscaling, retries,
  ETA/Countdown & rate limits disabled.
• I have tried reproducing the issue after downgrading
  and/or upgrading Celery and its dependencies.

Related Issues and Possible Duplicates

Related Issues

#6271

Possible Duplicates

• None

Environment & Settings

Celery version:

Steps to Reproduce

Required Dependencies

• Minimal Python Version: N/A or Unknown
• Minimal Celery Version: 5.2
• Minimal Kombu Version: N/A or Unknown
• Minimal Broker Version: SQS
• Minimal Result Backend Version: N/A or Unknown
• Minimal OS and/or Kernel Version: N/A or Unknown
• Minimal Broker Client Version: N/A or Unknown
• Minimal Result Backend Client Version: N/A or Unknown

Python Packages

Other Dependencies

Minimally Reproducible Test Case

Expected Behavior

Using pip-tools to compile a requirements.txt should include the locked dependencies for the sqs-backend.
For example using this requirements.in file:

celery[sqs]

Compiled to a requirements.txt by:

pip-compile --generate-hashes --output-file requirements.txt requirements.in

This requirements.txt should include the pinned dependencies for the SQS backend (boto3, pycurl, urllib3) and installable with pip install --require-hashes -r reqs/requirements.txt .

Actual Behavior

Trying to install this requirements.txt by pip install --require-hashes -r reqs/requirements.txt will error since not all dependencies are pinned as required by the option require-hashes. I've tried this with the latest pip (21.3.1). and pip-tools (6.4.0).

$ pip install --require-hashes -r requirements.txt
Collecting amqp==5.0.6
  Using cached amqp-5.0.6-py3-none-any.whl (53 kB)
Collecting billiard==3.6.4.0
  Using cached billiard-3.6.4.0-py3-none-any.whl (89 kB)
Collecting celery[sqs]==5.2.0
  Using cached celery-5.2.0-py3-none-any.whl (404 kB)
Requirement already satisfied: click==8.0.3 in ./venv/lib/python3.10/site-packages (from -r requirements.txt (line 19)) (8.0.3)
Collecting click-didyoumean==0.3.0
  Using cached click_didyoumean-0.3.0-py3-none-any.whl (2.7 kB)
Collecting click-plugins==1.1.1
  Using cached click_plugins-1.1.1-py2.py3-none-any.whl (7.5 kB)
Collecting click-repl==0.2.0
  Using cached click_repl-0.2.0-py3-none-any.whl (5.2 kB)
Collecting kombu==5.2.1
  Using cached kombu-5.2.1-py3-none-any.whl (188 kB)
Collecting prompt-toolkit==3.0.22
  Using cached prompt_toolkit-3.0.22-py3-none-any.whl (374 kB)
Collecting pytz==2021.3
  Using cached pytz-2021.3-py2.py3-none-any.whl (503 kB)
Collecting six==1.16.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting vine==5.0.0
  Using cached vine-5.0.0-py2.py3-none-any.whl (9.4 kB)
Collecting wcwidth==0.2.5
  Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: setuptools in ./venv/lib/python3.10/site-packages (from celery[sqs]==5.2.0->-r requirements.txt (line 15)) (58.2.0)
Collecting urllib3>=1.26.7
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    urllib3>=1.26.7 from https://files.pythonhosted.org/packages/af/f4/524415c0744552cce7d8bf3669af78e8a069514405ea4fcbd0cc44733744/urllib3-1.26.7-py2.py3-none-any.whl#sha256=c4fdf4019605b6e5423637e01bc9fe4daef873709a7973e195ceba0a62bbc844 (from kombu==5.2.1->-r requirements.txt (line 39))

I tracked down the cause of this to #6271 where requirements/sqs.txt is modified to install kombu[sqs] instead of the explicit dependencies. See https://github.com/celery/celery/pull/6271/files#diff-d229b45a35f019d718901cd33e806e8d2fbecb8eb66d81fb00f8e481c56ea0b4

I under that it's much nicer to only specify these dependencies in one repository by using this pass through dependency but it appears like pip-tools (or even the resolver in pip?) can not handle it. I haven't really had the time to dig into the details but it might make sense to explicitly define the SQS dependencies in https://github.com/celery/celery/blob/master/requirements/extras/sqs.txt as is done for the other backends until this works upstream since pip-tools is a rather common tool to work with dependencies.