Skip to content

Add Google Cloud Storage (GCS) backend #8868

Merged
thedrow merged 8 commits intocelery:mainfrom
jether-energy:main
Feb 28, 2024
Merged

Add Google Cloud Storage (GCS) backend #8868
thedrow merged 8 commits intocelery:mainfrom
jether-energy:main

Conversation

@haimjether
Copy link
Contributor

  • Add extra google-cloud-storage requirements

  • Add gcs backend module

  • Add gcs backend to userguide configuration

  • Add gcs to backends in README

  • Add gcs app Namespace

  • Add configuration documentation

Note: Before submitting this pull request, please review our contributing
guidelines
.

Description

* Add extra google-cloud-storage requirements

* Add gcs backend module

* Add gcs backend to userguide configuration

* Add gcs to backends in README

* Add gcs app Namespace

* Add configuration documentation
@codecov
Copy link

codecov bot commented Feb 24, 2024

Codecov Report

Attention: Patch coverage is 91.20879% with 8 lines in your changes are missing coverage. Please review.

Project coverage is 81.30%. Comparing base (71fce1b) to head (e3aecc2).
Report is 3 commits behind head on main.

Files Patch % Lines
celery/backends/gcs.py 91.20% 5 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8868      +/-   ##
==========================================
+ Coverage   81.26%   81.30%   +0.04%     
==========================================
  Files         149      150       +1     
  Lines       18554    18645      +91     
  Branches     3166     3182      +16     
==========================================
+ Hits        15077    15159      +82     
- Misses       3191     3196       +5     
- Partials      286      290       +4     
Flag Coverage Δ
unittests 81.28% <91.20%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@thedrow thedrow merged commit 06e91d9 into celery:main Feb 28, 2024
'azureblockblob': 'celery.backends.azureblockblob:AzureBlockBlobBackend',
'arangodb': 'celery.backends.arangodb:ArangoDbBackend',
's3': 'celery.backends.s3:S3Backend',
'gs': 'celery.backends.gcs:GCSBackend',

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@haimjether @thedrow something I was wondering about during upgrade-review for celery - is this meant to be registered as gs or is that a typo of gcs?

Copy link
Contributor Author

@haimjether haimjether May 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jayaddison , it was meant to be registered as gs , to mimic the blobs path prefix on GCS. E.g
gs://my-bucket/my-folder

The module path is gcs.GCSBackend because Google's commercial product is "GCS" , as opposed to "S3" in AWS

IMHO this is similar to AWS s3 one.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 great, glad to know it's as-intended. Thanks @haimjether!

Copy link
Member

@auvipy auvipy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this misses the .. versionadded:: 5.4 annotation

@auvipy auvipy added this to the 5.4 milestone May 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants