Skip to content

Conversation

@turbaszek
Copy link
Member

@turbaszek turbaszek commented Dec 29, 2020

Todo:

  • add sensor
  • add example dag
  • add system test
  • add docs

^ Add meaningful description above

Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.

@boring-cyborg boring-cyborg bot added the provider:google Google (including GCP) related issues label Dec 29, 2020
@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

Copy link

@tolkjen tolkjen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for implementing the operators so quickly!

@github-actions
Copy link

github-actions bot commented Jan 7, 2021

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

github-actions bot commented Jan 7, 2021

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

github-actions bot commented Jan 7, 2021

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

github-actions bot commented Jan 8, 2021

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@turbaszek turbaszek marked this pull request as ready for review January 26, 2021 11:58
@turbaszek turbaszek requested a review from mik-laj January 26, 2021 12:00
@turbaszek
Copy link
Member Author

@TobKed would you mind taking a look?

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

Copy link
Contributor

@TobKed TobKed left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice! Good job!. I just left some minor comments

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@turbaszek turbaszek requested a review from potiuk January 27, 2021 15:06
@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@github-actions
Copy link

The Workflow run is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.

@turbaszek
Copy link
Member Author

@mik-laj is the docs build failure related? I see many errors but those seems to affect all packages not only google

@mik-laj
Copy link
Member

mik-laj commented Jan 28, 2021

@turbaszek

apache-airflow-providers-mongo                                       2                           1
intersphinx inventory '/opt/airflow/docs/_inventory_cache/mongodb/objects.inv' not fetchable due to <class 'FileNotFoundError'>: [Errno 2] No such file or directory: '/opt/airflow/docs/_inventory_cache/mongodb/objects.inv'

mongo vs mongodb

Can you check that all vendor names are consistently used?

@github-actions
Copy link

The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the full tests needed We need to run full set of tests for this PR to merge label Jan 28, 2021
@mik-laj
Copy link
Member

mik-laj commented Jan 28, 2021

I found a problem.

THIRD_PARTY_INDEXES = {
    'boto3': 'https://boto3.amazonaws.com/v1/documentation/api/latest',
    'celery': 'https://docs.celeryproject.org/en/stable',
    'hdfs': 'https://hdfscli.readthedocs.io/en/latest',
    'jinja2': 'https://jinja.palletsprojects.com/en/master',
    'mongodb': 'https://api.mongodb.com/python/current',
    'pandas': 'https://pandas.pydata.org/pandas-docs/stable',

https://api.mongodb.com/python/current this links doesn't works.

@mik-laj
Copy link
Member

mik-laj commented Jan 28, 2021

https://pymongo.readthedocs.io/en/latest/ this links works now.

@github-actions
Copy link

The Workflow run is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.

fixup! Rename operators and add docs

Apply suggestions from code review

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>

Update airflow/providers/google/cloud/hooks/workflows.py

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>

fixup! fixup! Rename operators and add docs

fixup! fixup! fixup! Rename operators and add docs

fixup! fixup! fixup! fixup! Rename operators and add docs

fixup! fixup! fixup! fixup! fixup! Rename operators and add docs

fixup! fixup! fixup! fixup! fixup! fixup! Rename operators and add docs

Update airflow/providers/google/cloud/example_dags/example_workflows.py

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>
@turbaszek turbaszek merged commit 6d6588f into apache:master Jan 28, 2021
@turbaszek turbaszek deleted the cloud-workflows branch January 28, 2021 19:35
potiuk pushed a commit that referenced this pull request Mar 3, 2021
Add Google Cloud Workflows Operators, system test, example and sensor

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>
(cherry picked from commit 6d6588f)
potiuk pushed a commit that referenced this pull request Mar 3, 2021
Add Google Cloud Workflows Operators, system test, example and sensor

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>
(cherry picked from commit 6d6588f)
potiuk pushed a commit that referenced this pull request Mar 3, 2021
Add Google Cloud Workflows Operators, system test, example and sensor

Co-authored-by: Tobiasz Kędzierski <tobiasz.kedzierski@polidea.com>
(cherry picked from commit 6d6588f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full tests needed We need to run full set of tests for this PR to merge provider:google Google (including GCP) related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants