Skip to content

Commit 2e65c70

Browse files
potiukpotiuk
authored
Merging of the google ads vendored-in code. (#30399)
Following google ads disabling v11 of their API, we are vendoring in google adds code. This is the first commit that just vendors in the original code of google-ads v. 20.0.0 It will be followed by a second PR where the code will be modified following the new package. Separating those two commits will help in case we have a need to migrate to another version of the library as then we will just revert this one and apply the second and resolve conflicts.
1 parent 3c9369a commit 2e65c70

File tree

1,308 files changed

+196775
-40
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,308 files changed

+196775
-40
lines changed

‎.coveragerc‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ omit =
2727
airflow/migrations/**
2828
airflow/providers/**/example_dags/**
2929
airflow/www/node_modules/**
30+
airflow/providers/google/ads/_vendor/**
3031

3132
[report]
3233
skip_empty = True

‎.pre-commit-config.yaml‎

Lines changed: 38 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ repos:
3434
name: Add TOC for Markdown and RST files
3535
files:
3636
^CONTRIBUTING\.md$|^README\.md$|^UPDATING.*\.md$|^chart/UPDATING.*\.md$|^dev/.*\.md$|^dev/.*\.rst$
37-
exclude: ^airflow/_vendor/
37+
exclude: ^.*/_vendor/
3838
args:
3939
- "--maxlevel"
4040
- "2"
@@ -47,7 +47,7 @@ repos:
4747
exclude: |
4848
(?x)
4949
^\.github/|
50-
^airflow/_vendor/
50+
^.*/_vendor/
5151
args:
5252
- --comment-style
5353
- "/*||*/"
@@ -56,7 +56,7 @@ repos:
5656
- --fuzzy-match-generates-todo
5757
- id: insert-license
5858
name: Add license for all RST files
59-
exclude: ^\.github/.*$|^airflow/_vendor/|newsfragments/.*\.rst$
59+
exclude: ^\.github/.*$|^.*/_vendor/|newsfragments/.*\.rst$
6060
args:
6161
- --comment-style
6262
- "||"
@@ -67,7 +67,7 @@ repos:
6767
- id: insert-license
6868
name: Add license for all CSS/JS/JSX/PUML/TS/TSX files
6969
files: \.(css|jsx?|puml|tsx?)$
70-
exclude: ^\.github/.*$|^airflow/_vendor/
70+
exclude: ^\.github/.*$|^.*/_vendor/
7171
args:
7272
- --comment-style
7373
- "/*!| *| */"
@@ -77,7 +77,7 @@ repos:
7777
- id: insert-license
7878
name: Add license for all JINJA template files
7979
files: ^airflow/www/templates/.*\.html$
80-
exclude: ^\.github/.*$|^airflow/_vendor/
80+
exclude: ^\.github/.*$|^.*/_vendor/
8181
args:
8282
- --comment-style
8383
- "{#||#}"
@@ -86,7 +86,7 @@ repos:
8686
- --fuzzy-match-generates-todo
8787
- id: insert-license
8888
name: Add license for all Shell files
89-
exclude: ^\.github/.*$|^airflow/_vendor/|^dev/breeze/autocomplete/.*$
89+
exclude: ^\.github/.*$|^.*/_vendor/|^dev/breeze/autocomplete/.*$
9090
files: \.bash$|\.sh$
9191
args:
9292
- --comment-style
@@ -96,7 +96,7 @@ repos:
9696
- --fuzzy-match-generates-todo
9797
- id: insert-license
9898
name: Add license for all Python files
99-
exclude: ^\.github/.*$|^airflow/_vendor/
99+
exclude: ^\.github/.*$|^.*/_vendor/
100100
files: \.py$|\.pyi$
101101
args:
102102
- --comment-style
@@ -106,7 +106,7 @@ repos:
106106
- --fuzzy-match-generates-todo
107107
- id: insert-license
108108
name: Add license for all XML files
109-
exclude: ^\.github/.*$|^airflow/_vendor/
109+
exclude: ^\.github/.*$|^.*/_vendor/
110110
files: \.xml$
111111
args:
112112
- --comment-style
@@ -116,7 +116,7 @@ repos:
116116
- --fuzzy-match-generates-todo
117117
- id: insert-license
118118
name: Add license for all YAML files
119-
exclude: ^\.github/.*$|^airflow/_vendor/
119+
exclude: ^\.github/.*$|^.*/_vendor/
120120
types: [yaml]
121121
files: \.ya?ml$
122122
args:
@@ -128,7 +128,7 @@ repos:
128128
- id: insert-license
129129
name: Add license for all Markdown files
130130
files: \.md$
131-
exclude: ^\.github/.*$|PROVIDER_CHANGES.*\.md$|^airflow/_vendor/
131+
exclude: ^\.github/.*$|PROVIDER_CHANGES.*\.md$|^.*/_vendor/
132132
args:
133133
- --comment-style
134134
- "<!--|| -->"
@@ -137,7 +137,7 @@ repos:
137137
- --fuzzy-match-generates-todo
138138
- id: insert-license
139139
name: Add license for all other files
140-
exclude: ^\.github/.*$|^airflow/_vendor/
140+
exclude: ^\.github/.*$|^.*/_vendor/
141141
args:
142142
- --comment-style
143143
- "|#|"
@@ -152,7 +152,7 @@ repos:
152152
- id: black
153153
name: Run black (Python formatter)
154154
args: [--config=./pyproject.toml]
155-
exclude: ^airflow/_vendor/|^airflow/contrib/
155+
exclude: ^.*/_vendor/|^airflow/contrib/
156156
- repo: local
157157
hooks:
158158
- id: update-common-sql-api-stubs
@@ -181,7 +181,7 @@ repos:
181181
entry: ruff --fix --no-update-check --force-exclude
182182
additional_dependencies: ['ruff==0.0.226']
183183
files: \.pyi?$
184-
exclude: ^airflow/_vendor/
184+
exclude: ^.*/_vendor/
185185
- repo: https://github.com/asottile/blacken-docs
186186
rev: 1.13.0
187187
hooks:
@@ -204,28 +204,28 @@ repos:
204204
name: Detect accidentally committed debug statements
205205
- id: check-builtin-literals
206206
name: Require literal syntax when initializing Python builtin types
207-
exclude: ^airflow/_vendor/
207+
exclude: ^.*/_vendor/
208208
- id: detect-private-key
209209
name: Detect if private key is added to the repository
210210
exclude: ^docs/apache-airflow-providers-ssh/connections/ssh.rst$
211211
- id: end-of-file-fixer
212212
name: Make sure that there is an empty line at the end
213-
exclude: ^airflow/_vendor/|^docs/apache-airflow/img/.*\.dot|^docs/apache-airflow/img/.*\.sha256
213+
exclude: ^.*/_vendor/|^docs/apache-airflow/img/.*\.dot|^docs/apache-airflow/img/.*\.sha256
214214
- id: mixed-line-ending
215215
name: Detect if mixed line ending is used (\r vs. \r\n)
216-
exclude: ^airflow/_vendor/
216+
exclude: ^.*/_vendor/
217217
- id: check-executables-have-shebangs
218218
name: Check that executables have shebang
219-
exclude: ^airflow/_vendor/
219+
exclude: ^.*/_vendor/
220220
- id: check-xml
221221
name: Check XML files with xmllint
222-
exclude: ^airflow/_vendor/
222+
exclude: ^.*/_vendor/
223223
- id: trailing-whitespace
224224
name: Remove trailing whitespace at end of line
225-
exclude: ^airflow/_vendor/|^docs/apache-airflow/img/.*\.dot|^images/breeze/output.*$
225+
exclude: ^.*/_vendor/|^docs/apache-airflow/img/.*\.dot|^images/breeze/output.*$
226226
- id: fix-encoding-pragma
227227
name: Remove encoding header from Python files
228-
exclude: ^airflow/_vendor/
228+
exclude: ^.*/_vendor/
229229
args:
230230
- --remove
231231
- id: pretty-format-json
@@ -242,26 +242,26 @@ repos:
242242
hooks:
243243
- id: rst-backticks
244244
name: Check if RST files use double backticks for code
245-
exclude: ^airflow/_vendor/
245+
exclude: ^.*/_vendor/
246246
- id: python-no-log-warn
247247
name: Check if there are no deprecate log warn
248-
exclude: ^airflow/_vendor/
248+
exclude: ^.*/_vendor/
249249
- repo: https://github.com/adrienverge/yamllint
250250
rev: v1.29.0
251251
hooks:
252252
- id: yamllint
253253
name: Check YAML files with yamllint
254254
entry: yamllint -c yamllint-config.yml --strict
255255
types: [yaml]
256-
exclude: ^.*airflow\.template\.yaml$|^.*init_git_sync\.template\.yaml$|^airflow/_vendor/|^chart/(?:templates|files)/.*\.yaml$|openapi/.*\.yaml$|^\.pre-commit-config\.yaml$
256+
exclude: ^.*airflow\.template\.yaml$|^.*init_git_sync\.template\.yaml$|^.*/_vendor/|^chart/(?:templates|files)/.*\.yaml$|openapi/.*\.yaml$|^\.pre-commit-config\.yaml$
257257
- repo: https://github.com/ikamensh/flynt
258258
rev: '0.77'
259259
hooks:
260260
- id: flynt
261261
name: Run flynt string format converter for Python
262262
exclude: |
263263
(?x)
264-
^airflow/_vendor/
264+
^.*/_vendor/
265265
args:
266266
# If flynt detects too long text it ignores it. So we set a very large limit to make it easy
267267
# to split the text by hand. Too long lines are detected by flake8 (below),
@@ -278,7 +278,7 @@ repos:
278278
The word(s) should be in lowercase." && exec codespell "$@"' --
279279
language: python
280280
types: [text]
281-
exclude: ^airflow/_vendor/|^airflow/www/static/css/material-icons\.css$|^images/.*$|^RELEASE_NOTES\.txt$|^.*package-lock\.json$
281+
exclude: ^.*/_vendor/|^airflow/www/static/css/material-icons\.css$|^images/.*$|^RELEASE_NOTES\.txt$|^.*package-lock\.json$
282282
args:
283283
- --ignore-words=docs/spelling_wordlist.txt
284284
- --skip=airflow/providers/*/*.rst,airflow/www/*.log,docs/*/commits.rst,docs/apache-airflow/tutorial/pipeline_example.csv,*.min.js,*.lock,INTHEWILD.md
@@ -290,7 +290,7 @@ repos:
290290
entry: ./scripts/ci/pre_commit/pre_commit_replace_bad_characters.py
291291
language: python
292292
types: [file, text]
293-
exclude: ^airflow/_vendor/|^clients/gen/go\.sh$|^\.gitmodules$
293+
exclude: ^.*/_vendor/|^clients/gen/go\.sh$|^\.gitmodules$
294294
additional_dependencies: ['rich>=12.4.4']
295295
- id: lint-openapi
296296
name: Lint OpenAPI using spectral
@@ -419,7 +419,7 @@ repos:
419419
entry: "^\\s*from\\s+\\."
420420
pass_filenames: true
421421
files: \.py$
422-
exclude: ^airflow/_vendor/|^docs/|^tests/
422+
exclude: ^.*/_vendor/|^docs/|^tests/
423423
- id: check-urlparse-usage-in-code
424424
language: pygrep
425425
name: Don't use urlparse in code
@@ -483,6 +483,7 @@ repos:
483483
^airflow/www/static/css/bootstrap-theme.css$|
484484
^airflow/www/static/js/types/api-generated.ts$|
485485
^airflow/www/templates/appbuilder/flash.html$|
486+
^.*/_vendor/|
486487
^dev/|
487488
^docs/README.rst$|
488489
^docs/apache-airflow-providers-amazon/secrets-backends/aws-ssm-parameter-store.rst$|
@@ -535,7 +536,7 @@ repos:
535536
files: >
536537
(?x)
537538
^airflow/providers/.*\.py$
538-
exclude: ^airflow/_vendor/
539+
exclude: ^.*/_vendor/
539540
- id: check-decorated-operator-implements-custom-name
540541
name: Check @task decorator implements custom_operator_name
541542
language: python
@@ -556,28 +557,28 @@ repos:
556557
to avoid import cycles.
557558
entry: "from airflow\\.utils\\.db import.* (provide_session|create_session)"
558559
files: \.py$
559-
exclude: ^airflow/_vendor/
560+
exclude: ^.*/_vendor/
560561
pass_filenames: true
561562
- id: check-incorrect-use-of-LoggingMixin
562563
language: pygrep
563564
name: Make sure LoggingMixin is not used alone
564565
entry: "LoggingMixin\\(\\)"
565566
files: \.py$
566-
exclude: ^airflow/_vendor/
567+
exclude: ^.*/_vendor/
567568
pass_filenames: true
568569
- id: check-daysago-import-from-utils
569570
language: pygrep
570571
name: Make sure days_ago is imported from airflow.utils.dates
571572
entry: "(airflow\\.){0,1}utils\\.dates\\.days_ago"
572573
files: \.py$
573-
exclude: ^airflow/_vendor/
574+
exclude: ^.*/_vendor/
574575
pass_filenames: true
575576
- id: check-start-date-not-used-in-defaults
576577
language: pygrep
577578
name: "'start_date' not to be defined in default_args in example_dags"
578579
entry: "default_args\\s*=\\s*{\\s*(\"|')start_date(\"|')|(\"|')start_date(\"|'):"
579580
files: \.*example_dags.*\.py$
580-
exclude: ^airflow/_vendor/
581+
exclude: ^.*/_vendor/
581582
pass_filenames: true
582583
- id: check-apache-license-rat
583584
name: Check if licenses are OK for Apache
@@ -723,7 +724,7 @@ repos:
723724
language: python
724725
pass_filenames: true
725726
files: .*\.schema\.json$
726-
exclude: ^airflow/_vendor/
727+
exclude: ^.*/_vendor/
727728
require_serial: true
728729
additional_dependencies: ['jsonschema>=3.2.0,<5.0', 'PyYAML==5.3.1', 'requests==2.25.0']
729730
- id: lint-json-schema
@@ -808,7 +809,7 @@ repos:
808809
language: python
809810
pass_filenames: true
810811
files: \.py$
811-
exclude: ^airflow/_vendor/
812+
exclude: ^.*/_vendor/
812813
additional_dependencies: ['rich>=12.4.4']
813814
- id: lint-chart-schema
814815
name: Lint chart/values.schema.json file
@@ -868,7 +869,7 @@ repos:
868869
language: python
869870
pass_filenames: true
870871
files: \.py$
871-
exclude: ^airflow/_vendor/
872+
exclude: ^.*/_vendor/
872873
additional_dependencies: ['rich>=12.4.4', 'astor']
873874
- id: create-missing-init-py-files-tests
874875
name: Create missing init.py files in tests
@@ -899,14 +900,15 @@ repos:
899900
language: python
900901
entry: ./scripts/ci/pre_commit/pre_commit_mypy.py --namespace-packages
901902
files: \.py$
902-
exclude: ^airflow/_vendor/|^airflow/migrations|^airflow/providers|^dev|^docs|^provider_packages|^tests/providers|^tests/system/providers
903+
exclude: ^.*/_vendor/|^airflow/migrations|^airflow/providers|^dev|^docs|^provider_packages|^tests/providers|^tests/system/providers
903904
require_serial: true
904905
additional_dependencies: ['rich>=12.4.4', 'inputimeout']
905906
- id: run-mypy
906907
name: Run mypy for providers
907908
language: python
908909
entry: ./scripts/ci/pre_commit/pre_commit_mypy.py --namespace-packages
909910
files: ^airflow/providers/.*\.py$|^tests/providers/\*\.py$|^tests/system/providers/\*\.py$
911+
exclude: ^.*/_vendor/
910912
require_serial: true
911913
additional_dependencies: ['rich>=12.4.4', 'inputimeout']
912914
- id: run-mypy

‎.rat-excludes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,7 @@ tmp
103103

104104
# Vendored-in code
105105
_vendor
106+
./airflow/providers/google/ads/_vendor
106107

107108
# generated doc-only-changes.txt
108109
.latest-doc-only-change.txt

‎airflow/providers/google/ads/_vendor/README.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Vendor package
2+
3+
## What vendored packages are for
4+
5+
The `_vendor` package is foreseen for vendoring in packages, that we have to modify ourselves
6+
because authors of the packages do not have time to modify them themselves. This is often temporary
7+
and once the packages implement fixes that we need, and then we remove the packages from
8+
the `_vendor` package.
9+
10+
All Vendored libraries must follow these rules:
11+
12+
1. Vendored libraries must be pure Python--no compiling (so that we do not have to release multi-platform airflow packages on PyPI).
13+
2. Source code for the libary is included in this directory.
14+
3. License must be included in this repo and in the [LICENSE](../../LICENSE) file and in the
15+
[licenses](../../licenses) folder.
16+
4. Requirements of the library become requirements of airflow core.
17+
5. Version of the library should be included in the [vendor.md](vendor.md) file.
18+
6. No modifications to the library may be made in the initial commit.
19+
7. Apply the fixes necessary to use the vendored library as separate commits - each package separately,
20+
so that they can be cherry-picked later if we upgrade the vendored package. Changes to airflow code to
21+
use the vendored packages should be applied as separate commits/PRs.
22+
8. The `_vendor` packages should be excluded from any refactorings, static checks and automated fixes.
23+
24+
## Adding and upgrading a vendored package
25+
26+
Way to vendor a library or update a version:
27+
28+
1. Update ``vendor.md`` with the library, version, and SHA256 (`pypi` provides hashes as of recently)
29+
2. Remove all old files and directories of the old version.
30+
3. Replace them with new files (only replace relevant python packages:move LICENSE )
31+
* move licence files to [licenses](../../licenses) folder
32+
* remove README and any other supporting files (they can be found in PyPI)
33+
* make sure to add requirements from setup.py to airflow's setup.py with appropriate comment stating
34+
why the requirements are added and when they should be removed
35+
4. If you replace previous version, re-apply historical fixes from the "package" folder by
36+
cherry-picking them.
37+

‎airflow/providers/google/ads/_vendor/__init__.py‎

Whitespace-only changes.

‎airflow/providers/google/ads/_vendor/googleads/__init__.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
# Copyright 2018 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
import google.ads.googleads.client
16+
import google.ads.googleads.errors
17+
import google.ads.googleads.util
18+
19+
VERSION = "20.0.0"

0 commit comments

Comments
 (0)