feat: add get_bq_config_path() to _cloud_sdk.py by wj-chen · Pull Request #1358 · googleapis/google-auth-library-python

wj-chen · 2023-07-24T21:19:13Z

Will be used by the Cloud CLI's config to write a credential file for the bq tool to use. Part of the work to support BQ CLI's auth library upgrade from oauth2client to google-auth.

wj-chen · 2023-07-24T21:38:48Z

I believe the kokoro failures are unrelated to my change, but this is my first time making a change to this repo so please advise.

clundin25 · 2023-07-24T22:18:42Z

google/auth/_cloud_sdk.py


-def get_config_path():
-    """Returns the absolute path the the Cloud SDK's configuration directory.
+def get_config_path(config_directory=_CONFIG_DIRECTORY):


Maybe we should wrap this function with a gcloud_get_config_path so the intent is clearly documented.

Would it be a large change to modify existing uses of get_config_path?

Updated, not too much overhead to modify existing uses of get_config_path

clundin25 · 2023-07-25T18:06:50Z

@wj-chen I refreshed the test credentials. The next test run should have all the noise removed.

wj-chen · 2023-07-25T19:50:40Z

@wj-chen I refreshed the test credentials. The next test run should have all the noise removed.

Thank you! There appears to be lint errors. Running nox -s lint has always given me the following output, do you have suggestions?

nox > Running session lint
nox > Missing interpreters will error by default on CI systems.
nox > Session lint skipped: Python interpreter 3.8 not found.

clundin25 · 2023-07-25T20:38:51Z

The interpreter for the lint step uses Python 3.8.

I suggest installing it via pyenv.

Here is a sample that will give you a shell that has 3.8.

$ pyenv install 3.8.15
$ pyenv shell 3.8.15
$ python --version # Confirm the interpreter is 3.8.

Now the active shell should have access to 3.8.

wj-chen · 2023-07-26T04:38:30Z

The interpreter for the lint step uses Python 3.8.

I suggest installing it via pyenv.

Here is a sample that will give you a shell that has 3.8.
$ pyenv install 3.8.15
$ pyenv shell 3.8.15
$ python --version # Confirm the interpreter is 3.8.
Now the active shell should have access to 3.8.

Thanks! Linter would run now but would result in an AttributionError:

      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
        self.set_undefined_options('install',('install_layout','install_layout'))
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
        setattr(self, dst_option, getattr(src_cmd_obj, src_option))
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
        raise AttributeError(attr)
    AttributeError: install_layout
    [end of output]

pyenv --version: 2.3.23
Also tried reinstalling setuptools (currently version 68.0) and setting SETUPTOOLS_USE_DISTUTILS=stdlib etc. as suggested online to no avail.
Looks like a version mismatch. Is there another python 3.8 minor version to try?

clundin25 · 2023-07-26T16:23:10Z

Do you mind sharing the full trace with me either here or offline? I don't think I have enough info to help debug.

wj-chen · 2023-07-26T23:46:13Z

Do you mind sharing the full trace with me either here or offline? I don't think I have enough info to help debug.

Apologies that this has turned into a troubleshooting session but much appreciate your assistance! Sharing the stack trace here for visibility:

wjchen@wjchen:/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python$ python --version
Python 3.8.15
wjchen@wjchen:/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python$ nox -s lint
nox > Running session lint
nox > Creating virtual environment (virtualenv) using python3.8 in .nox/lint
nox > python -m pip install flake8 flake8-import-order docutils click==8.0.4 black==19.3b0
nox > python -m pip install -e .
nox > Command python -m pip install -e . failed with exit code 1:
Obtaining file:///google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting cachetools<6.0,>=2.0.0 (from google-auth==2.22.0)
  Using cached cachetools-5.3.1-py3-none-any.whl (9.3 kB)
Collecting pyasn1-modules>=0.2.1 (from google-auth==2.22.0)
  Using cached pyasn1_modules-0.3.0-py2.py3-none-any.whl (181 kB)
Collecting rsa<5,>=3.1.4 (from google-auth==2.22.0)
  Using cached rsa-4.9-py3-none-any.whl (34 kB)
Collecting urllib3<2.0 (from google-auth==2.22.0)
  Using cached urllib3-1.26.16-py2.py3-none-any.whl (143 kB)
Collecting pyasn1<0.6.0,>=0.4.6 (from pyasn1-modules>=0.2.1->google-auth==2.22.0)
  Using cached pyasn1-0.5.0-py2.py3-none-any.whl (83 kB)
Installing collected packages: urllib3, pyasn1, cachetools, rsa, pyasn1-modules, google-auth
  Running setup.py develop for google-auth
    error: subprocess-exited-with-error
    
    × python setup.py develop did not run successfully.
    │ exit code: 1
    ╰─> [39 lines of output]
        running develop
        /google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/easy_install.py:146: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        /google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
          warnings.warn(
        Traceback (most recent call last):
          File "<string>", line 2, in <module>
          File "<pip-setuptools-caller>", line 34, in <module>
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/setup.py", line 52, in <module>
            setup(
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
            return distutils.core.setup(**attrs)
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
            return run_commands(dist)
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
            dist.run_commands()
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
            self.run_command(cmd)
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/dist.py", line 1213, in run_command
            super().run_command(command)
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
            cmd_obj.ensure_finalized()
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
            self.finalize_options()
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/develop.py", line 52, in finalize_options
            easy_install.finalize_options(self)
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/easy_install.py", line 292, in finalize_options
            self.set_undefined_options(
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 293, in set_undefined_options
            src_cmd_obj.ensure_finalized()
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
            self.finalize_options()
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
            self.set_undefined_options('install',('install_layout','install_layout'))
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
            setattr(self, dst_option, getattr(src_cmd_obj, src_option))
          File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
            raise AttributeError(attr)
        AttributeError: install_layout
        [end of output]
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× python setup.py develop did not run successfully.
│ exit code: 1
╰─> [39 lines of output]
    running develop
    /google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/easy_install.py:146: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    /google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/setup.py", line 52, in <module>
        setup(
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/__init__.py", line 108, in setup
        return distutils.core.setup(**attrs)
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
        return run_commands(dist)
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
        dist.run_commands()
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
        self.run_command(cmd)
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/dist.py", line 1213, in run_command
        super().run_command(command)
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
        cmd_obj.ensure_finalized()
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
        self.finalize_options()
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/develop.py", line 52, in finalize_options
        easy_install.finalize_options(self)
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/easy_install.py", line 292, in finalize_options
        self.set_undefined_options(
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 293, in set_undefined_options
        src_cmd_obj.ensure_finalized()
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 111, in ensure_finalized
        self.finalize_options()
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/command/install_lib.py", line 17, in finalize_options
        self.set_undefined_options('install',('install_layout','install_layout'))
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 296, in set_undefined_options
        setattr(self, dst_option, getattr(src_cmd_obj, src_option))
      File "/google/src/cloud/wjchen/osconfig-agent-cider/google3/experimental/users/wjchen/google-auth-library-python/.nox/lint/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
        raise AttributeError(attr)
    AttributeError: install_layout
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
nox > Session lint failed.

clundin25 · 2023-07-27T16:45:07Z

Ah interesting. I do not have this issue.

Can you try working from your cloudtop's file system, instead of Piper?

wj-chen · 2023-07-28T04:44:53Z

Ah interesting. I do not have this issue.

Can you try working from your cloudtop's file system, instead of Piper?

Thank you! I managed to resolve it. Turned out to be a faulty nox installation and it worked after python3 -m pip install --user nox.

Linter failure has been fixed. default_explicit_authorized_user-3.7 and default_explicit_authorized_user_explicit_project-3.7 tests failed again though...

wj-chen · 2023-07-31T18:11:25Z

Thanks for the credential refresh! The checks have passed.

clundin25 · 2023-07-31T18:54:33Z

google/auth/_cloud_sdk.py


-def get_config_path():
-    """Returns the absolute path the the Cloud SDK's configuration directory.
+def get_config_path(config_directory=_CONFIG_DIRECTORY_GCLOUD):


A cursory look suggests this is only used in gcloud's code base.

Can you own making sure that there is no regression when this library is updated in Piper?

Yes of course. I was actually making a change in the gcloud code base first and that's when I realized the path definitions happen here. How often do you merge into google3 and when will this change be expected to land?

The merge is manual. I can provide instructions for how you can do it

Sounds good. Could you help me reach @sai-sunder-s for a review so I can proceed?

Hi @clundin25, I've received an approval from @sai-sunder-s and am ready for merge instructions now.

wj-chen · 2023-08-03T22:07:24Z

Hi @sai-sunder-s , let me know if you have any questions about this change

)" This reverts commit 9f52f66.

This reverts commit 9f52f66.

feat: add get_bq_config_path() to _cloud_sdk.py

6ec1bd8

wj-chen requested a review from a team July 24, 2023 21:19

wj-chen requested a review from a team as a code owner July 24, 2023 21:19

clundin25 reviewed Jul 24, 2023

View reviewed changes

wj-chen and others added 2 commits July 25, 2023 17:16

add get_gcloud_config_path()

b5266f6

chore: Refresh system test creds.

0e5ff2e

clundin25 added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 25, 2023

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Jul 25, 2023

add default parameter value to get_config_path()

946c63a

fix lint errors

5bbc9d2

wj-chen force-pushed the main branch from a7b6f3b to 5bbc9d2 Compare July 28, 2023 04:25

chore: Refresh system test creds.

8e2b920

clundin25 added the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 31, 2023

gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Jul 31, 2023

clundin25 reviewed Jul 31, 2023

View reviewed changes

clundin25 approved these changes Jul 31, 2023

View reviewed changes

clundin25 requested a review from sai-sunder-s July 31, 2023 18:54

sai-sunder-s approved these changes Aug 8, 2023

View reviewed changes

clundin25 merged commit 9f52f66 into googleapis:main Aug 8, 2023

release-please bot mentioned this pull request Aug 8, 2023

chore(main): release 2.23.0 #1350

Merged

clundin25 added a commit to clundin25/google-auth-library-python that referenced this pull request Aug 9, 2023

Revert "feat: add get_bq_config_path() to _cloud_sdk.py (googleapis#1358

44ef04a

)" This reverts commit 9f52f66.

clundin25 added a commit that referenced this pull request Aug 9, 2023

Revert "feat: add get_bq_config_path() to _cloud_sdk.py (#1358)" (#1367)

a4ec88c

This reverts commit 9f52f66.

wj-chen mentioned this pull request Aug 17, 2023

feat: add a new config path to _cloud_sdk.py #1372

Closed

Conversation

wj-chen commented Jul 24, 2023

Uh oh!

wj-chen commented Jul 24, 2023

Uh oh!

clundin25 Jul 24, 2023

Choose a reason for hiding this comment

Uh oh!

wj-chen Jul 25, 2023

Choose a reason for hiding this comment

Uh oh!

clundin25 commented Jul 25, 2023

Uh oh!

wj-chen commented Jul 25, 2023

Uh oh!

clundin25 commented Jul 25, 2023

Uh oh!

wj-chen commented Jul 26, 2023

Uh oh!

clundin25 commented Jul 26, 2023

Uh oh!

wj-chen commented Jul 26, 2023

Uh oh!

clundin25 commented Jul 27, 2023

Uh oh!

wj-chen commented Jul 28, 2023

Uh oh!

wj-chen commented Jul 31, 2023

Uh oh!

clundin25 Jul 31, 2023

Choose a reason for hiding this comment

Uh oh!

wj-chen Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

clundin25 Aug 1, 2023

Choose a reason for hiding this comment

Uh oh!

wj-chen Aug 7, 2023

Choose a reason for hiding this comment

Uh oh!

wj-chen Aug 8, 2023

Choose a reason for hiding this comment

Uh oh!

wj-chen commented Aug 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants