Skip to content
This repository was archived by the owner on Apr 1, 2026. It is now read-only.

feat!: automatically register the BigQuery magics when bigframes is imported#1231

Closed
tswast wants to merge 3 commits intomainfrom
b384564457-magics
Closed

feat!: automatically register the BigQuery magics when bigframes is imported#1231
tswast wants to merge 3 commits intomainfrom
b384564457-magics

Conversation

@tswast
Copy link
Copy Markdown
Contributor

@tswast tswast commented Dec 18, 2024

BREAKING-CHANGE: notebooks that mix the %%bigquery / %%bqsql magics and the bigframes module will use bigframes for DataFrame output by default
Release-As: 1.30.0

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes internal issue 384564457 🦕

…mported

BREAKING-CHANGE: notebooks that mix the %%bigquery / %%bqsql magics and the bigframes module will use bigframes for DataFrame output by default
@product-auto-label product-auto-label bot added the size: xl Pull request size is extra large. label Dec 18, 2024
@review-notebook-app
Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. label Dec 18, 2024
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: xl Pull request size is extra large. labels Jan 7, 2025
Comment on lines +23 to +30
if bigquery_magics.context.credentials is not None:
# The %%bigquery magics must have been run before BigQuery DataFrames
# was imported. In this case, we don't want to break any existing
# notebooks, so don't make any BigQuery DataFrames changes to the
# magics.
return

bigquery_magics.context = Context()
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DELETE ME

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, just delete this last part. Set the engine and default variable if we make it this far (e.g. the magics haven't been used yet)

"""Called by IPython when this module is loaded as an IPython extension."""
bigquery_magics.load_ipython_extension(ipython)

if bigquery_magics.context.credentials is not None:
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pandas-gbq sets the default credentials in-memory after the first run, but it doesn't look like the magics have any such feature yet.

TODO: bigquery-magics to cache the credentials and/or client objects after the first.


def load_ipython_extension(ipython):
"""Called by IPython when this module is loaded as an IPython extension."""
bigquery_magics.load_ipython_extension(ipython)
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, check if it is registered already before calling this and break early.

https://github.com/googleapis/python-bigquery-magics/blob/cbb3e44fcdb6aec7cb04d0460a34501f0fd6d271/bigquery_magics/__init__.py#L22

from bigframes.version import __version__


def load_ipython_extension(ipython):
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding an unload function that undoes any bigframes-specific configuration of %%bigquery

See: unload_ipython_extension in https://ipython.readthedocs.io/en/stable/config/extensions/#writing-extensions

@tswast
Copy link
Copy Markdown
Contributor Author

tswast commented Jan 22, 2025

Decided against this to:

  1. Reduce number of dependencies the bigframes has
  2. Prevent accidentally breaking existing notebooks that mix the magics and bigframes code

@tswast tswast closed this Jan 22, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

api: bigquery Issues related to the googleapis/python-bigquery-dataframes API. size: m Pull request size is medium.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant