Skip to content

Add notebook for per key models#28327

Merged
damccorm merged 7 commits intoapache:masterfrom
damccorm:users/damccorm/keyedMhNotebook
Oct 3, 2023
Merged

Add notebook for per key models#28327
damccorm merged 7 commits intoapache:masterfrom
damccorm:users/damccorm/keyedMhNotebook

Conversation

@damccorm
Copy link
Copy Markdown
Contributor

@damccorm damccorm commented Sep 6, 2023

RENDERED

This should not be merged until 2.51 is released, it was tested by replacing the beam install with a clone/install from the Beam repo though.

Resolves #27628


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

@damccorm damccorm marked this pull request as ready for review September 6, 2023 14:57
@damccorm damccorm marked this pull request as draft September 6, 2023 14:58
@damccorm damccorm marked this pull request as ready for review September 6, 2023 15:10
@damccorm
Copy link
Copy Markdown
Contributor Author

damccorm commented Sep 6, 2023

R: @riteshghorse @rszper

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Sep 6, 2023

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control

Copy link
Copy Markdown
Contributor

@riteshghorse riteshghorse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@rszper
Copy link
Copy Markdown
Contributor

rszper commented Sep 8, 2023

Should we also include this in the ReadMe list? https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/README.md

@damccorm
Copy link
Copy Markdown
Contributor Author

damccorm commented Sep 8, 2023

Should we also include this in the ReadMe list? https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/README.md

Yes, done!

"source": [
"## Postprocess our results\n",
"\n",
"The `RunInference` transform returns a Tuple of the original key and a `PredictionResult` object that contains the original example and the inference. From that, we will extract the data we care about. We will then group this data by the original example in order to compare each model's prediction."
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"The `RunInference` transform returns a Tuple of the original key and a `PredictionResult` object that contains the original example and the inference. From that, we will extract the data we care about. We will then group this data by the original example in order to compare each model's prediction."
"The `RunInference` transform returns the following items:\n",
"\n",
"* A tuple of the original key\n",
"* A `PredictionResult` object that contains the original example and the inference\n",
"\n",
"Use those outputs to extract the relevant data. Then, to compare each model's prediction, group this data by the original example."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That changes the meaning (which means it definitely wasn't clear 😅 ). Let me know if it reads better now

Copy link
Copy Markdown
Contributor Author

@damccorm damccorm Sep 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the relevant diff - 056d6d2 - the other commit is directly applying all other suggestions

damccorm and others added 3 commits September 8, 2023 14:49
Co-authored-by: Rebecca Szper <98840847+rszper@users.noreply.github.com>
@damccorm
Copy link
Copy Markdown
Contributor Author

damccorm commented Oct 3, 2023

I'm going to merge this in the current pull from master form so that it can be easily used for demos. I'll follow up to update the beam install once 2.51 is released

@damccorm damccorm merged commit 12b0714 into apache:master Oct 3, 2023
@damccorm damccorm deleted the users/damccorm/keyedMhNotebook branch October 3, 2023 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request]: Support loading a different model/key for RunInference

3 participants