[BEAM-13430] Fix provided configuration by removing extendsFrom for implementation #16688

emilymye · 2022-02-01T19:40:30Z

https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L1597-L1598 is pulling in provided dependencies and generating them as "compile" dependencies

Based on https://cwiki.apache.org/confluence/display/BEAM/Gradle+Tips, this will add provided to the same classpaths

R: @lukecwik @kennknowles @dpcollins-google

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

`ValidatesRunner` compliance status (on master branch)

Lang	ULR	Twister2
Go	---	---
Java
Python	---	---
XLang		---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy

emilymye · 2022-02-01T20:46:50Z

Run Java_Examples_Dataflow PreCommit

emilymye · 2022-02-01T20:46:59Z

Run Java_PVR_Flink_Docker PreCommit

codecov · 2022-02-01T21:06:14Z

Codecov Report

Merging #16688 (4d6a477) into master (b9c4919) will increase coverage by 8.99%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #16688      +/-   ##
==========================================
+ Coverage   74.63%   83.63%   +8.99%     
==========================================
  Files         654      452     -202     
  Lines       81956    62146   -19810     
==========================================
- Hits        61170    51975    -9195     
+ Misses      19799    10171    -9628     
+ Partials      987        0     -987

Impacted Files	Coverage Δ
sdks/python/apache_beam/utils/interactive_utils.py	`92.68% <0.00%> (-2.44%)`	⬇️
...hon/apache_beam/runners/worker/bundle_processor.py	`93.26% <0.00%> (-0.25%)`	⬇️
sdks/go/pkg/beam/util.go
sdks/go/pkg/beam/core/runtime/pipelinex/util.go
sdks/go/pkg/beam/core/runtime/exec/coder.go
sdks/go/pkg/beam/core/runtime/exec/reshuffle.go
sdks/go/pkg/beam/core/runtime/exec/fn_arity.go
sdks/go/pkg/beam/core/runtime/exec/datasink.go
sdks/go/pkg/beam/core/runtime/graphx/user.go
sdks/go/pkg/beam/core/runtime/exec/pcollection.go
... and 195 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9c4919...4d6a477. Read the comment docs.

runners/flink/flink_runner.gradle

emilymye · 2022-02-01T22:26:58Z

Run Java PreCommit

emilymye · 2022-02-01T22:27:26Z

Run Java_Examples_Dataflow_Java11 PreCommit

emilymye · 2022-02-01T22:27:41Z

Run Java_PVR_Flink_Docker PreCommit

emilymye · 2022-02-02T06:45:23Z

Run Java PostCommit

emilymye · 2022-02-02T06:48:04Z

Run Java Flink PortableValidatesRunner Batch

emilymye · 2022-02-02T06:48:16Z

Run Spark ValidatesRunner

emilymye · 2022-02-02T06:48:22Z

Run Java Spark PortableValidatesRunner Batch

emilymye · 2022-02-02T06:48:30Z

Run SQL PostCommit

emilymye · 2022-02-02T07:14:11Z

Run Java PreCommit

kennknowles

This makes sense. I always found that line suspicious, but I don't know enough about how we are (ab)using Gradle.

emilymye · 2022-02-02T18:19:41Z

Run Java PreCommit

…mplementation (apache#16688)

github-actions bot added the build label Feb 1, 2022

dpcollins-google approved these changes Feb 1, 2022

View reviewed changes

lukecwik requested changes Feb 1, 2022

View reviewed changes

buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy Outdated Show resolved Hide resolved

github-actions bot added dataflow extensions flink hadoop io java runners sorter spark sql labels Feb 1, 2022

emilymye requested a review from lukecwik February 1, 2022 20:46

lukecwik reviewed Feb 1, 2022

View reviewed changes

runners/flink/flink_runner.gradle Outdated Show resolved Hide resolved

emilymye force-pushed the fiximplementation branch from 689a829 to a26f06d Compare February 1, 2022 22:20

github-actions bot removed sorter spark labels Feb 1, 2022

emilymye requested a review from lukecwik February 1, 2022 22:20

emilymye added 3 commits February 2, 2022 06:36

revert from implementation extends to compile/runtime

65e3636

remove testCompileOnly

93df492

add back testImplementation where required

70d992a

emilymye force-pushed the fiximplementation branch from a26f06d to 70d992a Compare February 2, 2022 06:36

emilymye requested a review from kennknowles February 2, 2022 06:44

emilymye mentioned this pull request Feb 2, 2022

[BEAM-13430] Re-add provided configuration #16552

Merged

4 tasks

add back hcatalog testImplementation

4d6a477

github-actions bot added the hcatalog label Feb 2, 2022

kennknowles approved these changes Feb 2, 2022

View reviewed changes

emilymye merged commit 6e2409f into apache:master Feb 2, 2022

emilymye deleted the fiximplementation branch February 2, 2022 19:36

emilymye added a commit to emilymye/beam that referenced this pull request Feb 2, 2022

[BEAM-13430] Fix provided configuration by removing extendsFrom for i…

6b6b8d1

…mplementation (apache#16688)

emilymye mentioned this pull request Feb 2, 2022

[release-2.36.0][BEAM-13430] Fix provided configuration #16704

Merged

4 tasks

emilymye added a commit to emilymye/beam that referenced this pull request Feb 2, 2022

[BEAM-13430] Fix provided configuration by removing extendsFrom for i…

e326e31

…mplementation (apache#16688)

[BEAM-13430] Fix provided configuration by removing extendsFrom for implementation #16688

[BEAM-13430] Fix provided configuration by removing extendsFrom for implementation #16688

Uh oh!

Conversation

emilymye commented Feb 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

R: @lukecwik @kennknowles @dpcollins-google

ValidatesRunner compliance status (on master branch)

Examples testing status on various runners

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

Uh oh!

Uh oh!

emilymye commented Feb 1, 2022

Uh oh!

emilymye commented Feb 1, 2022

Uh oh!

codecov bot commented Feb 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

emilymye commented Feb 1, 2022

Uh oh!

emilymye commented Feb 1, 2022

Uh oh!

emilymye commented Feb 1, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

kennknowles left a comment

Choose a reason for hiding this comment

Uh oh!

emilymye commented Feb 2, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

emilymye commented Feb 1, 2022 •

edited

Loading

`ValidatesRunner` compliance status (on master branch)

codecov bot commented Feb 1, 2022 •

edited

Loading