[BEAM-13430] Remove propdeps and replace with compileOnly #16308

dpcollins-google · 2021-12-21T15:17:28Z

Still generate provided in the output POM.
Remove some unnecessary "provided" dependencies from examples.
add a dependency on runners:core-java from runners:java-fn-execution which was previously implicit
Remove usage of JacksonFactory which is deprecated, replace with GsonFactory
Fix dependencies of sdks:java:extensions:sql and other packages with issues revealed by this change

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
Update CHANGES.md with noteworthy changes.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

`ValidatesRunner` compliance status (on master branch)

Lang	ULR	Twister2
Go	---	---
Java
Python	---	---
XLang		---

Examples testing status on various runners

Lang	ULR	Dataflow	Flink	Samza	Spark	Twister2
Go	---	---	---	---	---	---	---
Java	---		---	---	---	---	---
Python	---	---	---	---	---	---	---
XLang	---	---	---	---	---	---	---

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Go	Java	Python

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website	Whitespace	Typescript
Non-portable
Portable	---			---	---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

codecov · 2021-12-21T16:02:12Z

Codecov Report

Merging #16308 (7afea6c) into master (c4dd8f4) will increase coverage by 37.52%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master   #16308       +/-   ##
===========================================
+ Coverage   46.15%   83.68%   +37.52%     
===========================================
  Files         197      447      +250     
  Lines       19522    61715    +42193     
===========================================
+ Hits         9011    51645    +42634     
- Misses       9540    10070      +530     
+ Partials      971        0      -971

Impacted Files	Coverage Δ
sdks/go/pkg/beam/util/shimx/generate.go
sdks/go/pkg/beam/core/graph/coder/registry.go
sdks/go/pkg/beam/core/runtime/harness/statemgr.go
sdks/go/pkg/beam/core/metrics/store.go
sdks/go/pkg/beam/transforms/stats/count.go
sdks/go/pkg/beam/io/textio/textio.go
sdks/go/pkg/beam/io/filesystem/memfs/memory.go
sdks/go/pkg/beam/internal/errors/errors.go
sdks/go/pkg/beam/core/funcx/fn.go
sdks/go/pkg/beam/core/runtime/exec/status.go
... and 634 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4dd8f4...7afea6c. Read the comment docs.

sdks/java/extensions/sql/datacatalog/build.gradle

TheNeuralBit · 2021-12-21T18:57:35Z

retest this please

TheNeuralBit · 2021-12-21T19:18:55Z

retest this please

Still generate <scope>provided</scope> in the output POM

dpcollins-google · 2021-12-21T19:21:19Z

retest this please

TheNeuralBit · 2021-12-21T19:27:28Z

retest this please

Still generate <scope>provided</scope> in the output POM

TheNeuralBit · 2021-12-21T20:38:21Z

Change to SQL test for BEAM-13506 LGTM

TheNeuralBit

LGTM

Noting here that @dpcollins-google locally verified that generated poms still include provided scope where necessary.

TheNeuralBit · 2021-12-21T22:02:08Z

Python precommit failure looks like a flake

lukecwik · 2021-12-29T22:40:35Z

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?

Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

aaltay · 2022-01-05T04:44:07Z

What was the reasoning behind going with compileOnly instead of creating a new provided configuration?

Note there were a couple of compileOnly dependencies before this change that now will be marked as provided such as all the compileOnlyAnnotations within BeamModulePlugin.groovy.

@lukecwik - should this be a jira tracked as part of https://issues.apache.org/jira/browse/BEAM-13430 ?

dpcollins-google force-pushed the gradle7 branch from cfa00ca to 721b671 Compare December 21, 2021 16:26

aaltay reviewed Dec 21, 2021

View reviewed changes

sdks/java/extensions/sql/datacatalog/build.gradle Outdated Show resolved Hide resolved

dpcollins-google force-pushed the gradle7 branch from b7d30c1 to 248929d Compare December 21, 2021 17:47

aaltay approved these changes Dec 21, 2021

View reviewed changes

dpcollins-google added 8 commits December 21, 2021 14:20

[BEAM-13430] Remove propdeps and replace with compileOnly

f9741af

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

6d4f4cf

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

81c56ac

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

011e489

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

4de55c3

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

e5c46d1

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

8e10e0c

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

9620cde

Still generate <scope>provided</scope> in the output POM

dpcollins-google force-pushed the gradle7 branch from 8c8d247 to 9620cde Compare December 21, 2021 19:20

dpcollins-google added 2 commits December 21, 2021 15:29

[BEAM-13430] Remove propdeps and replace with compileOnly

49044d2

Still generate <scope>provided</scope> in the output POM

[BEAM-13430] Remove propdeps and replace with compileOnly

7afea6c

Still generate <scope>provided</scope> in the output POM

TheNeuralBit approved these changes Dec 21, 2021

View reviewed changes

TheNeuralBit merged commit 2e72638 into apache:master Dec 21, 2021

emilymye mentioned this pull request Jan 18, 2022

[BEAM-13430] Re-add provided configuration #16552

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[BEAM-13430] Remove propdeps and replace with compileOnly #16308

[BEAM-13430] Remove propdeps and replace with compileOnly #16308

Uh oh!

dpcollins-google commented Dec 21, 2021 •

edited

Loading

Uh oh!

codecov bot commented Dec 21, 2021 •

edited

Loading

Uh oh!

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

dpcollins-google commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit left a comment

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

lukecwik commented Dec 29, 2021

Uh oh!

aaltay commented Jan 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[BEAM-13430] Remove propdeps and replace with compileOnly #16308

[BEAM-13430] Remove propdeps and replace with compileOnly #16308

Uh oh!

Conversation

dpcollins-google commented Dec 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ValidatesRunner compliance status (on master branch)

Examples testing status on various runners

Post-Commit SDK/Transform Integration Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

GitHub Actions Tests Status (on master branch)

Uh oh!

codecov bot commented Dec 21, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

dpcollins-google commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

TheNeuralBit left a comment

Choose a reason for hiding this comment

Uh oh!

TheNeuralBit commented Dec 21, 2021

Uh oh!

lukecwik commented Dec 29, 2021

Uh oh!

aaltay commented Jan 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dpcollins-google commented Dec 21, 2021 •

edited

Loading

`ValidatesRunner` compliance status (on master branch)

codecov bot commented Dec 21, 2021 •

edited

Loading