[SPARK-44544][INFRA][FOLLOWUP] Force run run_python_packaging_tests
#42173
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
Author
|
|
aadb28c to
b2311f3
Compare
Contributor
Author
b2311f3 to
47f1604
Compare
HyukjinKwon
reviewed
Jul 27, 2023
dev/run-tests.py
Outdated
Member
There was a problem hiding this comment.
Hm, to run pip packaging test, Spark has to be built first but here is before we build. I think it won't work correctly?
Contributor
Author
There was a problem hiding this comment.
oops, let me double check, thanks!
47f1604 to
8d10199
Compare
zhengruifeng
commented
Jul 27, 2023
| # in PySpark, since the Python Packaging Tests is only enabled within this module. | ||
| # This module is the smallest Python test modules, it only contains 1 test file | ||
| # and normally take < 2 seconds, so the additional cost is small. | ||
| "python/", |
Contributor
Author
There was a problem hiding this comment.
cc @HyukjinKwon I find it is pretty hard to me to decouple the packaging tests from other pyspark tests. what about this workaround?
HyukjinKwon
reviewed
Jul 27, 2023
HyukjinKwon
approved these changes
Jul 27, 2023
Co-authored-by: Hyukjin Kwon <gurwls223@gmail.com>
run_python_packaging_tests even if pyspark-errors is skippedrun_python_packaging_tests
LuciferYang
approved these changes
Jul 27, 2023
panbingkun
approved these changes
Jul 27, 2023
zhengruifeng
added a commit
that referenced
this pull request
Jul 27, 2023
### What changes were proposed in this pull request? run `run_python_packaging_tests` when there are any changes in PySpark ### Why are the changes needed? #42146 make CI run `run_python_packaging_tests` only within `pyspark-errors` (see https://github.com/apache/spark/actions/runs/5666118302/job/15359190468 and https://github.com/apache/spark/actions/runs/5668071930/job/15358091003)  but I ignored that `pyspark-errors` maybe skipped (because no related source changes), so the `run_python_packaging_tests` maybe also skipped unexpectedly (see https://github.com/apache/spark/actions/runs/5666523657/job/15353485731)  this PR is to run `run_python_packaging_tests` even if `pyspark-errors` is skipped ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? updated CI Closes #42173 from zhengruifeng/infra_followup. Lead-authored-by: Ruifeng Zheng <ruifengz@apache.org> Co-authored-by: Ruifeng Zheng <ruifengz@foxmail.com> Signed-off-by: Ruifeng Zheng <ruifengz@apache.org> (cherry picked from commit f794734) Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
zhengruifeng
added a commit
that referenced
this pull request
Jul 27, 2023
### What changes were proposed in this pull request? run `run_python_packaging_tests` when there are any changes in PySpark ### Why are the changes needed? #42146 make CI run `run_python_packaging_tests` only within `pyspark-errors` (see https://github.com/apache/spark/actions/runs/5666118302/job/15359190468 and https://github.com/apache/spark/actions/runs/5668071930/job/15358091003)  but I ignored that `pyspark-errors` maybe skipped (because no related source changes), so the `run_python_packaging_tests` maybe also skipped unexpectedly (see https://github.com/apache/spark/actions/runs/5666523657/job/15353485731)  this PR is to run `run_python_packaging_tests` even if `pyspark-errors` is skipped ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? updated CI Closes #42173 from zhengruifeng/infra_followup. Lead-authored-by: Ruifeng Zheng <ruifengz@apache.org> Co-authored-by: Ruifeng Zheng <ruifengz@foxmail.com> Signed-off-by: Ruifeng Zheng <ruifengz@apache.org> (cherry picked from commit f794734) Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
Contributor
Author
|
thank you guys! merged to master/branch-3.5/branch-3.4 |
viirya
pushed a commit
to viirya/spark-1
that referenced
this pull request
Oct 19, 2023
### What changes were proposed in this pull request? run `run_python_packaging_tests` when there are any changes in PySpark ### Why are the changes needed? apache#42146 make CI run `run_python_packaging_tests` only within `pyspark-errors` (see https://github.com/apache/spark/actions/runs/5666118302/job/15359190468 and https://github.com/apache/spark/actions/runs/5668071930/job/15358091003)  but I ignored that `pyspark-errors` maybe skipped (because no related source changes), so the `run_python_packaging_tests` maybe also skipped unexpectedly (see https://github.com/apache/spark/actions/runs/5666523657/job/15353485731)  this PR is to run `run_python_packaging_tests` even if `pyspark-errors` is skipped ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? updated CI Closes apache#42173 from zhengruifeng/infra_followup. Lead-authored-by: Ruifeng Zheng <ruifengz@apache.org> Co-authored-by: Ruifeng Zheng <ruifengz@foxmail.com> Signed-off-by: Ruifeng Zheng <ruifengz@apache.org> (cherry picked from commit f794734) Signed-off-by: Ruifeng Zheng <ruifengz@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
run
run_python_packaging_testswhen there are any changes in PySparkWhy are the changes needed?
#42146 make CI run
run_python_packaging_testsonly withinpyspark-errors(see https://github.com/apache/spark/actions/runs/5666118302/job/15359190468 and https://github.com/apache/spark/actions/runs/5668071930/job/15358091003)but I ignored that
pyspark-errorsmaybe skipped (because no related source changes), so therun_python_packaging_testsmaybe also skipped unexpectedly (see https://github.com/apache/spark/actions/runs/5666523657/job/15353485731)this PR is to run
run_python_packaging_testseven ifpyspark-errorsis skippedDoes this PR introduce any user-facing change?
no
How was this patch tested?
updated CI