Skip to content

[SPARK-44522][BUILD] Upgrade scala-xml to 2.2.0#42119

Closed
LuciferYang wants to merge 4 commits intoapache:masterfrom
LuciferYang:scala-xml-220
Closed

[SPARK-44522][BUILD] Upgrade scala-xml to 2.2.0#42119
LuciferYang wants to merge 4 commits intoapache:masterfrom
LuciferYang:scala-xml-220

Conversation

@LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Jul 24, 2023 •
edited
Loading

What changes were proposed in this pull request?

This pr aims to upgrade scala-xml from 2.1.0 to 2.2.0.

Why are the changes needed?

The new version bring some bug fix like:

The full release notes as follows:

Does this PR introduce any user-facing change?

No

How was this patch tested?

LuciferYang
LuciferYang commented Jul 24, 2023
View reviewed changes
.github/workflows/build_and_test.yml Outdated
HIVE_PROFILE: ${{ matrix.hive }}
GITHUB_PREV_SHA: ${{ github.event.before }}
SPARK_LOCAL_IP: localhost
SCALA_PROFILE: scala2.13
Copy link
Contributor Author

@LuciferYang LuciferYang Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test scala 2.13, will revert after test pass

@LuciferYang LuciferYang changed the title Test Scala xml 2.2.0 [SPARK-44522][BUILD] Upgrade scala-xml to 2.2.0 Jul 24, 2023
@github-actions github-actions bot removed the INFRA label Jul 24, 2023
dongjoon-hyun
dongjoon-hyun approved these changes Jul 24, 2023
View reviewed changes
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM (Pending CIs). Thank you, @LuciferYang .

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jul 24, 2023

For scala 2.13, all scala test passed.

https://github.com/LuciferYang/spark/runs/15278359785

pyspark-pandas-slow-connect failure is well known

Building wheels for collected packages: pyspark
  Building wheel for pyspark (setup.py): started
  Building wheel for pyspark (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [1724 lines of output]
      /tmp/tmp.Ex2gmKU9bV/3.9/lib/python3.9/site-packages/setuptools/dist.py:755: SetuptoolsDeprecationWarning: Invalid dash-separated options
      !!
      
              ********************************************************************************
              Usage of dash-separated 'description-file' will not be supported in future
              versions. Please use the underscore name 'description_file' instead.
      
Error: No space left on device : '/home/runner/runners/2.306.0/_diag/pages/9b7af01d-9255-4597-a103-2b1fbff3f418_551a9d66-8149-5c0e-5e40-9b55ab3d9173_1.log'

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jul 24, 2023 •
edited
Loading

For scala 2.13, all scala test passed.

https://github.com/LuciferYang/spark/runs/15278359785

pyspark-pandas-slow-connect failure is well known

Building wheels for collected packages: pyspark
  Building wheel for pyspark (setup.py): started
  Building wheel for pyspark (setup.py): finished with status 'error'
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [1724 lines of output]
      /tmp/tmp.Ex2gmKU9bV/3.9/lib/python3.9/site-packages/setuptools/dist.py:755: SetuptoolsDeprecationWarning: Invalid dash-separated options
      !!
      
              ********************************************************************************
              Usage of dash-separated 'description-file' will not be supported in future
              versions. Please use the underscore name 'description_file' instead.
      
Error: No space left on device : '/home/runner/runners/2.306.0/_diag/pages/9b7af01d-9255-4597-a103-2b1fbff3f418_551a9d66-8149-5c0e-5e40-9b55ab3d9173_1.log'

Seems https://github.com/apache/spark/pull/42115/files try to fix this

@srowen
Copy link
Member

srowen commented Jul 24, 2023

Looks OK just re-run tests

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jul 24, 2023 •
edited
Loading

Looks OK just re-run tests

OK, it's a known issue, need to wait #42115 to fix the issue , the failed task cannot be retried successfully currently

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jul 26, 2023

rebase

@srowen srowen closed this in 43b753a Jul 27, 2023
@srowen
Copy link
Member

srowen commented Jul 27, 2023

Merged to master

@LuciferYang
Copy link
Contributor Author

LuciferYang commented Jul 27, 2023

Thanks @srowen

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes

Assignees

No one assigned

Labels

BUILD

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@LuciferYang @srowen @dongjoon-hyun