[SPARK-47645][BUILD][CORE][SQL][YARN] Make Spark build with `-release` instead of `-target` by LuciferYang · Pull Request #45716 · apache/spark

LuciferYang · 2024-03-26T06:02:18Z

What changes were proposed in this pull request?

This pr makes the following changes to allow Spark to build with -release instead of -target:

Use MethodHandle instead of direct calls to sun.security.action.GetBooleanAction and sun.util.calendar.ZoneInfo, because they are not exports APIs.
Channels.newReader is used instead of ``,StreamDecoder.forDecoder because StreamDecoder.forDecoder is also not `exports` APIs.

  public static Reader newReader(ReadableByteChannel ch,
                                   CharsetDecoder dec,
                                   int minBufferCap)
    {
        Objects.requireNonNull(ch, "ch");
        return StreamDecoder.forDecoder(ch, dec.reset(), minBufferCap);
    }

Adjusted the import of java.io._ in yarn/Client.scala to fix the compilation error:

Error: ] /home/runner/work/spark/spark/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:20: object FileSystem is not a member of package java.io

Replaced -target with -release in pom.xml and SparkBuild.scala, and removed the -source option, because using -release is sufficient.
Upgrade scala-maven-plugin from 4.7.1 to 4.8.1 to fix the error [ERROR] -release cannot be less than -target when executing build/mvn clean install -DskipTests -Djava.version=21

Why are the changes needed?

After Scala 2.13.9, the compile option -target has been deprecated, it is recommended to use -release:

Make -release more useful, deprecate -target, align with Scala 3 scala/scala#9982

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

LuciferYang · 2024-03-26T06:07:53Z

.github/workflows/build_and_test.yml

+        # export JAVA_VERSION=${{ matrix.java }}
        # It uses Maven's 'install' intentionally, see https://github.com/apache/spark/pull/26414.
-        ./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pkubernetes -Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud -Djava.version=${JAVA_VERSION/-ea} install
+        ./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pkubernetes -Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud install


Mainly used to verify the cross-compilation scenario of Maven, using Java 21 to compile with -release 17

pom.xml

LuciferYang · 2024-03-28T09:03:03Z

sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala

    val timeZoneOffset = TimeZone.getDefault match {
-      case zoneInfo: ZoneInfo => zoneInfo.getOffsetsByWall(localMillis, null)
-      case timeZone: TimeZone => timeZone.getOffset(localMillis - timeZone.getRawOffset)
+      case zoneInfo: TimeZone if zoneInfo.getClass.getName == zoneInfoClassName =>


for fix:

Error: ] /home/runner/work/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:27: object util is not a member of package sun Error: ] /home/runner/work/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:218: not found: type ZoneInfo Error: ] /home/runner/work/spark/spark/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala:218: value getOffsetsByWall is not a member of java.util.TimeZone

Maybe we can just use val timeZoneOffset = TimeZone.getDefault.getOffset(localMillis) , but I'm not sure which case can verify the compatibility issue with 2.4

Let me investigate whether there are other open APIs that can be used as a substitute here.

LuciferYang · 2024-03-28T10:26:27Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala

    val decoder = Charset.forName(enc).newDecoder()

-    StreamDecoder.forDecoder(byteChannel, decoder, decodingBufferSize)
+    Channels.newReader(byteChannel, decoder, decodingBufferSize)


for fix:

Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala:27: object nio is not a member of package sun Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala:61: not found: type StreamDecoder Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala:67: not found: value StreamDecoder Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/CreateJacksonParser.scala:27: Unused import

LuciferYang · 2024-03-28T10:26:41Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/CreateXmlParser.scala

    val decoder = Charset.forName(enc).newDecoder()

-    StreamDecoder.forDecoder(byteChannel, decoder, decodingBufferSize)
+    Channels.newReader(byteChannel, decoder, decodingBufferSize)


for fix:

Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/CreateXmlParser.scala:27: object nio is not a member of package sun Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/CreateXmlParser.scala:78: not found: type StreamDecoder Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/CreateXmlParser.scala:84: not found: value StreamDecoder Error: ] /home/runner/work/spark/spark/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/CreateXmlParser.scala:27: Unused import

LuciferYang · 2024-03-28T12:19:33Z

resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala

 package org.apache.spark.deploy.yarn

-import java.io.{FileSystem => _, _}
+import java.io.{File, FileFilter, FileNotFoundException, FileOutputStream, InterruptedIOException, IOException, OutputStreamWriter}


fix

Error: ] /home/runner/work/spark/spark/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala:20: object FileSystem is not a member of package java.io

LuciferYang · 2024-03-28T14:27:26Z

core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala

+    val mh = lookup.unreflectConstructor(constructor)
+    val action = mh.invoke("sun.io.serialization.extendedDebugInfo")
+      .asInstanceOf[PrivilegedAction[Boolean]]
+    !AccessController.doPrivileged(action).booleanValue()


Use MethodHandle to ensure logical consistency. Considering that sun.security is not an exported package, we can also directly use !JBoolean.getBoolean("sun.io.serialization.extendedDebugInfo").

LuciferYang · 2024-03-29T06:27:49Z

pom.xml

    <scalatest-maven-plugin.version>2.2.0</scalatest-maven-plugin.version>
-    <!-- don't upgrade scala-maven-plugin to version 4.7.2 or higher, see SPARK-45144 for details -->
-    <scala-maven-plugin.version>4.7.1</scala-maven-plugin.version>
+    <scala-maven-plugin.version>4.8.1</scala-maven-plugin.version>


To fix Error: -release cannot be less than -target, it seems necessary to upgrade to use 4.8.1

LuciferYang · 2024-03-29T06:57:04Z

If it's not a good time to do this work now, please let me know and I will close this PR.
If the PR needs to be split, such as submitting the code change part first, then changing the compile parameters and upgrading the plugin, please let me know too.

dongjoon-hyun · 2024-03-29T15:56:41Z

Could you make the CI happy, @LuciferYang ?

LuciferYang · 2024-03-29T16:12:04Z

Could you make the CI happy, @LuciferYang ?

re-triggered the failed task, let's see if it can pass

dongjoon-hyun

+1, LGTM.

cc @srowen , @JoshRosen , @HyukjinKwon , too.

dongjoon-hyun · 2024-04-01T04:28:08Z

Merged to master. Thank you, @LuciferYang and all.

LuciferYang · 2024-04-01T04:47:28Z

Thanks @dongjoon-hyun ~

mihailomilosevic2001 · 2024-04-01T10:43:08Z

This PR seems to break local environment for maven in IntelliJ. Couple of us are getting this response when trying to run anything using maven, even after rebuilding.

java.lang.ExceptionInInitializerError
	at org.apache.spark.sql.catalyst.optimizer.ComputeCurrentTime$.apply(finishAnalysis.scala:111)
	at org.apache.spark.sql.catalyst.optimizer.ComputeCurrentTime$.apply(finishAnalysis.scala:108)
	at org.apache.spark.sql.catalyst.optimizer.Optimizer$FinishAnalysis$.$anonfun$apply$1(Optimizer.scala:306)
	at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183)
	at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179)
	at scala.collection.immutable.List.foldLeft(List.scala:79)
	at org.apache.spark.sql.catalyst.optimizer.Optimizer$FinishAnalysis$.apply(Optimizer.scala:306)
	at org.apache.spark.sql.catalyst.optimizer.Optimizer$FinishAnalysis$.apply(Optimizer.scala:286)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$2(RuleExecutor.scala:222)
	at scala.collection.immutable.ArraySeq.foldLeft(ArraySeq.scala:222)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1(RuleExecutor.scala:219)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$execute$1$adapted(RuleExecutor.scala:211)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.execute(RuleExecutor.scala:211)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.$anonfun$executeAndTrack$1(RuleExecutor.scala:182)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker$.withTracker(QueryPlanningTracker.scala:89)
	at org.apache.spark.sql.catalyst.rules.RuleExecutor.executeAndTrack(RuleExecutor.scala:182)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$optimizedPlan$1(QueryExecution.scala:166)
	at org.apache.spark.sql.catalyst.QueryPlanningTracker.measurePhase(QueryPlanningTracker.scala:138)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$2(QueryExecution.scala:233)
	at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:571)
	at org.apache.spark.sql.execution.QueryExecution.$anonfun$executePhase$1(QueryExecution.scala:233)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:918)
	at org.apache.spark.sql.execution.QueryExecution.executePhase(QueryExecution.scala:232)
	at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:162)
	at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:158)
	at org.apache.spark.sql.execution.QueryExecution.assertOptimized(QueryExecution.scala:176)
	at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:196)
	at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:193)
	at org.apache.spark.sql.execution.QueryExecution.simpleString(QueryExecution.scala:252)
	at org.apache.spark.sql.execution.QueryExecution.org$apache$spark$sql$execution$QueryExecution$$explainString(QueryExecution.scala:298)
	at org.apache.spark.sql.execution.QueryExecution.explainString(QueryExecution.scala:266)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$6(SQLExecution.scala:138)
	at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:241)
	at org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId0$1(SQLExecution.scala:116)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:918)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId0(SQLExecution.scala:72)
	at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:196)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.$anonfun$applyOrElse$1(QueryExecution.scala:120)
	at org.apache.spark.sql.execution.QueryExecution$.withInternalError(QueryExecution.scala:571)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:119)
	at org.apache.spark.sql.execution.QueryExecution$$anonfun$eagerlyExecuteCommands$1.applyOrElse(QueryExecution.scala:109)
	at org.apache.spark.sql.catalyst.trees.TreeNode.$anonfun$transformDownWithPruning$1(TreeNode.scala:442)
	at org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(origin.scala:83)
	at org.apache.spark.sql.catalyst.trees.TreeNode.transformDownWithPruning(TreeNode.scala:442)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.org$apache$spark$sql$catalyst$plans$logical$AnalysisHelper$$super$transformDownWithPruning(LogicalPlan.scala:34)
	at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning(AnalysisHelper.scala:330)
	at org.apache.spark.sql.catalyst.plans.logical.AnalysisHelper.transformDownWithPruning$(AnalysisHelper.scala:326)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:34)
	at org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.transformDownWithPruning(LogicalPlan.scala:34)
	at org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:418)
	at org.apache.spark.sql.execution.QueryExecution.eagerlyExecuteCommands(QueryExecution.scala:109)
	at org.apache.spark.sql.execution.QueryExecution.commandExecuted$lzycompute(QueryExecution.scala:96)
	at org.apache.spark.sql.execution.QueryExecution.commandExecuted(QueryExecution.scala:94)
	at org.apache.spark.sql.Dataset.<init>(Dataset.scala:224)
	at org.apache.spark.sql.Dataset$.$anonfun$ofRows$1(Dataset.scala:95)
	at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:918)
	at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:92)
	at org.apache.spark.sql.Dataset.withPlan(Dataset.scala:4453)
	at org.apache.spark.sql.Dataset.createOrReplaceTempView(Dataset.scala:3978)
	at org.apache.spark.sql.connector.DatasourceV2SQLBase.$anonfun$$init$$1(DatasourceV2SQLBase.scala:47)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
	at org.scalatest.BeforeAndAfter.runTest(BeforeAndAfter.scala:210)
	at org.scalatest.BeforeAndAfter.runTest$(BeforeAndAfter.scala:203)
	at org.apache.spark.sql.CollationSuite.runTest(CollationSuite.scala:37)
	at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:269)
	at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
	at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396)
	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:269)
	at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:268)
	at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1564)
	at org.scalatest.Suite.run(Suite.scala:1114)
	at org.scalatest.Suite.run$(Suite.scala:1096)
	at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1564)
	at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:273)
	at org.scalatest.SuperEngine.runImpl(Engine.scala:535)
	at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:273)
	at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:272)
	at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:69)
	at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213)
	at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210)
	at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208)
	at org.apache.spark.sql.CollationSuite.org$scalatest$BeforeAndAfter$$super$run(CollationSuite.scala:37)
	at org.scalatest.BeforeAndAfter.run(BeforeAndAfter.scala:273)
	at org.scalatest.BeforeAndAfter.run$(BeforeAndAfter.scala:271)
	at org.apache.spark.sql.CollationSuite.run(CollationSuite.scala:37)
	at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:47)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13(Runner.scala:1321)
	at org.scalatest.tools.Runner$.$anonfun$doRunRunRunDaDoRunRun$13$adapted(Runner.scala:1315)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1315)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24(Runner.scala:992)
	at org.scalatest.tools.Runner$.$anonfun$runOptionallyWithPassFailReporter$24$adapted(Runner.scala:970)
	at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1481)
	at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:970)
	at org.scalatest.tools.Runner$.run(Runner.scala:798)
	at org.scalatest.tools.Runner.run(Runner.scala)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2or3(ScalaTestRunner.java:43)
	at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:26)
Caused by: java.lang.IllegalAccessException: symbolic reference class is not accessible: class sun.util.calendar.ZoneInfo, from interface org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module @5cee5251)
	at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:894)
	at java.base/java.lang.invoke.MethodHandles$Lookup.checkSymbolicClass(MethodHandles.java:3787)
	at java.base/java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:3747)
	at java.base/java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:2767)
	at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.$init$(SparkDateTimeUtils.scala:206)
	at org.apache.spark.sql.catalyst.util.DateTimeUtils$.<clinit>(DateTimeUtils.scala:41)
	... 102 more```

I am looking into how to fix this for me locally, but haven't found a solution yet.

LuciferYang · 2024-04-01T10:45:38Z

@mihailom-db Could you provide a specific build command? The maven compilation in the GA test has passed.

LuciferYang · 2024-04-01T10:57:38Z

Could you try the operation above? I think this should not be a new issue.

mihailomilosevic2001 · 2024-04-01T11:02:51Z

This seems to have solved the problem. Thanks.

### What changes were proposed in this pull request? This PR aims to remove `ignore.symbol.file` Javac option. ### Why are the changes needed? Since Apache Spark 4.0.0, we are building with `-release` which means `Javac` always uses `ct.sym`, a stripped-down version of the standard library (like rt.jar in older JDKs) containing only the necessary class stubs and signature data for the documented APIs of a specific Java release. In other words, `ignore.symbol.file` option is ignored now and we can remove it safely. - #45716 ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Pass the CIs. **BEFORE** ``` $ git grep ignore.symbol.file | wc -l 4 ``` **AFTER** ``` $ git grep ignore.symbol.file | wc -l 0 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes #52802 from dongjoon-hyun/SPARK-54100. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

### What changes were proposed in this pull request? This PR aims to remove `ignore.symbol.file` Javac option. ### Why are the changes needed? Since Apache Spark 4.0.0, we are building with `-release` which means `Javac` always uses `ct.sym`, a stripped-down version of the standard library (like rt.jar in older JDKs) containing only the necessary class stubs and signature data for the documented APIs of a specific Java release. In other words, `ignore.symbol.file` option is ignored now and we can remove it safely. - apache#45716 ### Does this PR introduce _any_ user-facing change? No behavior change. ### How was this patch tested? Pass the CIs. **BEFORE** ``` $ git grep ignore.symbol.file | wc -l 4 ``` **AFTER** ``` $ git grep ignore.symbol.file | wc -l 0 ``` ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#52802 from dongjoon-hyun/SPARK-54100. Authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>

init

76487bb

LuciferYang marked this pull request as draft March 26, 2024 06:02

github-actions bot added BUILD INFRA labels Mar 26, 2024

check cross compile

50bdbb6

LuciferYang commented Mar 26, 2024

View reviewed changes

This comment was marked as outdated.

Sign in to view

test 4.8.1 first

10ded51

LuciferYang commented Mar 28, 2024

View reviewed changes

pom.xml Show resolved Hide resolved

fix

ca3e008

github-actions bot added the CORE label Mar 28, 2024

LuciferYang added 2 commits March 28, 2024 13:36

fix 2

2acd948

fix ZoneInfo

c7e4650

github-actions bot added the SQL label Mar 28, 2024

LuciferYang commented Mar 28, 2024

View reviewed changes

LuciferYang added 2 commits March 28, 2024 17:07

fix ZoneInfo

f10c43a

fix sun.nio

1abaf67

LuciferYang commented Mar 28, 2024

View reviewed changes

fix yarn

7547558

github-actions bot added the YARN label Mar 28, 2024

LuciferYang commented Mar 28, 2024

View reviewed changes

LuciferYang added 2 commits March 28, 2024 21:14

use -release

4fcfa19

revert

48ca8eb

github-actions bot removed the INFRA label Mar 28, 2024

LuciferYang changed the title ~~Test scala-maven-plugin 4.9.1~~ Test compile scala with -release 17 Mar 28, 2024

use mh

35d6349

LuciferYang commented Mar 28, 2024

View reviewed changes

LuciferYang changed the title ~~Test compile scala with -release 17~~ [WIP]Test compile scala with -release 17 Mar 28, 2024

LuciferYang changed the title ~~[WIP]Test compile scala with -release 17~~ [WIP] Make Scala code build successful with -release 17 Mar 28, 2024

fix -release cannot be less than -target

08c7cf3

LuciferYang changed the title ~~[WIP] Make Scala code build successful with -release 17~~ [WIP] Make Spark build successful with -release 17 Mar 29, 2024

LuciferYang changed the title ~~[WIP] Make Spark build successful with -release 17~~ [WIP] Make Spark build with -release instead of -target Mar 29, 2024

remove outdated comments

b669078

LuciferYang commented Mar 29, 2024

View reviewed changes

LuciferYang changed the title ~~[WIP] Make Spark build with -release instead of -target~~ [SPARK-47645][BUILD][CORE][SQL][YARN] Make Spark build with -release instead of -target Mar 29, 2024

LuciferYang marked this pull request as ready for review March 29, 2024 06:57

dongjoon-hyun approved these changes Mar 29, 2024

View reviewed changes

dongjoon-hyun closed this in 4c1405d Apr 1, 2024

LuciferYang mentioned this pull request Apr 2, 2024

[WIP][SQL] Refactor the implementation of SparkDateTimeUtils#toJavaDate #45817

Closed

dongjoon-hyun mentioned this pull request Oct 30, 2025

[SPARK-54100][BUILD] Remove ignore.symbol.file Javac option #52802

Closed

Conversation

LuciferYang commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Was this patch authored or co-authored using generative AI tooling?

Uh oh!

LuciferYang Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

LuciferYang Mar 28, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang Apr 1, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang Mar 28, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang Mar 28, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang Mar 28, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang Mar 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LuciferYang Mar 29, 2024

Choose a reason for hiding this comment

Uh oh!

LuciferYang commented Mar 29, 2024

Uh oh!

dongjoon-hyun commented Mar 29, 2024

Uh oh!

LuciferYang commented Mar 29, 2024

Uh oh!

dongjoon-hyun left a comment

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Apr 1, 2024

Uh oh!

LuciferYang commented Apr 1, 2024

Uh oh!

mihailomilosevic2001 commented Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LuciferYang commented Apr 1, 2024

Uh oh!

LuciferYang commented Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mihailomilosevic2001 commented Apr 1, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LuciferYang commented Mar 26, 2024 •

edited

Loading

LuciferYang Mar 28, 2024 •

edited

Loading

mihailomilosevic2001 commented Apr 1, 2024 •

edited

Loading

LuciferYang commented Apr 1, 2024 •

edited

Loading