Skip to content

standlone模式 oracle到hive1,有时候直接成功 ,有时候失败,经常找不到hive sink #1166

@LeonYoah

Description

@LeonYoah

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

jdk: java version "1.8.0_212
chunjun包是我git clone gitee地址的源码 然后编译 ,然后解压target目录下的 chunjun-assembly-1.12-SNAPSHOT-chunjun-dist.tar.gz 然后解压后吧名字改成 chujun-dist , 他里面有这些 bin ,lib等等 ,然后配置环境变量 为 /opt/module/chunjun-dist
,开始执行local模式,然后报错找不到lib, 发现是他自动指向的lib路径不对,

/usr/java/jdk1.8.0_212/bin/java -cp /../lib/* com.dtstack.chunjun.client.Launcher -job /opt/module/chunjun2/job/oracle_to_hive_test2_insert.json -mode standalone -jobType sync -chunjunDistDir -flinkConfDir /opt/module/flink-1.12.7/conf -flinkLibDir /opt/module/flink-1.12.7/lib
错误: 找不到或无法加载主类 ....lib.binfmt.d

发现他自动指向的lib是上一次目录,然后我就在chunjun-dist目录之上建立一个chunjun目录,该目录下我放了bin,lib和chunuun-dist
修改环境变量为/opt/moudle/chunjun/chujun-dist ,然后在chunjun目录下执行
sh ./bin/chunjun-standalone.sh -job /opt/module/chunjun/job/oracle_to_hive_test2_insert.json

成功了 然后就出现这个问题了

Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader

不仅如此,我试了编译后的chunjun目录,也出现同样的问题

What you expected to happen

2022-08-19 10:28:17.587 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Received JobGraph submission 2e7d1eb4efd2636364f74213e24baf26 (Flink_Job).
2022-08-19 10:28:17.587 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.dispatcher.StandaloneDispatcher - Submitting job 2e7d1eb4efd2636364f74213e24baf26 (Flink_Job).
2022-08-19 10:28:17.591 [cluster-io-thread-4] INFO org.apache.flink.runtime.rpc.akka.AkkaRpcService - Starting RPC endpoint for org.apache.flink.runtime.jobmaster.JobMaster at akka://flink/user/rpc/jobmanager_5 .
2022-08-19 10:28:17.592 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - Initializing job Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.593 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - Using restart back off time strategy NoRestartBackoffTimeStrategy for Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.593 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - start to buildGraph for job Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.600 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - trying to download shipFile from blobServer for job Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.601 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - Running initialization on master for job Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.679 [cluster-io-thread-4] INFO com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat - timeZone = sun.util.calendar.ZoneInfo[id="Asia/Shanghai",offset=28800000,dstSavings=0,useDaylight=false,transitions=29,lastRule=null]
2022-08-19 10:28:17.714 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - Successfully ran initialization on master in 113 ms.
2022-08-19 10:28:17.784 [cluster-io-thread-4] INFO c.dtstack.chunjun.connector.oracle.source.OracleInputFormat - createInputSplitsInternal success, splits is [
{
"mod": 0,
"isPolling": false,
"splitStrategy": "mod",
"rangeEndLocationOperator": " < ",
"partitionNumber": 0,
"totalNumberOfPartitions": 1
}
]
2022-08-19 10:28:17.785 [cluster-io-thread-4] INFO o.a.flink.runtime.scheduler.adapter.DefaultExecutionTopology - Built 1 pipelined regions in 0 ms
2022-08-19 10:28:17.786 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - No state backend has been configured, using default (Memory / JobManager) MemoryStateBackend (data in heap memory / checkpoints to JobManager) (checkpoints: 'null', savepoints: 'null', asynchronous: TRUE, maxStateSize: 5242880)
2022-08-19 10:28:17.787 [cluster-io-thread-4] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - No checkpoint found during restore.
2022-08-19 10:28:17.787 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobMaster - Using failover strategy org.apache.flink.runtime.executiongraph.failover.flip1.RestartPipelinedRegionFailoverStrategy@26ff5a2e for Flink_Job (2e7d1eb4efd2636364f74213e24baf26).
2022-08-19 10:28:17.787 [cluster-io-thread-4] INFO org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl - JobManager runner for job Flink_Job (2e7d1eb4efd2636364f74213e24baf26) was granted leadership with session id 00000000-0000-0000-0000-000000000000 at akka.tcp://flink@192.168.1.166:6123/user/rpc/jobmanager_5.
2022-08-19 10:28:17.788 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.JobMaster - Starting execution of job Flink_Job (2e7d1eb4efd2636364f74213e24baf26) under job master id 00000000000000000000000000000000.
2022-08-19 10:28:17.789 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.JobMaster - Starting scheduling with scheduling strategy [org.apache.flink.runtime.scheduler.strategy.PipelinedRegionSchedulingStrategy]
2022-08-19 10:28:17.789 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Flink_Job (2e7d1eb4efd2636364f74213e24baf26) switched from state CREATED to RUNNING.
2022-08-19 10:28:17.790 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: oraclesourcefactory (1/1) (358a30f63a48e9a4816a6a1b5681b968) switched from CREATED to SCHEDULED.
2022-08-19 10:28:17.790 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Sink: hivesinkfactory (1/1) (97c72f1302943a417c84bb8188272a16) switched from CREATED to SCHEDULED.
2022-08-19 10:28:17.791 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Cannot serve slot request, no ResourceManager connected. Adding as pending request [SlotRequestId{d54de734b6d4a10ac346a92602f89bc9}]
2022-08-19 10:28:17.792 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.JobMaster - Connecting to ResourceManager akka.tcp://flink@192.168.1.166:6123/user/rpc/resourcemanager_*(00000000000000000000000000000000)
2022-08-19 10:28:17.793 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.jobmaster.JobMaster - Resolved ResourceManager address, beginning registration
2022-08-19 10:28:17.794 [flink-akka.actor.default-dispatcher-1009] INFO o.a.flink.runtime.resourcemanager.StandaloneResourceManager - Registering job manager 00000000000000000000000000000000@akka.tcp://flink@192.168.1.166:6123/user/rpc/jobmanager_5 for job 2e7d1eb4efd2636364f74213e24baf26.
2022-08-19 10:28:17.796 [flink-akka.actor.default-dispatcher-1007] INFO o.a.flink.runtime.resourcemanager.StandaloneResourceManager - Registered job manager 00000000000000000000000000000000@akka.tcp://flink@192.168.1.166:6123/user/rpc/jobmanager_5 for job 2e7d1eb4efd2636364f74213e24baf26.
2022-08-19 10:28:17.797 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.JobMaster - JobManager successfully registered at ResourceManager, leader id: 00000000000000000000000000000000.
2022-08-19 10:28:17.797 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.jobmaster.slotpool.SlotPoolImpl - Requesting new slot [SlotRequestId{d54de734b6d4a10ac346a92602f89bc9}] and profile ResourceProfile{UNKNOWN} with allocation id 9eb8fa0ebb0c69b7b196ae2f74175c8e from resource manager.
2022-08-19 10:28:17.797 [flink-akka.actor.default-dispatcher-1007] INFO o.a.flink.runtime.resourcemanager.StandaloneResourceManager - Request slot with profile ResourceProfile{UNKNOWN} for job 2e7d1eb4efd2636364f74213e24baf26 with allocation id 9eb8fa0ebb0c69b7b196ae2f74175c8e.
2022-08-19 10:28:17.846 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: oraclesourcefactory (1/1) (358a30f63a48e9a4816a6a1b5681b968) switched from SCHEDULED to DEPLOYING.
2022-08-19 10:28:17.846 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Source: oraclesourcefactory (1/1) (attempt #0) with attempt id 358a30f63a48e9a4816a6a1b5681b968 to 192.168.1.167:46197-be6535 @ 192.168.1.167 (dataPort=44947) with allocation id 9eb8fa0ebb0c69b7b196ae2f74175c8e
2022-08-19 10:28:17.876 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Sink: hivesinkfactory (1/1) (97c72f1302943a417c84bb8188272a16) switched from SCHEDULED to DEPLOYING.
2022-08-19 10:28:17.876 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Deploying Sink: hivesinkfactory (1/1) (attempt #0) with attempt id 97c72f1302943a417c84bb8188272a16 to 192.168.1.167:46197-be6535 @ 192.168.1.167 (dataPort=44947) with allocation id 9eb8fa0ebb0c69b7b196ae2f74175c8e
2022-08-19 10:28:18.111 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Sink: hivesinkfactory (1/1) (97c72f1302943a417c84bb8188272a16) switched from DEPLOYING to RUNNING.
2022-08-19 10:28:18.121 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: oraclesourcefactory (1/1) (358a30f63a48e9a4816a6a1b5681b968) switched from DEPLOYING to RUNNING.
2022-08-19 10:28:18.293 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Sink: hivesinkfactory (1/1) (97c72f1302943a417c84bb8188272a16) switched from RUNNING to FAILED on 192.168.1.167:46197-be6535 @ 192.168.1.167 (dataPort=44947).
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150)
at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317)
... 6 common frames omitted
2022-08-19 10:28:18.295 [flink-akka.actor.default-dispatcher-1009] INFO o.a.f.r.e.f.flip1.RestartPipelinedRegionFailoverStrategy - Calculating tasks to restart to recover the failed task 0a448493b4782967b150582570326227_0.
2022-08-19 10:28:18.296 [flink-akka.actor.default-dispatcher-1009] INFO o.a.f.r.e.f.flip1.RestartPipelinedRegionFailoverStrategy - 2 tasks should be restarted to recover the failed task 0a448493b4782967b150582570326227_0.
2022-08-19 10:28:18.296 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Flink_Job (2e7d1eb4efd2636364f74213e24baf26) switched from state RUNNING to FAILING.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233)
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224)
at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215)
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666)
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89)
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:517)
at akka.actor.Actor.aroundReceive$(Actor.scala:515)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150)
at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317)
... 6 common frames omitted
2022-08-19 10:28:18.300 [flink-akka.actor.default-dispatcher-1009] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: oraclesourcefactory (1/1) (358a30f63a48e9a4816a6a1b5681b968) switched from RUNNING to CANCELING.
2022-08-19 10:28:18.415 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Source: oraclesourcefactory (1/1) (358a30f63a48e9a4816a6a1b5681b968) switched from CANCELING to CANCELED.
2022-08-19 10:28:18.417 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.executiongraph.ExecutionGraph - Job Flink_Job (2e7d1eb4efd2636364f74213e24baf26) switched from state FAILING to FAILED.
org.apache.flink.runtime.JobException: Recovery is suppressed by NoRestartBackoffTimeStrategy
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118)
at org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:80)
at org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:233)
at org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:224)
at org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:215)
at org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:666)
at org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89)
at org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:446)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
at org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:517)
at akka.actor.Actor.aroundReceive$(Actor.scala:515)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
at akka.actor.ActorCell.invoke(ActorCell.scala:561)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
at akka.dispatch.Mailbox.run(Mailbox.scala:225)
at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load user class: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
ClassLoader info: URL ClassLoader:
Class not resolvable through given classloader.
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:331)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.(OperatorChain.java:150)
at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:519)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: com.dtstack.chunjun.connector.hive.sink.HiveOutputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClassWithoutExceptionHandling(FlinkUserCodeClassLoader.java:64)
at org.apache.flink.util.ChildFirstClassLoader.loadClassWithoutExceptionHandling(ChildFirstClassLoader.java:74)
at org.apache.flink.util.FlinkUserCodeClassLoader.loadClass(FlinkUserCodeClassLoader.java:48)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$SafetyNetWrapperClassLoader.loadClass(FlinkUserCodeClassLoaders.java:172)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:76)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:615)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:600)
at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:587)
at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:541)
at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:317)
... 6 common frames omitted
2022-08-19 10:28:18.421 [flink-akka.actor.default-dispatcher-1012] INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Stopping checkpoint coordinator for job 2e7d1eb4efd2636364f74213e24baf26.
2022-08-19 10:28:18.422 [flink-akka.actor.default-dispatcher-1012] INFO o.a.f.runtime.checkpoint.StandaloneCompletedCheckpointStore - Shutting down

How to reproduce

oracle 到hive1.x

使用chunjun-assembly-1.12-SNAPSHOT-chunjun-dist.tar.gz这个包|

或者使用编译后的chunjun目录执行 standlone模式 ,oracle到hive1

Anything else

No response

Version

master

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions