Created
September 25, 2021 19:46
-
-
Save nsivabalan/f261c6d3e0e3a4178533e922272e162e to your computer and use it in GitHub Desktop.
TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflictCOW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2021-09-25T18:58:59.3537396Z 259585 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Acquiring lock succeeded for 004 | |
| 2021-09-25T18:58:59.3587949Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - MDT. applying commit 004 | |
| 2021-09-25T18:58:59.3589640Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - for partition 2016/03/15 | |
| 2021-09-25T18:58:59.3598936Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file .8f467155-779b-4a45-9b2a-bd4313afbbbb-1_001.log.2_0-73-97, size 14064 | |
| 2021-09-25T18:58:59.3600180Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - for partition 2015/03/16 | |
| 2021-09-25T18:58:59.3601164Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file .73acfaf0-1231-4db5-a47f-253e93567e06-0_001.log.2_1-73-98, size 11117 | |
| 2021-09-25T18:58:59.3602057Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - for partition 2015/03/17 | |
| 2021-09-25T18:58:59.3602984Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file .73acfaf0-1231-4db5-a47f-253e93567e06-1_001.log.2_2-73-99, size 5814 | |
| 2021-09-25T18:58:59.3604007Z 259590 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file .8f467155-779b-4a45-9b2a-bd4313afbbbb-0_001.log.2_3-73-100, size 5224 | |
| 2021-09-25T18:58:59.5670126Z 259798 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Beginning transaction for 004 true | |
| 2021-09-25T18:58:59.5672130Z 259798 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - REsolving conflict for 004 true | |
| 2021-09-25T18:58:59.5673338Z 259798 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Going to commit for 004 true | |
| 2021-09-25T18:58:59.8071249Z 260038 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Commit complete for 004 true | |
| 2021-09-25T18:58:59.8072642Z 260038 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Ended transaction for 004 true | |
| 2021-09-25T18:58:59.9470519Z 260178 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - ABWC. committing 004 | |
| 2021-09-25T18:58:59.9471914Z 260178 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - for partition 2016/03/15 | |
| 2021-09-25T18:58:59.9474569Z 260179 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - file info 8f467155-779b-4a45-9b2a-bd4313afbbbb-1, path 2016/03/15/.8f467155-779b-4a45-9b2a-bd4313afbbbb-1_001.log.2_0-73-97, total bytes written 14064 | |
| 2021-09-25T18:58:59.9475773Z 260179 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - for partition 2015/03/16 | |
| 2021-09-25T18:58:59.9478065Z 260179 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - file info 73acfaf0-1231-4db5-a47f-253e93567e06-0, path 2015/03/16/.73acfaf0-1231-4db5-a47f-253e93567e06-0_001.log.2_1-73-98, total bytes written 11117 | |
| 2021-09-25T18:58:59.9479332Z 260179 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - for partition 2015/03/17 | |
| 2021-09-25T18:58:59.9481683Z 260179 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - file info 73acfaf0-1231-4db5-a47f-253e93567e06-1, path 2015/03/17/.73acfaf0-1231-4db5-a47f-253e93567e06-1_001.log.2_2-73-99, total bytes written 5814 | |
| 2021-09-25T18:58:59.9483328Z 260180 [main] WARN org.apache.hudi.client.AbstractHoodieWriteClient - file info 8f467155-779b-4a45-9b2a-bd4313afbbbb-0, path 2015/03/17/.8f467155-779b-4a45-9b2a-bd4313afbbbb-0_001.log.2_3-73-100, total bytes written 5224 | |
| 2021-09-25T18:59:00.1248756Z 260356 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Unlocking 004 | |
| 2021-09-25T18:59:00.1603301Z 260391 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 677670 FSBLocks. is lock exists false lock path /tmp/junit8749113977980180/dataset/.hoodie/.locks/acquired | |
| 2021-09-25T18:59:00.1604439Z 260391 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 677670 FSBL. test : outside of while loop | |
| 2021-09-25T18:59:00.1635426Z 260395 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 677670 inside sync block : false | |
| 2021-09-25T18:59:00.1636301Z 260395 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 677670 moving onto acquiring lock | |
| 2021-09-25T18:59:00.1710948Z 260402 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 677670 acquired lock and returning true file | |
| 2021-09-25T18:59:00.1712126Z 260402 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Acquiring lock succeeded for 20210925185900 | |
| 2021-09-25T18:59:00.4284258Z 260657 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Unlocking 20210925185900 | |
| 2021-09-25T18:59:01.1766842Z 261407 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 523598 FSBLocks. is lock exists false lock path /tmp/junit8749113977980180/dataset/.hoodie/.locks/acquired | |
| 2021-09-25T18:59:01.1768419Z 261408 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 523598 FSBL. test : outside of while loop | |
| 2021-09-25T18:59:01.1796151Z 261411 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 523598 inside sync block : false | |
| 2021-09-25T18:59:01.1802826Z 261411 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 523598 moving onto acquiring lock | |
| 2021-09-25T18:59:01.1897370Z 261419 [main] WARN org.apache.hudi.client.transaction.FileSystemBasedLockProviderTestClass - 523598 acquired lock and returning true file | |
| 2021-09-25T18:59:01.1898651Z 261419 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Acquiring lock succeeded for 20210925185900 | |
| 2021-09-25T18:59:01.1924954Z 261424 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - MDT. applying commit 20210925185900 | |
| 2021-09-25T18:59:01.1928595Z 261424 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - for partition 2016/03/15 | |
| 2021-09-25T18:59:01.1932335Z 261424 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file 9d99b238-4c2f-4476-bf1a-70b375304688-0_0-94-122_20210925185900.parquet, size 448968 | |
| 2021-09-25T18:59:01.1942441Z 261426 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - for partition 2015/03/17 | |
| 2021-09-25T18:59:01.1946216Z 261426 [main] WARN org.apache.hudi.metadata.HoodieTableMetadataUtil - new file 5130b5ec-34c6-44ec-b587-7571b53352f3-0_1-94-123_20210925185900.parquet, size 446357 | |
| 2021-09-25T18:59:01.3993663Z 261630 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Beginning transaction for 20210925185900 true | |
| 2021-09-25T18:59:01.3995166Z 261630 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - REsolving conflict for 20210925185900 true | |
| 2021-09-25T18:59:01.3996322Z 261630 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Going to commit for 20210925185900 true | |
| 2021-09-25T18:59:01.7011258Z 261932 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Commit complete for 20210925185900 true | |
| 2021-09-25T18:59:01.7012747Z 261932 [main] WARN org.apache.hudi.table.action.commit.BaseCommitActionExecutor - Ended transaction for 20210925185900 true | |
| 2021-09-25T18:59:02.1479350Z 262379 [main] WARN org.apache.hudi.client.SparkRDDWriteClient - Clustering successfully on commit 20210925185900 | |
| 2021-09-25T18:59:02.1480965Z 262379 [main] WARN org.apache.hudi.client.transaction.lock.LockManager - Unlocking 20210925185900 | |
| 2021-09-25T18:59:02.3913751Z 262622 [qtp1740290827-16282] ERROR org.apache.hudi.timeline.service.RequestHandler - Got runtime exception servicing request partition=2016%2F03%2F15&basepath=%2Ftmp%2Fjunit8749113977980180%2Fdataset&fileid=8f467155-779b-4a45-9b2a-bd4313afbbbb-1&lastinstantts=001&timelinehash=a789cfb83c536edfd5edbdb9ee98037a7535f0813c641448ae03ebd84b96fb39 | |
| 2021-09-25T18:59:02.3915298Z java.lang.IllegalArgumentException: Last known instant from client was 001 but server has the following timeline [[001__deltacommit__COMPLETED], [004__deltacommit__COMPLETED], [20210925185900__replacecommit__COMPLETED]] | |
| 2021-09-25T18:59:02.3916212Z at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40) | |
| 2021-09-25T18:59:02.3916976Z at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:510) | |
| 2021-09-25T18:59:02.3917755Z at io.javalin.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22) | |
| 2021-09-25T18:59:02.3918405Z at io.javalin.Javalin.lambda$addHandler$0(Javalin.java:606) | |
| 2021-09-25T18:59:02.3919036Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:46) | |
| 2021-09-25T18:59:02.3919707Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:17) | |
| 2021-09-25T18:59:02.3920371Z at io.javalin.core.JavalinServlet$service$1.invoke(JavalinServlet.kt:143) | |
| 2021-09-25T18:59:02.3921037Z at io.javalin.core.JavalinServlet$service$2.invoke(JavalinServlet.kt:41) | |
| 2021-09-25T18:59:02.3954631Z at io.javalin.core.JavalinServlet.service(JavalinServlet.kt:107) | |
| 2021-09-25T18:59:02.3955582Z at io.javalin.core.util.JettyServerUtil$initialize$httpHandler$1.doHandle(JettyServerUtil.kt:72) | |
| 2021-09-25T18:59:02.3956352Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) | |
| 2021-09-25T18:59:02.3957054Z at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) | |
| 2021-09-25T18:59:02.3957993Z at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668) | |
| 2021-09-25T18:59:02.3958733Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) | |
| 2021-09-25T18:59:02.3959436Z at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) | |
| 2021-09-25T18:59:02.3960121Z at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) | |
| 2021-09-25T18:59:02.3960803Z at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61) | |
| 2021-09-25T18:59:02.3961501Z at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) | |
| 2021-09-25T18:59:02.3962212Z at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) | |
| 2021-09-25T18:59:02.3962850Z at org.eclipse.jetty.server.Server.handle(Server.java:502) | |
| 2021-09-25T18:59:02.3963477Z at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) | |
| 2021-09-25T18:59:02.3964139Z at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) | |
| 2021-09-25T18:59:02.3964862Z at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) | |
| 2021-09-25T18:59:02.3965541Z at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) | |
| 2021-09-25T18:59:02.3966191Z at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) | |
| 2021-09-25T18:59:02.3966882Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) | |
| 2021-09-25T18:59:02.3968325Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) | |
| 2021-09-25T18:59:02.3969060Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) | |
| 2021-09-25T18:59:02.3969714Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) | |
| 2021-09-25T18:59:02.3970377Z at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) | |
| 2021-09-25T18:59:02.3971202Z at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) | |
| 2021-09-25T18:59:02.3971831Z at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) | |
| 2021-09-25T18:59:02.3972377Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.3973836Z 262624 [Executor task launch worker for task 140] ERROR org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor - Error upserting bucketType UPDATE for partition :0 | |
| 2021-09-25T18:59:02.3974616Z org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.3975286Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.3975983Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.3976575Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.3977316Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.3978137Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.3978946Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.3979687Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.3981725Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.3982927Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.3984271Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.3985382Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.3986484Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.3987049Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.3987607Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.3988177Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.3988717Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.3989268Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.3989835Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.3990369Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.3990910Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.3991459Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.3992031Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.3992638Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.3993218Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.3993824Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.3994467Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.3995017Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.3995533Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.3996150Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.3996796Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.3997390Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.3998210Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.3999004Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.3999570Z ... 26 more | |
| 2021-09-25T18:59:02.4000531Z 262624 [Executor task launch worker for task 140] ERROR org.apache.spark.executor.Executor - Exception in task 0.0 in stage 117.0 (TID 140) | |
| 2021-09-25T18:59:02.4001238Z org.apache.hudi.exception.HoodieUpsertException: Error upserting bucketType UPDATE for partition :0 | |
| 2021-09-25T18:59:02.4001989Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:309) | |
| 2021-09-25T18:59:02.4002873Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.4005643Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4006227Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4006801Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4007353Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4007878Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4008337Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4008741Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4009175Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4009639Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4010045Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4010563Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4011033Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4011432Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4011853Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.4012279Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.4012723Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.4013207Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.4013662Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.4014142Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.4014629Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.4015061Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4015460Z Caused by: org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.4016013Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.4016587Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.4017069Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.4017666Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.4018367Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.4018824Z ... 22 more | |
| 2021-09-25T18:59:02.4019146Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.4019725Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.4020249Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.4020718Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.4021273Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.4021954Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.4022411Z ... 26 more | |
| 2021-09-25T18:59:02.4023689Z 262626 [task-result-getter-0] WARN org.apache.spark.scheduler.TaskSetManager - Lost task 0.0 in stage 117.0 (TID 140, localhost, executor driver): org.apache.hudi.exception.HoodieUpsertException: Error upserting bucketType UPDATE for partition :0 | |
| 2021-09-25T18:59:02.4024623Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:309) | |
| 2021-09-25T18:59:02.4025313Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.4025956Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4026534Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4027091Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4027656Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4028184Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4028633Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4029135Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4029572Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4030019Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4030435Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4030869Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4031313Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4031732Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4032154Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.4032580Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.4033024Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.4033509Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.4033971Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.4034438Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.4034934Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.4035360Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4035746Z Caused by: org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.4036314Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.4036887Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.4037352Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.4037958Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.4039731Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.4040213Z ... 22 more | |
| 2021-09-25T18:59:02.4040541Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.4041054Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.4041578Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.4042036Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.4042599Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.4043721Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.4044186Z ... 26 more | |
| 2021-09-25T18:59:02.4044315Z | |
| 2021-09-25T18:59:02.4046896Z 262626 [task-result-getter-0] ERROR org.apache.spark.scheduler.TaskSetManager - Task 0 in stage 117.0 failed 1 times; aborting job | |
| 2021-09-25T18:59:02.4398143Z 262669 [qtp1740290827-16284] ERROR org.apache.hudi.timeline.service.RequestHandler - Got runtime exception servicing request partition=2015%2F03%2F17&basepath=%2Ftmp%2Fjunit8749113977980180%2Fdataset&fileid=8f467155-779b-4a45-9b2a-bd4313afbbbb-0&lastinstantts=001&timelinehash=a789cfb83c536edfd5edbdb9ee98037a7535f0813c641448ae03ebd84b96fb39 | |
| 2021-09-25T18:59:02.4399326Z java.lang.IllegalArgumentException: Last known instant from client was 001 but server has the following timeline [[001__deltacommit__COMPLETED], [004__deltacommit__COMPLETED], [20210925185900__replacecommit__COMPLETED]] | |
| 2021-09-25T18:59:02.4400023Z at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40) | |
| 2021-09-25T18:59:02.4400593Z at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:510) | |
| 2021-09-25T18:59:02.4401315Z at io.javalin.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22) | |
| 2021-09-25T18:59:02.4401760Z at io.javalin.Javalin.lambda$addHandler$0(Javalin.java:606) | |
| 2021-09-25T18:59:02.4402203Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:46) | |
| 2021-09-25T18:59:02.4402679Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:17) | |
| 2021-09-25T18:59:02.4403132Z at io.javalin.core.JavalinServlet$service$1.invoke(JavalinServlet.kt:143) | |
| 2021-09-25T18:59:02.4403602Z at io.javalin.core.JavalinServlet$service$2.invoke(JavalinServlet.kt:41) | |
| 2021-09-25T18:59:02.4404052Z at io.javalin.core.JavalinServlet.service(JavalinServlet.kt:107) | |
| 2021-09-25T18:59:02.4404546Z at io.javalin.core.util.JettyServerUtil$initialize$httpHandler$1.doHandle(JettyServerUtil.kt:72) | |
| 2021-09-25T18:59:02.4405063Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) | |
| 2021-09-25T18:59:02.4405565Z at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) | |
| 2021-09-25T18:59:02.4406058Z at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668) | |
| 2021-09-25T18:59:02.4406552Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) | |
| 2021-09-25T18:59:02.4407061Z at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) | |
| 2021-09-25T18:59:02.4407563Z at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) | |
| 2021-09-25T18:59:02.4408032Z at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61) | |
| 2021-09-25T18:59:02.4408533Z at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) | |
| 2021-09-25T18:59:02.4409045Z at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) | |
| 2021-09-25T18:59:02.4409490Z at org.eclipse.jetty.server.Server.handle(Server.java:502) | |
| 2021-09-25T18:59:02.4410070Z at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) | |
| 2021-09-25T18:59:02.4410541Z at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) | |
| 2021-09-25T18:59:02.4411056Z at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) | |
| 2021-09-25T18:59:02.4411542Z at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) | |
| 2021-09-25T18:59:02.4411997Z at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) | |
| 2021-09-25T18:59:02.4412492Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) | |
| 2021-09-25T18:59:02.4413011Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) | |
| 2021-09-25T18:59:02.4413544Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) | |
| 2021-09-25T18:59:02.4414063Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) | |
| 2021-09-25T18:59:02.4414608Z at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) | |
| 2021-09-25T18:59:02.4415158Z at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) | |
| 2021-09-25T18:59:02.4415661Z at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) | |
| 2021-09-25T18:59:02.4416072Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4457026Z 262671 [Executor task launch worker for task 143] ERROR org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor - Error upserting bucketType UPDATE for partition :3 | |
| 2021-09-25T18:59:02.4457719Z org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.4458283Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.4458850Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.4459334Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.4460144Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.4460841Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.4461528Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.4462165Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4462748Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4463480Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4464046Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4464578Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4465024Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4465443Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4465877Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4466323Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4466739Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4467170Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4467627Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4468029Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4468449Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.4468968Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.4469416Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.4469901Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.4470355Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.4470823Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.4471327Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.4471749Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4472128Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.4472634Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.4473157Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.4473622Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.4474187Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.4474852Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.4475313Z ... 26 more | |
| 2021-09-25T18:59:02.4476192Z 262674 [task-result-getter-1] WARN org.apache.spark.scheduler.TaskSetManager - Lost task 3.0 in stage 117.0 (TID 143, localhost, executor driver): TaskKilled (Stage cancelled) | |
| 2021-09-25T18:59:02.4879015Z 262719 [qtp1740290827-16286] ERROR org.apache.hudi.timeline.service.RequestHandler - Got runtime exception servicing request partition=2015%2F03%2F17&basepath=%2Ftmp%2Fjunit8749113977980180%2Fdataset&fileid=73acfaf0-1231-4db5-a47f-253e93567e06-1&lastinstantts=001&timelinehash=a789cfb83c536edfd5edbdb9ee98037a7535f0813c641448ae03ebd84b96fb39 | |
| 2021-09-25T18:59:02.4880381Z java.lang.IllegalArgumentException: Last known instant from client was 001 but server has the following timeline [[001__deltacommit__COMPLETED], [004__deltacommit__COMPLETED], [20210925185900__replacecommit__COMPLETED]] | |
| 2021-09-25T18:59:02.4881090Z at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40) | |
| 2021-09-25T18:59:02.4881626Z at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:510) | |
| 2021-09-25T18:59:02.4882136Z at io.javalin.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22) | |
| 2021-09-25T18:59:02.4882570Z at io.javalin.Javalin.lambda$addHandler$0(Javalin.java:606) | |
| 2021-09-25T18:59:02.4883019Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:46) | |
| 2021-09-25T18:59:02.4883490Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:17) | |
| 2021-09-25T18:59:02.4883947Z at io.javalin.core.JavalinServlet$service$1.invoke(JavalinServlet.kt:143) | |
| 2021-09-25T18:59:02.4884423Z at io.javalin.core.JavalinServlet$service$2.invoke(JavalinServlet.kt:41) | |
| 2021-09-25T18:59:02.4884880Z at io.javalin.core.JavalinServlet.service(JavalinServlet.kt:107) | |
| 2021-09-25T18:59:02.4885360Z at io.javalin.core.util.JettyServerUtil$initialize$httpHandler$1.doHandle(JettyServerUtil.kt:72) | |
| 2021-09-25T18:59:02.4885895Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) | |
| 2021-09-25T18:59:02.4886391Z at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) | |
| 2021-09-25T18:59:02.4886872Z at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668) | |
| 2021-09-25T18:59:02.4887385Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) | |
| 2021-09-25T18:59:02.4887889Z at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) | |
| 2021-09-25T18:59:02.4888387Z at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) | |
| 2021-09-25T18:59:02.4888957Z at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61) | |
| 2021-09-25T18:59:02.4889458Z at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) | |
| 2021-09-25T18:59:02.4889972Z at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) | |
| 2021-09-25T18:59:02.4890417Z at org.eclipse.jetty.server.Server.handle(Server.java:502) | |
| 2021-09-25T18:59:02.4890846Z at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) | |
| 2021-09-25T18:59:02.4891312Z at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) | |
| 2021-09-25T18:59:02.4891816Z at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) | |
| 2021-09-25T18:59:02.4892313Z at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) | |
| 2021-09-25T18:59:02.4892770Z at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) | |
| 2021-09-25T18:59:02.4893251Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) | |
| 2021-09-25T18:59:02.4893791Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) | |
| 2021-09-25T18:59:02.4894337Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) | |
| 2021-09-25T18:59:02.4894865Z at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) | |
| 2021-09-25T18:59:02.4895405Z at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) | |
| 2021-09-25T18:59:02.4895956Z at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) | |
| 2021-09-25T18:59:02.4896458Z at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) | |
| 2021-09-25T18:59:02.4896869Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4898001Z 262720 [Executor task launch worker for task 142] ERROR org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor - Error upserting bucketType UPDATE for partition :2 | |
| 2021-09-25T18:59:02.4898683Z org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.4899309Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.4899880Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.4900362Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.4901001Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.4901706Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.4902376Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.4903010Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4904163Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.4904727Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4905292Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.4905814Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4906280Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4906684Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4907118Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4907580Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4907983Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4908515Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.4908972Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.4909374Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.4909795Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.4910223Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.4910668Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.4911151Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.4911608Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.4912079Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.4912580Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.4913009Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.4913393Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.4913902Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.4914420Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.4914890Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.4915441Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.4916107Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.4916558Z ... 26 more | |
| 2021-09-25T18:59:02.4949509Z 262726 [task-result-getter-2] WARN org.apache.spark.scheduler.TaskSetManager - Lost task 2.0 in stage 117.0 (TID 142, localhost, executor driver): TaskKilled (Stage cancelled) | |
| 2021-09-25T18:59:02.4981256Z 262729 [qtp1740290827-16285] ERROR org.apache.hudi.timeline.service.RequestHandler - Got runtime exception servicing request partition=2015%2F03%2F16&basepath=%2Ftmp%2Fjunit8749113977980180%2Fdataset&fileid=73acfaf0-1231-4db5-a47f-253e93567e06-0&lastinstantts=001&timelinehash=a789cfb83c536edfd5edbdb9ee98037a7535f0813c641448ae03ebd84b96fb39 | |
| 2021-09-25T18:59:02.4982646Z java.lang.IllegalArgumentException: Last known instant from client was 001 but server has the following timeline [[001__deltacommit__COMPLETED], [004__deltacommit__COMPLETED], [20210925185900__replacecommit__COMPLETED]] | |
| 2021-09-25T18:59:02.4983778Z at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:40) | |
| 2021-09-25T18:59:02.4985162Z at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:510) | |
| 2021-09-25T18:59:02.4985849Z at io.javalin.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22) | |
| 2021-09-25T18:59:02.4992178Z at io.javalin.Javalin.lambda$addHandler$0(Javalin.java:606) | |
| 2021-09-25T18:59:02.4992813Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:46) | |
| 2021-09-25T18:59:02.4994400Z at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:17) | |
| 2021-09-25T18:59:02.4995044Z at io.javalin.core.JavalinServlet$service$1.invoke(JavalinServlet.kt:143) | |
| 2021-09-25T18:59:02.4995621Z at io.javalin.core.JavalinServlet$service$2.invoke(JavalinServlet.kt:41) | |
| 2021-09-25T18:59:02.4996202Z at io.javalin.core.JavalinServlet.service(JavalinServlet.kt:107) | |
| 2021-09-25T18:59:02.4996817Z at io.javalin.core.util.JettyServerUtil$initialize$httpHandler$1.doHandle(JettyServerUtil.kt:72) | |
| 2021-09-25T18:59:02.4997475Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) | |
| 2021-09-25T18:59:02.4998079Z at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) | |
| 2021-09-25T18:59:02.4998695Z at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668) | |
| 2021-09-25T18:59:02.4999561Z at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) | |
| 2021-09-25T18:59:02.5000193Z at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) | |
| 2021-09-25T18:59:02.5000802Z at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) | |
| 2021-09-25T18:59:02.5001410Z at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:61) | |
| 2021-09-25T18:59:02.5002035Z at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) | |
| 2021-09-25T18:59:02.5002673Z at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) | |
| 2021-09-25T18:59:02.5003240Z at org.eclipse.jetty.server.Server.handle(Server.java:502) | |
| 2021-09-25T18:59:02.5003795Z at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370) | |
| 2021-09-25T18:59:02.5004384Z at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) | |
| 2021-09-25T18:59:02.5005030Z at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) | |
| 2021-09-25T18:59:02.5005637Z at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) | |
| 2021-09-25T18:59:02.5006218Z at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) | |
| 2021-09-25T18:59:02.5006820Z at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) | |
| 2021-09-25T18:59:02.5007464Z at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) | |
| 2021-09-25T18:59:02.5007995Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.5009217Z 262730 [Executor task launch worker for task 141] ERROR org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor - Error upserting bucketType UPDATE for partition :1 | |
| 2021-09-25T18:59:02.5009997Z org.apache.hudi.exception.HoodieRemoteException: Server Error | |
| 2021-09-25T18:59:02.5010667Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:297) | |
| 2021-09-25T18:59:02.5011458Z at org.apache.hudi.io.HoodieAppendHandle.init(HoodieAppendHandle.java:135) | |
| 2021-09-25T18:59:02.5012078Z at org.apache.hudi.io.HoodieAppendHandle.doAppend(HoodieAppendHandle.java:349) | |
| 2021-09-25T18:59:02.5012812Z at org.apache.hudi.table.action.deltacommit.AbstractSparkDeltaCommitActionExecutor.handleUpdate(AbstractSparkDeltaCommitActionExecutor.java:84) | |
| 2021-09-25T18:59:02.5013639Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.handleUpsertPartition(BaseSparkCommitActionExecutor.java:302) | |
| 2021-09-25T18:59:02.5014430Z at org.apache.hudi.table.action.commit.BaseSparkCommitActionExecutor.lambda$execute$ecf5068c$1(BaseSparkCommitActionExecutor.java:153) | |
| 2021-09-25T18:59:02.5015224Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.5015906Z at org.apache.spark.api.java.JavaRDDLike$$anonfun$mapPartitionsWithIndex$1.apply(JavaRDDLike.scala:102) | |
| 2021-09-25T18:59:02.5016607Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.5017301Z at org.apache.spark.rdd.RDD$$anonfun$mapPartitionsWithIndex$1$$anonfun$apply$25.apply(RDD.scala:853) | |
| 2021-09-25T18:59:02.5017931Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.5018620Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.5019189Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.5021387Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.5021852Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.5022255Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.5022685Z at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52) | |
| 2021-09-25T18:59:02.5023146Z at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324) | |
| 2021-09-25T18:59:02.5023858Z at org.apache.spark.rdd.RDD.iterator(RDD.scala:288) | |
| 2021-09-25T18:59:02.5024281Z at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90) | |
| 2021-09-25T18:59:02.5024711Z at org.apache.spark.scheduler.Task.run(Task.scala:123) | |
| 2021-09-25T18:59:02.5025154Z at org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408) | |
| 2021-09-25T18:59:02.5025637Z at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360) | |
| 2021-09-25T18:59:02.5026096Z at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414) | |
| 2021-09-25T18:59:02.5026563Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) | |
| 2021-09-25T18:59:02.5027064Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) | |
| 2021-09-25T18:59:02.5027488Z at java.lang.Thread.run(Thread.java:748) | |
| 2021-09-25T18:59:02.5027882Z Caused by: org.apache.http.client.HttpResponseException: Server Error | |
| 2021-09-25T18:59:02.5028383Z at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:69) | |
| 2021-09-25T18:59:02.5028905Z at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) | |
| 2021-09-25T18:59:02.5029372Z at org.apache.http.client.fluent.Response.returnContent(Response.java:97) | |
| 2021-09-25T18:59:02.5029923Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:179) | |
| 2021-09-25T18:59:02.5030588Z at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestFileSlice(RemoteHoodieTableFileSystemView.java:293) | |
| 2021-09-25T18:59:02.5031049Z ... 26 more | |
| 2021-09-25T18:59:02.5031923Z 262730 [task-result-getter-3] WARN org.apache.spark.scheduler.TaskSetManager - Lost task 1.0 in stage 117.0 (TID 141, localhost, executor driver): TaskKilled (Stage cancelled) | |
| 2021-09-25T18:59:02.5626296Z 262791 [main] WARN org.apache.hudi.testutils.HoodieClientTestHarness - Closing file-system instance used in previous test-run | |
| 2021-09-25T18:59:03.2267594Z [ERROR] Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 76.223 s <<< FAILURE! - in org.apache.hudi.client.TestHoodieClientMultiWriter | |
| 2021-09-25T18:59:03.2274352Z [ERROR] org.apache.hudi.client.TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflictCOW Time elapsed: 13.977 s <<< ERROR! | |
| 2021-09-25T18:59:03.2275419Z java.util.concurrent.ExecutionException: java.lang.RuntimeException: org.apache.hudi.exception.HoodieHeartbeatException: Unable to generate heartbeat | |
| 2021-09-25T18:59:03.2276344Z at org.apache.hudi.client.TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflict(TestHoodieClientMultiWriter.java:240) | |
| 2021-09-25T18:59:03.2277275Z at org.apache.hudi.client.TestHoodieClientMultiWriter.testMultiWriterWithAsyncTableServicesWithConflictCOW(TestHoodieClientMultiWriter.java:130) | |
| 2021-09-25T18:59:03.2278143Z Caused by: java.lang.RuntimeException: org.apache.hudi.exception.HoodieHeartbeatException: Unable to generate heartbeat | |
| 2021-09-25T18:59:03.2279050Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testMultiWriterWithAsyncTableServicesWithConflict$5(TestHoodieClientMultiWriter.java:218) | |
| 2021-09-25T18:59:03.2279888Z Caused by: org.apache.hudi.exception.HoodieHeartbeatException: Unable to generate heartbeat | |
| 2021-09-25T18:59:03.2280643Z at org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithInserts(TestHoodieClientMultiWriter.java:298) | |
| 2021-09-25T18:59:03.2281522Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testMultiWriterWithAsyncTableServicesWithConflict$5(TestHoodieClientMultiWriter.java:215) | |
| 2021-09-25T18:59:03.2282300Z Caused by: org.apache.hadoop.util.Shell$ExitCodeException: | |
| 2021-09-25T18:59:03.2283413Z chmod: cannot access '/tmp/junit6151631519916678816/dataset/.hoodie/.heartbeat/007': No such file or directory | |
| 2021-09-25T18:59:03.2283876Z | |
| 2021-09-25T18:59:03.2284478Z at org.apache.hudi.client.TestHoodieClientMultiWriter.createCommitWithInserts(TestHoodieClientMultiWriter.java:298) | |
| 2021-09-25T18:59:03.2285585Z at org.apache.hudi.client.TestHoodieClientMultiWriter.lambda$testMultiWriterWithAsyncTableServicesWithConflict$5(TestHoodieClientMultiWriter.java:215) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment