Skip to content

Instantly share code, notes, and snippets.

@nsivabalan
Last active July 28, 2021 05:39
Show Gist options
  • Save nsivabalan/4e58690cd778082a3601aa690b508548 to your computer and use it in GitHub Desktop.
Save nsivabalan/4e58690cd778082a3601aa690b508548 to your computer and use it in GitHub Desktop.
0: jdbc:hive2://hiveserver:10000> select symbol, max(ts) from stock_ticks_mor_rt group by symbol HAVING symbol = 'GOOG';
WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Error: org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257)
at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840)
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) (state=08S01,code=2)
2021-07-28T05:35:23,816 INFO [HiveServer2-Background-Pool: Thread-55]: utils.HoodieInputFormatUtils (HoodieInputFormatUtils.java:getTableMetaClientForBasePath(334)) - Reading hoodie metadata from path hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:23,816 INFO [HiveServer2-Background-Pool: Thread-55]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(105)) - Loading HoodieTableMetaClient from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:23,820 INFO [HiveServer2-Background-Pool: Thread-55]: table.HoodieTableConfig (HoodieTableConfig.java:<init>(171)) - Loading table properties from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/.hoodie/hoodie.properties
2021-07-28T05:35:23,825 INFO [HiveServer2-Background-Pool: Thread-55]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(125)) - Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=PARQUET) from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:23,846 INFO [HiveServer2-Background-Pool: Thread-55]: timeline.HoodieActiveTimeline (HoodieActiveTimeline.java:<init>(115)) - Loaded instants upto : Option{val=[20210728053453__deltacommit__COMPLETED]}
2021-07-28T05:35:23,847 INFO [HiveServer2-Background-Pool: Thread-55]: view.FileSystemViewManager (FileSystemViewManager.java:createInMemoryFileSystemViewWithTimeline(182)) - Creating InMemory based view for basePath hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:23,850 INFO [HiveServer2-Background-Pool: Thread-55]: view.AbstractTableFileSystemView (AbstractTableFileSystemView.java:resetFileGroupsReplaced(230)) - Took 2 ms to read 0 instants, 0 replaced file groups
2021-07-28T05:35:23,855 INFO [HiveServer2-Background-Pool: Thread-55]: util.ClusteringUtils (ClusteringUtils.java:getAllFileGroupsInPendingClusteringPlans(128)) - Found 0 files in pending clustering operations
2021-07-28T05:35:24,501 WARN [HiveServer2-Background-Pool: Thread-55]: utils.HoodieRealtimeInputFormatUtils (HoodieRealtimeInputFormatUtils.java:lambda$getRealtimeSplits$9(104)) - TEST_LOG. Setting virtual key HoodieVirtualKeyInfo{recordKeyField='key', partitionPathField='date', recordFieldIndex=12, partitionPathIndex=13}
2021-07-28T05:35:24,504 INFO [HiveServer2-Background-Pool: Thread-55]: view.AbstractTableFileSystemView (AbstractTableFileSystemView.java:lambda$ensurePartitionLoadedCorrectly$9(274)) - Building file system view for partition (2018/08/31)
2021-07-28T05:35:24,508 INFO [HiveServer2-Background-Pool: Thread-55]: view.AbstractTableFileSystemView (AbstractTableFileSystemView.java:addFilesToView(149)) - addFilesToView: NumFiles=3, NumFileGroups=1, FileGroupsCreationTime=1, StoreTimeTaken=0
2021-07-28T05:35:24,522 WARN [HiveServer2-Background-Pool: Thread-55]: utils.HoodieRealtimeInputFormatUtils (HoodieRealtimeInputFormatUtils.java:lambda$null$7(150)) - TEST_LOG. realtimeFileSplit
2021-07-28T05:35:24,526 WARN [HiveServer2-Background-Pool: Thread-55]: realtime.HoodieRealtimeFileSplit (HoodieRealtimeFileSplit.java:<init>(57)) - Instantiating HoodieRealtimeFileSplit :: HoodieVirtualKeyInfo{recordKeyField='key', partitionPathField='date', recordFieldIndex=12, partitionPathIndex=13}
2021-07-28T05:35:24,530 INFO [HiveServer2-Background-Pool: Thread-55]: view.AbstractTableFileSystemView (AbstractTableFileSystemView.java:resetFileGroupsReplaced(230)) - Took 0 ms to read 0 instants, 0 replaced file groups
2021-07-28T05:35:24,533 INFO [HiveServer2-Background-Pool: Thread-55]: util.ClusteringUtils (ClusteringUtils.java:getAllFileGroupsInPendingClusteringPlans(128)) - Found 0 files in pending clustering operations
2021-07-28T05:35:24,533 INFO [HiveServer2-Background-Pool: Thread-55]: utils.HoodieRealtimeInputFormatUtils (HoodieRealtimeInputFormatUtils.java:getRealtimeSplits(165)) - Returning a total splits of 1
2021-07-28T05:35:24,534 INFO [HiveServer2-Background-Pool: Thread-55]: io.HiveInputFormat (HiveInputFormat.java:getSplits(569)) - number of splits 1
2021-07-28T05:35:24,535 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:24,535 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:24,536 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:24,536 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:24,554 INFO [HiveServer2-Background-Pool: Thread-55]: mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(200)) - number of splits:1
2021-07-28T05:35:24,620 INFO [HiveServer2-Background-Pool: Thread-55]: mapreduce.JobSubmitter (JobSubmitter.java:printTokens(289)) - Submitting tokens for job: job_local1579169718_0001
2021-07-28T05:35:25,188 INFO [HiveServer2-Background-Pool: Thread-55]: mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:symlink(201)) - Creating symlink: /tmp/hadoop-root/mapred/local/1627450524786/hoodie-hadoop-mr-bundle.jar <- /opt/hive/bin/hoodie-hadoop-mr-bundle.jar
2021-07-28T05:35:25,194 INFO [HiveServer2-Background-Pool: Thread-55]: mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:setup(165)) - Localized file:/var/hoodie/ws/docker/hoodie/hadoop/hive_base/target/hoodie-hadoop-mr-bundle.jar as file:/tmp/hadoop-root/mapred/local/1627450524786/hoodie-hadoop-mr-bundle.jar
2021-07-28T05:35:25,265 INFO [HiveServer2-Background-Pool: Thread-55]: mapred.LocalDistributedCacheManager (LocalDistributedCacheManager.java:makeClassLoader(234)) - file:/tmp/hadoop-root/mapred/local/1627450524786/hoodie-hadoop-mr-bundle.jar
2021-07-28T05:35:25,283 INFO [HiveServer2-Background-Pool: Thread-55]: mapreduce.Job (Job.java:submit(1345)) - The url to track the job: http://localhost:8080/
2021-07-28T05:35:25,286 INFO [Thread-53]: mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(498)) - OutputCommitter set in config org.apache.hadoop.hive.ql.io.HiveFileFormatUtils$NullOutputCommitter
2021-07-28T05:35:25,288 INFO [Thread-53]: mapred.LocalJobRunner (LocalJobRunner.java:createOutputCommitter(516)) - OutputCommitter is org.apache.hadoop.hive.ql.io.HiveFileFormatUtils$NullOutputCommitter
2021-07-28T05:35:25,288 INFO [HiveServer2-Background-Pool: Thread-55]: exec.Task (SessionState.java:printInfo(1111)) - Job running in-process (local Hadoop)
2021-07-28T05:35:25,306 INFO [Thread-53]: mapred.LocalJobRunner (LocalJobRunner.java:runTasks(475)) - Waiting for map tasks
2021-07-28T05:35:25,317 INFO [LocalJobRunner Map Task Executor #0]: mapred.LocalJobRunner (LocalJobRunner.java:run(251)) - Starting task: attempt_local1579169718_0001_m_000000_0
2021-07-28T05:35:25,401 INFO [LocalJobRunner Map Task Executor #0]: mapred.Task (Task.java:initialize(620)) - Using ResourceCalculatorProcessTree : [ ]
2021-07-28T05:35:25,414 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:updateJobWithSplit(485)) - Processing split: org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat:HoodieRealtimeFileSplit{DataPath=hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_0-16-211_20210728053415.parquet, deltaLogPaths=[hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/.8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_20210728053415.log.1_0-16-211], maxCommitTime='20210728053453', basePath='hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor'}
2021-07-28T05:35:25,423 INFO [LocalJobRunner Map Task Executor #0]: exec.Utilities (Utilities.java:getBaseWork(407)) - PLAN PATH = file:/tmp/root/43454311-cbd4-4a02-b5fd-87cfcc2f896a/hive_2021-07-28_05-35-17_794_8926989242932757063-2/-mr-10005/8f90f766-b571-4ee2-b72b-937490a5ae15/map.xml
2021-07-28T05:35:25,424 INFO [LocalJobRunner Map Task Executor #0]: exec.SerializationUtilities (SerializationUtilities.java:deserializePlan(490)) - Deserializing MapWork using kryo
2021-07-28T05:35:25,501 INFO [LocalJobRunner Map Task Executor #0]: exec.Utilities (Utilities.java:getBaseWork(467)) - Deserialized plan (via FILE) - name: null size: 7.28KB
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(140)) - TEST_LOG. realtime split's virtual key info false,
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(142)) - TEST_LOG. job conf. use meta fields null
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(143)) - TEST_LOG. job conf. record key field null
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(144)) - TEST_LOG. job conf. partition path field null
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(145)) - TEST_LOG. job conf. record key index : null
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(146)) - TEST_LOG. job conf. partition path index : null
2021-07-28T05:35:25,503 INFO [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:addProjectionToJobConf(110)) - Before adding Hoodie columns, Projections :ts,symbol, Ids :6,7
2021-07-28T05:35:25,503 WARN [LocalJobRunner Map Task Executor #0]: utils.HoodieRealtimeInputFormatUtils (HoodieRealtimeInputFormatUtils.java:addRequiredProjectionFields(237)) - TEST_LOG. ------ Adding default meta cols to project
2021-07-28T05:35:25,504 INFO [LocalJobRunner Map Task Executor #0]: realtime.HoodieParquetRealtimeInputFormat (HoodieParquetRealtimeInputFormat.java:getRecordReader(148)) - Creating record reader with readCols :ts,symbol,_hoodie_record_key,_hoodie_commit_time,_hoodie_partition_path, Ids :6,7,2,0,3
2021-07-28T05:35:25,518 INFO [LocalJobRunner Map Task Executor #0]: exec.Utilities (Utilities.java:getBaseWork(407)) - PLAN PATH = file:/tmp/root/43454311-cbd4-4a02-b5fd-87cfcc2f896a/hive_2021-07-28_05-35-17_794_8926989242932757063-2/-mr-10005/8f90f766-b571-4ee2-b72b-937490a5ae15/map.xml
2021-07-28T05:35:25,554 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:25,554 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:25,554 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:25,554 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:25,991 INFO [LocalJobRunner Map Task Executor #0]: Configuration.deprecation (Configuration.java:logDeprecation(1285)) - mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id
2021-07-28T05:35:26,119 INFO [LocalJobRunner Map Task Executor #0]: zlib.ZlibFactory (ZlibFactory.java:<clinit>(49)) - Successfully loaded & initialized native-zlib library
2021-07-28T05:35:26,121 INFO [LocalJobRunner Map Task Executor #0]: compress.CodecPool (CodecPool.java:getDecompressor(184)) - Got brand-new decompressor [.gz]
2021-07-28T05:35:26,315 INFO [LocalJobRunner Map Task Executor #0]: realtime.HoodieRealtimeRecordReader (HoodieRealtimeRecordReader.java:constructRecordReader(69)) - Enabling merged reading of realtime records for split HoodieRealtimeFileSplit{DataPath=hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_0-16-211_20210728053415.parquet, deltaLogPaths=[hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/.8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_20210728053415.log.1_0-16-211], maxCommitTime='20210728053453', basePath='hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor'}
2021-07-28T05:35:26,319 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:<init>(61)) - cfg ==> ts,symbol,_hoodie_record_key,_hoodie_commit_time,_hoodie_partition_path
2021-07-28T05:35:26,319 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:<init>(62)) - columnIds ==> 6,7,2,0,3
2021-07-28T05:35:26,319 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:<init>(63)) - partitioningColumns ==> dt
2021-07-28T05:35:26,320 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(105)) - Loading HoodieTableMetaClient from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,323 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableConfig (HoodieTableConfig.java:<init>(171)) - Loading table properties from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/.hoodie/hoodie.properties
2021-07-28T05:35:26,324 WARN [HiveServer2-Background-Pool: Thread-55]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2021-07-28T05:35:26,325 INFO [HiveServer2-Background-Pool: Thread-55]: exec.Task (SessionState.java:printInfo(1111)) - 2021-07-28 05:35:26,309 Stage-1 map = 0%, reduce = 0%
2021-07-28T05:35:26,330 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(125)) - Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=PARQUET) from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,332 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:<init>(66)) - usesCustomPayload ==> true
2021-07-28T05:35:26,333 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(105)) - Loading HoodieTableMetaClient from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,336 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableConfig (HoodieTableConfig.java:<init>(171)) - Loading table properties from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/.hoodie/hoodie.properties
2021-07-28T05:35:26,341 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(125)) - Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=PARQUET) from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,346 INFO [LocalJobRunner Map Task Executor #0]: timeline.HoodieActiveTimeline (HoodieActiveTimeline.java:<init>(115)) - Loaded instants upto : Option{val=[20210728053453__deltacommit__COMPLETED]}
2021-07-28T05:35:26,403 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:init(91)) - Writer Schema From Log => {
"type" : "record",
"name" : "stock_ticks",
"fields" : [ {
"name" : "_hoodie_commit_time",
"type" : [ "null", "string" ],
"doc" : "",
"default" : null
}, {
"name" : "_hoodie_commit_seqno",
"type" : [ "null", "string" ],
"doc" : "",
"default" : null
}, {
"name" : "_hoodie_record_key",
"type" : [ "null", "string" ],
"doc" : "",
"default" : null
}, {
"name" : "_hoodie_partition_path",
"type" : [ "null", "string" ],
"doc" : "",
"default" : null
}, {
"name" : "_hoodie_file_name",
"type" : [ "null", "string" ],
"doc" : "",
"default" : null
}, {
"name" : "volume",
"type" : "long"
}, {
"name" : "ts",
"type" : "string"
}, {
"name" : "symbol",
"type" : "string"
}, {
"name" : "year",
"type" : "int"
}, {
"name" : "month",
"type" : "string"
}, {
"name" : "high",
"type" : "double"
}, {
"name" : "low",
"type" : "double"
}, {
"name" : "key",
"type" : "string"
}, {
"name" : "date",
"type" : "string"
}, {
"name" : "close",
"type" : "double"
}, {
"name" : "open",
"type" : "double"
}, {
"name" : "day",
"type" : "string"
} ]
}
2021-07-28T05:35:26,414 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:constructHiveOrderedSchema(115)) - Hive Columns : _hoodie_commit_time,_hoodie_commit_seqno,_hoodie_record_key,_hoodie_partition_path,_hoodie_file_name,volume,ts,symbol,year,month,high,low,key,date,close,open,day,dt,BLOCK__OFFSET__INSIDE__FILE,INPUT__FILE__NAME,ROW__ID
2021-07-28T05:35:26,414 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:constructHiveOrderedSchema(117)) - Hive Columns : _hoodie_commit_time,_hoodie_commit_seqno,_hoodie_record_key,_hoodie_partition_path,_hoodie_file_name,volume,ts,symbol,year,month,high,low,key,date,close,open,day,dt,BLOCK__OFFSET__INSIDE__FILE,INPUT__FILE__NAME,ROW__ID
2021-07-28T05:35:26,417 INFO [LocalJobRunner Map Task Executor #0]: realtime.AbstractRealtimeRecordReader (AbstractRealtimeRecordReader.java:init(108)) - About to read compacted logs [hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/.8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_20210728053415.log.1_0-16-211] for base split hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_0-16-211_20210728053415.parquet, projecting cols [_hoodie_commit_time, _hoodie_record_key, _hoodie_partition_path, ts, symbol]
2021-07-28T05:35:26,417 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(105)) - Loading HoodieTableMetaClient from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,421 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableConfig (HoodieTableConfig.java:<init>(171)) - Loading table properties from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/.hoodie/hoodie.properties
2021-07-28T05:35:26,426 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(125)) - Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=PARQUET) from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,435 INFO [LocalJobRunner Map Task Executor #0]: Configuration.deprecation (Configuration.java:logDeprecation(1285)) - mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb
2021-07-28T05:35:26,436 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(105)) - Loading HoodieTableMetaClient from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,439 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableConfig (HoodieTableConfig.java:<init>(171)) - Loading table properties from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/.hoodie/hoodie.properties
2021-07-28T05:35:26,444 INFO [LocalJobRunner Map Task Executor #0]: table.HoodieTableMetaClient (HoodieTableMetaClient.java:<init>(125)) - Finished Loading Table of type MERGE_ON_READ(version=1, baseFileFormat=PARQUET) from hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor
2021-07-28T05:35:26,466 INFO [LocalJobRunner Map Task Executor #0]: timeline.HoodieActiveTimeline (HoodieActiveTimeline.java:<init>(115)) - Loaded instants upto : Option{val=[20210728053453__deltacommit__COMPLETED]}
2021-07-28T05:35:26,474 INFO [LocalJobRunner Map Task Executor #0]: log.AbstractHoodieLogRecordScanner (AbstractHoodieLogRecordScanner.java:scan(151)) - Scanning log file HoodieLogFile{pathStr='hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/.8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_20210728053415.log.1_0-16-211', fileLen=0}
2021-07-28T05:35:26,481 INFO [LocalJobRunner Map Task Executor #0]: log.AbstractHoodieLogRecordScanner (AbstractHoodieLogRecordScanner.java:scan(178)) - Reading a data block from file hdfs://namenode:8020/user/hive/warehouse/stock_ticks_mor/2018/08/31/.8102aa48-b494-4de1-b00b-7ad1b960c3bd-0_20210728053415.log.1_0-16-211 at instant 20210728053453
2021-07-28T05:35:26,481 INFO [LocalJobRunner Map Task Executor #0]: log.AbstractHoodieLogRecordScanner (AbstractHoodieLogRecordScanner.java:scan(267)) - Merging the final data blocks
2021-07-28T05:35:26,481 INFO [LocalJobRunner Map Task Executor #0]: log.AbstractHoodieLogRecordScanner (AbstractHoodieLogRecordScanner.java:processQueuedBlocksForInstant(340)) - Number of remaining logblocks to merge 1
2021-07-28T05:35:26,565 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:26,565 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:26,565 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:26,565 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:26,634 INFO [LocalJobRunner Map Task Executor #0]: collection.BitCaskDiskMap (BitCaskDiskMap.java:initFile(136)) - Spilling to file location /tmp/4771140d-7739-4b8b-9c82-3b9c8c09513f in host (192.168.16.9) with hostname (hiveserver)
2021-07-28T05:35:26,638 INFO [LocalJobRunner Map Task Executor #0]: collection.ExternalSpillableMap (ExternalSpillableMap.java:put(210)) - Estimated Payload size => 440
2021-07-28T05:35:26,646 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(98)) - Number of log files scanned => 1
2021-07-28T05:35:26,646 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(99)) - MaxMemoryInBytes allowed for compaction => 805306368
2021-07-28T05:35:26,647 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(100)) - Number of entries in MemoryBasedMap in ExternalSpillableMap => 99
2021-07-28T05:35:26,647 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(101)) - Total size in bytes of MemoryBasedMap in ExternalSpillableMap => 43560
2021-07-28T05:35:26,647 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(103)) - Number of entries in BitCaskDiskMap in ExternalSpillableMap => 0
2021-07-28T05:35:26,647 INFO [LocalJobRunner Map Task Executor #0]: log.HoodieMergedLogRecordScanner (HoodieMergedLogRecordScanner.java:performScan(104)) - Size of file spilled to disk => 0
2021-07-28T05:35:26,656 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:runOldMapper(439)) - numReduceTasks: 1
2021-07-28T05:35:26,714 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:setEquator(1205)) - (EQUATOR) 0 kvi 26214396(104857584)
2021-07-28T05:35:26,714 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:init(998)) - mapreduce.task.io.sort.mb: 100
2021-07-28T05:35:26,714 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:init(999)) - soft limit at 83886080
2021-07-28T05:35:26,714 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:init(1000)) - bufstart = 0; bufvoid = 104857600
2021-07-28T05:35:26,714 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:init(1001)) - kvstart = 26214396; length = 6553600
2021-07-28T05:35:26,719 INFO [LocalJobRunner Map Task Executor #0]: mapred.MapTask (MapTask.java:createSortingCollector(403)) - Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
2021-07-28T05:35:26,724 INFO [LocalJobRunner Map Task Executor #0]: mr.ExecMapper (ExecMapper.java:configure(79)) - conf classpath = [file:/opt/hive/lib/commons-logging-1.2.jar, file:/opt/hive/lib/google-http-client-jackson2-1.15.0-rc.jar, file:/opt/hive/lib/asm-tree-3.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/log4j-1.2.17.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/httpclient-4.5.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/jersey-core-1.9.jar, file:/opt/hive/lib/mapdb-1.0.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/hadoop-nfs-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jersey-core-1.9.jar, file:/opt/hive/lib/commons-compress-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/junit-4.11.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-common-2.8.4.jar, file:/opt/hive/lib/avatica-metrics-1.8.0.jar, file:/opt/hive/lib/http-client-1.0.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/curator-client-2.7.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/asm-3.2.jar, file:/opt/hive/lib/log4j-slf4j-impl-2.6.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-registry-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/hadoop-auth-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/stax-api-1.0-2.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/javax.inject-1.jar, file:/opt/hive/lib/hive-testutils-2.3.3.jar, file:/opt/hive/lib/jackson-dataformat-smile-2.4.6.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar, file:/opt/hive/lib/log4j-web-2.6.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-io-2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/guice-3.0.jar, file:/opt/hive/lib/aether-util-0.9.0.M2.jar, file:/opt/hive/lib/log4j-api-2.6.2.jar, file:/opt/hive/lib/jetty-all-7.6.0.v20120127.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.4.jar, file:/opt/hive/lib/hbase-common-1.1.1.jar, file:/opt/hive/lib/druid-processing-0.9.2.jar, file:/opt/hive/lib/maven-repository-metadata-3.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/xz-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/guice-servlet-3.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar, file:/opt/hive/lib/javax.el-3.0.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jersey-json-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar, file:/opt/hive/lib/aether-api-0.9.0.M2.jar, file:/opt/hive/lib/libfb303-0.9.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/aopalliance-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/hadoop-hdfs-client-2.8.4.jar, file:/opt/hive/lib/hive-cli-2.3.3.jar, file:/opt/hive/lib/joda-time-2.8.1.jar, file:/opt/hive/lib/hive-hplsql-2.3.3.jar, file:/opt/hive/lib/hbase-hadoop2-compat-1.1.1-tests.jar, file:/opt/hive/lib/postgresql-9.4.1208.jre7.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-cli-1.2.jar, file:/opt/hive/lib/log4j-core-2.6.2.jar, file:/opt/hive/lib/maven-scm-provider-svn-commons-1.4.jar, file:/opt/hive/lib/datanucleus-rdbms-4.1.19.jar, file:/opt/hive/lib/hive-llap-server-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/log4j-1.2.17.jar, file:/opt/hive/lib/hive-llap-common-2.3.3-tests.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-math-2.2.jar, file:/opt/hive/lib/druid-server-0.9.2.jar, file:/opt/hive/lib/jcodings-1.0.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.4-tests.jar, file:/opt/hive/lib/snappy-java-1.0.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/paranamer-2.3.jar, file:/opt/hive/lib/jetty-servlets-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-collections-3.2.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/javax.inject-1.jar, file:/opt/hive/lib/maven-scm-provider-svnexe-1.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.8.4.jar, file:/opt/hive/conf/, file:/opt/hive/lib/rhino-1.7R5.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jaxb-api-2.2.2.jar, file:/opt/hive/bin/file, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/java-xmlbuilder-0.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/htrace-core4-4.0.1-incubating.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/guava-11.0.2.jar, file:/opt/hive/lib/geronimo-jta_1.1_spec-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/json-io-2.5.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar, file:/opt/hive/lib/druid-hdfs-storage-0.9.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-api-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jsr305-3.0.0.jar, file:/opt/hive/lib/disruptor-3.3.0.jar, file:/opt/hive/lib/dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar, file:/opt/hive/lib/jackson-datatype-guava-2.4.6.jar, file:/opt/hive/lib/metrics-core-2.2.0.jar, file:/opt/hive/lib/icu4j-4.8.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/fst-2.50.jar, file:/opt/hive/lib/hive-beeline-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/activation-1.1.jar, file:/opt/hive/lib/hive-shims-0.23-2.3.3.jar, file:/opt/hive/lib/hive-serde-2.3.3.jar, file:/opt/hive/lib/javax.inject-1.jar, file:/opt/hive/lib/airline-0.7.jar, file:/opt/hive/lib/bonecp-0.8.0.RELEASE.jar, file:/opt/hive/lib/hive-shims-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.8.4.jar, file:/opt/hive/lib/jetty-util-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar, file:/opt/hive/lib/jackson-databind-2.6.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar, file:/opt/hive/lib/jetty-continuation-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jersey-core-1.9.jar, file:/opt/hive/lib/jcommander-1.32.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-native-client-2.8.4.jar, file:/opt/hive/lib/jetty-server-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-codec-1.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-beanutils-1.7.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/slf4j-api-1.7.10.jar, file:/opt/hive/lib/aircompressor-0.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/netty-3.6.2.Final.jar, file:/opt/hive/lib/servlet-api-2.5-6.1.14.jar, file:/opt/hive/lib/aether-connector-file-0.9.0.M2.jar, file:/opt/hive/lib/hive-accumulo-handler-2.3.3.jar, file:/opt/hive/lib/jackson-module-jaxb-annotations-2.4.6.jar, file:/opt/hive/lib/jline-2.12.jar, file:/opt/hive/lib/commons-compiler-2.7.6.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/httpcore-4.4.4.jar, file:/opt/hive/lib/javax.servlet-api-3.1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-logging-1.1.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/hadoop-annotations-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.8.4.jar, file:/opt/hive/lib/calcite-druid-1.10.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar, file:/opt/hive/lib/ant-1.9.1.jar, file:/opt/hive/lib/protobuf-java-2.5.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/hadoop-common-2.8.4-tests.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/commons-io-2.4.jar, file:/opt/hive/lib/geronimo-jaspic_1.0_spec-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/json-smart-1.3.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-codec-1.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-client-2.8.4-tests.jar, file:/opt/hive/lib/mysql-connector-java.jar, file:/opt/hive/lib/guava-14.0.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar, file:/opt/hive/lib/parquet-hadoop-bundle-1.8.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/snappy-java-1.0.4.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/protobuf-java-2.5.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/activation-1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/avro-1.7.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/guava-11.0.2.jar, file:/opt/hive/lib/accumulo-trace-1.6.0.jar, file:/opt/hive/lib/netty-3.6.2.Final.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jsr305-3.0.0.jar, file:/opt/hive/lib/maven-model-3.1.1.jar, file:/opt/hive/lib/jpam-1.1.jar, file:/opt/hive/lib/httpclient-4.4.jar, file:/opt/hive/lib/jersey-server-1.14.jar, file:/opt/hive/lib/jsp-2.1-6.1.14.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar, file:/opt/hive/lib/hbase-hadoop2-compat-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jetty-6.1.26.jar, file:/opt/hive/lib/druid-console-0.0.2.jar, file:/opt/hive/lib/jersey-client-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/xmlenc-0.52.jar, file:/opt/hive/lib/hbase-hadoop-compat-1.1.1.jar, file:/opt/hive/lib/javax.el-api-3.0.0.jar, file:/opt/hive/lib/commons-cli-1.2.jar, file:/opt/hive/lib/asm-commons-3.1.jar, file:/opt/hadoop-2.8.4/etc/hadoop/, file:/opt/hive/lib/jsp-api-2.1.jar, file:/opt/hive/lib/hive-vector-code-gen-2.3.3.jar, file:/opt/hive/lib/paranamer-2.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/stax-api-1.0-2.jar, file:/opt/hive/lib/maven-scm-api-1.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/aopalliance-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar, file:/opt/hive/lib/log4j-jul-2.5.jar, file:/opt/hive/lib/hive-llap-client-2.3.3.jar, file:/opt/hive/lib/jackson-jaxrs-base-2.4.6.jar, file:/opt/hive/lib/tempus-fugit-1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/mockito-all-1.8.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-2.8.4.jar, file:/opt/hive/lib/calcite-core-1.10.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jersey-core-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/servlet-api-2.5.jar, file:/opt/hive/lib/accumulo-core-1.6.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-collections-3.2.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jcip-annotations-1.0-1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/log4j-1.2.17.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar, file:/opt/hive/lib/json-path-2.1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jsch-0.1.54.jar, file:/opt/hive/lib/spymemcached-2.11.7.jar, file:/opt/hive/lib/transaction-api-1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jetty-sslengine-6.1.26.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/curator-recipes-2.7.1.jar, file:/opt/hive/lib/jackson-jaxrs-smile-provider-2.4.6.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jersey-client-1.9.jar, file:/opt/hive/lib/commons-codec-1.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/curator-client-2.7.1.jar, file:/opt/hive/lib/tesla-aether-0.0.5.jar, file:/opt/hive/lib/bytebuffer-collections-0.2.5.jar, file:/opt/hive/lib/javax.servlet-3.0.0.v201112011016.jar, file:/opt/hive/lib/commons-math3-3.6.1.jar, file:/opt/hive/lib/maven-settings-builder-3.1.1.jar, file:/opt/hive/lib/classmate-1.0.0.jar, file:/opt/hive/lib/jetty-client-9.2.5.v20141112.jar, file:/opt/hive/lib/metrics-json-3.1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/netty-3.6.2.Final.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-codec-1.4.jar, file:/opt/hive/lib/ST4-4.0.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/tools/lib/hadoop-distcp-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jets3t-0.9.0.jar, file:/opt/hive/lib/maven-aether-provider-3.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar, file:/opt/hive/lib/hive-exec-2.3.3.jar, file:/opt/hive/lib/curator-client-2.7.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-compress-1.4.1.jar, file:/opt/hive/lib/maxminddb-0.2.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jettison-1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-lang-2.6.jar, file:/opt/hive/lib/jsp-api-2.0.jar, file:/opt/hive/lib/jackson-datatype-joda-2.4.6.jar, file:/opt/hive/lib/postgresql-metadata-storage-0.9.2.jar, file:/opt/hive/lib/velocity-1.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/okhttp-2.4.0.jar, file:/opt/hive/lib/avro-1.7.7.jar, file:/opt/hadoop-2.8.4/contrib/capacity-scheduler/*.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/paranamer-2.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar, file:/opt/hive/lib/netty-all-4.0.52.Final.jar, file:/opt/hive/lib/libthrift-0.9.3.jar, file:/opt/hive/lib/hive-common-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar, file:/opt/hive/lib/hive-shims-common-2.3.3.jar, file:/opt/hive/lib/jackson-jaxrs-1.9.13.jar, file:/opt/hive/lib/jasper-compiler-5.5.23.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jsr305-3.0.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jetty-6.1.26.jar, file:/opt/hive/lib/jta-1.1.jar, file:/opt/hive/lib/commons-pool2-2.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/servlet-api-2.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-logging-1.1.3.jar, file:/opt/hive/lib/jsp-api-2.1-6.1.14.jar, file:/opt/hive/lib/jamon-runtime-2.3.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-common-2.8.4.jar, file:/opt/hive/lib/htrace-core-3.1.0-incubating.jar, file:/opt/hive/lib/jersey-guice-1.19.jar, file:/opt/hive/lib/jetty-util-6.1.26.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/htrace-core4-4.0.1-incubating.jar, file:/opt/hive/lib/commons-io-2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/hadoop-annotations-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar, file:/opt/hive/lib/commons-lang3-3.1.jar, file:/opt/hive/lib/slider-core-0.90.2-incubating.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-cli-1.2.jar, file:/opt/hive/lib/derbyclient-10.11.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-io-2.4.jar, file:/opt/hive/lib/commons-lang-2.6.jar, file:/opt/hive/lib/commons-collections-3.2.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar, file:/opt/hive/lib/lz4-1.3.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-lang-2.6.jar, file:/opt/hive/lib/accumulo-fate-1.6.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/xmlenc-0.52.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.8.4.jar, file:/opt/hive/lib/hive-hbase-handler-2.3.3.jar, file:/opt/hive/lib/druid-api-0.9.2.jar, file:/opt/hive/lib/aether-spi-0.9.0.M2.jar, file:/opt/hive/lib/hive-llap-tez-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-net-3.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar, file:/opt/hive/lib/servlet-api-2.4.jar, file:/opt/hive/lib/jackson-annotations-2.6.0.jar, file:/opt/hive/lib/hive-druid-handler-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/api-util-1.0.0-M20.jar, file:/opt/hive/lib/config-magic-0.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/asm-3.2.jar, file:/opt/hive/lib/hive-hcatalog-server-extensions-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/zookeeper-3.4.6.jar, file:/opt/hive/lib/hive-hcatalog-core-2.3.3.jar, file:/opt/hive/lib/datanucleus-api-jdo-4.2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/commons-compress-1.4.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.8.4.jar, file:/opt/hive/lib/mysql-metadata-storage-0.9.2.jar, file:/opt/hive/lib/jackson-core-2.6.5.jar, file:/opt/hive/lib/eigenbase-properties-1.1.5.jar, file:/opt/hive/lib/curator-recipes-2.7.1.jar, file:/opt/hive/lib/geronimo-annotation_1.0_spec-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/hadoop-common-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-2.8.4-tests.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-io-2.4.jar, file:/opt/hive/lib/hbase-common-1.1.1-tests.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-native-client-2.8.4-tests.jar, file:/opt/hive/lib/hbase-client-1.1.1.jar, file:/opt/hive/lib/jettison-1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/xz-1.0.jar, file:/opt/hive/lib/jetty-io-9.2.5.v20141112.jar, file:/opt/hive/lib/plexus-utils-3.0.15.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/servlet-api-2.5.jar, file:/opt/hive/lib/RoaringBitmap-0.5.18.jar, file:/opt/hive/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar, file:/opt/hive/lib/hive-storage-api-2.4.0.jar, file:/opt/hive/lib/hive-shims-scheduler-2.3.3.jar, file:/opt/hive/lib/geoip2-0.4.0.jar, file:/opt/hive/lib/opencsv-2.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/guice-3.0.jar, file:/opt/hive/lib/compress-lzf-1.0.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar, file:/opt/hive/lib/hbase-server-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/guava-11.0.2.jar, file:/opt/hive/lib/hbase-procedure-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-client-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/okio-1.4.0.jar, file:/opt/hive/lib/groovy-all-2.4.4.jar, file:/opt/hive/lib/gson-2.2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/curator-framework-2.7.1.jar, file:/opt/hive/lib/ant-launcher-1.9.1.jar, file:/opt/hive/lib/json-1.8.jar, file:/opt/hive/lib/datanucleus-core-4.1.17.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar, file:/opt/hive/lib/hive-service-2.3.3.jar, file:/opt/hive/lib/jetty-6.1.26.jar, file:/opt/hive/lib/antlr-runtime-3.5.2.jar, file:/opt/hive/lib/metrics-jvm-3.1.0.jar, file:/opt/hive/lib/commons-el-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-math3-3.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/java-util-1.9.0.jar, file:/opt/hive/lib/hive-llap-common-2.3.3.jar, file:/opt/hive/lib/emitter-0.3.6.jar, file:/opt/hive/lib/findbugs-annotations-1.3.9-1.jar, file:/opt/hive/lib/commons-math-2.2.jar, file:/opt/hive/lib/maven-model-builder-3.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-configuration-1.6.jar, file:/var/hoodie/ws/docker/hoodie/hadoop/hive_base/target/hoodie-hadoop-mr-bundle.jar, file:/opt/hive/lib/jackson-jaxrs-json-provider-2.4.6.jar, file:/opt/hive/lib/jetty-proxy-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/curator-test-2.7.1.jar, file:/opt/hive/lib/hbase-protocol-1.1.1.jar, file:/opt/hive/lib/joni-2.1.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar, file:/opt/hive/lib/asm-3.1.jar, file:/opt/hive/lib/maven-settings-3.1.1.jar, file:/opt/hive/lib/httpcore-4.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jersey-server-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/avro-1.7.4.jar, file:/opt/hive/lib/guice-servlet-4.1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar, file:/opt/hive/lib/derbynet-10.11.1.1.jar, file:/opt/hive/lib/accumulo-start-1.6.0.jar, file:/opt/hive/lib/hive-service-rpc-2.3.3.jar, file:/opt/hive/lib/javolution-5.5.1.jar, file:/opt/hive/lib/wagon-provider-api-2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/asm-3.2.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/log4j-1.2.17.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-lang-2.6.jar, file:/opt/hive/lib/curator-x-discovery-2.11.0.jar, file:/opt/hive/lib/jdbi-2.63.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.8.4.jar, file:/opt/hive/lib/ivy-2.4.0.jar, file:/opt/hive/lib/jdo-api-3.0.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/hamcrest-core-1.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/asm-3.2.jar, file:/opt/hive/lib/mail-1.4.1.jar, file:/opt/hive/lib/commons-dbcp2-2.0.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jersey-server-1.9.jar, file:/opt/hive/lib/hive-llap-ext-client-2.3.3.jar, file:/opt/hive/lib/jetty-http-9.2.5.v20141112.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-tests-2.8.4.jar, file:/opt/hive/lib/commons-dbcp-1.4.jar, file:/opt/hive/lib/zookeeper-3.4.6.jar, file:/opt/hive/lib/jetty-servlet-9.2.5.v20141112.jar, file:/opt/hive/lib/hbase-annotations-1.1.1.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/jersey-server-1.9.jar, file:/opt/hive/lib/extendedset-1.3.10.jar, file:/opt/hive/lib/curator-framework-2.7.1.jar, file:/opt/hive/lib/jetty-security-9.2.5.v20141112.jar, file:/opt/hive/lib/activation-1.1.jar, file:/opt/hive/lib/aether-impl-0.9.0.M2.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar, file:/opt/hive/lib/aether-connector-okhttp-0.0.9.jar, file:/opt/hive/lib/irc-api-1.0-0014.jar, file:/opt/hive/lib/postgresql-jdbc.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/zookeeper-3.4.6.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/commons-digester-1.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jetty-6.1.26.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/junit-4.11.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/commons-cli-1.2.jar, file:/opt/hive/lib/hbase-prefix-tree-1.1.1.jar, file:/opt/hive/lib/jboss-logging-3.1.3.GA.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar, file:/opt/hive/lib/ant-1.6.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/gson-2.2.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/javassist-3.18.1-GA.jar, file:/opt/hive/lib/commons-httpclient-3.0.1.jar, file:/opt/hive/lib/stax-api-1.0.1.jar, file:/opt/hive/lib/guice-multibindings-4.1.0.jar, file:/opt/hive/lib/hive-jdbc-2.3.3.jar, file:/opt/hive/lib/okhttp-1.0.2.jar, file:/opt/hive/lib/server-metrics-0.2.8.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jetty-util-6.1.26.jar, file:/opt/hadoop-2.8.4/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.8.4.jar, file:/opt/hive/lib/super-csv-2.2.0.jar, file:/opt/hive/lib/HikariCP-2.5.1.jar, file:/opt/hive/lib/jetty-sslengine-6.1.26.jar, file:/opt/hive/lib/janino-2.7.6.jar, file:/opt/hive/lib/plexus-interpolation-1.19.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jersey-json-1.9.jar, file:/opt/hive/lib/hive-contrib-2.3.3.jar, file:/opt/hive/lib/jackson-xc-1.9.13.jar, file:/opt/hive/lib/org.abego.treelayout.core-1.0.1.jar, file:/opt/hive/lib/commons-vfs2-2.0.jar, file:/opt/hive/lib/calcite-linq4j-1.10.0.jar, file:/opt/hive/lib/javax.jdo-3.2.0-m3.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-client-2.8.4.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jackson-xc-1.9.13.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jetty-util-6.1.26.jar, file:/opt/hive/lib/avatica-1.8.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/lib/jersey-server-1.9.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.8.4.jar, file:/opt/hive/lib/commons-pool-1.5.4.jar, file:/opt/hive/lib/jsr305-3.0.0.jar, file:/opt/hive/lib/jasper-runtime-5.5.23.jar, file:/opt/hadoop-2.8.4/share/hadoop/common/lib/jsp-api-2.1.jar, file:/opt/hive/lib/hive-metastore-2.3.3.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/xz-1.0.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/, file:/opt/hive/lib/log4j-1.2-api-2.6.2.jar, file:/opt/hive/lib/java-util-0.27.10.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jettison-1.1.jar, file:/opt/hive/lib/hibernate-validator-5.1.3.Final.jar, file:/opt/hive/lib/antlr4-runtime-4.5.jar, file:/opt/hadoop-2.8.4/share/hadoop/yarn/lib/jersey-guice-1.9.jar, file:/opt/hive/lib/orc-core-1.3.3.jar, file:/opt/hive/lib/hive-jdbc-handler-2.3.3.jar, file:/opt/hive/lib/derby-10.10.2.0.jar, file:/opt/hive/lib/regexp-1.3.jar, file:/opt/hive/lib/metrics-core-3.1.0.jar, file:/opt/hive/lib/druid-common-0.9.2.jar, file:/opt/hive/lib/validation-api-1.1.0.Final.jar, file:/opt/hadoop-2.8.4/share/hadoop/hdfs/hadoop-hdfs-nfs-2.8.4.jar]
2021-07-28T05:35:26,724 INFO [LocalJobRunner Map Task Executor #0]: mr.ExecMapper (ExecMapper.java:configure(81)) - thread classpath = [file:/tmp/hadoop-root/mapred/local/1627450524786/hoodie-hadoop-mr-bundle.jar]
2021-07-28T05:35:26,724 INFO [LocalJobRunner Map Task Executor #0]: exec.Utilities (Utilities.java:getBaseWork(407)) - PLAN PATH = file:/tmp/root/43454311-cbd4-4a02-b5fd-87cfcc2f896a/hive_2021-07-28_05-35-17_794_8926989242932757063-2/-mr-10005/8f90f766-b571-4ee2-b72b-937490a5ae15/map.xml
2021-07-28T05:35:26,728 INFO [LocalJobRunner Map Task Executor #0]: exec.MapOperator (Operator.java:initialize(334)) - Initializing operator MAP[0]
2021-07-28T05:35:26,737 INFO [LocalJobRunner Map Task Executor #0]: mr.ExecMapper (ExecMapper.java:configure(107)) -
<MAP>Id =0
<Children>null
<\Children>
<Parent><\Parent>
<\MAP>
2021-07-28T05:35:26,742 INFO [LocalJobRunner Map Task Executor #0]: exec.TableScanOperator (Operator.java:initialize(334)) - Initializing operator TS[0]
2021-07-28T05:35:26,742 INFO [LocalJobRunner Map Task Executor #0]: exec.FilterOperator (Operator.java:initialize(334)) - Initializing operator FIL[9]
2021-07-28T05:35:26,752 INFO [LocalJobRunner Map Task Executor #0]: exec.SelectOperator (Operator.java:initialize(334)) - Initializing operator SEL[2]
2021-07-28T05:35:26,753 INFO [LocalJobRunner Map Task Executor #0]: exec.SelectOperator (SelectOperator.java:initializeOp(73)) - SELECT struct<_hoodie_commit_time:string,_hoodie_commit_seqno:string,_hoodie_record_key:string,_hoodie_partition_path:string,_hoodie_file_name:string,volume:bigint,ts:string,symbol:string,year:int,month:string,high:double,low:double,key:string,date:string,close:double,open:double,day:string,dt:string>
2021-07-28T05:35:26,753 INFO [LocalJobRunner Map Task Executor #0]: exec.GroupByOperator (Operator.java:initialize(334)) - Initializing operator GBY[4]
2021-07-28T05:35:26,762 INFO [LocalJobRunner Map Task Executor #0]: exec.GroupByOperator (GroupByOperator.java:computeMaxEntriesHashAggr(436)) - Max hash table memory: 126877696 bytes
2021-07-28T05:35:26,766 INFO [LocalJobRunner Map Task Executor #0]: exec.GroupByOperator (GroupByOperator.java:initializeOp(417)) - isTez: false isLlap: false numExecutors: 1 maxMemory: 253755392
2021-07-28T05:35:26,766 INFO [LocalJobRunner Map Task Executor #0]: exec.ReduceSinkOperator (Operator.java:initialize(334)) - Initializing operator RS[5]
2021-07-28T05:35:26,767 INFO [LocalJobRunner Map Task Executor #0]: exec.ReduceSinkOperator (ReduceSinkOperator.java:initializeOp(219)) - Using tag = -1
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.MapOperator (Operator.java:close(690)) - Closing operator MAP[0]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.MapOperator (Operator.java:logStats(946)) - DESERIALIZE_ERRORS:0, RECORDS_IN:0,
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.TableScanOperator (Operator.java:close(690)) - Closing operator TS[0]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.FilterOperator (Operator.java:close(690)) - Closing operator FIL[9]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.SelectOperator (Operator.java:close(690)) - Closing operator SEL[2]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.GroupByOperator (Operator.java:close(690)) - Closing operator GBY[4]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.ReduceSinkOperator (Operator.java:close(690)) - Closing operator RS[5]
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.ReduceSinkOperator (ReduceSinkOperator.java:closeOp(547)) - RS[5]: records written - 0
2021-07-28T05:35:26,811 INFO [LocalJobRunner Map Task Executor #0]: exec.ReduceSinkOperator (Operator.java:logStats(946)) - RECORDS_OUT_INTERMEDIATE:0,
2021-07-28T05:35:26,813 INFO [Thread-53]: mapred.LocalJobRunner (LocalJobRunner.java:runTasks(483)) - map task executor complete.
2021-07-28T05:35:26,813 WARN [Thread-53]: mapred.LocalJobRunner (LocalJobRunner.java:run(587)) - job_local1579169718_0001
java.lang.Exception: java.io.IOException: java.lang.NullPointerException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:489) ~[hadoop-mapreduce-client-common-2.8.4.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:549) ~[hadoop-mapreduce-client-common-2.8.4.jar:?]
Caused by: java.io.IOException: java.lang.NullPointerException
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:199) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:185) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:52) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) ~[hadoop-mapreduce-client-common-2.8.4.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
Caused by: java.lang.NullPointerException
at org.apache.hudi.hadoop.realtime.RealtimeCompactedRecordReader.next(RealtimeCompactedRecordReader.java:112) ~[hoodie-hadoop-mr-bundle.jar:0.9.0-SNAPSHOT]
at org.apache.hudi.hadoop.realtime.RealtimeCompactedRecordReader.next(RealtimeCompactedRecordReader.java:49) ~[hoodie-hadoop-mr-bundle.jar:0.9.0-SNAPSHOT]
at org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader.next(HoodieRealtimeRecordReader.java:79) ~[hoodie-hadoop-mr-bundle.jar:0.9.0-SNAPSHOT]
at org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader.next(HoodieRealtimeRecordReader.java:36) ~[hoodie-hadoop-mr-bundle.jar:0.9.0-SNAPSHOT]
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:360) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116) ~[hive-exec-2.3.3.jar:2.3.3]
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:199) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:185) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:52) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) ~[hadoop-mapreduce-client-core-2.8.4.jar:?]
at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:270) ~[hadoop-mapreduce-client-common-2.8.4.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_212]
2021-07-28T05:35:27,327 WARN [HiveServer2-Background-Pool: Thread-55]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2021-07-28T05:35:27,335 ERROR [HiveServer2-Background-Pool: Thread-55]: exec.Task (SessionState.java:printError(1126)) - Ended Job = job_local1579169718_0001 with errors
2021-07-28T05:35:27,338 ERROR [Thread-75]: exec.Task (SessionState.java:printError(1126)) - Error during job, obtaining debugging information...
2021-07-28T05:35:27,338 INFO [Thread-75]: Configuration.deprecation (Configuration.java:logDeprecation(1285)) - mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2021-07-28T05:35:27,359 ERROR [HiveServer2-Background-Pool: Thread-55]: ql.Driver (SessionState.java:printError(1126)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
2021-07-28T05:35:27,360 INFO [HiveServer2-Background-Pool: Thread-55]: ql.Driver (SessionState.java:printInfo(1111)) - MapReduce Jobs Launched:
2021-07-28T05:35:27,366 WARN [HiveServer2-Background-Pool: Thread-55]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2021-07-28T05:35:27,368 INFO [HiveServer2-Background-Pool: Thread-55]: ql.Driver (SessionState.java:printInfo(1111)) - Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL
2021-07-28T05:35:27,369 INFO [HiveServer2-Background-Pool: Thread-55]: ql.Driver (SessionState.java:printInfo(1111)) - Total MapReduce CPU Time Spent: 0 msec
2021-07-28T05:35:27,369 INFO [HiveServer2-Background-Pool: Thread-55]: ql.Driver (Driver.java:execute(2050)) - Completed executing command(queryId=root_20210728053517_938b3d8c-ed32-4a52-a86b-d73475255bdf); Time taken: 5.068 seconds
2021-07-28T05:35:27,381 ERROR [HiveServer2-Background-Pool: Thread-55]: operation.Operation (SQLOperation.java:run(352)) - Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:380) ~[hive-service-2.3.3.jar:2.3.3]
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:257) ~[hive-service-2.3.3.jar:2.3.3]
at org.apache.hive.service.cli.operation.SQLOperation.access$800(SQLOperation.java:91) ~[hive-service-2.3.3.jar:2.3.3]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:348) ~[hive-service-2.3.3.jar:2.3.3]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_212]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_212]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1840) ~[hadoop-common-2.8.4.jar:?]
at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:362) ~[hive-service-2.3.3.jar:2.3.3]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]
2021-07-28T05:35:27,388 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,388 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:27,388 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,389 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:27,390 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,390 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:27,390 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,390 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:27,403 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,404 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:35:27,404 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: operation.OperationManager (OperationManager.java:closeOperation(284)) - Closing operation: OperationHandle [opType=EXECUTE_STATEMENT, getHandleIdentifier()=deaa8edb-9cf8-4eaf-8ff2-eb3bb16cdabb]
2021-07-28T05:35:27,404 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: exec.ListSinkOperator (Operator.java:close(690)) - Closing operator LIST_SINK[10]
2021-07-28T05:35:27,406 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:35:27,407 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: thrift.ThriftCLIService (ThriftBinaryCLIService.java:deleteContext(137)) - Session disconnected without closing properly.
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: thrift.ThriftCLIService (ThriftBinaryCLIService.java:deleteContext(141)) - Closing the session: SessionHandle [43454311-cbd4-4a02-b5fd-87cfcc2f896a]
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: service.CompositeService (SessionManager.java:closeSession(446)) - Session closed, SessionHandle [43454311-cbd4-4a02-b5fd-87cfcc2f896a], current sessions:0
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:updateThreadName(421)) - Updating thread name to 43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:36:16,943 INFO [43454311-cbd4-4a02-b5fd-87cfcc2f896a HiveServer2-Handler-Pool: Thread-46]: conf.HiveConf (HiveConf.java:getLogIdVar(3944)) - Using the default value passed in for log id: 43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:36:16,943 INFO [HiveServer2-Handler-Pool: Thread-46]: session.HiveSessionImpl (HiveSessionImpl.java:cleanupSessionLogDir(769)) - Operation log session directory is deleted: /tmp/root/operation_logs/43454311-cbd4-4a02-b5fd-87cfcc2f896a
2021-07-28T05:36:16,944 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:resetThreadName(432)) - Resetting thread name to HiveServer2-Handler-Pool: Thread-46
2021-07-28T05:36:16,952 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:dropPathAndUnregisterDeleteOnExit(838)) - Deleted directory: /tmp/hive/root/43454311-cbd4-4a02-b5fd-87cfcc2f896a on fs with scheme hdfs
2021-07-28T05:36:16,952 INFO [HiveServer2-Handler-Pool: Thread-46]: session.SessionState (SessionState.java:dropPathAndUnregisterDeleteOnExit(838)) - Deleted directory: /tmp/root/43454311-cbd4-4a02-b5fd-87cfcc2f896a on fs with scheme file
2021-07-28T05:36:16,953 INFO [HiveServer2-Handler-Pool: Thread-46]: hive.metastore (HiveMetaStoreClient.java:close(564)) - Closed a connection to metastore, current connections: 1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment