Last active
December 10, 2020 21:50
-
-
Save bentito/3c64e21d3d0ad6f121b6de3c4cb81d4e to your computer and use it in GitHub Desktop.
Hadoop + Hive Build/Runtime issues for Metering Operator -- trying to get the tip of Hadoop 3.3 branch and latest Hive master (Oct 15, 2020) to work to see if will fix a bug related to S3 bucket contents deletion
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| │ Exception in thread "main" 2020-10-13T17:23:10.464510059Z java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V2020-10-13T17:23:10.464518738Z │ | |
| │ at org.apache.hadoop.conf.Configuration.set(Configuration.java:1382)2020-10-13T17:23:10.464616395Z │ | |
| │ at org.apache.hadoop.conf.Configuration.set(Configuration.java:1363)2020-10-13T17:23:10.464630106Z │ | |
| │ at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)2020-10-13T17:23:10.464669465Z │ | |
| │ at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554) │ | |
| │ at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)2020-10-13T17:23:10.464698814Z │ | |
| │ at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4047)2020-10-13T17:23:10.464708189Z │ | |
| │ at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:4010) │ | |
| │ at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7002)2020-10-13T17:23:10.464737483Z │ | |
| │ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) │ | |
| │ at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)2020-10-13T17:23:10.464765419Z │ | |
| │ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) │ | |
| │ at java.lang.reflect.Method.invoke(Method.java:498)2020-10-13T17:23:10.464793618Z │ | |
| │ at org.apache.hadoop.util.RunJar.run(RunJar.java:323)2020-10-13T17:23:10.464803494Z │ | |
| │ at org.apache.hadoop.util.RunJar.main(RunJar.java:236) │ | |
| │ stream closed |
With the above hive.non-managed-table-writes-enabled: true was able to see data appear in reportdatasources for S3 mode.
But report failed. In reporting-opr errors look like:
│ Message: error occurred while generating report: Failed to execute query for Report table hive.metering.report_btofel_pod_request_cpu_cores_immed: presto: │
│ query failed (200 OK): "io.prestosql.spi.PrestoException: localhost:9083"
In Presto errors look like:
Caused by: org.apache.thrift.TApplicationException: Required field 'engine' is unset! Struct:PartitionsStatsRequest(dbName:metering, tblName:datasource_btofel_pod_limit_cpu_ │
│ at org.apache.thrift.TApplicationException.read(TApplicationException.java:111)
or
│ 2020-10-22T14:28:15.136Z ERROR SplitRunner-6-108 io.prestosql.execution.executor.TaskExecutor Error processing Split 20201022_142804_00054_2yc4a.0.0-16 (start = 6120586.921251, wall = 10245 ms, cpu = 0 │
│ ms, wait = 4 ms, calls = 4): HIVE_METASTORE_ERROR: Required field 'engine' is unset! Struct:PartitionsStatsRequest(dbName:metering, tblName:datasource_btofel_pod_limit_memory_bytes, colNames:[amount, timestamp, ti │
│ eprecision, labels[], partNames:[dt=2020-10-22], engine:null)
Need to get another baseline that reports are actually working with HDFS storage...
reports failing with Hive 4.0.0-SNAPSHOT, trying again with Hive 3.1.2
Error on podman build Hive:
STEP 59: COPY --from=build /build/packaging/target/apache-hive-$HIVE_VERSION-bin/apache-hive-$HIVE_VERSION-bin $HIVE_HOME
Error: error dry-running "COPY --from=build /build/packaging/target/apache-hive-$HIVE_VERSION-bin/apache-hive-$HIVE_VERSION-bin $HIVE_HOME": no files found mat$
hing "/home/mail_reknew_gmail_com/.local/share/containers/storage/overlay/d0802985299d299688537d7744ab87fccc365d45a268fa3f9cfd5ebf7c8748ae/merged/build/packagi$
g/target/apache-hive--bin/apache-hive--bin": no such file or directory
HDFS:
With logLevel: debug on the metastore container, grep for report_btofel_pod_request_cpu_cores_immed:
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_p │
│ od_request_cpu_cores_immed │
☝️ after deleting the report
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName │
│ :metering, owner:hadoop, createTime:1603807397, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), Field │
│ Schema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema( │
│ name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.or │
│ c.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache. │
│ hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], │
│ skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"tr │
│ ue","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_ve │
│ rsion=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT │
│ , createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true) │
│ , PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop │
│ , grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER) │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_po │
│ d_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603807397, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, typ │
│ e:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:st │
│ ring, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputForma │
│ t:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:n │
│ ull, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewe │
│ dColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_ │
│ STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":" │
│ true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[Pr │
│ ivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, gr │
│ antorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELE │
│ TE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER) │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO utils.FileUtils: Creating directory if it doesn't exist: hdfs://hdfs-namenode-0.hdfs-namenode:9820/user/hive/warehouse/metering.db │
│ /report_btofel_pod_request_cpu_cores_immed │
☝️ after applying the report again
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_ │
│ pod_request_cpu_cores_immed │
│ 20/10/27 14:03:19 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:19 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_ │
│ pod_request_cpu_cores_immed
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu │
│ _cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.meterin │
│ g.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_ │
│ pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu │
│ _cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.meterin │
│ g.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod_request_cpu_cores_immed newtbl │
│ =report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod │
│ _request_cpu_cores_immed newtbl=report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO utils.MetaStoreUtils: Updating table stats for report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:20 [pool-12-thread-197]: INFO utils.MetaStoreUtils: Updated size of table report_btofel_pod_request_cpu_cores_immed to 6171 │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:46 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_p │
│ od_request_cpu_cores_immed
HDFS:
time period around 1st report delete (regex: 14:00:2[0-9]):
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Cleaning up thread local RawStore... │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Cleaning up thread local RawStore... │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Done cleaning up thread local RawStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Done cleaning up thread local RawStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Done cleaning up thread local RawStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Done cleaning up thread local RawStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: WARN metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.ObjectStore: ObjectStore, initialize called │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.ObjectStore: Initialized ObjectStore │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:00:28 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_p │
│ od_request_cpu_cores_immed
HDFS:
time period around report applied 2nd time (regex: 14:03:1[7-8]):
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Cleaning up thread local RawStore... │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Cleaning up thread local RawStore... │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Done cleaning up thread local RawStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Done cleaning up thread local RawStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Done cleaning up thread local RawStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Done cleaning up thread local RawStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: WARN metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.ObjectStore: ObjectStore, initialize called │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.ObjectStore: Initialized ObjectStore │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_po │
│ d_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_database: @hive#metering │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_database: @hive#metering │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName │
│ :metering, owner:hadoop, createTime:1603807397, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), Field │
│ Schema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema( │
│ name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.or │
│ c.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache. │
│ hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[], skewedColValues:[], │
│ skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"tr │
│ ue","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_ve │
│ rsion=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT │
│ , createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true) │
│ , PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop │
│ , grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER) │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_po │
│ d_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603807397, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, typ │
│ e:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:st │
│ ring, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputForma │
│ t:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:n │
│ ull, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:[], sortCols:[], parameters:{}, skewedInfo:SkewedInfo(skewe │
│ dColNames:[], skewedColValues:[], skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:[], parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_ │
│ STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":" │
│ true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[Pr │
│ ivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, gr │
│ antorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELE │
│ TE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER) │
│ 20/10/27 14:03:17 [pool-12-thread-6]: INFO utils.FileUtils: Creating directory if it doesn't exist: hdfs://hdfs-namenode-0.hdfs-namenode:9820/user/hive/warehouse/metering.db │
│ /report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_ │
│ pod_request_cpu_cores_immed │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore │
│ 20/10/27 14:03:17 [pool-12-thread-197]: WARN metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.ObjectStore: ObjectStore, initialize called
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.ObjectStore: Initialized ObjectStore │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: Cleaning up thread local RawStore... │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Cleaning up thread local RawStore... │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO metastore.HiveMetaStore: 196: Done cleaning up thread local RawStore │
│ 20/10/27 14:03:17 [pool-12-thread-197]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=Done cleaning up thread local RawStore │
│ 20/10/27 14:03:18 [pool-12-thread-3]: INFO metastore.HiveMetaStore: 3: source:0:0:0:0:0:0:0:1 get_config_value: name=metastore.batch.retrieve.max defaultValue=50 │
│ 20/10/27 14:03:18 [pool-12-thread-3]: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_config_value: name=metastore.batch.retriev │
│ e.max defaultValue=50
S3: log grepped for: pod_request_cpu_cores_immed:
20/10/27 15:31:06 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:06 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:06 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603812666, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:, sortCols:, parameters:{}, skewedInfo:SkewedInfo(skewedColNames:, skewedColValues:, skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:, parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER)
20/10/27 15:31:06 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603812666, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:, sortCols:, parameters:{}, skewedInfo:SkewedInfo(skewedColNames:, skewedColValues:, skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:, parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER)
20/10/27 15:31:06 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:06 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:08 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:08 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: WARN metastore.MetaStoreDirectSql: Failed to execute [select "COLUMN_NAME", "COLUMN_TYPE", "LONG_LOW_VALUE", "LONG_HIGH_VALUE", "DOUBLE_LOW_VALUE", "DOUBLE_HIGH_VALUE", "BIG_DECIMAL_LOW_VALUE", "BIG_DECIMAL_HIGH_VALUE", "NUM_NULLS", "NUM_DISTINCTS", '', "AVG_COL_LEN", "MAX_COL_LEN", "NUM_TRUES", "NUM_FALSES", "LAST_ANALYZED" from "TAB_COL_STATS" where "CAT_NAME" = ? and "DB_NAME" = ? and "TABLE_NAME" = ? and "COLUMN_NAME" in (?,?,?,?,?,?)] with parameters [hive, metering, report_btofel_pod_request_cpu_cores_immed, period_start, period_end, pod, namespace, node, pod_request_cpu_core_seconds
20/10/27 15:31:10 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod_request_cpu_cores_immed newtbl=report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod_request_cpu_cores_immed newtbl=report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO utils.MetaStoreUtils: Updating table stats for report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:10 [pool-10-thread-7: INFO utils.MetaStoreUtils: Updated size of table report_btofel_pod_request_cpu_cores_immed to 17092
20/10/27 15:31:50 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:31:50 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 drop_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:15 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:15 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:15 [pool-10-thread-6: INFO metastore.HiveMetaStore: 6: source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603812735, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:, sortCols:, parameters:{}, skewedInfo:SkewedInfo(skewedColNames:, skewedColValues:, skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:, parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER)
20/10/27 15:32:15 [pool-10-thread-6: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 create_table: Table(tableName:report_btofel_pod_request_cpu_cores_immed, dbName:metering, owner:hadoop, createTime:1603812735, lastAccessTime:0, retention:0, sd:StorageDescriptor(cols:[FieldSchema(name:period_start, type:timestamp, comment:null), FieldSchema(name:period_end, type:timestamp, comment:null), FieldSchema(name:pod, type:string, comment:null), FieldSchema(name:namespace, type:string, comment:null), FieldSchema(name:node, type:string, comment:null), FieldSchema(name:pod_request_cpu_core_seconds, type:double, comment:null)], location:null, inputFormat:org.apache.hadoop.hive.ql.io.orc.OrcInputFormat, outputFormat:org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat, compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null, serializationLib:org.apache.hadoop.hive.ql.io.orc.OrcSerde, parameters:{serialization.format=1}), bucketCols:, sortCols:, parameters:{}, skewedInfo:SkewedInfo(skewedColNames:, skewedColValues:, skewedColValueLocationMaps:{}), storedAsSubDirectories:false), partitionKeys:, parameters:{totalSize=0, numRows=0, rawDataSize=0, COLUMN_STATS_ACCURATE={"BASIC_STATS":"true","COLUMN_STATS":{"namespace":"true","node":"true","period_end":"true","period_start":"true","pod":"true","pod_request_cpu_core_seconds":"true"}}, numFiles=0, bucketing_version=2}, viewOriginalText:null, viewExpandedText:null, tableType:MANAGED_TABLE, privileges:PrincipalPrivilegeSet(userPrivileges:{hadoop=[PrivilegeGrantInfo(privilege:INSERT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:SELECT, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:UPDATE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true), PrivilegeGrantInfo(privilege:DELETE, createTime:-1, grantor:hadoop, grantorType:USER, grantOption:true)]}, groupPrivileges:null, rolePrivileges:null), temporary:false, catName:hive, ownerType:USER)
20/10/27 15:32:15 [pool-10-thread-6: INFO utils.FileUtils: Creating directory if it doesn't exist: s3a://btofel-s3-test-2/metering.db/report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:15 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:15 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:17 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:17 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table : tbl=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 get_table_statistics_req: table=hive.metering.report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO metastore.HiveMetaStore: 7: source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod_request_cpu_cores_immed newtbl=report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO HiveMetaStore.audit: ugi=hadoop ip=0:0:0:0:0:0:0:1 cmd=source:0:0:0:0:0:0:0:1 alter_table: hive.metering.report_btofel_pod_request_cpu_cores_immed newtbl=report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO utils.MetaStoreUtils: Updating table stats for report_btofel_pod_request_cpu_cores_immed
20/10/27 15:32:18 [pool-10-thread-7: INFO utils.MetaStoreUtils: Updated size of table report_btofel_pod_request_cpu_cores_immed to 5436
With debugging actually turned on: (full log with report present, then delete, then delete again) https://pastebin.com/Yeb5Ev8x
Red Hat link to full debug log: https://drive.google.com/file/d/1K9bY6rI0ezNRB3Y1zYIsRTgXUH68djFm/view?usp=sharing
More recently we'd like to upgrade protobuf. As such, moving to Hadoop rel/release-3.3.0
full rebase results in broken container build for Metering's Hadoop fork.
Now trying, checkout release and cherry-pick on all our commits.
Container build works at the 3.3.0 release commit.
So trying bisect strategy (in practice, but not git bisect)
* = build worked
** = trying/tried
*** = failed
0aa8f69ed5818798d4897cbf7a707dc604673b8e (HEAD -> origin/from_rel-3.3.0, origin/metering-rel-rel-3.3.0, metering-rel-rel-3.3.0) Container mage build to Hadoop 3.3.0, pb 3.7.1
87e2066cc8246feb78303990d948a57d9cff1269 Allow hadoop to use ubi7 for base image in CI
356f0ddd422be0c60a77409c0d6be32296138eea Updating Dockerfile.rhel baseimages to mach ocp-build-data config
e6fefeb7e035df4dbb6c2631e206a73e0109c121 Prevent cmake symlink creation if file is present
97618a82600b49ee1b039004fb64fcccd2c3a2c0 Ensure the jmx_prometheus_javaagent jar is moved to the correct destination path.
2f730859a95f2fdbd05bc9547acb40e393c17612 Allow undeclared variables
5d30807074e1420e220d4acd7a550ed6e98fd15b Restoring Dockerfile
0c879f141e6c7160fff0b0555e6cc72b4881a7d6 Use a rhel7 builder to ensure that protobuf 2.5.0 is available
fc2c1aeb64e79c0794d808a93b7c5740a0a65ca4 Symlink Dockerfile->Dockerfile.rhel
e16976ec1f6b8911f7772b755d986f0370bc41a8 Create & use conditional build script
ea67c6affc1dd12aa9ad17d4425357fbeced26dd support s390x in mapreduce-client-nativetask
6cde32cd2e257aebb9e663747ab451d35b55e5ee Update OWNERS.
c49ba3b0e7eb23d8e789ed168478bd3fab5eeb02 Dockerfile: Update the Dockerfile maintainer to be the mailing list.
d4b003f0eeac8f6da0744a93f3186413a47d0b58 Dockerfile: Stop escalating /etc/passwd permissions.
c675e659f1b1e21221db070fadca2c987f0a23ed Revert "Bug 1793313: Stop giving /etc/passwd incorrect priviledges."
44d23ce634a772fef799cd14dca96f1475910df4 Dockerfile: Update the maintainers to be the Metering mailing list.
7f6cad9f2f8e32bed781549ab68f81dfd3e79341 Dockerfile: Stop giving /etc/passwd incorrect priviledges.
6d6714fa0c4be9d3cc57ed471fe8c0372efb152a OWNERS: Update the owners file.
6de36829dc0e86003a5934158efe6a3e0ae855a2 Dockerfile.rhel: Fix location of gcs-connector jar
3c18170165390099de189234a95bf123620296e5 Dockerfile*: Make Java root truststore $JAVA_HOME/lib/security/cacerts writable by GID 0
7eff300748bb359829c85d9da23b9b601d50d68d Update docker-build-rhel.sh to use new REPO_DIR var
cd9a97fcaea5c60dd79ed04b0f372abbffe3557a Dockerfile.rhel: Add GCS connector JAR
e8a2636c8cf4b1fd76fdb982997d2d0745439e46 Dockerfile*: Remove yum update
07c3f0b4d1d860088036f8304dcfde746a4c77d2 Dockerfile.okd: Remove faq/epel since origin-base include OCP RPMs
bf4a5ca2ef62ae31cad2b0170d8186169e871263 Dockerfile.okd: Use shaded GCS jar in OKD image
f7262c5b60e803cfb234de3c9b4847d00ddf81a8 Dockerfile*: Add tini
6d16cc660237a750883aeb1c04d29987ce7512cd Add Brett Tofel
4df4c19c0e036138c6969e852f17c6830dc15305 Dockerfile.okd: Use origin-base as base image
8286bf134f1708cccafa943645591767313d97c5 Dockerfile*: Run yum update before yum install
0b95af37f15bc9ec0d769e1cb71e12784f63fb43 Dockerfile.okd: Use shaded gcs connector jar
cb75cb3075871dd0ff9fbb50c75e13ce48aeedf7 added shaded version
14cc1256c71125d54bf375de3541be9e38ba966d Dockerfile.okd: Add GCS connector
35da5c5b261d7bbb2ac5a3eb1baeedb425642888 Dockerfile.okd: Install epel-release from file, not repos
c1a9252e2d1602047133a0d29f4a2f1b8fcd9ec0 Dockerfile: Add OKD Dockerfile based on UBI
ab62f4ba62078e8dbf360179ed805cea85604d7b added gcs-connector for hadoop
b75387922182523b4f4535727d0e54a9b8311122 Add faq to image
14521f3381100dd22dff8e8c92ef3cb8d1fc87cd Dockerfile*: Add openssl
de767d353e0ff7b159c621f2205cb8a7ebaf5243 Dockerfile*: Add rsync to enable usage with oc rsync.
0a29d3387b7a559ed4858de28fbba4d6da7dc138 Dockerfile*: Set WORKDIR to $HADOOP_HOME
d48d5b159f059b25f0676c5c7f85711590121270 Dockerfile*: Add Prometheus JMX agent
f3d3e8d5ed8822afa207bbdb2fa9bf7af125c93b Dockerfile.rhel: Fix missing HADOOP_LOG_DIR variable
cee247c5603c1f260d9dcf0b04e9908d5ab4d89e Dockerfile*: Remove hadoop home symlink, just put everything into /opt/hadoop
a0a0e5828912e2a17af46e30197f8e9010eec554 Update OWNERS to remove tschuy and add timflannagan1
86a56ce31045328ceedbf6e4255c8e021f52ec29 Update image names/tags in scripts
4a770fb160c06b92227fa16e5d82bcf0486e51d5 Add OWNERS file
f9e2d8519d84e356c6cc4331b8dee40bbc6210d7 Switch to explicitly using COPY instead of whitelist based dockerignore
1ce62085bcffb291de554f02719c8329465618ae .dockerignore: Ignore everything not explicitly required for building
4f0f927132d609afa1541ccc5e1a2f06095461b3 Dockerfile*: Minor updates for image builder/clarity
532db247406ec8746f6ad8e9952eb2eda1cd1df8 Dockerfile*: Remove existing networkaddress.cache property options from java.security config file before setting new value
b07ebf19440bd10c442e706e2788d092ad8c4e51 Dockerfile*: Pass --setopt=skip_missing_names_on_install=False to yum install
82c50885fe0bf38c53b1311c1899f41d6b4ccdbd Remove spaces from java.security networkaddress values
ee4b4fa9687b0177a6dfa6ddb607bb7c6cfab976 Dockerfile*: Install openjdk-devel for debugging tools
3944b2b7e5dc7d3fc27b180b36dfacef4a07ad3c Dockerfile*: Tell JVM to not cache DNS results
ccf7dbdb4db4ef955cd418732f86fd0dc15270be Update docker-build-rhel.sh to tag image with short name also
fc1f62a66fe3e7e529959060e12fb185111d2b19 Create hadoop user in image
dd809b1e195fba89376e0b3fa99c9f384fa7ef5d Fix syntax in rhel dockerfile
f7c923433ade8abfa6b2067cba6376d073975e1c Update rhel Dockerfile to use rhel instead of centos and add build scripts
5f7aa442d5ae8697c02cdbd33f9aa7a98a3092ac Rename Dockerfile.rhel7 to Dockerfile.rhel
*** 848a4d3f309ad93e27544de9d59bd68a7f8a6dda Add rhel7 dockerfile
37b95da45b217082aa4db4215de99de4d5e600bc make dockerignore recursively ignore target/ dirs
14a6cc9e05e4f5fe29978ac8fa6c183991a36fbd hadoop-dist: Set artifact classifier to bin
80e495a643e47873a37b81f3c2e6aee4df8697c3 hadoop-dist: Don't skip deploy for hadoop-dist project
db91f4e1d8e544d6d5256b64d017b86b3d797f0a Add exclusion at top level hadoop-project on jersey-guice targeting javax.inject:javax.inject
270cd730f21bce1b334a2fd8c8aa57ca34e2d35d Dockerfile: Skip using tar and copy distribution directory directly
37a0fb5b43673e7a95b532105251933846f725cd Add Dockerfile
ef94f643bd7a41d804cda7d9982d19072ab7bf6b Fix invalid xml caused by xlint compiler argument in pom.xml
e2cb081ba93ad9515d3b49812ff252318e6866f4 Revert "Fix potential FSImage corruption. Contributed by Ekanth Sethuramalingam & Arpit Agarwal."
10d5b075b5702c7f423124e19c1b25abb87178de HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds. Contributed by Gabor Bota
e4bfcae57231580a511d471cafc74b1972057752 HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon.
1b1e4a4739795fd89c63f3a940392acbac280e43 YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. Contributed by Chandni Singh
*** 6ae4d5cc9122bb80c3c73db04d9adb2562ae5846 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan.
e9e4d3d5e5bde26435870def3628d4340cf6dc94 YARN-8259. Improve privileged docker container liveliness checks. Contributed by Shane Kumpf
0b7883f1691902cef1adbe1a3313189fba4372ff YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda)
72c6df97a2e85862f33fce9ece729d1a3bd908af Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg.
052932a619edd6d969c69cc14eb03c989b1b3699 YARN-6919. Add default volume mount list. Contributed by Eric Badger
48fd523d57704c66382c05d0f47663dbbf673c1a YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe.
bc14e87262a00ecce1a3616a8b94404c9bbadeac YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger
84b87a6d11b10d4b876b7b10e1063d159aa64978 YARN-7530. Refactored YARN service API project location. Contributed by Chandni Singh
*** 6ceb3a0f58df7950ef6284a3c5ece5dc5349c2dc Add 2.9.1 release notes and changes documents
5db37206bb2a5c73f7da9db803eecd6854b98c65 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah."
a764336a21aab5ce9ac6e6b9bda12542008b62f9 YARN-7973. Added ContainerRelaunch feature for Docker containers. Contributed by Shane Kumpf
*** b57546cfb63c58f61dbc2a7a190de75bb1264b46 YARN-8209. Fixed NPE in Yarn Service deletion. Contributed by Eric Badger
576c8846ad18be1abb0495f16e4f305ced3d5f31 YARN-8064. Bugfix for clean up container-executor cmd helper file. Contributed by Eric Badger
cbc4b0a8f968f12aadfb4c57716536d9d8072fc2 YARN-7142. Support placement policy in yarn native services. (Gour Saha via wangda)
** dd0c7df233d4bf60eff70cd6dc920fc1bbc58cd0 HDFS-13215. RBF: Move Router to its own module.
* aa96f1871bfd858f9bac59cf2a81ec470da649af Updated the index as per 3.3.0 release
Trying again:
a23509823c0f0df9cd9f04a33e8050297d922755 (HEAD -> from_rel-3.3.0, origin/from_rel-3.3.0) Update image names/tags in scripts
5531ef14e727b555cb99aa5d523512ada20c80d0 Add OWNERS file
a5c964d8e5cc0f43fba64430e8be08a9dc8b095a Switch to explicitly using COPY instead of whitelist based dockerignore
a5cfb0c1e8a1795641425a8301694e942a474d30 .dockerignore: Ignore everything not explicitly required for building
fb23be1eea06af901c8ae6e6923e12cb3511a01f Dockerfile*: Minor updates for image builder/clarity
ed19db17d057aaa9aeb5fe922f9983c2361bfc7a Dockerfile*: Remove existing networkaddress.cache property options from java.security config file before setting new value
f50cd8d38dcb703c8121ac186e1e5a03ac9216fc Dockerfile*: Pass --setopt=skip_missing_names_on_install=False to yum install
9fa80d32c30d6463af7986960da17e62b7a94427 Remove spaces from java.security networkaddress values
7323ee0c10ad4e5cc5bd4cef5ac165c42686f543 Dockerfile*: Install openjdk-devel for debugging tools
1523ce4cfef0259e93e9892f77cacd6d6c6da555 Dockerfile*: Tell JVM to not cache DNS results
b4abe2e23685ec6255df2580397779c042966501 Update docker-build-rhel.sh to tag image with short name also
785feb09695ea47384a35ee631112f5bc4dd8752 Create hadoop user in image
9252a92231d07492ea8a9dbaa4f44f7acccacb09 Fix syntax in rhel dockerfile
c5e9f9562b0aa5f9633a7d8f1ddfd773a5848b27 Update rhel Dockerfile to use rhel instead of centos and add build scripts
75fb6dc964f963bbf53f9ba0933e779f4fd60847 Rename Dockerfile.rhel7 to Dockerfile.rhel
*** 8b43fa7038a2733e3c9b7774b6a7dca3fae25544 Add rhel7 dockerfile
64263d4976899c51e390852180694a207040c2c8 make dockerignore recursively ignore target/ dirs
c289d42333906addca3e59076dc5612943a15dfb hadoop-dist: Set artifact classifier to bin
9cb8450d5e44746d4888a5e37b53faf8d5df0f2f hadoop-dist: Don't skip deploy for hadoop-dist project
*** 0242adc8c5f35aff7baec4187e7178293334e01a Exclude org.jsr-305:ri frmo shading in hadoop-client-runtime
^^^ broken ^^^
* e4476f6e108907a76c2fbbaa39544d1f75e994ce Add exclusion at top level hadoop-project on jersey-guice targeting javax.inject:javax.inject
* c75a270372026390b797389de57ea94b793e704d Dockerfile: Skip using tar and copy distribution directory directly
* b01be28d2e591506933d22b454e3503f62b3e84d Add Dockerfile
* 1772b677796c19d4a76b3f84124fdeba029969f3 Fix invalid xml caused by xlint compiler argument in pom.xml
* aa96f1871bfd858f9bac59cf2a81ec470da649af Updated the index as per 3.3.0 release
b064f09bd687cdecbbfc8af5db487d834182049f Preparing for 3.3.0 Release
a883752df1ca5b8f2523e77179033bcb889ab80c HDFS-15421. IBR leak causes standby NN to be stuck in safe mode.
This patch allows 0242adc8c5f35aff7baec4187e7178293334e01a to build:
diff --git a/hadoop-client-modules/hadoop-client-runtime/pom.xml b/hadoop-client-modules/hadoop-client-runtime/pom.xml
index f8dda3db20..b9eb68feda 100644
--- a/hadoop-client-modules/hadoop-client-runtime/pom.xml
+++ b/hadoop-client-modules/hadoop-client-runtime/pom.xml
@@ -13,16 +13,16 @@
limitations under the License. See accompanying LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-project</artifactId>
- <version>3.1.1</version>
+ <version>3.3.0</version>
<relativePath>../../hadoop-project</relativePath>
</parent>
<artifactId>hadoop-client-runtime</artifactId>
- <version>3.1.1</version>
+ <version>3.3.0</version>
<packaging>jar</packaging>
<description>Apache Hadoop Client</description>
@@ -157,7 +157,12 @@
<!-- Leave javax APIs that are stable -->
<!-- the jdk ships part of the javax.annotation namespace, so if we want to relocate this we'll have to care it out by class :( -->
<exclude>com.google.code.findbugs:jsr305</exclude>
- <exclude>org.jsr-305:ri</exclude>
+ <exclude>io.dropwizard.metrics:metrics-core</exclude>
+ <exclude>org.eclipse.jetty:jetty-servlet</exclude>
+ <exclude>org.eclipse.jetty:jetty-security</exclude>
+ <exclude>org.ow2.asm:*</exclude>
+ <!-- Leave bouncycastle unshaded because it's signed with a special Oracle certificate so it can be a custom JCE security provider -->
+ <exclude>org.bouncycastle:*</exclude>
</excludes>
</artifactSet>
<filters>
And allows master for metering hadoop to build after checkout upstream/rel/release-3.3.0 and cherry-pick all metering related commits onto the branch.
modified metering for newer hadoop/hive, meant for 3.3 branch tip. but hopefully will work with the rel/release-3.3.0 tagged version:
bin/deploy-metering install --repo quay.io/btofel/metering-operator --tag metering-operator-hadoop33-mods
TO DO:
podman build -f Dockerfile -t quay.io/btofel/metering-hive:3.1.2-hadoop-rel-3.3.0 .
Above was resolved by changes to pom around exclusions.
With 3.3.0 Hadoop + 3.1.2 Hive deploy, we have an startup problem:
│ 20/11/12 19:03:37 [main]: INFO conf.MetastoreConf: Found configuration file file:/opt/hive/conf/hive-site.xml │
│ 20/11/12 19:03:37 [main]: INFO conf.MetastoreConf: Unable to find config file hivemetastore-site.xml │
│ 20/11/12 19:03:37 [main]: INFO conf.MetastoreConf: Found configuration file null │
│ 20/11/12 19:03:37 [main]: INFO conf.MetastoreConf: Unable to find config file metastore-site.xml │
│ 20/11/12 19:03:37 [main]: INFO conf.MetastoreConf: Found configuration file null │
│ Exception in thread "main" 2020-11-12T19:03:37.329610002Z java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/l │
│ at org.apache.hadoop.conf.Configuration.set(Configuration.java:1380) │
│ at org.apache.hadoop.conf.Configuration.set(Configuration.java:1361) │
│ at org.apache.hadoop.hive.metastore.conf.MetastoreConf.lambda$newMetastoreConf$1(MetastoreConf.java:1191) │
│ at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) │
│ at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) │
│ at java.util.Iterator.forEachRemaining(Iterator.java:116) │
│ at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) │
│ at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) │
│ at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) │
│ at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)2020-11-12T19:03:37.329860362Z │
│ at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) │
│ at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)2020-11-12T19:03:37.329910228Z │
│ at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)2020-11-12T19:03:37.329931256Z │
│ at org.apache.hadoop.hive.metastore.conf.MetastoreConf.newMetastoreConf(MetastoreConf.java:1188)2020-11-12T19:03:37.329956148Z │
│ at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8770)2020-11-12T19:03:37.329994081Z │
│ at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) │
│ at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)2020-11-12T19:03:37.330023524Z │
│ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)2020-11-12T19:03:37.330047776Z │
│ at java.lang.reflect.Method.invoke(Method.java:498)2020-11-12T19:03:37.330062836Z │
│ at org.apache.hadoop.util.RunJar.run(RunJar.java:323)2020-11-12T19:03:37.330087858Z │
│ at org.apache.hadoop.util.RunJar.main(RunJar.java:236)2020-11-12T19:03:37.330101452Z
Above problem seems to be multiple guava lib problem:
And from Hive container:
bash-4.4$ find / -name "guava.*jar" -print
/opt/hive/lib/guava-19.0.jar
/opt/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar
/opt/hadoop/share/hadoop/hdfs/lib/guava-27.0-jre.jar
/opt/hadoop/share/hadoop/yarn/csi/lib/guava-20.0.jar
Suggested fix like:
$ rm /opt/shared/apache-hive-3.1.2-bin/lib/guava-19.0.jar
$ cp /opt/shared/hadoop-3.2.1/share/hadoop/hdfs/lib/guava-27.0-jre.jar /opt/shared/apache-hive-3.1.2-bin/lib/
Having a problem where Hadoop version created by checkout tag rel/release-3.3.0 and cherry pick all "our" commits creates a fully working Hadoop and tests okay with Hive 3.1.2 (through Metering deploy and reportdatasources data being added).
But... PR has needs-rebase, likely due to these not being the same:
hadoop from_rel-3.3.0 $ git show-ref --heads -s origin master
6ace76f403981964e7b6714530f5e01948c10b09
hadoop from_rel-3.3.0 $ git merge-base origin/master from_rel-3.3.0
49c747ab187d0650143205ba57ca19607ec4c6bd
Trying to rebase instead onto 3.2.1, but we run into:
Caused by: org.apache.maven.plugin.MojoExecutionException: org.apache.maven.plugin.MojoExecutionException: protoc version is 'libprotoc 3.7.1', expected version is '2.5.0'
and the whole point of this move is to move to newer protobuf
But this rebased onto 3.2.1 still fails with error I was also seeing on the bigger jump to 3.3.0:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project hadoop-common: Compilation failure: Compilation failure:
[ERROR] /build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java:[162,38] cannot find symbol
[ERROR] symbol: class AbstractDelegationTokenSelector
[ERROR] location: class org.apache.hadoop.crypto.key.kms.KMSClientProvider
[ERROR] /build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java:[1010,35] cannot find symbol
[ERROR] symbol: method selectToken(org.apache.hadoop.io.Text,java.util.Collection<org.apache.hadoop.security.token.Token<? extends org.apache.hadoop.security.token.TokenIdentifier>
>)
[ERROR] location: variable INSTANCE of type org.apache.hadoop.crypto.key.kms.KMSClientProvider.TokenSelector
Above is possible related to this commit:
752092a8608 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah."
A New Beginning....
Successfully used roughly the kube/master Dockerfile and rebased from 3.1.1 to 3.1.4 just to see how that would go and maybe more nearly align with what has to be the next stop for this change, 3.2.0
https://github.com/bentito/hadoop/blob/yarb_rel_3.1.4/Dockerfile
rebase to 3.1.4 and 3.2.0 common ancestor compiled successfully:
git pull --rebase upstream a39296260f8c77f3808e27b43c623a0edebe4a17
This branch is: bentito/yarb_rel_3.1.4-3.2.0
After the, now, shorter rebase from ☝️ to upstream/branch_3.2.0 seeing this error on build:
[ERROR] /build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:[529,54] cannot find symbol
Built 3.3.0 with a "our" Dockerfile that moves:
cmake -> 3.19.1;
mvn -> 3.3.9;
protobuf -> 3.7.1;
this branch is: hadoop-3.3.0-from-3.2.2
So we still face a needs-rebase tag. Going with
git cherry-pick 2b9a8c1d3a2^..6ace76f4039
onto hadoop-3.3.0-from-3.2.2 to pickup everything of "ours" from kube-reporting/hadoop/master
Seeing this as before on this path:
Caused by: org.apache.hadoop.hive.metastore.api.MetaException: User hadoop is not allowed to perform this API call ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Successfully running the Hadoop 3.3/Hive:latest with modified Metering as per above comments. Seeing data in
reportdatasourcefor HDFS deploy. No data for s3 deploy (aka: the one we care about for this bug). So trying a workaround suggested by @timflannagan1 . Addhive.non-managed-table-writes-enabled: trueThis change would be in:
charts/openshift-metering/templates/presto/presto-catalog-config-secret.yamlvia
_presto_helpers.tpl