Hadoop + Hive Build/Runtime issues for Metering Operator -- trying to get the tip of Hadoop 3.3 branch and latest Hive master (Oct 15, 2020) to work to see if will fix a bug related to S3 bucket contents deletion

gistfile1.txt

│ Exception in thread "main" 2020-10-13T17:23:10.464510059Z java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V2020-10-13T17:23:10.464518738Z        │
│     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1382)2020-10-13T17:23:10.464616395Z                                                                                                                 │
│     at org.apache.hadoop.conf.Configuration.set(Configuration.java:1363)2020-10-13T17:23:10.464630106Z                                                                                                                 │
│     at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)2020-10-13T17:23:10.464669465Z                                                                                                                         │
│     at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)                                                                                                                                                │
│     at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)2020-10-13T17:23:10.464698814Z                                                                                                                         │
│     at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4047)2020-10-13T17:23:10.464708189Z                                                                                                               │
│     at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:4010)                                                                                                                                                 │
│     at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:7002)2020-10-13T17:23:10.464737483Z                                                                                                      │
│     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)                                                                                                                                                     │
│     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)2020-10-13T17:23:10.464765419Z                                                                                                     │
│     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)                                                                                                                           │
│     at java.lang.reflect.Method.invoke(Method.java:498)2020-10-13T17:23:10.464793618Z                                                                                                                                  │
│     at org.apache.hadoop.util.RunJar.run(RunJar.java:323)2020-10-13T17:23:10.464803494Z                                                                                                                                │
│     at org.apache.hadoop.util.RunJar.main(RunJar.java:236)                                                                                                                                                             │
│ stream closed

Above is possible related to this commit:

752092a8608 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah."

A New Beginning....
Successfully used roughly the kube/master Dockerfile and rebased from 3.1.1 to 3.1.4 just to see how that would go and maybe more nearly align with what has to be the next stop for this change, 3.2.0

https://github.com/bentito/hadoop/blob/yarb_rel_3.1.4/Dockerfile

rebase to 3.1.4 and 3.2.0 common ancestor compiled successfully:

git pull --rebase upstream  a39296260f8c77f3808e27b43c623a0edebe4a17

This branch is: bentito/yarb_rel_3.1.4-3.2.0

After the, now, shorter rebase from ☝️ to upstream/branch_3.2.0 seeing this error on build:

[ERROR] /build/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java:[529,54] cannot find symbol

Built 3.3.0 with a "our" Dockerfile that moves:
cmake -> 3.19.1;
mvn -> 3.3.9;
protobuf -> 3.7.1;
this branch is: hadoop-3.3.0-from-3.2.2

So we still face a needs-rebase tag. Going with

git cherry-pick 2b9a8c1d3a2^..6ace76f4039

onto hadoop-3.3.0-from-3.2.2 to pickup everything of "ours" from kube-reporting/hadoop/master

Seeing this as before on this path:

Caused by: org.apache.hadoop.hive.metastore.api.MetaException: User hadoop is not allowed to perform this API call                                                                                                 ```