Skip to content

Instantly share code, notes, and snippets.

@posulliv
Created October 12, 2020 14:46
Show Gist options
  • Save posulliv/937b13b35fb23b16ac413e11e11f61b5 to your computer and use it in GitHub Desktop.
Save posulliv/937b13b35fb23b16ac413e11e11f61b5 to your computer and use it in GitHub Desktop.
Patch for Apache Atlas version 2.1.0 to allow Hive Metastore hook to work correctly with Presto.
--- addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveMetastoreHookImpl.java 2020-10-12 09:54:29.000000000 -0400
+++ addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveMetastoreHookImpl.java.presto 2020-10-12 10:44:24.000000000 -0400
@@ -17,6 +17,8 @@
*/
package org.apache.atlas.hive.hook;
+import com.google.common.collect.MapDifference;
+import com.google.common.collect.Maps;
import org.apache.atlas.hive.hook.events.*;
import org.apache.atlas.hook.AtlasHook;
import org.apache.commons.lang.StringUtils;
@@ -94,8 +96,7 @@
context.setOperation(ALTERTABLE_RENAME);
} else if (isColumnRename(oldTable, newTable, context)) {
context.setOperation(ALTERTABLE_RENAMECOL);
- } else if(isAlterTableProperty(tableEvent, "last_modified_time") ||
- isAlterTableProperty(tableEvent, "transient_lastDdlTime")) {
+ } else if(isAlterTableProperty(tableEvent)) {
context.setOperation(ALTERTABLE_PROPERTIES); // map other alter table operations to ALTERTABLE_PROPERTIES
}
@@ -192,19 +193,14 @@
return isColumnRename;
}
- private boolean isAlterTableProperty(AlterTableEvent tableEvent, String propertyToCheck) {
- final boolean ret;
- String oldTableModifiedTime = tableEvent.getOldTable().getParameters().get(propertyToCheck);
- String newTableModifiedTime = tableEvent.getNewTable().getParameters().get(propertyToCheck);
+ private boolean isAlterTableProperty(AlterTableEvent tableEvent) {
+ final MapDifference<String, String> differences = Maps.difference(
+ tableEvent.getOldTable().getParameters(),
+ tableEvent.getNewTable().getParameters());
-
- if (oldTableModifiedTime == null) {
- ret = newTableModifiedTime != null;
- } else {
- ret = !oldTableModifiedTime.equals(newTableModifiedTime);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("HiveMetastoreHook.isAlterTableProperty({}): truely alter table event.", differences.toString());
}
-
- return ret;
-
+ return !differences.areEqual();
}
}
@tooptoop4
Copy link

@posulliv did u manage to capture lineage for presto views in Atlas? been trying to track down pPanda-beta/presto-utils#1

@TheNextCEO
Copy link

Would this work with Trino? @posulliv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment