diff --git a/HowToKerberize.md b/HowToKerberize.md new file mode 100644 index 000000000..d1176c8cf --- /dev/null +++ b/HowToKerberize.md @@ -0,0 +1,106 @@ +![Bee waggle-dancing on a hive.](logo.png "Federating Hive Meta Stores.") + +# Additional instructions to use Waggle Dance in a Kerberized environment + + +### Process + +In a Kerberos environment a client make a request to Waggle Dance which in turn requests the proxy user's token from the metastore and then uses this token to communicate with the metastore. + +This is necessary in certain scenarios that need authentication - for example the `create_table` API that requires the proxy user to create HDFS directories. + +![Kerberos Process.](kerberos-process.png "Kerberos Process") + +In addition, because Kerberos authentication requires a delegation-token to proxy as other users, the proxy user of the session is shared globally. This means we need to make all Hive Metastores share a set of delegation-token storage so that a single delegation-token can be authenticated by multiple Metastores. + +**One solution is to use Zookeeper to store tokens for all Hive Metastores** + +### Prerequisites + +* Kerberized cluster: + active KDC, + some required properties in configuration files of Hadoop services +* User account with privileges in kerberos environment +* Zookeeper to store delegation-token (Recommended) + +### Configuration + +Waggle Dance does not read Hadoop's `core-site.xml` so a general property providing Kerberos auth should be added to +the Hive configuration file `hive-site.xml`: + +``` + + hadoop.security.authentication + KERBEROS + +``` + + +Waggle Dance also needs a keytab file to communicate with the Metastore so the following properties should be present: +``` + + hive.metastore.sasl.enabled + true + + + hive.metastore.kerberos.principal + hive/_HOST@YOUR_REALM.COM + + + hive.metastore.kerberos.keytab.file + /etc/hive.keytab + +``` + +In addition, all metastores need to use the Zookeeper shared token: +``` + + hive.cluster.delegation.token.store.class + org.apache.hadoop.hive.thrift.ZooKeeperTokenStore + + + hive.cluster.delegation.token.store.zookeeper.connectString + zk1:2181,zk2:2181,zk3:2181 + + + hive.cluster.delegation.token.store.zookeeper.znode + /hive/token + +``` + +If you are intending to use a Beeline client, the following properties may be valuable: +``` + + hive.server2.transport.mode + http + + + hive.server2.authentication + KERBEROS + + + hive.server2.authentication.kerberos.principal + hive/_HOST@YOUR_REALM.COM + + + hive.server2.authentication.kerberos.keytab + /etc/hive.keytab + + + hive.server2.enable.doAs + false + +``` + + +### Running + +Waggle Dance should be started by a privileged user with a fresh keytab. + +If Waggle Dance throws a GSS exception, you have problem with the keytab file. +Try to perform `kdestroy` and `kinit` operations and check the keytab file ownership flags. + +If the Metastore throws an exception with code -127, Waggle Dance is probably using the wrong authentication policy. +Check the values in `hive-conf.xml` and make sure that HIVE_HOME and HIVE_CONF_DIR are defined. + +Don't forget to restart hive services! diff --git a/kerberos-process.png b/kerberos-process.png new file mode 100644 index 000000000..992c8ee10 Binary files /dev/null and b/kerberos-process.png differ diff --git a/pom.xml b/pom.xml index 45c6a96c3..4165ebed3 100644 --- a/pom.xml +++ b/pom.xml @@ -51,10 +51,32 @@ 6.0.0 1.18.24 3.12.0 + 2.13.0 + + + org.apache.curator + curator-client + ${curator.version} + + + org.apache.curator + curator-framework + ${curator.version} + + + org.apache.curator + curator-recipes + ${curator.version} + + + org.apache.curator + curator-x-discovery + ${curator.version} + com.google.guava guava diff --git a/waggle-dance-core/pom.xml b/waggle-dance-core/pom.xml index 2deeb3552..b00c2bee0 100644 --- a/waggle-dance-core/pom.xml +++ b/waggle-dance-core/pom.xml @@ -144,6 +144,17 @@ org.apache.hive hive-service + ${hive.version} + + + org.eclipse.jetty + jetty-server + + + org.eclipse.jetty + jetty-runner + + diff --git a/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/mapping/service/impl/PrefixBasedDatabaseMappingService.java b/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/mapping/service/impl/PrefixBasedDatabaseMappingService.java index a13cdd440..919254bc4 100644 --- a/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/mapping/service/impl/PrefixBasedDatabaseMappingService.java +++ b/waggle-dance-core/src/main/java/com/hotels/bdp/waggledance/mapping/service/impl/PrefixBasedDatabaseMappingService.java @@ -90,10 +90,16 @@ public PrefixBasedDatabaseMappingService( private void add(AbstractMetaStore metaStore) { MetaStoreMapping metaStoreMapping = metaStoreMappingFactory.newInstance(metaStore); + DatabaseMapping databaseMapping = createDatabaseMapping(metaStoreMapping); if (metaStore.getFederationType() == PRIMARY) { primaryDatabaseMapping = databaseMapping; + if (!metaStoreMapping.isAvailable()) { + throw new WaggleDanceException( + String.format("Primary metastore is unavailable {}", metaStore.getRemoteMetaStoreUris()) + ); + } } mappingsByPrefix.put(metaStoreMapping.getDatabasePrefix(), databaseMapping);