Hive用户认证配置流程

  1. 首先确保Hadoop环境和Hive环境兼容且配置好,我这里Hive的元数据库使用的MySQL。
  2. 编辑$HIVE_HOME/conf/hive-site.xml,对server2适当配置并启动server。
    使用server2是因为HiveServer2支持多客户端的并发和认证,为开放API客户端如JDBC、ODBC提供更好的支持。server/server2区别以及配置
    当然也可以不配置server2相关内容,保留默认配置即可,需注意默认tcp port is 10000(default http port is 10001)
    启动server2:

    hive --service hiveserver2 &
    

    使用beeline连接,在没有配置用户的情况下任何人都可以连接,密码为空:

    $ beeline
    Beeline version 1.2.1.spark2 by Apache Hive
    beeline> !connect jdbc:hive2://localhost:10000 hadoop
    Connecting to jdbc:hive2://localhost:10000
    Enter password for jdbc:hive2://localhost:10000: 
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/usr/local/lib/spark-2.2.0/jars/slf4j-log4j12-1.7.16.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/usr/local/lib/hadoop-2.8.1/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    Connected to: Apache Hive (version 2.3.0)
    Driver: Hive JDBC (version 1.2.1.spark2)
    Transaction isolation: TRANSACTION_REPEATABLE_READ
    
  3. 编码完成CustomHiveServer2Auth类,以及Authentication file
    authentication file是保存明文用户名和MD5加密密码的文本文件,如下所示。该文件需要放置在server中,我这里放在$HIVE_HOME/conf下并命名为hive.server2.users.conf了。

    opticalix,88b6ed7f53340e0d45ff60f277e4c420
    other_username,88b6ed7f53340e0d45ff60f277e4c421
    

    CustomHiveServer2Auth类实现如下,参考博文见此

    package com.opticalix;
    
    import com.opticalix.utils.MD5Utils;
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.hive.conf.HiveConf;
    import org.apache.hive.service.auth.PasswdAuthenticationProvider;
    
    import javax.security.sasl.AuthenticationException;
    import java.io.BufferedReader;
    import java.io.File;
    import java.io.FileReader;
    import java.io.IOException;
    
    /**
     * 根据用户名密码文件,遍历看是否能匹配上一个用户。如果不能则抛出异常
     */
    public class CustomHiveServer2Auth implements PasswdAuthenticationProvider {
        public static final String HIVE_SERVER2_CUSTOM_AUTHENTICATION_FILE = "hive.server2.custom.authentication.file";
    
        @Override
        public void Authenticate(String username, String password) throws AuthenticationException {
            boolean ok = false;
            String passMd5 = new MD5Utils().md5(password);
            //load hive conf, get auth file location
            HiveConf hiveConf = new HiveConf();
            Configuration conf = new Configuration(hiveConf);
            String filePath = conf.get(HIVE_SERVER2_CUSTOM_AUTHENTICATION_FILE);
            System.out.println("hive.server2.custom.authentication.file [" + filePath + "] ..");
            //try match
            File file = new File(filePath);
            BufferedReader reader = null;
            try {
                reader = new BufferedReader(new FileReader(file));
                String tempString = null;
                while ((tempString = reader.readLine()) != null) {
                    System.out.println("read line=" + tempString);
                    String[] split = tempString.split(",", -1);
                    if (split.length != 2) continue;
                    //ok
                    if (split[0].equals(username) && split[1].equals(passMd5)) {
                        ok = true;
                        break;
                    }
                }
                reader.close();
            } catch (Exception e) {
                e.printStackTrace();
            } finally {
                if (reader != null) {
                    try {
                        reader.close();
                    } catch (IOException ignored) {
                    }
                }
            }
    
            //throw exception if not matched
            if (ok) {
                System.out.println("user [" + username + "] auth check ok .. ");
            } else {
                System.out.println("user [" + username + "] auth check fail .. ");
                throw new AuthenticationException("user [" + username + "] auth check fail .. ");
            }
        }
    }
    

    编码时需要添加必要的dependency,maven配置如下,版本号需要根据自己的情况而定:

    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-jdbc</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-metastore</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-exec</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-service</artifactId>
        <version>2.3.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>2.8.1</version>
    </dependency>
    
  4. 编辑$HIVE_HOME/conf/hive-site.xml,增加或修改以下property。
    其中authentication file和authentication class要根据自己的情况修改。

    <property>
      <name>hive.server2.authentication</name>
      <value>CUSTOM</value>
      <description>
        Expects one of [nosasl, none, ldap, kerberos, pam, custom].
        Client authentication types.
          NONE: no authentication check
          LDAP: LDAP/AD based authentication
          KERBEROS: Kerberos/GSSAPI authentication
          CUSTOM: Custom authentication provider
                  (Use with property hive.server2.custom.authentication.class)
          PAM: Pluggable authentication module
          NOSASL:  Raw transport
      </description>
    </property>
    <property>
      <name>hive.server2.custom.authentication.class</name>
      <value>com.opticalix.CustomHiveServer2Auth</value>
      <description>
        Custom authentication class. Used when property
        'hive.server2.authentication' is set to 'CUSTOM'. Provided class
        must be a proper implementation of the interface
        org.apache.hive.service.auth.PasswdAuthenticationProvider. HiveServer2
        will call its Authenticate(user, passed) method to authenticate requests.
        The implementation may optionally implement Hadoop's
        org.apache.hadoop.conf.Configurable class to grab Hive's Configuration object.
      </description>
    </property>
    <property>
      <name>hive.server2.custom.authentication.file</name>
      <value>/usr/local/lib/hive-2.3.0/conf/hive.server2.users.conf</value>
    </property>
    
  5. 打jar包。不需要dependencies一同打进jar。这里利用maven plugin: maven-assembly-plugin(Jar打包博文见此)。添加该plugin后执行命令mvn assembly:assembly即可在target中得到jar。然后
    利用scp命令将jar放在集群的$HIVE_HOME/lib,这样在hive启动时会自动加载lib中的jars。

  6. 重新启动hive-server2(如果刚才的hive server进程未结束,可以用kill -9 $pid来结束进程),并使用beeline连接,此时用刚才设置的用户名/密码才能连接成功:

    beeline> !connect jdbc:hive2://localhost:10000
    Connecting to jdbc:hive2://localhost:10000
    Enter username for jdbc:hive2://localhost:10000: opticalix
    Enter password for jdbc:hive2://localhost:10000: **************
    ...
    Connected to: Apache Hive (version 2.3.0)
    Driver: Hive JDBC (version 1.2.1.spark2)
    Transaction isolation: TRANSACTION_REPEATABLE_READ
    

    Server控制台也会输出log:

    hive.server2.custom.authentication.file [/usr/local/lib/hive-2.3.0/conf/hive.server2.users.conf] ..
    read line=opticalix,88b6ed7f53340e0d45ff60f277e4c420
    user [opticalix] auth check ok ..