大数据项目实践（五）——Hue安装

一、下载hue安装包

http://archive-primary.cloude...

二、安装依赖包

yum install gcc g++ libxml2-devel libxslt-devel cyrus-sasl-devel cyrus-sasl-gssapi mysql-devel python-devel python-setuptools sqlite-devel ant libsasl2-dev libsasl2-modules-gssapi-mit libkrb5-dev libtidy-0.99-0 mvn openldap-dev

三、编译

如果之前是编译过的，则删除

$ rm app.reg
$ rm -r build

[root@bigdata-pro01 hue-3.9.0-cdh5.10.2]# su bigdata
[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ pwd
/opt/modules/hue-3.9.0-cdh5.10.2
[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ make apps 

四、编译过程中遇到的问题

反复编译，直到成功，成功后，目录中多出两个文件：一个是

• 问题1

/usr/include/features.h:164:1: 警告：这是先前定义的位置 
src/_fastmath.c:36:18: 错误：gmp.h：没有那个文件或目录 
src/_fastmath.c:63: 错误：expected ‘)’ before ‘n’ 
src/_fastmath.c:66: 错误：expected ‘)’ before ‘m’ 
src/_fastmath.c:107: 错误：expected ‘)’ before ‘m’ 
src/_fastmath.c:148: 错误：expected specifier-qualifier-list before ‘mpz_t’ 
src/_fastmath.c:158: 错误：expected specifier-qualifier-list before ‘mpz_t’ 
src/_fastmath.c:196: 错误：expected declaration specifiers or ‘...’  before ‘mpz_t’ 
src/_fastmath.c:196: 错误：expected declaration specifiers or ‘...’  before ‘mpz_t’ 
......
src/_fastmath.c:1601: 警告：隐式声明函数‘mpz_gcd_ui’ 
src/_fastmath.c:1608: 警告：隐式声明函数‘mpz_divexact_ui’ 
error: command ‘gcc‘ failed with exit status 1 
make[2]: *** [/opt/hue/desktop/core/build/pycrypto-2.6.1/egg.stamp] 错误 1 
make[2]: Leaving directory `/opt/hue/desktop/core‘ 
make[1]: *** [.recursive-env-install/core] 错误 2 
make[1]: Leaving directory `/opt/hue/desktop‘ 
make: *** [desktop] 错误 2 

少个gmp
yum install gmp-devel

• 问题2

Modules/errors.h:8:18: error: lber.h: No such file or directory Modules/errors.h:9:18: error: ldap.h: No such file or directory

解决如下：

sudo apt-get install libsasl2-dev python-dev libldap2-dev libssl-dev
sudo apt-get install -y python-dev libldap2-dev libsasl2-dev libssl-dev
sudo yum install python-devel
sudo yum install openldap-devel

实际执行 sudo yum install openldap-devel 即可

五、配置

进入desktop目录

[bigdata@bigdata-pro01 hue-3.9.0-cdh5.10.2]$ cd desktop/
[bigdata@bigdata-pro01 desktop]$ ll
total 284
drwxr-xr-x  2 bigdata bigdata   4096 Jun 27  2017 conf
drwxr-xr-x  5 bigdata bigdata   4096 Aug 27 21:40 core
-rw-r--r--  1 bigdata bigdata 264192 Aug 27 22:24 desktop.db
drwxr-xr-x 15 bigdata bigdata   4096 Jun 27  2017 libs
drwxrwxr-x  2 bigdata bigdata   4096 Aug 27 22:23 logs
-rw-r--r--  1 bigdata bigdata   3467 Jun 27  2017 Makefile
[bigdata@bigdata-pro01 desktop]$ cd conf/
[bigdata@bigdata-pro01 conf]$ ll
total 64
-rw-r--r-- 1 bigdata bigdata 54575 Jun 27  2017 hue.ini
-rw-r--r-- 1 bigdata bigdata  1843 Jun 27  2017 log4j.properties
-rw-r--r-- 1 bigdata bigdata  1809 Jun 27  2017 log.conf
[bigdata@bigdata-pro01 conf]$ 

编辑hue.ini 文件

secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
http_host=bigdata-pro01.mars.com
http_port=8888
time_zone=Asia/Shanghai

修改desktop.db 权限

[bigdata@bigdata-pro01 desktop]$ sudo chmod o+w desktop.db 

启动hue 服务

[bigdata@bigdata-pro01 bin]$ pwd
/opt/modules/hue-3.9.0-cdh5.10.2/build/env/bin
[bigdata@bigdata-pro01 bin]$ ./supervisor 
[INFO] Not running as root, skipping privilege drop
starting server with options:
{'daemonize': False,
 'host': 'bigdata-pro01.mars.com',
 'pidfile': None,
 'port': 8888,
 'server_group': 'hue',
 'server_name': 'localhost',
 'server_user': 'hue',
 'ssl_certificate': None,
 'ssl_certificate_chain': None,
 'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
 'ssl_private_key': None,
 'threads': 40,
 'workdir': None}

进入web界面，浏览器地址输入：bigdata-pro01.mars.com:8888

创建超级用户，牢记用户名和密码，如 admin/admin
然后就进入主界面

六、Hue与HDFS集成

1.修改core-site.xml
在core-site.xml中添加以下配置：

<property>
        <name>hadoop.proxyuser.hue.hosts</name>
        <value>*</value>
    </property>
    
    <property>
        <name>hadoop.proxyuser.hue.groups</name>
        <value>*</value>
    </property>

2.修改hue.ini

添加以下内容：

fs_defaultfs=hdfs://hadoop-ha
webhdfs_url=http://bigdata-pro01.mars.com:50070/webhdfs/v1
hadoop_conf_dir=/opt/modules/hadoop-2.6.0-cdh5.10.2/etc/hadoop
hadoop_bin=/opt/modules/hadoop-2.6.0-cdh5.10.2/bin
hadoop_hdfs_home=/opt/modules/hadoop-2.6.0-cdh5.10.2

3.将core-site.xml分发到集群的其他每个机器上
重启hdfs

$HADOOP_HOME/etc/hadoop/stop-dfs.sh
$HADOOP_HOME/etc/hadoop/start-dfs.sh

重启hue服务

$HUE_HOME/build/env/bin/supervisor

浏览器查看：

七、Hue 和 yarn 集成

八、Hue 和 hive 集成

八、Hue 和 mysql 集成(hive metastore)

九、Hue 和 Hbase 集成

同时，Hbase启动thrift服务：
$HBASE_HOME/bin/hbase-daemon.sh start thrift