Table of Contents
Hue Stream Android latest 0.8.4 APK Download and Install. Sync colours from Android device screen directly to Philips Hue lights! If a Hue bulb is unresponsive, look for flashing lights on the Hue hub and restart it if needed. Sometimes a device will respond from the manufacturer's app when not responding from the Home app.
Hue is a graphical user interface to operate and develop applications forApache Hadoop. Hue applications are collected into a desktop-style environmentand delivered as a Web application, requiring no additional installation forindividual users.
This guide describes how to install and configure a Hue tarball. Forinformation about installing Hue packages, seeInstalling Hue.
There is also a companion SDK guide that describes how to developnew Hue applications:Hue SDK Documentation
Hue requires the Hadoop contained in Cloudera’s Distribution includingApache Hadoop (CDH), version 3 update 4 or later. |
Conventions Used in this Guide:
root
permission have a #
command prompt.root
permission have a $
command prompt.The following instructions describe how to install the Hue tarball on amulti-node cluster. You need to also install CDH and update someHadoop configuration files before running Hue.
Hue consists of a web service that runs on a special node in your cluster.Choose one node where you want to run Hue. This guide refers to that node asthe Hue Server. For optimal performance, this should be one of the nodeswithin your cluster, though it can be a remote node as long as there are nooverly restrictive firewalls. For small clusters of less than 10 nodes,you can use your existing master node as the Hue Server.
You can download the Hue tarball here:http://github.com/cloudera/hue/downloads/
Hue employs some Python modules which use native code and requirescertain development libraries be installed on your system. To install from thetarball, you must have the following installed:
Table 1. Required Dependencies
Redhat | Ubuntu |
---|---|
gcc | gcc |
libxml2-devel | libxml2-dev |
libxslt-devel | libxslt-dev |
cyrus-sasl-devel | libsasl2-dev |
cyrus-sasl-gssapi | libsasl2-modules-gssapi-mit |
mysql-devel | libmysqlclient-dev |
python-devel | python-dev |
python-setuptools | python-setuptools |
python-simplejson | python-simplejson |
sqlite-devel | libsqlite3-dev |
ant | ant |
Configure $PREFIX
with the path where you want to install Hue by running:
You can install Hue anywhere on your system, and run Hue as a non-root user.The Shell application needs root privileges to launch various sub-processes asthe logged in users.It is a good practice to create a new user for Hue and either install Hue inthat user’s home directory, or in a directory within /usr/share
.
Q: I moved my Hue installation from one directory to another and now Hue no. longer functions correctly.
A: Due to the use of absolute paths by some Python packages, you must run aseries of commands if you move your Hue installation. In the new location, run:
Q: Why does 'make install' compile other pieces of software? A: In order to ensure that Hue is stable on a variety of distributions andarchitectures, it installs a Python virtual environment which includes itsdependencies. This ensures that the software can depend on specific versionsof various Python libraries and you don’t have to be concerned about missingsoftware components.
To use Hue, you must install and run Hadoop from CDH3u4 or later. If youare not running this version of CDH or later, upgrade your cluster beforeproceeding.
Table 2. Dependency on CDH Components
Component | Required | Applications | Notes |
---|---|---|---|
HDFS | Yes | Core, Filebrowser | HDFS access through WebHdfs or HttpFS |
MR1 | No | JobBrowser, JobDesigner*, Beeswax* | Job information access through hue-plugins |
Yarn | No | JobDesigner*, Beeswax* | Transitive dependency via Hive or Oozie |
Oozie | No | JobDesigner | Oozie access through REST API |
Hive | No | Beeswax | Beeswax uses the Hive client libraries |
Flume | No | Shell | Optionally provides access to the Flume shell |
HBase | No | Shell | Optionally provides access to the HBase shell |
Pig | No | Shell | Optionally provides access to the Pig shell |
[*] Transitive dependency
You need to enable WebHdfs or run an HttpFS server. To turn on WebHDFS,add this to your hdfs-site.xml
and restart your HDFS cluster.Depending on your setup, your hdfs-site.xml
might be in /etc/hadoop/conf
.
You also need to add this to core-site.html
.
If you place your Hue Server outside the Hadoop cluster, you can runan HttpFS server to provide Hue access to HDFS. The HttpFS service requiresonly one port to be opened to the cluster.
Also add this in httpfs-site.xml
which might be in /etc/hadoop-httpfs/conf
.
Hue communicates with the JobTracker via the Hue plugins, which is a jarfile that you place in your MapReduce lib
directory.
If you JobTracker and Hue are located on the same host, copy it over.If you are using CDH3, your MapReduce library directory might be in /usr/lib/hadoop/lib
.
If you JobTracker runs on a different host, you need to scp
the Hue pluginsjar to the JobTracker host.
Then add this to your mapred-site.xml
and restart your JobTracker.Depending on your setup, your mapred-site.xml
might be in /etc/hadoop/conf
.
You can confirm that the plugins are running correctly by tailing the daemonlogs:
Hue submits MapReduce jobs to Oozie as the logged in user. You need toconfigure Oozie to accept the hue
user to be a proxyuser. Specify this inyour oozie-site.xml
(even in a non-secure cluster), and restart Oozie:
If you are setting $HADOOP_CLASSPATH
in your hadoop-env.sh
, be sureto set it in such a way that user-specified options are preserved. For example:
Correct:
Incorrect:
This enables certain components of Hue to add toHadoop’s classpath using the environment variable.
If your users are likely to be submitting jobs both using Hue and from thesame machine via the command line interface, they will be doing so as the hue
user if they’re using Hue and via their own user account on the command line.This leads to some contention on the directory specified by hadoop.tmp.dir
,which defaults to /tmp/hadoop-${user.name}
. Specifically, hadoop.tmp.dir
is used to unpack jars in bin/hadoop jar
. One work around to this isto set hadoop.tmp.dir
to /tmp/hadoop-${user.name}-${hue.suffix}
in thecore-site.xml file:
Unfortunately, when the variable is unset, you’ll end upwith directories named /tmp/hadoop-user_name-${hue.suffix}
in/tmp
. Despite that, Hue will still work.
The Beeswax server writes into a local directory on the Hue machinethat is specified by |
Hue currently requires that the machines within your cluster can connect toeach other freely over TCP. The machines outside your cluster must be able toopen TCP port 8888 on the Hue Server (or the configured Hue web HTTP port)to interact with the system.
Hue’s Beeswax application helps you use Hive to query your data.It depends on a Hive installation on your system. Please readthis section to ensure a proper integration.
Your Hive data is stored in HDFS, normally under /user/hive/warehouse
(or any path you specify as hive.metastore.warehouse.dir
in yourhive-site.xml
). Make sure this location exists and is writable bythe users whom you expect to be creating tables. /tmp
(on the local filesystem) must be world-writable, as Hive makes extensive use of it.
If you used the embedded Hive MetaStore functionality of Beeswax in Hue fromversions prior to Hue 1.2, read this section. Hue 1.2 includes changes in theHive MetaStore schema that are part of the Hive 0.7 release. If you want to useBeeswax in Hue 1.2, it is imperative that you upgrade the Hive MetaStore schemaby running the appropriate schema upgrade script located in the |
Familiarize yourself with the configuration options inhive-site.xml
. SeeHiveInstallation and Configuration.Having a hive-site.xml
is optional but often useful, particularly on settingup a metastore.You may store the hive-site.xml
in /etc/hue/conf
, or instructBeeswax to locate it using the hive_conf_dir
configuration variable. See/etc/hue/conf/hue.ini
.
In /etc/hue/conf/hue.ini
, modify hive_conf_dir
to point to thedirectory containing hive-site.xml
.
Hue ships with a default configuration that will work forpseudo-distributed clusters. If you are running on a real cluster, you mustmake a few changes to the /etc/hue/hue.ini
configuration file. Thefollowing sections describe the key configuration options you must make toconfigure Hue.
Listing all Configuration Options |
---|
To list all available configuration options, run: This commands outlines the various sections and options in the configuration,and provides help and information on the default values. |
Viewing Current Configuration Options |
---|
To view the current configuration from within Hue, open: |
Using Multiple Files to Store Your Configuration |
---|
Hue loads and merges all of the files with extension |
These configuration variables are under the [desktop]
section inthe /etc/hue/hue.ini
configuration file.
Hue uses a Spawning or a CherryPy web server (configurable). You can usethe following options to change the IP address and port that the web serverlistens on. The default setting is port 8888 on all configured IP addresses.
For security, you should also specify the secret key that is used for securehashing in the session store. Enter a long series of random characters(30 to 60 characters is recommended).
If you don’t specify a secret key, your session cookies will not besecure. Hue will run but it will also display error messages telling you toset the secret key. |
By default, the first user who logs in to Hue can choose anyusername and password and becomes an administrator automatically. Thisuser can create other user and administrator accounts. User information isstored in the Django database in the Django backend.
The authentication system is pluggable. For more information, see theHue SDK Documentation.
You can configure Hue to serve over HTTPS. To do so, you must install'pyOpenSSL' within Hue’s context and configure your keys.
To install pyOpenSSL
, from the root of your Hue installation path,do the following steps:
Run this command:
Configure Hue to use your private key by adding the followingoptions to the /etc/hue/hue.ini
configuration file:
Ideally, you would have an appropriate key signed by a Certificate Authority.If you’re just testing, you can create a self-signed key using the openssl
command that may be installed on your system:
Self-signed Certificates and File Uploads |
---|
To upload files using the Hue File Browser over HTTPS requiresusing a proper SSL Certificate. Self-signed certificates don’twork. |
These configuration variables are under the [hadoop]
section inthe /etc/hue/hue.ini
configuration file.
Hue only support one HDFS cluster currently. That cluster should be definedunder the [[[default]]]
sub-section.
fs.defaultFS
(aka fs.default.name
) in Hadoop configuration./usr/lib/hadoop
./usr/bin/hadoop
./etc/hadoop/conf
.Hue only support one MapReduce cluster currently. That cluster should be definedunder the [[[default]]]
sub-section. Note that JobBrowser only works with MR1.
true
. Hue will be submitting jobs to this MapReduce cluster./usr/lib/hadoop-0.20-mapreduce
for CDH packages. If submit_to
is true for this cluster, this config value becomes the $HADOOP_MAPRED_HOME
for BeeswaxServer and child shell processes./usr/bin/hadoop
./etc/hadoop/conf
.Hue only support one Yarn cluster currently. That cluster should be definedunder the [[[default]]]
sub-section.
true
. Hue will be submitting jobs to this Yarn cluster. But note that JobBrowser will not be able to show MR2 jobs./usr/lib/hadoop-mapreduce
for CDH packages. If submit_to
is true for this cluster, this config value becomes the $HADOOP_MAPRED_HOME
for BeeswaxServer and child shell processes./usr/bin/hadoop
./etc/hadoop/conf
.In the [beeswax]
section of the configuration file, you canoptionally specify the following:
localhost
, and therefore only serves local IPC clients.hive-site.xml
Hive configuration file.In the [jobsub]
section of the configuration file, you shouldspecify:
OOZIE_URL
environment variable for Oozie.In the [useradmin]
section of the configuration file, you canoptionally specify the following:
Hue can detect certain invalid configuration. It will show a red alert icon onthe top navigation bar.
To view the configuration of a running Hue instance, navigate tohttp://myserver:8888/dump_config
, also accessible through the Aboutapplication.
After your cluster is running with the plugins enabled, you can start Hue onyour Hue Server by running:
This will start several subprocesses, corresponding to the different Huecomponents. Your Hue installation is now running.
Now that you’ve installed and started Hue, you can feel free to skip aheadto the Using Hue section. Administrators may want to refer to thissection for more details about managing and operating a Hue installation.
Filebrowser requires Hue to be running as the hue user.
A script called supervisor
manages all Hue processes. The supervisor is awatchdog process — its only purpose is to spawn and monitor other processes.A standard Hue installation starts and monitors the following processes:
runcpserver
- a web server based on CherryPy that provides the core webfunctionality of Huebeeswax server
- a daemon that manages concurrent Hive queriesIf you have installed other applications into your Hue instance, you may seeother daemons running under the supervisor as well.
You can see the supervised processes running in the output of ps -f -u hue
:
Note that the supervisor automatically restarts these processes if they fail forany reason. If the processes fail repeatedly within a short time, the supervisoritself shuts down.
The Hue logs are found in /var/log/hue
, or in a logs
directory under yourHue installation root. Inside the log directory you can find:
access.log
file, which contains a log for all requests against the Hueweb server.supervisor.log
file, which contains log information for the supervisorprocess.supervisor.out
file, which contains the stdout and stderr for thesupervisor process..log
file for each supervised process described above, which containsthe logs for that process..out
file for each supervised process described above, which containsthe stdout and stderr for that process.If users on your cluster have problems running Hue, you can often find errormessages in these log files. If you are unable to start Hue from the initscript, the supervisor.log
log file can often contain clues.
In addition to logging INFO
level messages to the logs
directory, the Hueweb server keeps a small buffer of log messages at all levels in memory. You canview these logs by visiting http://myserver:8888/logs
. The DEBUG
levelmessages shown can sometimes be helpful in troubleshooting issues.
Hue requires a SQL database to store small amounts of data, including useraccount information as well as history of job submissions and Hive queries.By default, Hue is configured to use the embedded database SQLite for thispurpose, and should require no configuration or management by the administrator.However, MySQL is the recommended database to use. This section containsinstructions for configuring Hue to access MySQL and other databases.
The default SQLite database used by Hue is located in: /usr/share/hue/desktop/desktop.db
.You can inspect this database from the command line using the sqlite3
program. For example:
It is strongly recommended that you avoid making any modifications to thedatabase directly using SQLite, though this trick can be useful for managementor troubleshooting.
If you use the default SQLite database, then copy the desktop.db
file toanother node for backup. It is recommended that you back it up on a regularschedule, and also that you back it up before any upgrade to a new version ofHue.
Although SQLite is the default database type, some advanced users may preferto have Hue access an alternate database type. Note that if you elect toconfigure Hue to use an external database, upgrades may require more manualsteps in the future.
The following instructions are for MySQL, though you can also configure Hue towork with other common databases such as PostgreSQL and Oracle.
Tested Database Backends |
---|
Note that Hue has only been tested with SQLite and MySQL database backends. |
To configure Hue to store data in MySQL:
Create a new database in MySQL and grant privileges to a Hue user to manage this database.
To migrate your existing data to MySQL, use the following command to dump the existing database data to a text file. Note that using the '.json' extension is required.
Open the /etc/hue/hue.ini
file in a text editor. Directly below the [[database]]
line, add the following options (and modify accordingly for your MySQL setup):
As the Hue user, configure Hue to load the existing data and create the necessary database tables:
Your system is now configured and you can start the Hue server as normal.
After installation, you can use Hue by navigating to http://myserver:8888/
.The following login screen appears:
The Help application guides users through the various installed applications.
Hue is primarily tested on Firefox, Google Chrome and Safari on Windows, Mac,and Linux.
Your feedback is welcome. The best way to send feedback is to join themailing list, andsend e-mail, to hue-user@cloudera.org.
If you find that something doesn’t work, it’ll often be helpful to include logsfrom your server. (See the Hue Logging section. Please include thelogs as a zip (or cut and paste the ones that look relevant) and send those withyour bug reports.