The Job Designer application enables you to create and submit jobs to the Hadoop cluster. You can include variables with your jobs to enable you and other users to enter values for the variables when they run your job. The Job Designer supports the actions supported by Oozie: MapReduce, Streaming, Java, Pig, Hive, Sqoop, Shell, Ssh, DistCp, Fs, and Email.
Job Designer is one of the applications installed as part of Hue. For information about installing and configuring Hue, see Hue Installation in http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/CDH4-Installation-Guide.html.
In order to run DistCp, Streaming, Pig, Sqoop, and Hive jobs, Oozie must be configured to use the Oozie ShareLib. See Oozie Installation in http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/CDH4-Installation-Guide.html.
Click the Job Designer icon () in the navigation bar at the top of the Hue web page.
The Job Designs page opens in the browser.
A job design specifies several meta-level properties of a job, including the job design name, description, the executable scripts or classes, and any parameters for those scripts or classes.
You can filter the job designs that appear in the list by owner, name, type, and description.
To filter the Job Designs list:
Most job design types support all the settings listed in the following table. For job type specific settings, see: MapReduce, Streaming, Java, Pig, Hive, Sqoop, Shell, Ssh, DistCp, Fs, and Email.
All job design settings except Name and Description support the use of variables of the form $variable_name. When you run the job, a dialog box will appear to enable you to specify the values of the variables.
Name |
Identifies the job and its collection of properties and parameters. |
Description |
A description of the job. The description is displayed in the dialog box that appears if you specify variables for the job. |
Advanced |
|
Prepare |
Specifies paths to create or delete before starting the workflow job. |
Params |
Parameters to pass to a script or command. The parameters are expressed using the JSP 2.0 Specification (JSP.2.3) Expression Language, allowing variables, functions, and complex expressions as parameters. |
Job Properties |
Job properties. To set a property value, click Add Property.
|
Files |
Files to pass to the job. Equivalent to the Hadoop -files option. |
Archives |
Archives to pass to the job. Equivalent to the Hadoop -archives option. |
Hadoop streaming jobs enable you to create MapReduce functions in any non-Java language that reads standard Unix input and writes standard Unix output. For more information about Hadoop streaming jobs, see Hadoop Streaming.
Mapper |
The path to the mapper script or class. If the mapper file is not on the machines on the cluster, use the Files option to pass it as a part of job submission. Equivalent to the Hadoop -mapper option. |
Reducer |
The path to the reducer script or class. If the reducer file is not on the machines on the cluster, use the Files option to pass it as a part of job submission. Equivalent to the Hadoop -reducer option. |
A Fs job design consists of a command that operates on HDFS.
Delete path |
The path to delete. If it is a directory, it deletes recursively all its content and then deletes the directory. |
Create directory |
The path of a directory to create. |
Move file |
The source and destination paths to the file to be moved. |
Change permissions |
The path whose permissions are to be changed, the permissions, and an indicator of whether to change permission recursively. |
A job's input files must be uploaded to the cluster before you can submit the job.
To submit a job design:
After the job is complete, the Job Designer displays the results of the job. For information about displaying job results, see Displaying the Results of Submitting a Job.
To display the Job Submission History:
In the Job Designs window, click the History tab. The jobs are displayed in the Job Submissions History listed by Oozie job ID.
To display Job Details:
In the Job Submission History window, click an Oozie Job ID. The results of the job display: