1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
|
## Overview
This charm provides computation and storage resources for an Apache Hadoop
deployment, and is intended to be used only as a part of that deployment.
This document describes how this charm connects to and interacts with the
other components of the deployment.
## Provided Relations
### datanode (interface: dfs-slave)
This relation connects this charm to the apache-hadoop-hdfs-master charm.
It is a bi-directional interface, with the following keys being exchanged:
* Sent to hdfs-master:
* `private-address`: Address of this unit, to be registered as a DataNode
* Received from hdfs-master:
* `private-address`: Address of the HDFS master unit, to provide the NameNode
* `has_slave`: Flag indicating if HDFS has at least one DataNode
* `port`: Port for HDFS operations (IPC)
* `webhdfs-port`: Port for the NameNode web interface
### nodemanager (interface: mapred-slave)
This relation connects this charm to the apache-hadoop-yarn-master charm.
It is a bi-directional interface, with the following keys being exchanged:
* Sent to yarn-master:
* `private-address`: Address of this unit, to be registered as a NodeManager
* Received from yarn-master:
* `private-address`: Address of the YARN master unit, to provide the ResourceManager
* `has_slave`: Flag indicating if YARN has at least one NodeManager
* `port`: Port for YARN operations (IPC)
* `historyserver-port`: JobHistory port (IPC)
## Required Relations
*There are no required relations for this charm.*
## Manual Deployment
The easiest way to deploy the core Apache Hadoop platform is to use one of
the [apache bundles](https://jujucharms.com/u/bigdata-dev/#bundles).
However, to manually deploy the base Apache Hadoop platform without using one
of the bundles, you can use the following:
juju deploy apache-hadoop-hdfs-master hdfs-master
juju deploy apache-hadoop-hdfs-secondary secondary-namenode
juju deploy apache-hadoop-yarn-master yarn-master
juju deploy apache-hadoop-compute-slave compute-slave -n3
juju deploy apache-hadoop-plugin plugin
juju add-relation yarn-master hdfs-master
juju add-relation secondary-namenode hdfs-master
juju add-relation compute-slave yarn-master
juju add-relation compute-slave hdfs-master
juju add-relation plugin yarn-master
juju add-relation plugin hdfs-master
This will create a scalable deployment with separate nodes for each master,
and a three unit compute slave (NodeManager and DataNode) cluster. The master
charms also support co-locating using the `--to` option to `juju deploy` for
more dense deployments.
|