This repository has been archived by the owner on Mar 31, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 2
/
PreparingyourPivotalHDDeploymentforPivotalAX.html
269 lines (212 loc) · 13.2 KB
/
PreparingyourPivotalHDDeploymentforPivotalAX.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<!-- Always force latest IE rendering engine or request Chrome Frame -->
<meta content="IE=edge,chrome=1" http-equiv="X-UA-Compatible">
<!-- Use title if it's in the page YAML frontmatter -->
<title>Preparing your Pivotal HD Deployment for Pivotal AX | Pivotal CF Docs</title>
<link href="../stylesheets/master.css" media="screen,print" rel="stylesheet" type="text/css" />
<link href="../stylesheets/breadcrumbs.css" media="screen,print" rel="stylesheet" type="text/css" />
<link href="../stylesheets/search.css" media="screen,print" rel="stylesheet" type="text/css" />
<link href="../stylesheets/portal-style.css" media="screen,print" rel="stylesheet" type="text/css" />
<link href="../stylesheets/printable.css" media="print" rel="stylesheet" type="text/css" />
<script src="../javascripts/all.js" type="text/javascript"></script>
<link href='/images/favicon.ico' rel='shortcut icon'>
</head>
<body class="pivotalax pivotalax_PreparingyourPivotalHDDeploymentforPivotalAX">
<div class='wrap'>
<script src="//use.typekit.net/clb0qji.js" type="text/javascript"></script>
<script type="text/javascript">
try {
Typekit.load();
} catch (e) {
}
</script>
<script type="text/javascript">
document.domain = "gopivotal.com";
</script>
<header class="navbar desktop-only" id="nav">
<div class="navbar-inner">
<div class="container-fluid">
<div class="pivotal-logo--container">
<a class="pivotal-logo" href="http://gopivotal.com"><span></span></a>
</div>
<ul class="nav pull-right">
<li class="navbar-link">
<a href="http://www.gopivotal.com/paas" id="paas-nav-link">PaaS</a>
</li>
<li class="navbar-link">
<a href="http://www.gopivotal.com/big-data" id="big-data-nav-link">BIG DATA</a>
</li>
<li class="navbar-link">
<a href="http://www.gopivotal.com/agile" id="agile-nav-link">AGILE</a>
</li>
<li class="navbar-link">
<a href="http://www.gopivotal.com/oss" id="oss-nav-link">OSS</a>
</li>
<li class="nav-search">
<a href="http://www.gopivotal.com/search-pivotal"><span></span></a>
</li>
</ul>
</div>
<a href="http://www.gopivotal.com/contact">
<img id="get-started" src="http://www.gopivotal.com/sites/all/themes/gopo13/images/get-started.png">
</a>
</div>
</header>
<div class="main-wrap">
<div class="container-fluid">
<div class="container">
<div id="sub-nav" class="nav-container desktop-only">
<ul class="menu">
<li class="first expanded js-maintopic"><a id='home-nav-link' href="/pivotalax/index.html">Home</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/PivotalAXVersion1_0.html">Overview of Pivotal Analytics</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/InstallationGuide.html">Installation Guide</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/SystemRequirementsandSizingGuidelines.html">System Requirements and Sizing</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/AdministrationGuide.html">Administration Guide</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/DataSourceManagementGuide.html">Data Source Management Guide</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/DataVisualizationGuide.html">Data Visualization Guide</a></li>
<li class="first expanded js-maintopic"><a href="/pivotalax/DataIngestionAPIs.html">Data Ingestion APIs</a></li>
</ul>
</div><!--end of sub-nav-->
<div class="body-container content">
<!-- Google CSE Search Box -->
<script>
(function() {
var cx = '005804290371689664731:n1ix15a_ysy';
var gcse = document.createElement('script');
gcse.type = 'text/javascript';
gcse.async = true;
gcse.src = (document.location.protocol == 'https:' ? 'https:' : 'http:') +
'//www.google.com/cse/cse.js?cx=' + cx;
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(gcse, s);
})();
</script>
<gcse:search></gcse:search>
<ul class="breadcrumbs"><li><a href="./">Pivotal AX Version 1.0 Documentation</a></li> <li class="active"><span>Preparing your Pivotal HD Deployment for Pivotal AX</span></li></ul>
<h3 class="title-container">Preparing your Pivotal HD Deployment for Pivotal AX</h3>
<p>If your Pivotal HD deployment was installed using the Pivotal Command Center software, follow the steps in this section to prepare your PHD deployment for use with PIvotal AX.</p>
<p>If your Pivotal HD deployment was not installed using Pivotal Command Center, skip this section and continue with the steps in <a href="ConfiguringPivotalAX.html">Configuring Pivotal AX</a>.</p>
<p>To complete this procedure, you must have root access to all of the PHD nodes, without having to enter a password.</p>
<ol>
<li> Download the <code>adduserPHD.py </code>tool.</li>
<li> Log in to your Pivotal Command Center node using SSH as the root user. </li>
<li> Create a text file that lists all of the Pivotal HD nodes. Enter the IP address or host name for each node, one per line. Save the file with a name of your choosing. </li>
<li><p>Create a principal for the <strong>paadmin</strong> user in your Pivotal HD cluster: </p>
<ol>
<li> Log in to the Kerberos admin node.</li>
<li><p>Run the following command in the console to start the Kerberos command line:</p>
<p><code>kadmin.local</code></p></li>
<li><p>Run following command in the <code>kadmin.local</code> command shell: </p>
<p><code>addprinc -randkey paadmin@REALM</code></p>
<p>Where<em> <code>REALM</code> </em>is based on your Kerberos setting, for example <code>GOPIVOTAL.COM</code>.</p></li>
<li><p>Retrieve the keytab file using the following command:</p>
<p><code>kadmin.local: xst -norandkey -k paadmin.keytab paadmin@REALM</code></p>
<p>The file <code>paadmin.keytab</code> is written to the current working directory.</p></li>
<li><p>Exit the <code>kadmin.local</code> command shell.</p></li>
<li><p>Run following command to base64 encode the keytab file:</p>
<p><code>base64 paadmin.keytab</code></p>
<p>You will use this output in a later step by copying it and pasting the output in the <strong>Hadoop Kerberos Access Keytab File Content </strong>field in the Pivotal AX section of the Pivotal CF Operations Manager. It may be helpful to paste the output into a text file and save it for later use. See <a href="ConfiguringPivotalAX.html#ConfiguringPivotalAX-hd_kerberos_keytab">Configuring Pivotal AX</a>. </p></li>
</ol></li>
<li><p>Run the <code>adduser</code> tool. For example: </p>
<p><code>$ ./adduserPHD.py <hostfile> -g hadoop -H -I <cluster_name></code></p>
<p>Add the<code> -s</code> option if you are connecting to a secure PHD deployment.</p>
<p>(Where: </p>
<p><em><code><hostfile></code></em> is the file you created that contains the IP address or host name of each Hadoop node and</p>
<p><em><code><cluster_name></code></em> is the name of your Pivotal HD cluster.)</p>
<p>The tool prompts you for the login name, the full name of the user, and the password. Enter <code>paadmin </code>as the login name and create a password for this user. The tool also creates a <code>/user/paadmin</code> endpoint in HDFS. Set the <code>-g</code> option to <code>hadoop</code>, so that the <code>paadmin</code> user becomes a member of the <code>hadoop</code> group.</p></li>
<li><p>Verify these user and group changes by using SSH to log in to each PHD node and running the following command to make sure the <code>paadmin</code> user has been created and is a member of the <code>hadoop</code> group:</p>
<p><code>$ groups paadmin</code></p>
<p>The command should respond with the following:</p>
<p><code>$ paadmin : hadoop</code></p></li>
<li><p>Verify that the endpoint <code>/user/paadmin</code> has been created and is owned by <code>paadmin:hadoop</code> in HDFS by running the following command:</p>
<p><code>$ sudo -u hdfs hdfs dfs -ls /user</code></p>
<p>The command displays a directory listing that should look similar to the following output, indicating that the <code>/user/paadmin</code> endpoint has been created and is owned by <code>paadmin:hadoop:</code></p>
<p><code>$ drwxr-xr-x - paadmin hadoop 0 2013-10-31 14:39 /user/paadmin </code></p></li>
<li><p>Switch to the <code>gpadmin</code> user by running the following command:</p>
<p><code>$ su - gpadmin</code></p></li>
<li><p>Run the following command to retrieve the cluster configuration of your Pivotal HD deployment:</p>
<p><code>$ icm_client fetch-configuration -o <output directory path> -l <Pivotal HD cluster name></code></p>
<p>Where<em><code><output directory path></code></em> is the local directory where the <code>icm_client</code> command saves the cluster configuration file. The file name is <code>hdfs-site.xml</code>.</p></li>
<li><p>Open the <code>hdfs-site.xml</code> file in a text editor and locate the following property: <code>dfs.block.local-path-access.user</code>. The property is defined in an XML block that looks like the following:</p>
<pre><code><property>
<name>dfs.block.local-path-access.user</name>
<value>gpadmin,hdfs,mapred,yarn,hbase,hive</value>
<description>
specify the user allowed to do short circuit read
</description>
</property>
</code></pre></li>
<li><p>In the <code><value></code> element, add the <code>paadmin</code> user. The element now looks like the following:</p>
<pre><code><value>paadmin,gpadmin,hdfs,mapred,yarn,hbase,hive</value>
</code></pre></li>
<li><p>Save the file. </p></li>
<li><p>Run the following command to update the username on all nodes of your Pivotal HD cluster:</p>
<p><code>$ icm_client reconfigure -l <Pivotal HD cluster name> -c <input directory path></code></p>
<p>Where <em><code><input directory path></code></em> is the local directory where you saved the <code>hdfs-site.xml</code> file. </p>
<p>The command prompts your for the root password for the cluster nodes and also asks you to create a password for the <code>gpadmin</code> user. </p>
<p>Depending on the number of nodes in your cluster, this command may take 10 minutes or more to complete. When the update completes you will see a message similar to the following:</p>
<pre><code>Results:
phd3... [Success]
Cluster ID: 27
</code></pre></li>
<li><p>Restart the Pivotal HD cluster using the following command:</p>
<p><code>$ icm_client start -l <Pivotal HD cluster name></code></p>
<p>This command may take some time to complete. </p></li>
</ol>
<p>Continue the installation procedure with <a href="ConfiguringPivotalAX.html">Configuring Pivotal AX</a>.</p>
</div><!-- end of body-container content-->
</div><!-- end of container -->
</div><!--end of container-fluid-->
</div><!--end of main-wrap-->
<div class="site-footer desktop-only">
<div class="container-fluid">
<div class="site-footer-links">
<span class="version"><a href='/'>Pivotal Documentation</a></span>
<span>©
<script>
var d = new Date();
document.write(d.getFullYear());
</script>
<a href='http://gopivotal.com'>Pivotal Software</a> Inc. All Rights Reserved.
</span>
</div>
</div>
</div>
<script type="text/javascript">
(function() {
var didInit = false;
function initMunchkin() {
if(didInit === false) {
didInit = true;
Munchkin.init('625-IUJ-009');
}
}
var s = document.createElement('script');
s.type = 'text/javascript';
s.async = true;
s.src = document.location.protocol + '//munchkin.marketo.net/munchkin.js';
s.onreadystatechange = function() {
if (this.readyState == 'complete' || this.readyState == 'loaded') {
initMunchkin();
}
};
s.onload = initMunchkin;
document.getElementsByTagName('head')[0].appendChild(s);
})();
</script>
<script type="text/javascript">
var _gaq = _gaq || [];
_gaq.push(['_setAccount', 'UA-39702075-1']);
_gaq.push(['_setDomainName', 'gopivotal.com']);
_gaq.push(['_trackPageview']);
(function() {
var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
})();
</script>
</body>
</html>