Skip to content

objectrocket/ansible-hadoop

Folders and files

NameName
Last commit message
Last commit date
Dec 15, 2020
Sep 13, 2019
Jun 3, 2020
Jan 24, 2016
Dec 9, 2016
Dec 9, 2016
Mar 2, 2017
Mar 2, 2017
May 20, 2016
Feb 20, 2017
Jun 3, 2020
Jun 3, 2020
Mar 1, 2016
Mar 20, 2016
May 31, 2016
Mar 31, 2016
Mar 29, 2016
Jan 22, 2020
Jun 3, 2020
Jan 25, 2017
Feb 24, 2017
Mar 1, 2016
Mar 20, 2016
Nov 26, 2019
Feb 21, 2020

Repository files navigation

Build Status

ansible-hadoop

These Ansible playbooks will build a Hadoop cluster.

You can pre-build a Rackspace cloud environment or run the playbooks against an existing environment.


[Installation] (id:installation)

See INSTALL-ENV.md for installation and build instructions.

[Requirements] (id:requirements)

  • Ansible == 2.1.3.0 (2.2 is not supported at the moment)

  • Expects RHEL/CentOS 6/7 or Ubuntu 14 hosts.

  • Building the Rackspace Cloud environment requires the pyrax Python module: pyrax link.

[Features] (id:features)

  • It installs Hortonworks Data Platform using Ambari Blueprints.

  • It supports static inventory if the environment is pre-built (inventory/static file).

  • The data drives can be customized and can be put on top of Cloud Block Storage (partitioning is automatic).

  • If there are 2 or 3 masternodes, it will also enable HA NameNode.

  • Memory settings are scaled with the hardware configuration of the nodes.

[Inventory] (id:inventory)

  • The cloud environment requires the standard pyrax credentials file that looks like this:

    [rackspace_cloud]
    username = my_username
    api_key = 01234567890abcdef
    

    This file will be referenced in playbooks/group_vars/all (the rax_credentials_file variable).

    By default, the file is expected to be: ~/.raxpub.

  • When provisioning HDP on existing infrastructure edit inventory/static and add the nodes.

[Configuration files] (id:configuration)

To customize, change the variables under playbooks/group_vars folder:

  1. playbooks/group_vars/all: contains global cluster and cloud settings
  2. playbooks/group_vars/master-nodes: master-nodes configuration
  3. playbooks/group_vars/slave-nodes: slave-nodes configuration
  4. playbooks/group_vars/edge-nodes: edge-nodes configuration

For a one-node cluster, set cloud_nodes_count in master-nodes to 1 and cloud_nodes_count in slave-nodes to 0.

[Scripts] (id:scripts)

###provision_rax.sh

To provision a cloud environment, run the provision_rax.sh script after you've customized the variables under playbooks/group_vars:

bash provision_rax.sh

Continue with the HDP deployment steps here : HDP Install

About

Ansible playbooks for deploying Hortonworks Data Platform

Resources

Stars

Watchers

Forks

Packages

No packages published