Use smarter defaults for network interface for elasticsearch

[martinb3]

Currently, unless you override, eth1 is used. This causes ES to silently exit if eth1 doesn't exist. We should fall back to eth0.

[pangratz]

👍 I just fell over that

[joerg]

Me too. There actually is a default_interface in ohai, maybe this should be used.

[martinb3]

We originally defaulted to eth1 because we were mostly running in cloud environments where eth1 tended to be site-local and not connected to the internet / did not cost for bandwidth, so it seemed safer for cluster communication. We should investigate what the default interface is in ohai when running in popular clouds to see if it defaults to an internet-facing address or something else.

[joerg]

Ohai parses the output of "route -n", takes the line starting with "0.0.0.0" and get the default_gateway and default_interface from there. So this is definitely an "internet-facing device".
Nonetheless, the default_interface will be correct 80% of the times so it would be a safe bet. So maybe the best way would be to somehow check on which hosting provider we are and based on that set the attribute:

case node['custom_ohai_check']['cloud_provider']
  when 'rackspace'
    default['interface_to_use'] = 'eth1'
    default['something_else'] = 'blubb'
  when 'google'
    <someone else will have to do this and send a pull request>
  else
    default['interface_to_use'] = node['network']['default_interface']
    default['something_else'] = 'blubb'
end

https://github.com/chef/ohai/blob/master/lib/ohai/plugins/linux/network.rb#L332-L334

[patcon]

There's a suggested approach in #167 :)

(my assumption is that eth1 will always be private -- ie "local" in the cloud_v2-speak -- and otherwise use eth0, assumed to be public)