Skip to content

Instantly share code, notes, and snippets.

@ottomata
Created October 16, 2018 16:59
Show Gist options
  • Save ottomata/7c4845a57f4ac522ffbf38501c2dce81 to your computer and use it in GitHub Desktop.
Save ottomata/7c4845a57f4ac522ffbf38501c2dce81 to your computer and use it in GitHub Desktop.
In {T204177} we received a buncha new hardware. We'd like to use 5 of these as worker nodes as part of a new and separate Hadoop cluster that will run Presto. This cluster will be used to host public data that will be queryable from Cloud VPS networks.
This Hadoop cluster will be called the 'cloud-analytics'. It's Hadoop 'cluster name' in Hadoop configs will be 'cloud-analytics-eqiad', to match the naming convention we have been using for other clusters, e.g. Kafka, Zookeeper, Druid, etc.
The cloud-analytics nodes will also run other softwares (Hive, Presto, etc.). The workers will run on bare metal, but the masters can run on ganati VPS instances. I'll file a separate task for the ganeti instances.
This task is to rack and set up 5 worker nodes. They should be named ca-worker100[1-5] (cloud-analytics).
**These nodes should be in the Analytics VLAN**.
They should also be (eventually) accessible from Cloud VPS networks on restricted (TBD) ports.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment