vicenteg

12/30/2014 00:00:002OyrjyK7Cy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002L12ClmZCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002RGwwgj5Cy7BlzSbmZJFqb12Y9 BbdHr I1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002loTsdnBwCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002GtX9sEspCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002wqpErlPCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002ALiTlhcCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:0029qEafQzCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:0025r5ZrlECy7BlzSbm

vicenteg

On the client nodes (Redhat):

yum -y install https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpm

Ubuntu:

wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder_0.4.0_amd64.deb

vicenteg

Keybase proof

I hereby claim:

• I am vicenteg on github.
• I am vincegonzalez (https://keybase.io/vincegonzalez) on keybase.
• I have a public key whose fingerprint is 258C E435 86F2 91DB 28A8 3FCE CF04 4D56 A007 9D2D

To claim this, I am signing this object:

vicenteg

{
  "id": null,
  "title": "MapR Node Metrics",
  "originalTitle": "MapR Node Metrics",
  "tags": [],
  "style": "dark",
  "timezone": "browser",
  "editable": true,
  "hideControls": false,
  "sharedCrosshair": false,

vicenteg

/*
I'm interested in the EC2 instance types with the most stable spot price.
I want to give myself the best chance of having long-running instances at
the lowest possible price.

I'll get my data via the AWS CLI: 

aws ec2 describe-spot-price-history --product-description "Linux/UNIX" --output text > /tmp/spot_price_history.tsv

Not sure Excel will do too well with this much data, so I'll give Drill a shot.

vicenteg

#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# mapr_facts: get facts about a MapR cluster.
#
# Copyright 2014, Vince Gonzalez
# Vince Gonzalez <[email protected]>
#
# This software may be freely redistributed under the terms of the GNU
# general public license version 2.

vicenteg

Your Cluster
===

You can access MCS at these addresses:
{% for webserver in groups['webserver'] %}
  {%- if 'ansible_ec2_public_hostname' in ec2_facts.ansible_facts -%}
* https://{{hostvars[webserver].ec2_facts.ansible_facts.ansible_ec2_public_hostname}}:8443
  {%- else %}
* https://{{hostvars[webserver].ec2_facts.ansible_facts.ansible_ec2_local_ipv4}}:8443
  {%- endif %}

vicenteg

DROP TABLE IF EXISTS yelp_business;
DROP TABLE IF EXISTS yelp_reviews;

CREATE TABLE IF NOT EXISTS yelp_business
  (city STRING, review_count INT, name STRING, neighborhoods ARRAY<STRING>,
  type STRING, business_id STRING, full_address STRING, hours STRING,
  state STRING, longitude FLOAT, stars INT, latitude FLOAT, attributes STRING, open BOOLEAN, categories STRING)
  ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';

CREATE TABLE IF NOT EXISTS yelp_reviews

vicenteg

SET hive.exec.compress.output=true;
SET mapred.output.compression.codec=org.apache.hadoop.io.compress.SnappyCodec;
SET mapred.output.compression.type=BLOCK;

CREATE EXTERNAL TABLE IF NOT EXISTS X1_TEST_SNAPPY_B
LIKE yelp_business
LOCATION '/user/testsnappy/';

INSERT INTO TABLE X1_TEST_SNAPPY_B SELECT city,review_count,name,neighborhoods,business_id,full_address,hours,state,longitude,stars,latitude,attributes,open,categories FROM yelp_business;

vicenteg

import requests
import logging

user = "mapr"
password = "mapr"
host = "node1"
disk_list = [ '/dev/sde', '/dev/sdf', '/dev/sdg' ]


def add_disk():