vicenteg

Your Cluster
===

You can access MCS at these addresses:
{% for webserver in groups['webserver'] %}
  {%- if 'ansible_ec2_public_hostname' in ec2_facts.ansible_facts -%}
* https://{{hostvars[webserver].ec2_facts.ansible_facts.ansible_ec2_public_hostname}}:8443
  {%- else %}
* https://{{hostvars[webserver].ec2_facts.ansible_facts.ansible_ec2_local_ipv4}}:8443
  {%- endif %}

vicenteg

#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# mapr_facts: get facts about a MapR cluster.
#
# Copyright 2014, Vince Gonzalez
# Vince Gonzalez <[email protected]>
#
# This software may be freely redistributed under the terms of the GNU
# general public license version 2.

vicenteg

/*
I'm interested in the EC2 instance types with the most stable spot price.
I want to give myself the best chance of having long-running instances at
the lowest possible price.

I'll get my data via the AWS CLI: 

aws ec2 describe-spot-price-history --product-description "Linux/UNIX" --output text > /tmp/spot_price_history.tsv

Not sure Excel will do too well with this much data, so I'll give Drill a shot.

vicenteg

{
  "id": null,
  "title": "MapR Node Metrics",
  "originalTitle": "MapR Node Metrics",
  "tags": [],
  "style": "dark",
  "timezone": "browser",
  "editable": true,
  "hideControls": false,
  "sharedCrosshair": false,

vicenteg

Keybase proof

I hereby claim:

• I am vicenteg on github.
• I am vincegonzalez (https://keybase.io/vincegonzalez) on keybase.
• I have a public key whose fingerprint is 258C E435 86F2 91DB 28A8 3FCE CF04 4D56 A007 9D2D

To claim this, I am signing this object:

vicenteg

On the client nodes (Redhat):

yum -y install https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder-0.4.0-1.x86_64.rpm

Ubuntu:

wget https://download.elastic.co/logstash-forwarder/binaries/logstash-forwarder_0.4.0_amd64.deb

vicenteg

12/30/2014 00:00:002OyrjyK7Cy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002L12ClmZCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002RGwwgj5Cy7BlzSbmZJFqb12Y9 BbdHr I1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002loTsdnBwCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002GtX9sEspCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002wqpErlPCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:002ALiTlhcCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:0029qEafQzCy7BlzSbmZJFqb12Y9 BbdHr kmI1jY250owst370Y02/12/2015 13:45:0002/12/2015 13:45:00
12/30/2014 00:00:0025r5ZrlECy7BlzSbm

vicenteg

{
  "type": "file",
  "enabled": true,
  "connection": "file:///",
  "workspaces": {
    "root": {
      "location": "/",
      "writable": false,
      "defaultInputFormat": null
    },

vicenteg

#!/bin/sh

REGIONS="ap-southeast-2 ap-southeast-1 ap-northeast-1 us-east-1 us-west-1 us-west-2 eu-west-1 eu-central-1"

for region in $REGIONS; do 
  aws ec2 describe-spot-instance-requests --region $region ~/data/requests/spot-instance-requests-$region.json;
done

for region in $REGIONS; do 
  aws ec2 describe-instances --region $region > ~/data/instances/instances-$region.json;

vicenteg

Tweets

create or replace view MapRTweets as select
CAST(t.`dir3` as INT) as `hour`,
CAST(t.`dir2` as INT) as `day`,
CAST(t.`dir1` as INT) as `month`,
CAST(t.`dir0` as INT) as `year`,
CAST(t.`id` as BIGINT) as `id`,
CAST(t.`user`.`id` as BIGINT) as `user_id`,
CAST(t.`text` as VARCHAR(140)) as `tweet`,