kimsterv

http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore

Make the keystore:
keytool -genkeypair -alias sg -keyalg RSA -validity 7 -keystore keystore

Take a look at it:
keytool -list -v -keystore keystore

Sign it (makes it into a .cer):

kimsterv

import java.io.IOException;
import java.util.Map;

import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.InputFormat;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.RecordReader;
import org.apache.hadoop.mapreduce.lib.input.LineRecordReader;
import org.apache.pig.LoadFunc;
import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat;

kimsterv

Dude, you came to the exact right person.  He want's Lucchese.  I
wanted some cowboy boots for Christmas last year, and I went out of my
way to make sure they were Lucchese's.  They are $300 and comfortable
as hell, and they also make some higher end ones if interested.  Do
NOT fuck around with Justin or any of the other shitty brands you will
find at Western Warehouse.  Just trust me on this one.  Luchesse's are
the most classic mid-market (ie not $3,000) brand.  They are all
hand-made in San Antonio or El Paso.  Here's a video of them making
some: http://www.youtube.com/watch?v=OycogDJXIRc.  This shit is real.
They are so comfortable.

kimsterv

@Override
public void prepareToRead(@SuppressWarnings("rawtypes") RecordReader reader, PigSplit split)
throws IOException {
    mSplit = split;
    in = reader;
}

//add something like this to your getNext()
if (mSplit != null) {
    FileSplit fs = (FileSplit) mSplit.getWrappedSplit();

kimsterv

Melissa S.	yay fun
it's almost worse to spend the whole day in super air conditioned convention center. but JSM is statistician mecca. 5k+ go every year
4:05 PM
Casey C.	sounds like your odds will be pretty good
*rimshot*
4:10 PM
Jon R.	casey: it may be a *integral* part of her career
Casey C.	especially if her talk is a real outlier
Jon R.	She'll prob need to lim n->30 her talk, or she wont have any time for questions at the end
Casey C.	I bet they'll give her a 5% margin of error

kimsterv

package com.simplegeo.elephantgeo.pig.load;

/*
 * Licensed under the Apache License, Version 2.0 (the "License");
 *
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *

kimsterv

# Copyright 2011 Omniscale (http://omniscale.com)
# 
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# 
#     http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,

kimsterv

/*
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0

kimsterv

#!/bin/bash

for i in `hadoop fs -lsr /flume/incoming/2011/04 | awk '{print $8}'`; do 
        echo $i; hadoop fs -cat $i | gzip -t
        if [ $? -ne 0 ] && [ -n "`echo \"$i\" | grep 'gz$'`" ]; then
                `echo hadoop fs -rmr -skipTrash $i`
        fi
done

kimsterv

import simplejson as json
import sys
import re

load = open('/dev/stdin', 'r')
buf = ''
for line in load:
    buf += line
    if line.strip().endswith('}'):
        try: