msukmanowsky

Storm Transactional Topologies with Mongo DB State

Strict Transactional State

1. Batches for a given txid are always the same. Replays of batches for a txid will exact same set of tuples as the first time that batch was emitted for that txid.
2. There's no overlap between batches of tuples (tuples are in one batch or another, never multiple).
3. Every tuple is in a batch (no tuples are skipped)

Current DB document State

msukmanowsky

from flask.ext.wtf import Form
from flask.ext.wtf.file import FileField

import imghdr


class ImageFileRequired(object):
    """
    Validates that an uploaded file from a flask_wtf FileField is, in fact an
    image.  Better than checking the file extension, examines the header of

msukmanowsky

package com.parsely.pig.screens;

import java.io.IOException;

import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;

/**
 * This UDF takes a tuple with two positive ints specified 0 -> width
 * and 1 -> height and returns the aspect ratio for the resolution.

msukmanowsky

from collections import defaultdict
try:
    import cStringIO as StringIO
except ImportError:
    import StringIO


class EscapedLineReader(object):
    """Custom reader for files where we could have escaped new lines.