This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Compute score for decoded text in a CTC-trained neural network using TensorFlow: | |
1. decode text with best path decoding (or some other decoder) | |
2. feed decoded text into loss function | |
3. loss is negative logarithm of probability | |
Example data: two time-steps, 2 labels (0, 1) and the blank label (2). | |
Decoding results in [0] (i.e. string containing one entry for label 0). | |
The probability is the sum over all paths yielding [0], these are: [0, 0], [0, 2], [2, 0] | |
with probability |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import asyncio | |
@asyncio.coroutine | |
def open_file(name): | |
print("opening {}".format(name)) | |
return open(name) | |
@asyncio.coroutine | |
def close_file(file): | |
print("closing {}".format(file.name)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// load error messages from a log into memory | |
// then interactively search for various patterns | |
// base RDD | |
val lines = sc.textFile("log.txt") | |
// transformed RDDs | |
val errors = lines.filter(_.startsWith("ERROR")) | |
val messages = errors.map(_.split("\t")).map(r => r(1)) | |
messages.cache() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json, datetime | |
class RoundTripEncoder(json.JSONEncoder): | |
DATE_FORMAT = "%Y-%m-%d" | |
TIME_FORMAT = "%H:%M:%S" | |
def default(self, obj): | |
if isinstance(obj, datetime.datetime): | |
return { | |
"_type": "datetime", | |
"value": obj.strftime("%s %s" % ( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# An example of decoding/encoding datetime values in JSON data in Python. | |
# Code adapted from: http://broadcast.oreilly.com/2009/05/pymotw-json.html | |
# Copyright (c) 2023, Abhinav Upadhyay | |
# All rights reserved. | |
# | |
# Redistribution and use in source and binary forms, with or without | |
# modification, are permitted provided that the following conditions are met: |