Fabian Schreiber fabsta

[TOC]

Pure Python

Types

a = 2           # integer
b = 5.0         # float
c = 8.3e5       # exponential
d = 1.5 + 0.5j  # complex

Metadata

#Selecting a database	
USE database;	USE database;

#Listing databases	
SHOW DATABASES;	SHOW DATABASES;

sorting files by user

ls -l | sort -k3,3

find ignore errors
find / -name livy_server 2>/dev/null

model setup

from numpy.random import random, permutation
from scipy import misc, ndimage
from scipy.ndimage.interpolation import zoom

import keras
from keras import backend as K
from keras.utils.data_utils import get_file
from keras.models import Sequential, Model

from IPython.display import FileLink
FileLink(file_name)

Barplot (with order) https://rstudio-pubs-static.s3.amazonaws.com/7433_4537ea5073dc4162950abb715f513469.html

Input

array([[  1.9247e-01,   7.2496e-04,   3.7586e-05,   2.4820e-05,   8.0483e-01,   1.4839e-03,
          3.4440e-06,   4.3349e-04],
       [  7.4949e-02,   2.5567e-04,   9.0141e-05,   2.7097e-04,   3.8967e-01,   8.0172e-04,
          4.2277e-04,   5.3354e-01],
       [  7.3892e-02,   8.5835e-04,   4.3923e-05,   8.5646e-04,   4.6396e-01,   4.9485e-05,
          1.5451e-03,   4.5879e-01],
       [  8.8657e-01,   2.1959e-03,   9.6101e-05,   3.6997e-04,   6.2324e-02,   1.6894e-05,
          3.1924e-05,   4.8398e-02]], dtype=float32)

Observing Model Predictions

source: https://www.cs.utah.edu/~cmertin/dogs+cats+redux.html

First, we need to calculate the predictions on the validation set, since we know those labels, rather than looking at the test set. In [19]:

vgg.model.load_weights(latest_weights_filename)

In [20]:

About data augmentation

Keras comes with very convenient features for automating data augmentation. You simply define what types and maximum amounts of augmentation you want, and keras ensures that every item of every batch randomly is changed according to these settings. Here's how to define a generator that includes data augmentation: In [26]:

dim_ordering='tf' uses tensorflow dimension ordering, which is the same order as matplotlib uses for display. Therefore when just using for display purposes, this is more convenient

gen = image.ImageDataGenerator(rotation_range=10, width_shift_range=0.1,


	# Hive
	```

	```

	# Oracle

	```
	val oracle_db = sqlContext.load("jdbc", Map("url" -> "jdbc:oracle:thin:user/passwd@//server.com:1526/serviceID.com", "dbtable" -> "table"))