Today, many datas are geolocalised (meaning that they have a position in space). They're named GIS datas.
It's not rare that we need to do operations on those, such as aggregations, and there are many optimisations existing to do that.
| # The commands below are a guide to remove a large file that has been | |
| # accidentally committed to a Git repository's history. If the file is | |
| # larger than 100 MB, GitHub will prevent you from pushing your latest | |
| # commits. The annotated steps below should help you remove the large | |
| # file from your commit history, even if you've made new commit since. | |
| # Some Git users advise against rebasing. You can safely use it here | |
| # because you haven't published your changes yet. | |
| # So, you first need to rebase your current branch onto the point that |
Today, many datas are geolocalised (meaning that they have a position in space). They're named GIS datas.
It's not rare that we need to do operations on those, such as aggregations, and there are many optimisations existing to do that.
| from pyspark import SparkContext, SparkConf | |
| import json | |
| import argparse | |
| def fn_to_doc(line): | |
| try: | |
| doc = {} | |
| data = json.loads(line) | |
| doc['data'] = data |
| #!/bin/sh | |
| # Based on instructions found here: http://wiki.razuna.com/display/ecp/FFMpeg+Installation+on+CentOS+and+RedHat#FFMpegInstallationonCentOSandRedHat-InstallX264 | |
| if [ "`/usr/bin/whoami`" != "root" ]; then | |
| echo "You need to execute this script as root." | |
| exit 1 | |
| fi | |
| cat > /etc/yum.repos.d/centos.repo<<EOF |
| import scala.runtime.ScalaRunTime | |
| /** | |
| * A sample code to implement a customized case class manually (for Scala 2.9.2 final). | |
| * | |
| * Id has two properties: name and displayName. 'displayName' has default value that | |
| * is same value as 'name'. | |
| */ | |
| class Id private ( // make primary constructor not to be accessible. (standard case class can do) |
Prereq:
apt-get install zsh
apt-get install git-coreGetting zsh to work in ubuntu is weird, since sh does not understand the source command. So, you do this to install zsh
wget https://github.com/robbyrussell/oh-my-zsh/raw/master/tools/install.sh -O - | zsh