This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# 先にDocker community edition for windowsを下記URLからダウンロードしてインストール | |
# https://store.docker.com/editions/community/docker-ce-desktop-windows | |
# jupyterの入ったdockerイメージをダウンロード | |
docker pull jupyter/datascience-notebook | |
# dockerでjupyter notebookを起動 | |
docker run -d -p 8888:8888 -v "$(pwd):/home/jovyan/work" jupyter/datascience-notebook start-notebook.sh --NotebookApp.token='' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
filename="sampledata.csv" | |
rmax=5000 | |
pmax=150 | |
echo "" > $filename | |
for ((i=0; i < $rmax; i++)); do | |
row="${i}," | |
for ((j=0; j < $pmax; j++)); do | |
row="$row$(($RANDOM % 50))," | |
done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# 必要パッケージのインストール | |
sudo apt-get install git gcc make openssl libssl-dev libbz2-dev libreadline-dev libsqlite3-dev | |
# pyenvのインストール | |
cd | |
git clone git://github.com/yyuu/pyenv.git ${HOME}/.pyenv | |
cat << EOF >> ${HOME}/.profile | |
export PYENV_ROOT=\${HOME}/.pyenv | |
export PATH=\${PYENV_ROOT}/bin:\${PATH} | |
eval "\$(pyenv init -)" | |
EOF |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# pyenvのインストール | |
brew install pyenv | |
# anacondaのインストール | |
pyenv install anaconda3-5.2.0 | |
# 環境変数の設定 | |
cat << EOF >> ${HOME}/.bash_profile | |
export PYENV_ROOT=\${HOME}/.pyenv | |
export PATH=\${PYENV_ROOT}/bin:\${PATH} | |
eval "\$(pyenv init -)" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
script:↲ | |
- ./bin/activator test↲ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
./bin/spark-submit \ | |
--class <main-class> | |
--master <master-url> \ | |
--deploy-mode <deploy-mode> \ | |
... # other options | |
<application-jar> \ | |
[application-arguments] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
>>> textFile.filter(lambda line: "Spark" in line).count() # How many lines contain "Spark"? | |
15 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
./spark-ec2 -k <keypair> -i <key-file> -s <num-slaves> launch <cluster-name> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import sbt._ | |
import Keys._ | |
import play.Project._ | |
import de.johoop.jacoco4sbt._ | |
import JacocoPlugin._ | |
import de.johoop.findbugs4sbt._ | |
import de.johoop.findbugs4sbt.FindBugs._ | |
import de.johoop.cpd4sbt.CopyPasteDetector._ | |
object ApplicationBuild extends Build { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
apply plugin: "java" | |
apply plugin: "eclipse" | |
apply plugin: "maven" | |
apply plugin: "findbugs" | |
apply plugin: "pmd" | |
def defaultEncoding = 'UTF-8' | |
[compileJava, compileTestJava]*.options*.encoding = defaultEncoding | |
sourceCompatibility = 1.7 |
NewerOlder