Skip to content

Instantly share code, notes, and snippets.

@avigail-oron
avigail-oron / Aiakos server without docker
Last active July 10, 2018 11:23
Aiakos server without docker
//on a diff machine - install mysql, bind it to all addresses and create a user called 'aiakos' with all priveleges.
//prerequisites: python 3.6, pip
//also, need to have mysql clinet instlled:
sudo apt-get install libmysqlclient-dev
//get aiakos server git repo:
git clone https://gitlab.com/aiakos/aiakos
//set all environmnet variables that aiakos requires on all shells that will be used in the process:
@avigail-oron
avigail-oron / How run aiakos server against MariaDB alpine
Last active July 4, 2018 06:32
How run aiakos server against MariaDB alpine
//run a c docker container of aline that declares using port 3306:
docker run -it -p 3306:3306/tcp <docker-alpine-image-id> /bin/sh
//inside the docker container do the following:
apk update
apk add mysql mysql-client
DB_DATA_PATH="/var/lib/mysql"
DB_ROOT_PASS="mariadb_root_password"
DB_USER="mariadb_user"
DB_PASS="mariadb_user_password"
In it's initialization, python3 calls getrandom() sys call. this crashes the sgx lkl since there's a problem with the syscall.c file.
There are 2 ways to overcome that:
1. fix the sgx-lkl/sgx-lkl-musl/src/misc/syscall.c file to include the missing handling for this syscall
2. workaround it by setting the following env variable in the shell prior to invoking sgx lkl: export PYTHONHASHSEED=<some number>
The modified syscall.c contants:
#include "syscall.h"
#include <stdarg.h>
//prerequisites: python 3.5, pip
//clone sgx-lkl repo
//fix it's getrandom() problem using this fix: https://gist.github.com/avigail-oron/20261038c9f04df80845a45d5e573bad
//set it's iptables according to sgx-lkl instructions
//use the miniroot example app as a basis
//create a python miniapp using the following buildenv.sh:
#!/bin/sh
set -ex
PATH=/usr/sbin:/sbin:/usr/bin:/bin
The deployment is comprised of 3 main components:
=================================================
1. Aiakos provider - running in SGX-LKL
2. MariaDB - running in SGX-LKL
3. Client application - plain docker compose application (application container + mariadb container)
Setting up MariaDB
==================
Take the mariadb sgx-lkl image
@avigail-oron
avigail-oron / Setup Hadoop3 cluster
Last active December 27, 2018 07:33
Setup Hadoop3 cluster
OS: ubuntu 16.04
//To run on each node:
//====================
//Prerequisistes
//--------------
sudo apt install default-jdk
//verify java installed
java -version
@avigail-oron
avigail-oron / spark on yarn
Last active February 13, 2019 12:35
spark on yarn
//Build spark with YARN support, without hadoop dependencies
//(this will prevent clashes when distributing the spark assembly jar over yarn cluster)
build/mvn -Pyarn -Pbigtop-dist -Dhadoop.version=3.0.0 -Phadoop-provided -DskipTests clean package
[Does not work for me...
//Create a spark distribution
./make-distribution.sh --name <some name> --tgz -Pyarn -Pbigtop-dist -Dhadoop.version=3.0.0 -Phadoop-provided -DskipTests
]
//setup the same user in the cluster on your spark clinet machine:
//OS: Ubuntu 16.04
//install Qemu with ARM
sudo apt install qemu-system-aarch64
//create a tap device for the VM - it's part of same network
sudo ip tuntap add dev sptap0 mode tap user ubuntu
sudo ip link set dev sptap0 up
//connect the tap and physical nic via a brisge: