hypriot-testing.md

Testing Xings scripts on Hypriot

HypriotOS: root@black-pearl in ~/pitest on master
$ python discover_register_devices.py 
Device: C8:1E:E7:7C:7F:41 -> Wally's iPhone
[{'name': "Wally's iPhone", 'address': 'C8:1E:E7:7C:7F:41'}]
HypriotOS: root@black-pearl in ~/pitest on master
$ python find_discovered_devices.py 
[{'name': "Wally's iPhone", 'address': 'C8:1E:E7:7C:7F:41'}]
Found Device: C8:1E:E7:7C:7F:41 -> Wally's iPhone
HypriotOS: root@black-pearl in ~/pitest on master
$ cat bluetooth_devices.pkl 
(lp1
(dp2
S'name'
p3
S"Wally's iPhone"
p4
sS'address'
p5
S'C8:1E:E7:7C:7F:41'
p6

Show Docker is available

sa.HypriotOS: root@black-pearl in ~/pitest on master
$ docker info
Containers: 0
Images: 0
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Kernel Version: 3.18.11-hypriotos-v7+
Operating System: Raspbian GNU/Linux 7 (wheezy)
CPUs: 4
Total Memory: 925.1 MiB
Name: black-pearl
ID: AXXT:PXIX:IUSQ:WRZ7:KNMO:P5V3:SX4C:OQWM:A3DJ:NLW6:J5FF:7NZA
Debug mode (server): true
Debug mode (client): false
Fds: 9
Goroutines: 15
System Time: Thu Jul 23 22:39:21 CEST 2015
EventsListeners: 0
Init SHA1: 9183200cc532e132ff6eac70172636ccc5a33724
Init Path: /usr/lib/docker/dockerinit
Docker Root Dir: /var/lib/docker

Use Docker Semantics to save and export data.

We have to use specific ARM-based docker images.

HypriotOS: root@black-pearl in ~/pitest on master
$ docker run -it --name pihack-c2 hypriot/rpi-alpine-scratch sh
HypriotOS: root@black-pearl in ~/pitest on master
$ sudo docker exec pihack-c2 mkdir /data
HypriotOS: root@black-pearl in ~/pitest on master
$ sudo docker exec pihack-c2 touch /data/hi
HypriotOS: root@black-pearl in ~/pitest on master
$ sudo docker exec pihack-c2 ls /data
hi


HypriotOS: root@black-pearl in ~/pitest on master
$ docker ps
CONTAINER ID        IMAGE                               COMMAND             CREATED              STATUS              PORTS               NAMES
259e7f595408        hypriot/rpi-alpine-scratch:latest   "sh"                About a minute ago   Up About a minute                       pihack-c2                   pihack-c1  

We could even make a simple node or python rest server to except GET requests for data (optional)

Save and Export

Export (we can now send around this file to other Pis)

HypriotOS: root@black-pearl in ~/pitest on master
$ docker export -o exported-pihack-c2.tar pihack-c2

HypriotOS: root@black-pearl in ~/pitest on master
$ ls | grep exported
exported-pihack-c2.tar
HypriotOS: root@black-pearl in ~/pitest on master

Then import, once we have this file we can re-run it, and even add more data.

HypriotOS: root@black-pearl in ~/pitest on master
$ cat exported-pihack-c2.tar | docker import - pihack-c2:imported001
b0e4b22c2b9814398d95ed555ff90253881b9b093e0bbd9c839f8a195db9c959
HypriotOS: root@black-pearl in ~/pitest on master
HypriotOS: root@black-pearl in ~/pitest on master
$ docker images | grep imported
pihack-c2                    imported001         b0e4b22c2b98        17 seconds ago      4.971 MB
HypriotOS: root@black-pearl in ~/pitest on master

Run the import container

Data is still there, b/c we saved it the container FS and not via bind mounted volumes.

HypriotOS: root@black-pearl in ~/pitest on master
$ docker run -it pihack-c2:imported001 sh
/ # ls /data/
hi
/ # 

How can we add arbitrary data to the container while its running?

You can find the image here Or you can build the image here

Lets add some data to a file outisde of a container echo "Hello, this is a file" > original_file

Next, start a daemonized scratch container to serve as an isolated point for data

HypriotOS: root@black-pearl in ~
$ docker run -it -d --name ryans2 alpine-custom
5ce0cdaa0b84d755e7f94d953fb845c5e7630cc98faf52bffb7ce6a8d8c2ee18
HypriotOS: root@black-pearl in ~
$ 

Next whenw we want to send some data, we start a netcat session with a filename

HypriotOS: root@black-pearl in ~
$ docker exec ryans2 bash -c "nc -l 4444 > /root/received_data" &
[2] 11858
$ 

Next We can send data into the container via netcat

HypriotOS: root@black-pearl in ~
$ netcat 172.17.0.20 4444 < original_file 
[2]+  Done                    docker exec ryans2 bash -c "nc -l 4444 > /root/received_data"

To verify its inside the container, we can run cat in docker exec

HypriotOS: root@black-pearl in ~
$ docker exec ryans2 cat /root/received_data
Hello, this is a file

All of this is automated

First, import the Raspian ARM custom scratch image $ docker pull wallnerryan/rpi-alpine-custom

Start a new metadata container

Command ./add_metadata_container.sh <name-of-metadata-container>

HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ ./add_metadata_container.sh ryans3
340a3aec91a51435f45c16c471b45012359315fa6584c505135744c0aa1fb285
HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ docker ps
CONTAINER ID        IMAGE                                  COMMAND             CREATED              STATUS              PORTS               NAMES
340a3aec91a5        wallnerryan/rpi-alpine-custom:latest   "/bin/sh -c bash"   23 seconds ago       Up 22 seconds       4444/tcp            ryans3              

Load some data ( use optional parameters to aviod collision when trying more than one example of loading data into container)

Command

(minimum)
python load_data -f /path/to/file.txt -c <container-name>

(optional params)
python load_data -f /path/to/file.txt -o /path/inside/container/fordata/ -c <container-name> -n <filename-inside-container>

example

HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ echo "data is here" > data.txt
HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ python load_data.py -f /root/btpihack2015/tools/data.txt -c ryans3
Loading /root/btpihack2015/tools/data.txt
Into ryans3
At /home/received_data

Show data by running a "cat" inside the metadata container

HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ docker exec ryans3 cat /home/received_data
data is here

Export the metadata container into compressed file (around few hundred MB)

Command

python export_data -c <container-name>
python export_data -o <output-filename> -c <container-name>
python export_data -o <output-filename> -l <output-location (defaults to /opt/)> -c <container-name>

example

HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ python export_data.py -o ryans3-exported.tar -c ryans3
Exporting data from ryans3 
To location /opt/ryans3-exported.tar 

(This is where you would transport the tar)

Import the newly exported data

Command

python import_data.py -f </full/path/to/the.tar> -c <container-name>
python import_data.py -f </full/path/to/the.tar> -c <container-name> -t <tag-for-imported-container (defaults to imported000)>

example

HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ python import_data.py -f /opt/ryans3-exported.tar -c ryans3
Importing /opt/ryans3-exported.tar to ryans3:imported000
cb45516b809414656a2f806a9fa3c16b3880d63f04838370d155197a38cf188a

Run the newly imported container with data still in it

Command

./add_imported_metadata_container.sh <new-name-for-container> <name-of-imported-tar-image>:<tag-for-imported-tar-image>

example

$ ./add_imported_metadata_container.sh ryans3-imported ryans3 imported000
c030fb97592a74cefacf17089fed0e6e8e09901026e0b9975a8e5e348ce4a8dd
HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ docker ps | grep ryans3-imported
c030fb97592a        ryans3:imported000                     "bash"              11 seconds ago      Up 11 seconds                           ryans3-imported     
HypriotOS: root@black-pearl in ~/btpihack2015/tools on master*
$ docker exec ryans3-imported cat /home/received_data
data is here