2015-04-10-scientific-docker.md

                 ##            .          
           ## ## ##           ==          
        ## ## ## ## ##       ===
     /""""""""""""""""""\___/ ===-
~~~ {~~ ~~~~ ~~~ ~~~ ~~ ~~ ~ /  ===- ~~~
     \       o            __/           
      \____/           __/
       \______________/

        |          |
     __ |  __   __ | _  __   _
    /  \| /  \ /   |/  / _\ | 
    \__/| \__/ \__ |\_ \__  |
     for scientific computing

A Modest Proposal

  In addition to your regular packaging,
build containers for your simulation code

   This will help 'driveby downloads':
   start quickly, just to play around

Why?

      `""==,,__  
        `"==..__"=..__ _    _..-==""_
   -----     .-,`"=/ /\ \""/_)==""``
  / I'm \   ( (    | | | \/ |
  \lazy /    \ '.  |  \;  \ /
      \       |  \ |   |   ||
       \ ,-._.'  |_|   |   ||
        .\_/\     -'   ;   Y
       |  `  |        /    |-.
       '. __/_    _.-'     /'
   jgs        `'-.._____.-'

Installing dependencies suc^H^H^H is a pain

apt-get install cmake make libboost-all-dev
libxml2-dev libxml++2.6-dev libsqlite3-dev pip install -r requirements.txt # in a venv cmake ../src && make # hope it's all good

curl Miniconda.sh | bash conda install packages

  ___________________
 < that's not so bad >
  -------------------
         \   ^__^
          \  (oo)\_______
             (__)\       )\/\
                 ||----w |
                 ||     ||

But now repeat on multiple machines

    .--.          .--.          .--.     
   |o_o |        |o_o |        |o_o |    
   |:_/ |        |:_/ |        |:_/ |    
  //   \ \      //   \ \      //   \ \   
 (|     | )    (|     | )    (|     | )  
/'\_   _/`\   /'\_   _/`\   /'\_   _/`\  
\___)=(___/   \___)=(___/   \___)=(___/  

    .--.          .--.          .--.     
   |o_o |        |o_o |        |o_o |    
   |:_/ |        |:_/ |        |:_/ |    
  //   \ \      //   \ \      //   \ \   
 (|     | )    (|     | )    (|     | )  
/'\_   _/`\   /'\_   _/`\   /'\_   _/`\  
\___)=(___/   \___)=(___/   \___)=(___/  

There's no guarantee that your users' or 
 clusters' machines will have a working
      combination of dependencies.


     (unless they're running NixOS)

            Enter Docker

    _________
   < whales! >
    ---------
           \   ^__^
            \  (oo)\_______
               (__)\       )\/\
                   ||----w |
                   ||     ||

       Nope, containers.

Containers

Wikipedia:

LXC (Linux Containers) is an operating-system-level virtualization environment for running multiple isolated Linux systems (containers) on a single Linux control host.

    _____
   < wat >
    -----
           \   ^__^
            \  (oo)\_______
               (__)\       )\/\
                   ||----w |
                   ||     ||

What are containers?

Run multiple copies of 'Linux' at once.

Each app/container has its own:

• file system
  • libraries, binaries, etc
• PIDs
• network

    ___________
   < like VMs! >
    -----------
           \   ^__^
            \  (oo)\_______
               (__)\       )\/\
                   ||----w |
                   ||     ||

      sort of...

Virtual Machines

 ---------   ---------   ---------
 |  App  |   |  App  |   |  App  |
 |-------|   |-------|   |-------|
 | bins/ |   | bins/ |   | bins/ |
 | libs  |   | libs  |   | libs  |
 |-------|   |-------|   |-------|
 | Guest |   | Guest |   | Guest |
 |  OS   |   |  OS   |   |  OS   |
 |-------------------------------|
 |          Hypervisor           |
 |-------------------------------|
 |           Host OS             |
 |-------------------------------|
 |            Server             |
 ---------------------------------

Containers

 ---------   ---------   ---------
 |  App  |   |  App  |   |  App  |
 |-------|   |-------|   |-------|
 | bins/ |   | bins/ |   | bins/ |
 | libs  |   | libs  |   | libs  |
 |-------------------------------|
 |            Docker             |
 |-------------------------------|
 |           Host OS             |
 |-------------------------------|
 |            Server             |
 ---------------------------------

Many Containers

 ---------   ---------   ---------
 | A | A |   | A | A |   | A | A |
 | p | p |   | p | p |   | p | p |
 | p | p |   | p | p |   | p | p |
 |-------|   |-------|   |-------|
 | bins/ |   | bins/ |   | bins/ |
 | libs  |   | libs  |   | libs  |
 |-------------------------------|
 |            Docker             |
 |-------------------------------|
 |           Host OS             |
 |-------------------------------|
 |            Server             |
 ---------------------------------

Actually...

(Docker manages the containers, but otherwise isn't involved)

 ---------   ---------   ---------
 | A | A |   | A | A |   | A | A |
 | p | p |   | p | p |   | p | p |
 | p | p |   | p | p |   | p | p |
 |-------|   |-------|   |-------|
 | bins/ |   | bins/ |   | bins/ |
 | libs  |   | libs  |   | libs  |
 |-------------------------------|
 |           Host OS             |
 |-------------------------------|
 |            Server             |
 ---------------------------------

  _______________________________
 < so what do you propose again? >
  -------------------------------
         \   ^__^
          \  (oo)\_______
             (__)\       )\/\
                 ||----w |
                 ||     ||

A Modest Proposal

  In addition to your regular packaging,
build containers for your simulation code

    Running the code becomes a simple:
      docker run my-super-simulation

  _______________
 < what do I do? >
  ---------------
         \   ^__^
          \  (oo)\_______
             (__)\       )\/\
                 ||----w |
                 ||     ||

Building images

Simply...

  vim Dockerfile
  docker build -t .... TIME'S UP

Learn more

             docker.com

        Email me: 🐋 @♥☢.ws
([email protected] also works)

Docker and Containers Madison Meetup

A Modest Proposal, Jonathan Swift

Your precious code is a burden
    on its maintainers and
        the publick.
           Eat it.
    Rewrite in Go or Rust.