Unidata Docker Talk

.gitignore

*~

container.png

ship.png

unidata-docker-talk.org

Unidata Docker Talk

Docker

Source: Wikimedia

What is Docker?

• Borrows from the notion of shipping containers
• Docker is an open source platform for uniformly building, deploying & running software
• Linux only, so most applicable (but not limited) to server-side
• Barrier to entry is low

Containers versus VMs

Image Source

• Docker superficially resembles VM technology, but it is not a VM
• Docker container will weigh 1000^2 bytes versus 1000^3 bytes for a VM
• I think about Docker as smart namespacing of Linux w/ emphasis on re-use

Docker Features

• Lightweight OS isolation (or illusion of…)
• Portability (package & deploy your app on Docker locally or on cloud)
• Codified, version controlled, layered, reproducible environments
• Faster software life cycle
• Legos for cloud computing

Why is this Happening Now?

• The concept of virtualization is not new, but Docker popularized it
• Changes to Linux kernel circa 2009 enabled lightweight virtualization

Docker Goals

• Build once run anywhere
• Deployment of pre-configured applications to local or cloud environments
• Productivity (i.e., build an image to do X and quickly spin it up)
• Reproducible Science by way of defined, codified, version controlled environments
• Running software in isolated environments to not pollute host OS

Docker Installation

• All Linux package managers support Docker
• For development on OS X, use boot2docker VM (brew install)
• For Windows?

What are the parts of Docker?

• Docker daemon: Build, run, distribute containers
• Docker CLI client: Interface to the daemon
• Docker container: Namespaced OS, app related files
• Docker image: A snapshot of the container
• Docker registry: “git/github” for images
• Dockerfile: Simple mini-DSL for building images

Docker Containers

• Containers are an abstraction of the Linux kernel.
• Think about containers as namespacing and isolating the kernel
• Minimal OS, file system, CPU, memory, I/O, and network abstraction
• Light footprint compared to VMs

More about Containers

• Only files related to the application belong in container
• Containers can be configured to only use a certain about of resources
• Best Practice, only one process per container
• Point of clarification: container is an instance of an image

Dockerfile

• Makefile for Docker containers
• A Dockerfile is a formal description of the container
• A Dockerfile has its own simple, mini-DSL for describing a container
• A Dockerfile is a series of instructions written in DSL executed sequentially
• For example RAMADDA Dockerfile

Dockerfile DSL

• FROM <image>:<tag> Set the base image
• RUN Execute commands for building container
• ADD <src>... <dest> Add files from a URI
• COPY Similar to ADD (AFAIK)
• EXPOSE <port> The container will listen on specific port
• VOLUME Mounting a volume from the host OS
• ENV <key> <value> Set environment variables
• WORKDIR Set the working directory
• CMD Default execution process of the container. (There can only be one.)
• Several more instructions

Dockerfile Layered Images

• Each instruction in the Dockerfile is analogous to a commit in a VCS
• Each instruction, and therefore commit creates an intermediate image
• When building an image, look for Using cache
• Docker CLI operations are efficient b/c of reuse of intermediate images
• Intermediate images are stored in local repository and cache

Building an Image with a Dockerfile

• Dockerfile is used to build the container, docker build .
• Put the Dockerfile at the root of the local directory structure
• Put Dockerfile and related files in that directory structure
• Build a container with tag: docker build -t unidata/appxyz
• Building an image will put it local repository. See docker images
• See Unidata Dockerfiles

Docker Daemon & CLI

• To issue docker commands, first start daemon
• Run docker daemon a root: sudo docker -d &
• CLI commands passed along to Docker daemon
• Run docker commands w/ sudo
• There are ways around the sudo requirement (see web)
• (I am using boot2docker, so I will not be using sudo)

docker build

• Build a new image from the source code
• Example: docker build -t unidata/ramadda .
• -t Repository name (and optionally a tag)

docker images

• List all images (not containers)
• Example: docker images

docker run

• Run a command in a new container
• Example: docker run -i -t fedora /bin/bash
• -i Keep STDIN open even if not attached
• -t Allocate a pseudo-TTY
• Example: docker run --rm -i -t -P -v ~/repository:/data/repository unidata/ramadda:latest
• -P Publish all exposed ports to random ports on the host interfaces
• -v Bind mount a volume (e.g., from the host)

docker ps

• List all containers (not images)
• Example: docker ps

docker attach

• Attach to a running container
• Example: docker attach <id>

docker info

• Display system-wide information related to Docker
• Example: docker info

docker diff

• Inspect changes on a container’s file system
• Useful to see where and what has been installed

docker exec

• Run a command in a running container
• Quite useful to attach to running container
• Example: docker exec -it 8e864205c592 bash
• -i Keep STDIN open even if not attached
• -t Allocate a pseudo-TTY

Seeking help from Docker CLI

• docker <command> --help

Inter-Container Communication

• Docker container linking
• Mounts (e.g., data container mounts)
• Shared memory

Controlling Container Resource Consumption

https://docs.docker.com/reference/run/

Sharing at DockerHub

• DockerHub is, you guessed it, “github” but for images
• Share and build-upon images, judiciously
• docker pull Pull an image or a repository from the registry
• docker push Push an image or a repository to the registry
• Unidata on DockerHub

docker pull

• Pull an image or a repository from the registry
• docker pull fedora

docker push

• Push an image or a repository to the registry
• docker push unidata/ramadda

Example Dockerfile Revisited

• RAMADDA Dockerfile

Docker Security

• Do not run random Docker containers
• DockerHub maintains and vets trusted containers
• Only run trusted containers
• Docker has been somewhat secured internally with various stuff disabled within the container
• Docker isn’t sandboxing in a security sense. It’s sandboxing in a deployment sense
• Do not have to run Docker as root

Unidata in the Cloud

• Start building Docker images for Unidata technology offerings
• IDV, THREDDS, RAMADDA, EDEX/CAVE
• Use these images for the Unidata/UCAR cloud and data we distribute
• Educate our community on how to use these images for their purposes and data
• Make the concept of the Motherlode prototype real