NG: Image Federation

docker_image_federation.md

What is image layer federation?

Image federation is where dependent image layers are served from different servers. For example, an ISV builds on a Red Hat base image. The ISV layers are served from cdn.isv.com and the Red Hat layers are served from cdn.redhat.com.

The content-addressable v2 image format and registry makes this an ideal time to consider this model.

Why is it important? Who cares?

Many companies require to host their own bits. It's their control point. It's an important legal and provenance issue for them.

How does it work?

A simple example:

$ docker pull isv/app
bef54b8f8a2f <- served from cdn.redhat.com
8da983e1fdd5 <- served from cdn.isv.com

What might implementation look like?

Support pushing of metadata only. This assumes image has landed on CDN by another means.

$ docker push rhel7 --redirect-url https://cdn.redhat.com/registry/images/
bef54b8f8a2f <- pushing metadata only 

When an image based on the above is pushed the layer upload is skipped.

$ docker push isv/app
bef54b8f8a2f <- skipped, metadata already uploaded
8da983e1fdd5 <- layer pushed to registry

Example Implementation

This has been implemented in Crane, a component of Pulp. Red Hat uses this as its production registry. Crane is a read-only implementation of the docker registry protocol. Registry metadata (json) is created by the Pulp server.

Crane serves calls to /v1/repositories/<namespace>/<repository>/images|tags directly and then redirects (302) any calls to /v1/images/<image_id>/*.

In the following example note the two URL values.

Red Hat base image

{
    "images": [
        {
            "id": "bef54b8f8a2fdd221734f1da404d4c0a7d07ee9169b1443a338ab54236c8c91a"
        }
    ],
    "protected": true,
    "repo-registry-id": "rhel7",
    "repository": "redhat-rhel7",
    "tags": {
        "0-23": "bef54b8f8a2fdd221734f1da404d4c0a7d07ee9169b1443a338ab54236c8c91a",
        "latest": "bef54b8f8a2fdd221734f1da404d4c0a7d07ee9169b1443a338ab54236c8c91a"
    },
    "type": "pulp-docker-redirect",
    "url": "https://cdn.redhat.com/images/registry/",
    "version": 1
}

A child ISV image file redirects to another URL.

ISV image

{
    "images": [
        {
            "id": "8da983e1fdd58a2fdd221734f1da404d4c0a7d07ee9169b1443a338b8f8a2fdd"
        }
    ],
    "protected": true,
    "repo-registry-id": "isv/app",
    "repository": "isv-app",
    "tags": {
        "latest": "8da983e1fdd58a2fdd221734f1da404d4c0a7d07ee9169b1443a338b8f8a2fdd"
    },
    "type": "pulp-docker-redirect",
    "url": "https://cdn.isv.example.com/images/registry/",
    "version": 1
}