OCI Distribution Specification Extension: Uncompressed Blob Signaling

Status: Draft Proposal Version: 0.3.0 Last Updated: 2025-11-14

Abstract

This extension enables registries to serve uncompressed blob representations while maintaining cryptographic integrity verification. It defines a two-phase client-server negotiation mechanism using HTTP headers to signal capability and intent.

This specification serves two use cases:

Peer-to-Peer Image Serving: Local registries serving images from uncompressed storage (e.g., containers-storage) to other nodes, VMs, or containers on high-bandwidth local networks.
Production Registry Transition: Migration path for production registries adopting diffid-based storage, with HTTP Content-Encoding as the eventual standard for wire-level compression.

Problem Statement

The OCI image format uses content-addressed storage where blob digests are calculated from compressed layer data. This creates challenges when registries can only serve uncompressed data:

Peer-to-Peer Serving: Container storage systems like containers-storage only maintain uncompressed layer data (diffids), not original compressed blobs. When serving to local VMs or containers over high-bandwidth networks, recompression is wasteful.

Production Registry Migration: Registries wanting diffid-based storage for deduplication need a transition path while the ecosystem has mixed client capabilities.

Specification Extension

1. HTTP Header Definitions

1.1 Client Request Header

Clients MUST send the following header when requesting manifests if they support diffid-based blob fetching:

OCI-Accept-Uncompressed-Blobs: true

This header indicates the client understands diffids and can fetch blobs using uncompressed digests.

Example:

GET /v2/<name>/manifests/<reference> HTTP/1.1
OCI-Accept-Uncompressed-Blobs: true

1.2 Registry Response Header

Registries MAY include the following header when serving manifests:

OCI-Uncompressed-Blobs: <directive>

Valid directives:

only - Registry only has uncompressed blob representations available
preferred - Registry has uncompressed blobs and prefers clients request them
available - Registry has both compressed and uncompressed versions available

The header value is case-insensitive.

Example:

HTTP/1.1 200 OK
Content-Type: application/vnd.oci.image.manifest.v1+json
OCI-Uncompressed-Blobs: only

{...manifest JSON...}

1.3 Header Inclusion Requirements

Registries MUST include OCI-Uncompressed-Blobs only when the client sends OCI-Accept-Uncompressed-Blobs: true.

Registries MUST NOT include the header for clients that don't signal support, as those clients would fail when attempting to fetch blobs using manifest digests they interpret as regular blob digests.

Exception: For GET /v2/<name>/manifests/<digest> (pull-by-digest), registries CANNOT rewrite the manifest as it would change the digest. Registries MUST either:

Serve the original manifest with compressed digests (if available), OR
Return 404 or 400 with appropriate error message

2. Client Behavior

2.1 Manifest Request

Clients supporting uncompressed blob fetching MUST send:

GET /v2/<name>/manifests/<reference> HTTP/1.1
OCI-Accept-Uncompressed-Blobs: true

Clients MUST NOT send this header if they cannot handle diffid-based blob fetching.

2.2 Header Interpretation

For only directive:

Client MUST use the uncompressed digest (diffid) from the manifest's layer descriptor when requesting blobs
If no diffid is available in the descriptor, the client MUST use the digest field directly

For preferred directive:

Client SHOULD attempt to fetch blobs using the uncompressed digest (diffid)
If the uncompressed request returns 404, client MAY fall back to requesting the compressed digest

For available directive:

Client MAY choose compressed or uncompressed blobs based on its preferences

2.3 Blob Request Strategy

When OCI-Uncompressed-Blobs: only is present:

Extract the diffid from:
- The org.opencontainers.image.uncompressed annotation, OR
- The layer's digest field (if registry rewrites manifests)
Request: GET /v2/<name>/blobs/<diffid>
Verify received data matches the diffid

Example manifest with rewritten digests:

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {...},
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "size": 1048576,
      "digest": "sha256:ef567890..."
    }
  ]
}

The digest field contains the diffid, and mediaType indicates uncompressed tar data.

2.4 Pull-by-Digest Fallback

For GET /v2/<name>/manifests/sha256:abc... (digest reference), if the registry cannot serve the original manifest:

Client receives 404 or error
Client MAY retry individual blob requests:
- First attempt: use compressed digest from cached/known manifest
- On 404: fall back to diffid if available
This is best-effort; pull-by-digest with uncompressed-only registries is not guaranteed to work

2.5 Error Handling

When OCI-Uncompressed-Blobs: only is signaled and a diffid blob request fails:

Client SHOULD treat 404 as a fatal error
Client MUST NOT fall back to compressed digests when only is specified

3. Registry Behavior

3.1 Negotiation Logic

If client sends OCI-Accept-Uncompressed-Blobs: true:

Registry MAY include OCI-Uncompressed-Blobs header
Registry MAY rewrite manifest to use diffids (for tag-based pulls)
Client will use diffids for blob requests

If client does NOT send the header:

Registry MUST NOT include OCI-Uncompressed-Blobs header
Registry MUST serve original manifest with compressed digests (if available)
Registry MUST serve compressed blobs, OR return error if unavailable

This prevents old clients from failing when they interpret diffids as regular digests.

3.2 Pull-by-Digest Constraint

For GET /v2/<name>/manifests/<digest>:

Registry CANNOT rewrite manifest (would change digest)
Registry MUST serve original manifest or return error
OCI-Uncompressed-Blobs header MAY still be included to inform client of capabilities

3.3 Blob Response Requirements

When registry includes OCI-Uncompressed-Blobs: only:

MUST respond with 200 OK for blob requests using diffids
MUST respond with 404 Not Found for blob requests using compressed digests
MUST serve uncompressed blob data matching the diffid

3.4 Manifest Rewriting

Registries using OCI-Uncompressed-Blobs: only SHOULD rewrite manifests (when not pull-by-digest):

Replace layer digests with diffids
Update layer mediaTypes to application/vnd.oci.image.layer.v1.tar
Update layer sizes to match uncompressed sizes

Alternatively, registries MAY preserve original digests and add org.opencontainers.image.uncompressed annotations.

4. Security Considerations

4.1 Digest Integrity

Diffids provide the same cryptographic integrity guarantees as compressed digests. SHA-256 collision resistance ensures clients detect any tampering.

4.2 Downgrade Attacks

Clients MUST NOT be vulnerable to downgrade attacks where an attacker removes the OCI-Uncompressed-Blobs header and serves malicious compressed blobs. Mitigation: cache the header value per manifest digest.

4.3 Registry Trust

This extension maintains the existing trust model. Clients already trust registries to serve correct blob data. TLS and authentication remain the primary security mechanisms.

5. Examples

5.1 Complete Pull Flow

Step 1: Client requests manifest

GET /v2/example/app/manifests/v1.0 HTTP/1.1
Host: registry.example.com
Accept: application/vnd.oci.image.manifest.v1+json
OCI-Accept-Uncompressed-Blobs: true

Step 2: Registry responds

HTTP/1.1 200 OK
Content-Type: application/vnd.oci.image.manifest.v1+json
OCI-Uncompressed-Blobs: only

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "config": {...},
  "layers": [
    {
      "mediaType": "application/vnd.oci.image.layer.v1.tar",
      "size": 2048576,
      "digest": "sha256:layer1diffid789..."
    }
  ]
}

Step 3: Client requests blob using diffid

GET /v2/example/app/blobs/sha256:layer1diffid789... HTTP/1.1

Step 4: Registry streams uncompressed blob

HTTP/1.1 200 OK
Content-Type: application/vnd.oci.image.layer.v1.tar
Content-Length: 2048576

<uncompressed tar data>

Step 5: Client verifies

Calculate SHA-256 of received data
Verify it matches sha256:layer1diffid789...

5.2 Old Client (No Negotiation Header)

Request:

GET /v2/example/app/manifests/v1.0 HTTP/1.1
Host: registry.example.com

Response:

HTTP/1.1 404 Not Found

{"errors":[{"code":"MANIFEST_UNKNOWN","message":"compressed manifest not available"}]}

Registry cannot serve this client because it lacks compressed blobs and the client doesn't support diffid-based fetching.

6. Backward Compatibility

6.1 Legacy Client Behavior

Clients that don't send OCI-Accept-Uncompressed-Blobs: true:

Will NOT receive OCI-Uncompressed-Blobs header
Will receive original manifest with compressed digests (if available)
Will receive 404/error from uncompressed-only registries

This is expected. Legacy clients require compressed blobs.

6.2 Mixed-Mode Registries

Registries MAY support both formats:

Accept requests from both old and new clients
Serve compressed blobs to old clients
Serve uncompressed blobs to clients signaling support via header

6.3 Signed Images

Problem: Rewriting manifests breaks signatures covering manifest digests.

Solutions:

Mixed-mode registry: serve original manifests to signature-verifying clients
Verify signatures at registry level before rewriting, store verification metadata
Document that signature verification will fail for rewritten manifests

This requires further standardization work.

7. Relationship to HTTP Content-Encoding

The OCI community discusses using HTTP Content-Encoding for transport-level compression (distribution-spec #235, #301):

Content-Encoding Approach:

Manifests store uncompressed descriptors (digest = diffid)
Registries compress on-the-fly using Content-Encoding: gzip/zstd/br
Clients negotiate via Accept-Encoding
Verification happens after decompression, against diffid

For Peer-to-Peer Serving: The header-based signaling is the complete solution. Content-Encoding is unnecessary on high-bandwidth local networks.

For Production Registries: Content-Encoding is the eventual standard for WAN scenarios. The header-based approach provides a migration path during ecosystem adoption. Both can coexist.

8. Implementation Notes

8.1 cstor-dist Implementation

cstor-dist implements this specification for peer-to-peer serving:

Fetches manifest from containers-storage
Retrieves layer diffids and uncompressed sizes
Rewrites manifest to use diffids
Includes OCI-Uncompressed-Blobs: only header (when client signals support)
Serves uncompressed tar data

This is optimal for serving from storage systems that only maintain uncompressed data to local network clients.

8.2 Performance Considerations

Bandwidth: Uncompressed blobs are ~2.5x larger than typical compressed layers. Acceptable for local networks, prohibitive over WAN.

CPU: Clients and servers save CPU by avoiding compression/decompression.

Optimal Use Cases:

Peer-to-peer serving from local storage (cstor-dist, Kubernetes node-to-node)
VM/container scenarios with high-bandwidth host-to-guest connections
Local development environments

Use Cases Needing Content-Encoding:

Production registries over WAN
CDN-backed registries

9. References

Appendix A: Glossary

Blob: Content-addressable unit of data in a registry
Digest: Cryptographic hash (typically SHA-256) identifying a blob
Diffid: Digest of an uncompressed layer (tar file)
Layer: Filesystem layer in a container image
Manifest: JSON document describing an image's configuration and layers
Registry: Service implementing the OCI Distribution Specification

cgwalters/oci-spec-ext.md