SHACL Versioning Pattern for the Solid Ecosystem

gistfile1.txt

# SHACL Versioning Pattern for the Solid Ecosystem

## Context & Problem Statement

The Solid SHACL Shapes Catalogue (`solid/shapes`) already establishes that published shapes are **immutable contracts** — once merged, their validation rules must not be modified. Evolution happens through new shapes, not mutation of existing ones.

This is a strong foundation, but it doesn't yet address several questions that become critical as the catalogue grows:

- How do consumers know which shape is the "current" version of a concept?
- How do apps discover that a shape they depend on has a successor?
- How do we express the relationship between `AddressShape` and `AddressStrictShape` — is one a refinement, a replacement, or an alternative?
- How do we coordinate versioning across the ecosystem without creating a central bottleneck?

This document proposes a versioning pattern that builds on the catalogue's existing immutability principle and works within the constraints of Solid's decentralised architecture.

---

## Design Principles

1. **Immutability is non-negotiable.** Published shapes are never modified. This is already established and this pattern reinforces it.
2. **Shapes are contracts, not code.** Versioning semantics should reflect the impact on data consumers and producers, not internal development milestones.
3. **Decentralised discovery.** Any app should be able to determine version relationships by reading the shapes graph alone — no external registry required.
4. **Backward compatibility is the default.** New shapes should be additive where possible; breaking changes should be explicit and rare.
5. **Namespace stability.** The `https://solidproject.org/shapes/{domain}#` namespace should remain stable across versions.

---

## Versioning Strategy

### Level 1: Naming Convention (Current Catalogue Approach)

The existing catalogue already supports evolution through naming:

```turtle
address-shape:AddressShape           # Original
address-shape:AddressMinimalShape    # Lighter variant
address-shape:AddressStrictShape     # Stricter variant
```

**This works for variants** (alternative shapes for different validation needs) but does not express temporal succession or deprecation. It's the right approach for *coexisting alternatives*, not for *replacement*.

### Level 2: Version Annotation Metadata

For shapes that represent a deliberate evolution of an earlier shape, add annotation metadata using standard vocabularies:

```turtle
@prefix address-shape: <https://solidproject.org/shapes/address#> .
@prefix sh:    <http://www.w3.org/ns/shacl#> .
@prefix owl:   <http://www.w3.org/2002/07/owl#> .
@prefix dct:   <http://purl.org/dc/terms/> .
@prefix xsd:   <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs:  <http://www.w3.org/2000/01/rdf-schema#> .
@prefix vs:    <http://www.w3.org/2003/06/sw-vocab-status/ns#> .

# ── Original shape (still valid, never modified) ──────────────

address-shape:AddressShape
    a sh:NodeShape ;
    rdfs:label "Address Shape" ;
    rdfs:comment "Validates a postal address using vCard properties." ;
    dct:created "2026-04-01"^^xsd:date ;
    vs:term_status "stable" ;
    sh:targetClass vcard:Address ;
    # ... property shapes ...
    .

# ── Evolved shape (successor) ─────────────────────────────────

address-shape:AddressShape-v2
    a sh:NodeShape ;
    rdfs:label "Address Shape v2" ;
    rdfs:comment "Validates a postal address with structured sub-components." ;
    dct:created "2027-01-15"^^xsd:date ;
    dct:replaces address-shape:AddressShape ;
    vs:term_status "stable" ;
    sh:targetClass vcard:Address ;
    # ... evolved property shapes ...
    .
```

And on the original, once a successor exists:

```turtle
address-shape:AddressShape
    dct:isReplacedBy address-shape:AddressShape-v2 ;
    vs:term_status "archaic" ;
    .
```

### Key Metadata Properties

| Property | Usage | Source Vocabulary |
|---|---|---|
| `dct:created` | Date shape was published | Dublin Core Terms |
| `dct:replaces` | Points from new shape → predecessor | Dublin Core Terms |
| `dct:isReplacedBy` | Points from old shape → successor | Dublin Core Terms |
| `vs:term_status` | Lifecycle stage: `unstable`, `testing`, `stable`, `archaic` | W3C Vocab Status |
| `rdfs:comment` | Human-readable description of what changed | RDFS |
| `owl:versionInfo` | Optional free-text version string | OWL |

### Why Dublin Core `replaces`/`isReplacedBy`?

These are well-established, widely understood predicates already used in the Linked Data ecosystem. They express directional succession without requiring OWL reasoning. They're also already in use across W3C specifications and the broader vocabulary ecosystem, so Solid apps that understand Dublin Core (which most do via `solid-namespace`) can interpret them without new dependencies.

---

## Lifecycle Model

A shape progresses through these stages, expressed via `vs:term_status`:

```
unstable → testing → stable → archaic
```

| Stage | Meaning | Can be modified? |
|---|---|---|
| **unstable** | Draft, in active development. Not yet in the catalogue. | Yes (pre-merge only) |
| **testing** | Published to catalogue for community review. Apps may experiment. | No (immutable once merged) |
| **stable** | Endorsed for production use. Apps should depend on this. | No |
| **archaic** | Superseded by a successor. Still valid, but new apps should use successor. | No (only `dct:isReplacedBy` and `vs:term_status` annotation added) |

**Important:** Moving a shape to `archaic` is the *only* permitted modification to a published shape, and it only adds metadata — it never changes validation semantics.

---

## What Constitutes a Breaking Change?

In the Solid context, a "breaking change" is anything that would cause existing valid Pod data to fail validation, or would prevent an existing app from correctly reading/writing data. Specifically:

### Breaking (requires new shape):
- Adding mandatory properties (`sh:minCount` increased above 0)
- Removing properties that apps may have written
- Changing `sh:datatype` of an existing property
- Changing `sh:targetClass`
- Tightening cardinality (e.g. `[0..*]` → `[1..1]`)
- Changing `sh:nodeKind` (e.g. `sh:Literal` → `sh:IRI`)

### Non-breaking (allowed as annotation to existing shape):
- Adding `vs:term_status`
- Adding `dct:isReplacedBy`
- Adding `rdfs:comment` or `rdfs:seeAlso`
- Adding `owl:versionInfo`

### Requires a new shape variant (not a version bump):
- Adding new *optional* properties
- Adding alternative path constraints
- Loosening cardinality (e.g. `[1..1]` → `[0..*]`)

The distinction between "new version" (`-v2`) and "new variant" (`MinimalShape`, `StrictShape`) matters: versions imply succession; variants imply coexistence.

---

## Namespace & IRI Strategy

### Keep the namespace stable

The existing pattern `https://solidproject.org/shapes/{domain}#` should **not** include version numbers in the namespace:

```
# GOOD — stable namespace, version in the local name
https://solidproject.org/shapes/address#AddressShape
https://solidproject.org/shapes/address#AddressShape-v2

# AVOID — version in namespace creates proliferation
https://solidproject.org/shapes/address/v2#AddressShape
```

**Rationale:** Version-in-namespace means every new version creates a new namespace, which breaks prefix caching, complicates SPARQL queries, and makes discovery harder. The FOAF vocabulary's long-lived success with a fixed namespace demonstrates this principle.

### Use IRIs, not blank nodes, for all shapes

This is already implied by the catalogue's naming conventions but worth making explicit: every shape and every property shape should have a named IRI. This is critical for:
- Cross-version references (`dct:replaces`)
- App-level caching and dependency tracking
- `sh:deactivated` overrides in downstream shapes graphs

**Property shape naming convention:**

```turtle
address-shape:AddressShape-v2-streetAddress
    a sh:PropertyShape ;
    sh:path vcard:street-address ;
    # ...
    .
```

---

## Discovery Pattern

For apps that need to programmatically find the current version of a shape:

### Option A: Follow `dct:isReplacedBy` chain

An app holding a reference to `address-shape:AddressShape` can check whether it has been superseded:

```sparql
SELECT ?current WHERE {
    <https://solidproject.org/shapes/address#AddressShape> 
        dct:isReplacedBy* ?current .
    FILTER NOT EXISTS { ?current dct:isReplacedBy ?newer }
}
```

This traverses the replacement chain to find the terminal (current) shape.

### Option B: Shape index document

As the catalogue grows, maintain a lightweight index at a well-known location:

```turtle
# https://solidproject.org/shapes/index

<https://solidproject.org/shapes/address#AddressShape-v2>
    a sh:NodeShape ;
    vs:term_status "stable" ;
    dct:replaces <https://solidproject.org/shapes/address#AddressShape> .

<https://solidproject.org/shapes/person#PersonShape>
    a sh:NodeShape ;
    vs:term_status "stable" .
```

This gives apps a single document to fetch for the current state of all shapes.

---

## Relationship to Solid 2026 and the W3C Shapes Namespace

The `solid/shapes` catalogue currently uses:

```
https://solidproject.org/shapes/{domain}#
```

The earlier `solid/shapes` README also references the W3C path:

```
https://www.w3.org/ns/solid/shapes/{shortname}
```

**Recommendation:** Clarify whether these are intended to be the same shapes served from two locations (with one redirecting), or separate concerns. For versioning purposes, shapes should have exactly one canonical IRI. If W3C namespace publication is intended for "blessed" shapes that have reached `stable` status, this could be expressed as:

```turtle
address-shape:AddressShape-v2
    rdfs:isDefinedBy <https://solidproject.org/shapes/address> ;
    rdfs:seeAlso <https://www.w3.org/ns/solid/shapes/address> ;
    .
```

---

## Worked Example: Evolving the Address Shape

### Phase 1: Initial shape (Solid 2026 launch)

```turtle
@prefix address-shape: <https://solidproject.org/shapes/address#> .

address-shape:AddressShape
    a sh:NodeShape ;
    rdfs:label "Address Shape" ;
    dct:created "2026-04-01"^^xsd:date ;
    vs:term_status "stable" ;
    sh:targetClass vcard:Address ;
    sh:property address-shape:AddressShape-streetAddress ;
    sh:property address-shape:AddressShape-locality ;
    sh:property address-shape:AddressShape-postalCode ;
    sh:property address-shape:AddressShape-countryName ;
    .

address-shape:AddressShape-streetAddress
    a sh:PropertyShape ;
    sh:path vcard:street-address ;
    sh:datatype xsd:string ;
    sh:maxCount 1 ;
    .

# ... other property shapes ...
```

### Phase 2: Community feedback — apps need structured address lines

Rather than modifying `AddressShape`, a **variant** is created:

```turtle
address-shape:AddressStructuredShape
    a sh:NodeShape ;
    rdfs:label "Structured Address Shape" ;
    rdfs:comment "Address with separate fields for building, street, etc." ;
    dct:created "2026-09-01"^^xsd:date ;
    vs:term_status "testing" ;
    sh:targetClass vcard:Address ;
    # ... structured property shapes ...
    .
```

Both `AddressShape` and `AddressStructuredShape` remain valid and coexist.

### Phase 3: Breaking change needed — country code replaces country name

A new version is required because the datatype changes:

```turtle
address-shape:AddressShape-v2
    a sh:NodeShape ;
    rdfs:label "Address Shape v2" ;
    rdfs:comment "Uses ISO 3166-1 country codes instead of free-text country names." ;
    dct:created "2027-06-01"^^xsd:date ;
    dct:replaces address-shape:AddressShape ;
    vs:term_status "stable" ;
    sh:targetClass vcard:Address ;
    # ... updated property shapes ...
    .
```

And `AddressShape` is annotated (the only permitted change):

```turtle
address-shape:AddressShape
    dct:isReplacedBy address-shape:AddressShape-v2 ;
    vs:term_status "archaic" ;
    .
```

---

## Tooling Recommendations

1. **Sorted Turtle serialisation.** Use deterministic serialisation (e.g. rdflib's Sorted Turtle, or TopBraid's equivalent) for all `.ttl` files in the catalogue. This makes Git diffs meaningful and reviewable.

2. **CI validation.** The catalogue's build script should validate:
   - All shapes parse as valid SHACL
   - All shapes have required metadata (`rdfs:label`, `dct:created`, `vs:term_status`)
   - No `stable` or `archaic` shape has been modified (checksum comparison)
   - `dct:replaces`/`dct:isReplacedBy` links are bidirectional and consistent

3. **Shape diff tooling.** Consider integrating a SHACL diff tool into the PR review process that can automatically classify changes as breaking/non-breaking.

4. **Changelog generation.** Metadata in the shapes graph (`dct:created`, `dct:replaces`, `vs:term_status`) is sufficient to auto-generate a human-readable changelog.

---

## Summary

| Concern | Approach |
|---|---|
| Immutability | Shapes never change once published (existing policy) |
| Variants (coexisting alternatives) | Descriptive naming: `AddressMinimalShape`, `AddressStrictShape` |
| Versions (temporal succession) | Suffix naming: `AddressShape-v2`, linked via `dct:replaces` |
| Lifecycle status | `vs:term_status`: unstable → testing → stable → archaic |
| Namespace | Stable `https://solidproject.org/shapes/{domain}#` — no version in namespace |
| Discovery | `dct:isReplacedBy` chain traversal or shape index document |
| Breaking change definition | Anything that invalidates existing Pod data or breaks existing app reads/writes |
| Serialisation | Sorted Turtle for meaningful Git diffs |
| CI enforcement | Validate metadata completeness, immutability, and link consistency |