My aim is to define a set of rules that can be used to decide whether an incremental data change is backwardly compatible or not, list things to beware of for a particular change, and mitigations that can be put in place.
Clearly it is assumed that the services will be using API versioning, and that a major API version number change would allow any required change to that API. The scope of this document is to explore what changes can be made within a single API version.
There are four main release management models, each of which has its own requirements and challenges in versioning the APIs.
- Lock-step: Releases done together: manifest of services deployed and tested together. No backward compatibility required between releases and no discussion is included here. Development-time backward compatibility may however well still be needed (and that evolution model determined). Advantages of microservice decoupling lost.
- Server-first: The service provider is always upgraded before the clients. May become difficult to enforce if there are circular dependencies between services.
- Client-first: The client is updated ahead of the service implementation. Unlikely to be a system-wide release policy, however included for scenarios where external requirements may require this approach on an individual-case basis.
- Uncontrolled: Both clients and servers can be upgraded independently. Included mostly to highlight how difficult/impossible it is to perform incremental versioning in this environment.
In the examples below the words ‘mandatory’ and ‘optional’ relate to whether the field is required to exist, and be non-null for the recipient to process.
As the server is always upgraded ahead of the clients, the pattern can be used is similar to the Covariance/Contravariance model used in object subclasses, with the restrictions on evolving request data being different from those on evolving response data.
| Modification | Allowed | Details | Notes / Mitigation |
|---|---|---|---|
| None → Optional | ✅ | Server must implement default processing if field not present | |
| None → Mandatory | ❌ | Legacy clients will cause request processing to fail | - |
| Optional → Mandatory | ❌ | Legacy clients will cause request processing to fail | - |
| Mandatory → Optional | ✅ | New server default processing for missing optional only invoked for updated clients | - |
| Mandatory → None | ✅❓ | Client data will be ignored | Aggressive request validation will cause extraneous fields to fail the request |
| Optional → None | ✅❓ | Client data will be ignored | Aggressive request validation will cause extraneous fields to fail the request |
| Type change: More specialised 1 | ❌ | Legacy clients will send data that cause the requests to fail | Add new, optional, field that updated clients can use to process the more specific data. Remove old data-type on major version bump3 |
| Type change: More general 2 | ✅ | Server will process legacy client data in a more general way | - |
| Enumeration addition | ✅ | Legacy Clients will not send | - |
| Enumeration removal | ❓ | Only if field is optional | Legacy Clients will be supported if server is tolerantly implemented - removed field mapped to null. Review Impact |
| Enumeration change | ❌❓ | Conditions for removal apply as above | Safer to Add new, then remove legacy on version bump |
| Modification | Allowed | Details | Notes / Mitigation |
|---|---|---|---|
| None → Optional | ✅ | Legacy clients unaffected by new field | - |
| None → Mandatory | ✅ | Legacy clients unaffected by new field | - |
| Optional → Mandatory | ✅ | Legacy clients unaffected by new field behaviour | - |
| Mandatory → Optional | ❌ | Clients that depend on field may break | Audit Clients |
| Mandatory → None | ❌ | Clients that depend on field may break | Audit Clients |
| Optional → None | ❓ | As long as previous client processing scenarios do not require field | Audit Clients |
| Type change: More specialised 1 | ✅ | Legacy clients can process data as more general type | |
| Type change: More general 2 | ❌ | Legacy clients will receive unexpected data | Add new, optional, field that updated servers can use to return more general data. Remove old data-type on major version bump3 |
| Enumeration addition | ❓ | Only if field is optional | Legacy Clients will work only if tolerantly implemented - new field mapped to null |
| Enumeration removal | ✅ | No effect | - |
| Enumeration change | ❓ | Conditions for addition apply as above | Add, then remove legacy on version bump |
Where the clients are always upgraded ahead of the server. It is effectively the reverse of the server-first model above (the evolution model for the request becomes that for the response and vice versa).
| Modification | Allowed | Details | Notes / Mitigation |
|---|---|---|---|
| None → Optional | ✅ | Legacy server unaffected by new field | - |
| None → Mandatory | ✅ | Legacy server unaffected by new field | - |
| Optional → Mandatory | ✅ | Legacy server unaffected by new field behaviour | - |
| Mandatory → Optional | ❌ | Server that depends on field may break | Audit Server |
| Mandatory → None | ❌ | Server that depends on field may break | Audit Server |
| Optional → None | ❓ | As long as previous server processing scenarios do not require field | Audit Server |
| Type change: More specialised 1 | ✅ | Legacy server can process data as more general type | |
| Type change: More general 2 | ❌ | Legacy server will receive unexpected data | Add new, optional, field that updated clients can use to send more general data. Remove old data-type on major version bump3 |
| Enumeration addition | ❓ | Only if field is optional | Legacy server will work only if tolerantly implemented - new field mapped to null |
| Enumeration removal | ✅ | No effect | - |
| Enumeration change | ❓ | Conditions for addition apply as above | Add, then remove legacy on version bump |
| Modification | Allowed | Details | Notes / Mitigation |
|---|---|---|---|
| None → Optional | ✅ | Client must implement default processing if field not present | |
| None → Mandatory | ❌ | Legacy server will cause response processing to fail | - |
| Optional → Mandatory | ❌ | Legacy server will cause response processing to fail | - |
| Mandatory → Optional | ✅ | New client default processing for missing optional only invoked when server updated | - |
| Mandatory → None | ✅❓ | Server data will be ignored | Aggressive response validation will cause extraneous fields to fail processing |
| Optional → None | ✅❓ | Server data will be ignored | Aggressive response validation will cause extraneous fields to fail processing |
| Type change: More specialised 1 | ❌ | Legacy server will send data that cause the response processing to fail | Add new, optional, field that updated server can use to send the more specific data. Remove old data-type on major version bump3 |
| Type change: More general 2 | ✅ | Client will process legacy server data in a more general way | - |
| Enumeration addition | ✅ | Legacy Server will not send | - |
| Enumeration removal | ❓ | Only if field is optional | Legacy Server OK if server is tolerantly implemented - e.g. removed field mapped to null. Review Impact |
| Enumeration change | ❌❓ | Conditions for removal apply as above | Safer to Add new, then remove legacy on version bump |
| Modification | Allowed | Details | Notes / Mitigation |
|---|---|---|---|
| None → Optional | ✅ | Both sides can support when needed | - |
| None → Mandatory | ❌ | Will break if recipient-first | - |
| Optional → Mandatory | ❌ | Will break if recipient-first | - |
| Mandatory → Optional | ❌ | Will break if sender-first | Audit Recipient Impact |
| Mandatory → None | ❌ | Will break if sender-first | Audit Recipient Impact |
| Optional → None | ❓ | As long as not required by particular processing scenarios and recipient does not aggressively validate | Audit Recipient Impact |
| Type change: More specialised 1 | ❌ | Will break if recipient-first | Add new, optional, field that sender can use to send more specific data. Remove old data-type on major version bump3 |
| Type change: More general 2 | ❌ | Will break if sender-first | Add new, optional, field that sender can use to send more general data. Remove old data-type on major version bump3 |
| Enumeration addition | ❓ | Possible if field is optional | Recipient can accept only if tolerantly implemented - new field processed as null |
| Enumeration removal | ❓ | Possible if field is optional | Recipient can accept only if tolerantly implemented - old field processed as null |
| Enumeration change | ❌ | No | Mitigation: Create completely new (optional) enumeration field3 |
Footnotes:
1: Change of the types used in the data field to be more specific. For example, String --> Int
2: Change of the types used in the data field to be more general. For example, Int --> String
3: The Expand/Contract pattern, also documented by Martin Fowler as Parallel Change