Skip to content

Instantly share code, notes, and snippets.

@aimuz

aimuz/google.aip.dev-llm.txt

Created March 20, 2025 06:21
Show Gist options
  • Save aimuz/4d1b95dde39224163bbe6aa7cc91877b to your computer and use it in GitHub Desktop.
Save aimuz/4d1b95dde39224163bbe6aa7cc91877b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
google.aip.dev-llm.txt
>>>> 0001.md
---
id: 1
state: approved
created: 2018-08-20
placement:
category: meta
order: 10
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# AIP Purpose and Guidelines
As the corpus of Google APIs has grown and the API Governance team has grown to
meet the demand of supporting them, it is increasingly necessary to have a
corpus of documentation for API producers, reviewers, and other interested
parties to reference. The API style guide and introductory One Platform
documentation are intentionally terse and high-level. The AIP collection
provides a way to provide consistent documentation for API design guidance.
## What is an AIP?
AIP stands for **API Improvement Proposal**, which is a design document
providing high-level, concise documentation for API development. They are to
serve as the source of truth for API-related documentation at Google and the
means by which API teams discuss and come to consensus on API guidance. AIPs
are maintained as Markdown files in the [AIP GitHub repository][].
## Types of AIPs
There are several different types of AIPs, described below. The list of AIP
types may evolve over time as necessary.
### Guidance
These AIPs describe guidance on API design. These are provided as instruction
for API producers to help write simple, intuitive, and consistent APIs, and are
used by API reviewers as a basis for review comments.
### Process
These AIPs describe a process surrounding API design. They often affect the AIP
process itself and are used to enhance the way in which AIPs are handled.
## Stakeholders
As with any process there are many different stakeholders when it comes to
reviewing and working with AIPs. Below is a summary of the escalation path
starting with the API producer.
```graphviz
digraph d_front_back {
rankdir=BT;
ranksep=0.3;
node [ style="filled,solid" shape=box fontname="Roboto" ];
producer [ label="API Producer" ];
editors [ label="AIP Editors" ];
tl_infra [ label="Infrastructure TL" ];
tl_design [ label="Design TL" ];
tl [ label="TL" ];
producer -> editors;
editors -> tl_infra -> tl;
editors -> tl_design -> tl;
}
```
As noted in the diagram above, the TL is the final decision-maker on the AIP
process and the final point of escalation if necessary.
### Editors
The editors are the set of people who make decisions on AIPs. The general goal
is that the AIP process is collaborative and that we largely work on the basis
of consensus. However, a limited number of designated approvers is necessary,
and these Googlers will be approvers for each AIP in the general scope.
The list of AIP editors is currently:
- Angie Lin ([@alin04][])
- Jon Skeet ([@jskeet][])
- Jose Juan Zavala Iglesias ([@itsStrobe][])
- Louis Dejardin ([@loudej][])
- Noah Dietz ([@noahdietz][])
- Sam Levenick ([@slevenick][])
- Sam Woodard ([@shwoodard][])
The editors are also responsible for the administrative and editorial aspects
of shepherding AIPs and managing the AIP pipeline and workflow. They approve
PRs to AIPs, assign proposal numbers, manage the agenda, set AIP states, and so
forth. They also ensure that AIPs are readable (proper spelling, grammar,
sentence structure, markup, etc.).
AIP editorship is by invitation of the current editors.
## Domain-specific AIPs
Some AIPs may be specific to a particular domain (for example, only to APIs
within a certain PA, or even a certain team). In this situation, the group will
be given a particular block of AIPs to use in accordance with AIP-2, and the
applicable AIPs will clearly indicate their scope.
## States
At any given time, AIPs may exist in a variety of states as they work their way
through the process. The following is a summary of each state.
### Draft
The initial state for an AIP is the "Draft" state. This means that the AIP is
being discussed and iterated upon, primarily by the original authors. While the
editors _may_ get involved at this stage, it is not necessary.
**Note:** If significant, high-level iteration is required, it is recommended
to draft AIPs in a Google doc instead of a PR. AIPs that are migrated into the
AIP system from Google Docs **may** skip the draft state and go directly to
reviewing provided there is sufficient approval.
### Reviewing
Once discussion on an AIP has generally concluded, but before it is formally
accepted it moves to the "Reviewing" state. This means that the authors have
reached a general consensus on the proposal and the editors are now involved.
At this stage the editors may request changes or suggest alternatives to the
proposal before moving forward.
**Note:** As a formal matter, one AIP approver (other than the author) **must**
provide formal signoff to advance an AIP to the reviewing state. Additionally,
there **must not** be formal objections ("changes requested" on the GitHub PR)
from other approvers.
### Approved
Once an approved AIP has been agreed upon, it enters "approved" state and is
considered "best current practice".
**Note:** As a formal matter, two AIP approvers (other than the author)
**must** provide formal signoff to advance an AIP to the approved state.
Additionally, there **must not** be formal objections ("changes requested" on
the GitHub PR) from other approvers.
### Withdrawn
If an AIP is withdrawn by the author or champion, it enters "withdrawn" state.
AIPs that are withdrawn may be taken up by another champion.
### Rejected
If an AIP is rejected by the AIP editors, it enters "rejected" state. AIPs that
are rejected remain, and provide documentation and reference to inform future
discussions.
### Deferred
If an AIP has not been acted upon for a significant period of time, the editors
may mark it as "deferred".
### Replaced
If an AIP has been replaced by another AIP, it enters "replaced" state. AIP
editors are responsible to provide a notice explaining the replacement and
rationale (the replacement AIP should also clearly explain the rationale).
In general, API producers should rely primarily on AIPs in the "approved"
state.
## Workflow
The following workflow describes the process for proposing an AIP, and moving
an AIP from proposal to implementation to final acceptance.
### Overview
```graphviz
digraph d_front_back {
rankdir=LR;
node [ style="filled,solid" shape=box fontname="Roboto" ];
draft [ label="Draft" fillcolor="orange" ];
reviewing [ label="Reviewing" fillcolor="lightskyblue" ];
approved [ label="Approved" fillcolor="palegreen" ];
withdrawn [ label="Withdrawn" fillcolor="mistyrose" ];
rejected [ label="Rejected" fillcolor="mistyrose" ];
deferred [ label="Deferred" fillcolor="lightsteelblue" ];
replaced [ label="Replaced" fillcolor="lightsteelblue" ];
draft -> reviewing;
draft -> withdrawn [ style=dashed, color=mistyrose3 ];
draft -> rejected [ style=dashed, color=mistyrose3 ];
reviewing -> approved;
reviewing -> withdrawn [ style=dashed, color=mistyrose3 ];
reviewing -> rejected [ style=dashed, color=mistyrose3 ];
draft -> deferred [ style=dashed, color=lightsteelblue3 ];
reviewing -> deferred [ style=dashed, color=lightsteelblue3 ];
approved -> replaced [ style=dashed, color=lightsteelblue3 ];
reviewing -> replaced [ style=dashed, color=lightsteelblue3 ];
}
```
### Proposing an AIP
In order to propose an AIP, first [open an issue][] to circulate the
fundamental idea for initial feedback. It should generally be possible to
describe the idea in a couple of pages.
When proposing a new AIP or changes to an existing one, it is best to reference
prior art and/or example use cases that the proposal will impact, so as to
ensure that the proposal is grounded in a realistic problem space. So, proposals
**should** provide concrete references and/or well-defined examples. Appropriate
material includes, but is not limited to, the following:
- Existing external RFCs or standards
- A corpus of APIs that have aligned on a similar pattern e.g. `Search` methods
- A concrete use case that has yet to be solved that exists or could exist in
one or more APIs e.g. adding an AIP-202 Format for AIP-143 Unicode CLDR
region codes
Once ready, create a PR with a new file in the AIP directory using a file
titled `aip/new.md`. Ensure that the PR is editable by maintainers.
In most circumstances, the editors will assign the proposal an AIP number and
submit the PR with the AIP in the "Reviewing" state. The editors may reject an
AIP outright if they have an obvious reason to do so (e.g. the proposal was
already discussed and rejected in another AIP or is fundamentally unsound), in
which case the PR is not merged.
### Discussing an AIP
Once the PR is merged, the AIP author is responsible for championing the AIP on
a follow-up approval pull request. This means that the author is responsible
for pushing towards consensus around the proposal. This may involve a
discussion at the regularly scheduled meetings for the API Governance team.
The AIP author may modify the AIP over the course of discussion by submitting
follow-up commits to the PR.
### Accepting an AIP
The editors will work together to ensure that qualified proposals do not linger
in review.
To gain final approval, an AIP **must** be approved by, at minimum, the TL with
responsibility over the domain covered by the AIP (either design or
infrastructure) and at least one other editor, with no editors actively
requesting changes.
**Note:** If an AIP editor is the primary author of an AIP, then at least two
_other_ editors must approve it.
Once the AIP is approved, the editors will update the state of the AIP to
reflect this and submit the PR.
### Withdrawing or Rejecting an AIP
The author of an AIP may decide, after further consideration, that an AIP
should not advance. If so, the author may withdraw the AIP by updating the PR
adding a notice of withdrawal with an explanation of the rationale.
Additionally, the author may be unable to get consensus among the group and the
AIP editors may elect to reject the AIP. In this situation, the AIP editors
shall amend the PR adding a notice of rejection with an explanation of the
rationale. In both cases, the AIP editors update the state accordingly and
submit the PR.
### Replacing an AIP
In rare cases, it may be necessary to replace an AIP with another one. This is
not general practice: minor edits to approved AIPs are acceptable, and will be
the common way to tweak guidance. However, if new guidance fundamentally alters
the old guidance in some way, then the AIP editors shall create a new AIP that,
once approved, will replace the old one. The old one then enters "Replaced"
state, and will link to the new, current AIP.
## Changelog
- **2025-01-09**: Add requirement to include references/examples in proposals.
- **2024-09-04**: Updated names of current editors and remove TLs.
- **2023-05-10**: Updated names of current and editors and TLs.
- **2019-07-30**: Further clarified AIP quorum requirements.
- **2019-05-12**: Collapsed AIP approvers and editors into a single position,
relaxed approval rules from full quorum.
- **2019-05-04**: Updated the AIP to refer to GitHub processes, rather than
internal processes.
[aip github repository]: https://github.com/googleapis/aip
[open an issue]: https://github.com/googleapis/aip/issues
[@alin04]:https://github.com/alin04
[@jskeet]: https://github.com/jskeet
[@loudej]: https://github.com/loudej
[@noahdietz]: https://github.com/noahdietz
[@slevenick]: https://github.com/slevenick
[@shwoodard]: https://github.com/shwoodard
[@itsStrobe]: https://github.com/itsStrobe
>>>> 0002.md
---
id: 2
state: approved
created: 2018-08-23
placement:
category: meta
order: 20
---
# AIP Numbering
The AIP system provides a mechanism to index and have a single source of truth
for API Improvement Proposals, as well as iterate on them collaboratively and
transparently.
This document describes the AIP numbering system.
## Assigning AIP Numbers
The AIP editors (see [AIP-1](./0001.md)) are responsible for assigning a number
to each AIP when it is accepted as a draft for review. Importantly, _all_ AIPs
have numbers, not just approved ones. The [AIP Index](/) clearly delineates
which AIPs are approved and binding and which are under discussion.
The editors may decide to reserve a specific block of numbers for groups of
AIPs that are related in some way (for example, that are only scoped to a
specific subset of APIs).
Beyond this, AIP numbers are assigned arbitrarily. In general, the editors will
take the next AIP number off of the stack to assign to a draft AIP, but
occasionally may use a special/joke number if useful for mnemonic or other
reasons.
## AIP Blocks
Currently recognized blocks of AIP numbers are:
### Generally Applicable
- **1-99:** Reserved for meta-AIPs (generally process-related).
- **100-999:** General API design guidance
### Google Product Areas
- **2700-2799:** Apps (Google Workspace)
- **2500-2599:** Cloud
- **3000-3099:** Actions on Google
- **3200-3299:** Firebase
- **4100-4199:** Auth libraries
- **4200-4299:** Client libraries
- **4600-4699:** Geo
To request a block for a specific team that is publishing API guidance or
documentation germane to that specific team, reach out to api-editors@.
## Changelog
- **2019-10-03:** The 3000-3099 block was assigned to Actions on Google.
- **2019-01-26:** The general API design guidance block was expanded to include
100-199.
- **2018-10-24:** The 4600-4699 block was assigned to Google Geo.
- **2018-10-02:** The 2500-2599 block was assigned to Google Cloud Platform.
- **2018-10-02:** The 2700-2799 block was assigned to Google Workspace.
>>>> 0003.md
---
id: 3
state: approved
created: 2023-03-28
placement:
category: meta
order: 30
---
# AIP Versioning
This AIP defines the versioning scheme of the AIPs.
## Guidance
- The AIPs **must** be versioned by date, using the [ISO-8601 format of
`YYYY-MM-DD`](https://www.w3.org/TR/NOTE-datetime), that corresponds to the
date the version was added.
- AIPs versions **must** be available as a tag on the source control system used
to store the AIPs, of the format `v{version}`. Example: `v2023-03-28`.
- The AIPs **must** have a new version when there is a significant change to one
or more AIPs.
- Each AIP **must** include a changelog section, with the date the change was
made and a short description.
## Rationale
Versions serve as reference points to AIPs at a specific point in time. They are
crucial since guidance on an AIP can be reversed, or include significant changes
such that they are no longer similar to the original design. APIs using AIPs may
need to reference older AIP guidance to justify their design choice.
### Rationale for date-based versioning
With date-based versioning, a client can easily find the AIP guidance at the
time an API was authored.
### Why not individually versioned AIPs
An alternative to a universal version is to have specific versions attached to
each AIP.
AIPs often cross-reference one another. If each AIP had a specific version, then
cross-references would also have to specify specific versions of those
referenced AIPs to provide complete guidance.
>>>> 0008.md
---
id: 8
state: approved
created: 2019-05-28
placement:
category: meta
order: 40
---
# AIP Style and Guidance
AIP stands for **API Improvement Proposal**, which is a design document
providing high-level, concise documentation for API design and development. The
goal is for these documents to serve as the source of truth for API-related
documentation at Google and the way API teams discuss and come to consensus on
API guidance.
AIPs are most useful when they are clear and concise, and cover a single topic
or inquiry well. In the same way that AIPs describe consistent patterns and
style for use in APIs, they also _follow_ consistent patterns and style.
## Guidance
- AIPs **must** cover a single, discrete topic, and provide clear, actionable
guidance.
- AIPs **must not** duplicate or contradict guidance in another AIP.
- AIPs **may** also cover what _not_ to do, but **should not** cover _only_
anti-patterns.
- If AIP guidance is conditional (e.g. a design pattern such as Jobs), the
guidance **must** clearly explain under what conditions the guidance should be
followed.
Guidance contained within an AIP **must** be beneficial to one or more
types of clients or their authors, including but not limited to:
- Asset inventories which can be used to audit and analyze resources.
- Command line interfaces for exploration and simple automation.
- Custom controllers (e.g. auto-scalers) which poll live state and adjust
resource configuration accordingly.
- [IaC][] clients for orchestration and automation of multiple resources.
- Recommendation tools which provide guidance on which APIs are useful for
specific use cases, and how to use them.
- SDKs to interact with an API from a programming language, often used heavily
for data-plane operations.
- Security orchestration, automation, and remediation tools.
- Simple scripts to automate or orchestrate tasks.
- Test frameworks.
- Tools that operate on resource data at rest.
- Visual User Interfaces for visualization and one-off manual actions.
- Users.
Examples of improvements include:
- Requiring new proto annotations that enable more descriptive interfaces on
clients (e.g. requiring `singular` and `plural` on a `google.api.resource`
annotation).
AIP guidance **must not** be a significant detriment to a client's usability or
implementation difficulty, or maintenance difficulty.
Examples of detriments include:
- Introduction of a non-uniform pattern in a standard method such that all
clients must introduce additional code without sufficient benefit (e.g. List
behaves like this *except* for resources that start with the name Foo).
- Renames of well-established fields for minor improvements in readability (e.g.
rename `expire_time` to `lapse_time` since `lapse` is a common term in my
service).
While the length of AIPs will necessarily vary based on the complexity of the
question, most AIPs **should** be able to cover their content in roughly two
printed pages.
### File structure
AIPs **must** be written in Markdown, and **must** be named using their
four-digit number (example: `0008.md`). AIPs that serve a specific scope
**must** be in the subdirectory for that scope.
AIPs **must** have appropriate front matter.
```yaml
---
id: 8
state: reviewing
created: 2019-05-28
permalink: /8
redirect_from:
- /08
- /008
- /0008
---
```
Front matter for AIPs **must** include:
- The `aip` key:
- `id`: Required. The ID for the given AIP, as an integer.
- `state`: Required. The current state of the AIP, in all lower-case. The
valid states are listed in [AIP-1][], and common states are `draft`,
`reviewing`, and `approved`.
- `created`: Required. The ISO-8601 date (`yyyy-mm-dd`) when the AIP was
originally drafted, with no quotes.
- `updated`: The ISO-8601 date (`yyyy-mm-dd`) when the AIP was last revised.
- `scope`: The scope for the AIP. This **must** match the directory name for
that scope. Required for AIPs with IDs >= 1000, prohibited otherwise.
- The `permalink` key (required): This **must** be set to
`/{aip.scope}/{aip.id}`. If there is no scope, use `/{aip.id}` instead.
- The `redirect_from` key: This should include a list of any `/{aip.id}`
permutations that a reader would be likely to enter, including:
- `/{aip.id}` (for AIPs where the permalink includes the scope)
- AIP IDs with zero-padding, for each level of zero-padding up to four digits
(for example: `/08`, `/008`, `/0008`).
### Document structure
AIPs **must** begin with a top-level heading with the AIP's title (`# Title`).
The title **should** be a noun (not an imperative). For example, "Bad API
precedents" not "Avoid breaking API precedent".
AIPs **should** then begin with an introduction (with no additional heading),
followed by a `## Guidance` heading. If necessary, the AIP **may** include any
of the following after the guidance, in the following order:
- "Further reading" is a bulleted list of links to other AIPs that are useful to
fully understand the current AIP.
- "Appendices" covering further explanation in the same AIP. These are
relatively rare but are important in cases where an AIP requires a lot of
justification for the decision. Often this is primarily an explanation of
alternatives considered to help explain the guidance.
- "Changelog" is a bulleted list of changes made to the AIP since the first
writing.
The guidance section **may** include subsections that elaborate further on
details. Subsections will automatically create an entry in the table of
contents, and an anchor for citations.
Below is an example AIP shell that uses each major section:
```md
# AIP title
The introductory text explains the background and reason why the AIP exists. It
lays out the basic question, but does not tell the reader what to do.
## Guidance
The "guidance" section helps the reader know what to do. A common format for
the guidance section is a high-level imperative, followed by an example,
followed by a bulleted list explaining the example.
### Subsection
Individual subsections can be cited individually, and further elaborate
details.
## Rationale
The "rationale" section is optional, and helps the reader understand the
motivation behind specific guidance within the AIP.
Deeper explanations of design justification and tradeoffs **must** be in the
rationale instead of other sections, to ensure the rest of the document acts as
an easily actionable reference.
## History
The "history" section is optional, and documents events and context around a
significant edit to an AIP. For example, explanation of rewrite would be
included in this section
While the changelog is a dotted list of one-line summaries of changes to an AIP,
the history section should elaborate on significant events in a descriptive
format.
The section **must not** be used to exhaustively enumerate all changes. This
is what the changelog provides.
## Further reading
A bulleted list of (usually) other AIPs, in the following format:
- [AIP-1](./0001.md): AIP purpose and guidelines
## Changelog
A bulleted list of changes in reverse chronological order, using the following
format:
- **2020-02-18**: Specified ordering.
- **2019-07-01**: Added a subsection clarifying XYZ.
```
AIPs **should** attempt to follow this overall format if possible, but AIPs
**may** deviate from it if necessary (in particular, if the AIP would be more
difficult to understand, even for a reader already accustomed to reading AIPs in
the usual format).
**Note:** Except for the title, AIPs **must** only use the second heading level
(`##`) and above. AIPs **should** only use the second and third heading levels
(`##`, `###`).
### Requirement keywords
AIPs **should** use the following requirement level keywords: "MUST", "MUST
NOT", "SHOULD", "SHOULD NOT", and "MAY", which are to be interpreted as
described in [RFC 2119][].
When using these terms in AIPs, they **must** be lower-case and **bold**. These
terms **should not** be used in other ways.
If "SHOULD" or "SHOULD NOT" are used, they **must** include valid examples of
where other concerns may override the guidance.
**Important:** If rationale is used, it exists to provide background and a more
complete understanding, but **must not** contain guidance (and RFC-2119 terms
**must not** be used).
### Code examples
API design examples in AIPs **should** use [protocol buffers][]. Examples
**should** cover only enough syntax to explain the concept. When using RPCs in
examples, a `google.api.http` annotation **should** be included.
### Referencing AIPs
When AIPs reference other AIPs, the prosaic text **must** use the format
`AIP-XXXX` without zero-padding (e.g., `AIP-8`, not `AIP-0008`), and **must**
link to the relevant AIP. AIP links **may** point to a particular section of the
AIP if appropriate.
**Important:** AIP links **must** use the relative path to the file in the
repository (such as `./0008.md` for core AIPs, or `../0008.md` for AIPs in a
subdirectory); this ensures that the link works both on the AIP site, when
viewing the Markdown file on GitHub, using the local development server, or a
branch.
## Rationale
### Designing for a broad set of clients
API guidance, similar to any software, is most beneficial when there is a clear
purpose and target beneficiary.
The beneficiaries of improved API design are users. These users interact with
APIs via a variety of clients, depending on their use case as enumerated above.
API guidance must in turn consider the impact broadly across these clients.
[aip-1]: ./0001.md
[Infrastructure as Code]: ./0009.md#iac
[IaC]: ./0009.md#iac
[protocol buffers]: https://developers.google.com/protocol-buffers/
[rfc 2119]: https://www.ietf.org/rfc/rfc2119.txt
## Changelog
- **2023-05-20**: Increase API guidance scope to include broad set of clients.
- **2023-03-30**: Removed appendix, added rationale and history to the template.
- **2020-02-18**: Specified reverse chronological ordering for changelog items.
- **2019-08-23**: Added guidance for internal AIP links.
>>>> 0009.md
---
id: 9
state: approved
created: 2019-08-01
placement:
category: meta
order: 100
---
# Glossary
In the name of brevity, this AIP defines some common terminology here rather
than in each AIP individually.
## Guidance
The following terminology **should** be used consistently throughout AIPs.
### API
Application programming interface. This can be a local interface (such as a
client library) or a Network API (defined below).
### API backend
A set of servers and related infrastructure that implements the business logic
for an API service. An individual API backend server is often called an API
server.
### API consumer
The entity that consumes an API service. For Google APIs, it typically is a
Google project that owns the client application or the server resource.
### API definition
The definition of an API, usually defined in a Protocol Buffer service. An API
definition can be implemented by any number of API services.
### API frontend
A set of servers plus related infrastructure that provides common functionality
across API services, such as load balancing and authentication. An individual
API frontend server is often called an API proxy.
**Note:** the API frontend and the API backend may run next to each other or far
away from each other. In some cases, they can be compiled into a single
application binary and run inside a single process.
### API method
An individual operation within an API. It is typically represented in Protocol
Buffers by an `rpc` definition, and is mapped to a function in the API in most
programming languages.
### API producer
The entity that produces an API service. For Google APIs, it typically is a
Google team responsible for the API service.
### API product
An API service and its related components, such as Terms of Service,
documentation, client libraries, and service support, are collectively presented
to customers as a API product. For example, Google Calendar API.
**Note:** people sometimes refer to an API product simply as an API.
### API service
A deployed implementation of one or more APIs, exposed on one or more network
addresses, such as the Cloud Pub/Sub API.
### API service definition
The combination of API definitions (`.proto` files) and API service
configurations (`.yaml` files) used to define an API service. The schema for
Google API service definition is `google.api.Service`.
### API service endpoint
Refers to a network address that an API service uses to handle incoming API
requests. One API service may have multiple API service endpoints, such as
`https://pubsub.googleapis.com` and `https://content-pubsub.googleapis.com`.
### API service name
Refers to the logical identifier of an API service. Google APIs use RFC 1035 DNS
compatible names as their API service names, such as `pubsub.googleapis.com`.
### API title
Refers to the user-facing product title of an API service, such as "Cloud Pub/Sub
API".
### API request
A single invocation of an API method. It is often used as the unit for billing,
logging, monitoring, and rate limiting.
### API version
The version of an API or a group of APIs if they are defined together. An API
version is often represented by a string, such as "v1", and presents in API
requests and Protocol Buffers package names.
### Client
Clients are programs that perform a specific tasks by calling an API or generic
tools, such as CLIs, that expose the API in a user-accessible fashion or operate
on resource data at rest.
Examples of clients include the following:
- Command line interfaces
- Libraries, such as an SDK for a particular programming language
- Scripts that operates on a JSON representation of a resource after reading it
from an API
- Tools, such as a [Declarative clients][]
- Visual UIs, such as a web application
### Google API
A Network API exposed by a Google service. Most of these are hosted on the
`googleapis.com` domain. It does not include other types of APIs, such as client
libraries and SDKs.
### Declarative Clients
Declarative Clients, also known as Infrastructure as Code (IaC), describes a
category of clients that consumes a markup language or code that represents
resources exposed by an API, and executes the appropriate imperative actions to
drive the resource to that desired state. To determine what changes to make and
if a set of updates was successful a declarative client compares server side
resource attributes with client defined values. The comparison feature ensures
accuracy of a creation or an update but it requires services to treat the client
set fields as read-only and diligently preserve those values.
Examples of complexities that declarative clients abstract away include:
- Determining the appropriate imperative action (create / update / delete) to
achieve desired state.
- Ordering of these imperative actions.
[Terraform][] is an example of such a client.
### User
A human being which is using an API directly, such as with cURL. This term is
defined to differentiate usage in the AIPs between a human *user* and a
programmatic *client*.
### Network API
An API that operates across a network of computers. Network APIs communicate
using network protocols including HTTP, and are frequently produced by
organizations separate from those that consume them.
[Declarative clients]: #declarative-clients
[Terraform]: https://www.terraform.io/
## Changelog
- **2024-12-18**: Downcase headings and terms as per dev docs style
- **2024-10-23**: Add API Title entry
- **2023-07-24**: Rename IaC to Declarative Clients
- **2023-04-01**: Adding definition of IaC
- **2023-03-24**: Reformatting content to include anchor links.
>>>> 0100.md
---
id: 100
state: approved
created: 2018-08-27
placement:
category: process
order: 10
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# API Design Review FAQ
API design review exists to ensure a simple, intuitive, and consistent API
experience throughout our API corpus.
## Do I need API design approval?
**TL;DR:** You usually need API design approval if you are launching an API
that users can code against (either now or in the future) at the beta or GA
quality level.
API design review is fundamentally about ensuring we provide a simple and
consistent experience for our users, and therefore is only expected for APIs
that users code directly against.
The following flowchart illustrates whether or not your API needs to go through
the design review process:
```graphviz
digraph {
node [ style="filled,solid" shape=box fontname="Roboto" ];
graph [ splines=ortho, nodesep=0.2 ];
audience [ label="Who should code directly\nagainst this API?"
shape=diamond fillcolor=bisque ];
subgraph audience_responses {
rank = "same";
node [ shape=oval fillcolor=orange ];
googlers [ label="Googlers\nOnly" fillcolor=lightcoral ];
public [ label="Anyone" fillcolor=limegreen ];
partners [ label="Partners\nOnly" fillcolor=deepskyblue ];
partners -> public -> googlers [ style=invisible arrowhead=none ];
}
subgraph and_ever {
rank = "same";
node [ shape=diamond fillcolor=bisque ];
forever_partners [ label="Forever?" shape=diamond fillcolor=bisque ];
forever_googlers [ label="Forever?" shape=diamond fillcolor=bisque ];
}
subgraph forever_responses {
rank = "same";
node [ shape=oval fillcolor=orange ];
forever_partners_yes [ label="Yes" fillcolor=deepskyblue ];
forever_no [ label="No, anyone\neventually" fillcolor=limegreen ];
forever_googlers_yes [ label="Yes" fillcolor=lightcoral ];
}
release_level [ label="What release\nlevel?" shape=diamond fillcolor=bisque ];
# Not required is not in the subgraph with the other outcomes because
# it makes the graph much smaller if it can be higher in the image.
subgraph release_levels {
rank = "same";
node [ shape=oval ];
alpha [ label="Alpha" fillcolor=darkorange ];
beta [ label="Beta" fillcolor=goldenrod1 ];
ga [ label="GA" fillcolor=limegreen ];
alpha -> beta -> ga [ style=invisible arrowhead=none ];
}
changes [ label="Any changes\nfrom beta?" shape=diamond fillcolor=bisque ];
subgraph changes_bool {
rank = "same";
node [ shape=oval ];
changes_yes [ label="Yes" fillcolor=goldenrod1 ];
changes_no [ label="No" fillcolor=limegreen ];
}
subgraph outcome {
rank = "same";
node [ style="rounded,filled" ];
fyi [ label="FYI" fillcolor=lightblue ];
recommended [ label="Recommended" fillcolor=limegreen ];
required [ label="⚠ Required" fillcolor=goldenrod1 ];
not_required
[ label="Not Required" fillcolor=lightgrey style="rounded,filled" ];
recommended -> required -> fyi -> not_required
[style=invisible arrowhead=none];
}
audience -> googlers [ arrowhead=none style=dashed color=grey ];
audience -> partners [ arrowhead=none ];
audience -> public [ arrowhead=none ];
partners -> forever_partners;
googlers -> forever_googlers [ style=dashed color=grey ];
public -> release_level;
forever_partners -> forever_partners_yes [ arrowhead=none ];
forever_googlers -> forever_googlers_yes
[ arrowhead=none style=dashed color=grey ];
forever_partners -> forever_no [ arrowhead=none ];
forever_googlers -> forever_no [ arrowhead=none ];
forever_partners_yes -> recommended;
forever_googlers_yes -> not_required [ style=dashed color=grey ];
forever_no -> release_level;
release_level -> alpha [ arrowhead=none ];
release_level -> beta [ arrowhead=none ];
release_level -> ga [ arrowhead=none ];
alpha -> recommended;
beta -> required;
ga -> changes;
changes -> changes_yes [ arrowhead=none ];
changes -> changes_no [ arrowhead=none ];
changes_yes -> required;
changes_no -> fyi;
}
```
### Who should code directly against it?
One of the more complex questions is, "Who should code directly against this
API?" API design review is primarily concerned about the API's audience. This
means we care about who is permitted to write their own HTTP/gRPC calls against
the service, and who is able to see the documentation. (We do _not_ care about
questions such as whether the service is exposed on the public network.)
Design review is expected if the general public is intended to read
documentation and write code that interacts with the service.
The following situations **do not** require design review:
- An API which will only ever be used by Googlers, or internal tools (for
example, Pantheon).
- An API which will only ever be called by an executable program released by
Google (even if the API could be reverse-engineered from the executable).
- An API which will only ever be called by a single customer or small set of
customers under contract, and which will _never_ be made more widely
available. (Design review is still recommended in this case, but not
required.)
### Alpha
For alpha, API design review is optional but recommended. It may often make
sense to endeavor to get initial feedback from customers quickly, and launching
an alpha can be a way of gaining data to determine the best answer to some
usability questions; therefore, bypassing review may be expedient. On the other
hand, launching an alpha requires building an implementation which then takes
engineering effort to update if the API design review at the beta stage raises
concerns. Because API design review can precede implementation work, we
recommend a design review for alpha.
## Why is design review important?
**TL;DR:** Product excellence.
Our design review process exists to ensure that the APIs that we present to
customers are **simple**, **intuitive**, and **consistent**. Your reviewer
approaches your API from the standpoint of a naïve user, thinks through the
resources and actions that your API provides, and attempts to make the surface
as accessible and extensible as possible.
Your design reviewer is not only evaluating your API, but also checking to
ensure that your API is consistent with Google's existing corpus of APIs. Many
customers use multiple APIs, and therefore it is important that our conventions
and naming choices line up with customer expectations.
## What should I expect?
### How long does the review process take?
Reviewers make an effort to keep up with their assigned reviews and offer
feedback frequently, so as not to cause unnecessary delay, but it's generally
best to begin the review process early in case there are delays.
The design review process varies based on the size and complexity of the
underlying API surface:
- Incremental changes to existing APIs generally take a few days.
- Small APIs usually take around a week.
- Entirely new APIs with large surfaces tend to take no less than a week. In
cases with extraordinarily large surfaces (e.g., Cloud AutoML), reviews may
take a month or more to go through design review.
### How do reviewers approach my API?
API reviewers seek to approach your API the same way that your users will, by
focusing primarily on the API surface and its user-facing documentation. In an
ideal world, your API reviewer will ask the types of design questions that
users will ask (and nudge the API toward raising fewer of those questions in
the first place).
### What is precedent?
In general, we want Google APIs to be as consistent as possible. Once customers
learn their first Google API, it should be easier to learn the second (and then
the third, and so on) because we are using the same patterns consistently.
We refer to **precedent** to mean decisions that have already been made by
previous APIs, which generally should be binding upon newer APIs in similar
situations. The most common example of this is naming: we have a list of
[standard fields][] that dictate how we use common terms like `name`,
`create_time`, and so forth, and which also dictates that we always attach _the
same_ name to the same concept.
Precedent also applies to _patterns_. All APIs should implement pagination the
same way. Ditto for long-running operations, import and export, and so on. Once
a pattern has been established, we seek to implement that pattern the same way
wherever it is germane.
## What should I do?
### ...if I have a launch on a tight deadline?
The best thing that you can do is to engage design review as early as possible.
Additionally, make your reviewers aware of your timeline so that they are
aware, and can endeavor to provide you the best possible service. We _want_ you
to make your deadline if at all possible.
For time-sensitive _alpha_ launches, an API **may** launch without receiving
design review approval. Such launches **must** be limited to a known set of
users. In this case, the reviewers will provide notes for the API team to take
under consideration for subsequent stages.
**Warning:** Launching an API in alpha with an incomplete design review **does
not** enshrine that API's decisions. Design review will be required to promote
the API to beta, and API reviewers will block your beta launch if there are
issues.
For launch stages after alpha, the API design review is mandatory due to its
impact on user experience across the board. Your team's inconsistencies affect
more than just your team.
In some cases, there is a difficult choice to be made between product
excellence and either engineering effort or deadlines. These are difficult
business decisions and we understand that they are sometimes necessary;
however, a director or VP must make an explicit choice to put these other
concerns ahead of product excellence when choosing to bypass design review or
disregard reviewers' feedback.
### ...to make my review go faster?
A few tips:
- Begin API review as early as possible, and follow up frequently.
- Run the [API linter][] beforehand. (If you are disabling the linter at any
point, explain why. Reviewers often find that the linter is disabled because
it did its job.)
- Ensure that every message, RPC, and field is _usefully_ commented. Comments
should be in valid American English and say something meaningful.
- If your API reviewer asks you to explain something, add the explanation _in
the proto comments_, rather than the code review conversation. This will very
often save you a round trip.
### ...if one of my API reviewers is unresponsive?
Reach out to the reviewer on Chat. If that fails, reach out to the other
reviewer, who will coordinate accordingly. If that fails also, escalate
according to [AIP-1][].
### ...if I have a design question?
The first places to look at the [API style guide][], the [AIP index][], and
other public APIs within Google. Other public APIs are particularly valuable;
it is common that someone has encountered a situation that is germane to your
question.
### ...if I have a question not covered there?
Reach out to [email protected] with your question.
This generally works best when you are seeking guidance on a specific question
related to API design, and when you clearly explain your use case and provide
examples.
**Note:** The membership of this list comprises almost exclusively volunteers,
who spend the majority of their time doing something else. We do our best to be
responsive, but please be patient with us.
### ...if a question is complex and languishing in a CL?
While the code review interface is the best way to resolve questions when
practical, sometimes there are issues that are sufficiently complicated that
working them out in the code review tool is not feasible. In this situation,
reach out to your reviewers and ask to schedule a meeting. In general, most
issues can be discussed in 30 minutes.
When this happens, make sure that someone documents what is discussed in the
CL, so that the history is preserved.
### ...if my API needs to violate a standard?
Clearly document (using an internal comment in the proto) that you are
violating an API design guideline and your rationale for doing so. This comment
**must** be prefixed with `aip.dev/not-precedent`.
In general, your rationale for the design guideline violation **should** be in
accordance with one of the enumerated reasons listed in [AIP-200][]. If it is
not, work together with your API reviewer to determine the right thing to do.
### ...if a reviewer is bringing up a previously-settled issue?
If you have a different reviewer from your API's previous stages, this might
happen. In general, the best approach is simply to reference the code review
where the issue was decided. Reviewers want to avoid causing you churn, and
therefore usually give deference to previous reviews. This is usually
sufficient to resolve the question promptly.
Occasionally, the reviewer may believe that the previous reviewer made a
significant mistake, and that correcting it is important. In this case, you and
your reviewer should work together to determine the best course of action.
### ...if the team and the reviewers strongly disagree?
Escalate according to [AIP-1][].
## Does my PA or team have any particular guidelines?
The Cloud PA has specific guidelines to ensure additional uniformity across
Cloud, and Cloud APIs have their own reviewer pool. Other teams may adopt
similar (but not necessarily identical) rules and systems. Some teams that
produce multiple APIs (for example, machine learning) may also have guidelines
that apply to that group of APIs.
In all cases, we endeavor to make these guidelines available as AIPs; the
higher AIP numbers are reserved for specific PA and team use (see [AIP-2][]),
and these AIPs are listed in the [AIP index][].
[aip-1]: ./0001.md
[aip-2]: ./0002.md
[aip-200]: ./0200.md
[aip index]: /
[api linter]: https://github.com/googleapis/api-linter
[api style guide]: https://cloud.google.com/apis/design/
[standard fields]: https://cloud.google.com/apis/design/standard_fields
>>>> 0111.md
---
id: 111
state: approved
created: 2023-06-17
placement:
category: api-concepts
order: 50
---
# Planes
Resources and methods on an API can be divided into the *plane* that they reside
or perform operations upon. For the context of APIs, the following planes are
defined:
- Management plane: a uniform, resource-oriented API that primarily
configures and allows retrieval of resources.
- Data plane: a heterogenous API (ideally resource-oriented) that reads and
write user data. Often connects to entities provisioned by the management
plane, such as virtual machines.
The term "plane" was originally used in networking architecture. Although system
and network architecture often defines additional planes (e.g. control plane or
power planes), as the AIPs are focused on the interface, they are not defined in
this AIP.
## Guidance
### Management Plane
Management resources and methods exist primarily to provision, configure, and
audit the resources that the data plane interfaces with.
As an example, the following are considered management resources for a cloud
provider:
- virtual machines
- virtual private networks
- virtual disks
- a blob store instance
- a project or account
### Data Plane
Methods on the data plane operate on user data in a variety of data formats, and
generally interface with a resource provisioned via a management plane API.
Examples of data plane methods include:
- writing and reading rows in a table
- pushing to or pulling from a message queue
- uploading blobs to or downloading blobs from a blob store instance
Data plane APIs **may** be heterogenous across a larger API surface, due to
requirements including high throughput, low latency, or the need to adhere to an
existing interface specification (e.g. ANSI SQL).
- For convenience, resources and methods that operate on the data plane **may**
expose themselves via resource-oriented management APIs. If so, those resources
and methods **must** adhere to the requirements of the management plane as
specified in the other AIPs ([AIP-131][] through [AIP-135][]).
### Major distinctions between management and data plane
- [Declarative clients][] operate on the management plane exclusively.
- Data planes are often on the critical path of user-facing functionality, and
therefore:
- Have higher availabilty requirements than management planes.
- Are more performance-sensitive than management planes.
- Require higher-throughput than management planes.
[AIP-131]: ./0131.md
[AIP-135]: ./0136.md
[Declarative clients]: ./0009.md#declarative-clients
## Changelog
- **2023-06-10**: Introduction of this AIP.
>>>> 0121.md
---
id: 121
state: approved
created: 2019-01-26
placement:
category: resource-design
order: 10
---
# Resource-oriented design
Resource-oriented design is a pattern for specifying [RPC][] APIs, based on
several high-level design principles (most of which are common to recent public
HTTP APIs):
- The fundamental building blocks of an API are individually-named _resources_
(nouns) and the relationships and hierarchy that exist between them.
- A small number of standard _methods_ (verbs) provide the semantics for most
common operations. However, custom methods are available in situations where
the standard methods do not fit.
- Stateless protocol: Each interaction between the client and the server is
independent, and both the client and server have clear roles.
Readers might notice similarities between these principles and some principles
of [REST][]; resource-oriented design borrows many principles from REST, while
also defining its own patterns where appropriate.
## Guidance
When designing an API, consider the following (roughly in logical order):
- The resources (nouns) the API will provide
- The relationships and hierarchies between those resources
- The schema of each resource
- The methods (verbs) each resource provides, relying as much as possible on
the standard verbs.
### Resources
A resource-oriented API **should** generally be modeled as a resource
hierarchy, where each node is either a simple resource or a collection of
resources.
A _collection_ contains resources of _the same type_. For example, a publisher
has the collection of books that it publishes. A resource usually has fields,
and resources may have any number of sub-resources (usually collections).
**Note:** While there is some conceptual alignment between storage systems and
APIs, a service with a resource-oriented API is not necessarily a database, and
has enormous flexibility in how it interprets resources and methods. API
designers **should not** expect that their API will be reflective of their
database schema. In fact, having an API that is identical to the underlying
database schema is actually an anti-pattern, as it tightly couples the surface
to the underlying system.
### Methods
Resource-oriented APIs emphasize resources (data model) over the methods
performed on those resources (functionality). A typical resource-oriented API
exposes a large number of resources with a small number of methods on each
resource. The methods can be either the standard methods ([Get][], [List][],
[Create][], [Update][], [Delete][]), or [custom methods][].
If the request to or the response from a standard method (or a custom method in
the same *service*) **is** the resource or **contains** the resource, the
resource schema for that resource across all methods **must** be the same.
| Standard method | Request | Response |
| --------------- | --------------------- | ---------------------- |
| Create | Contains the resource | Is the resource |
| Get | None | Is the resource |
| Update | Contains the resource | Is the resource |
| Delete | None | None |
| List | None | Contains the resources |
*The table above describes each standard method's relationship to the resource,
where "None" indicates that the resource neither **is** nor **is contained** in
the request or the response*
A resource **must** support at minimum [Get][]: clients must be
able to validate the state of resources after performing a mutation such
as [Create][], [Update][], or [Delete][].
A resource **must** also support [List][], except for [singleton resources][]
where more than one resource is not possible.
**Note:** A custom method in resource-oriented design does _not_ entail
defining a new or custom HTTP verb. Custom methods use traditional HTTP verbs
(usually `POST`) and define the custom verb in the URI.
APIs **should** prefer standard methods over custom methods; the purpose of
custom methods is to define functionality that does not cleanly map to any of
the standard methods. Custom methods offer the same design freedom as
traditional RPC APIs, which can be used to implement common programming
patterns, such as database transactions, import and export, or data analysis.
### Strong Consistency
For methods that operate on the [management plane][], the completion of those
operations (either successful or with an error, long-running operation, or
synchronous) **must** mean that the state of the resource's existence and all
user-settable values have reached a steady-state.
[Output only][] values unrelated to the resource [state][] **should** also have
reached a steady-state for values that are related to the resource [state][].
Examples include:
- Following a successful create that is the latest mutation on a resource, a get
request for a resource **must** return the resource.
- Following a successful update that is the latest mutation on a resource, a get
request for a resource **must** return the final values from the update
request.
- Following a successful delete that is the latest mutation on a resource, a get
request for a resource **must** return `NOT_FOUND` (or the resource with the
`DELETED` state value in the case of [soft delete][])
Clients of resource-oriented APIs often need to orchestrate multiple operations
in sequence (e.g. create resource A, create resource B which depends on A), and
ensuring that resources immediately reflect steady user state after an operation
is complete ensures clients can rely on method completion as a signal to begin
the next operation.
[Output only][] fields ideally would follow the same guidelines, but as
these fields can often represent a resource's live state, it's sometimes
necessary for these values to change after a successful mutation operation to
reflect a state change.
### Stateless protocol
As with most public APIs available today, resource-oriented APIs **must**
operate over a [stateless protocol][]: The fundamental behavior of any
individual request is independent of other requests made by the caller.
This is to say, each request happens in isolation of other requests made by that
client or another, and resources exposed by an API are directly addressable
without needing to apply a series of specific requests to "reach" the desired
resource.
In an API with a stateless protocol, the server has the responsibility for
persisting data, which may be shared between multiple clients, while clients
have sole responsibility and authority for maintaining the application state.
### Cyclic References
The relationship between resources, such as with [resource references][],
**must** be representable via a [directed acyclic graph][]. The parent-child
relationship also **must** be acyclic, and as per [AIP-124][] a given resource
instance will only have one canonical parent resource.
A cyclic relationship between resources increases the complexity of managing
resources. Consider resources A and B that refer to
each other. The process to create said resources are:
1. create resource A without a reference to B. Retrieve id for resource A.
2. create resource B with a reference to A. Retrieve id for resource B.
3. update resource A with the reference to B.
The delete operation may also become more complex, due to reasoning about which
resource must be dereferenced first for a successful deletion.
This requirement does not apply to relationships that are expressed via
[output only][] fields, as they do not require the user to specify the values
and in turn do not increase resource management complexity.
[AIP-124]: ./0124.md
[create]: ./0133.md
[custom methods]: ./0136.md
[delete]: ./0135.md
[directed acyclic graph]: https://en.wikipedia.org/wiki/Directed_acyclic_graph
[get]: ./0131.md
[list]: ./0132.md
[management plane]: ./0111.md#management-plane
[output only]: ./0203.md#output-only
[rest]: https://en.wikipedia.org/wiki/Representational_state_transfer
[resource references]: ./0122.md#fields-representing-another-resource
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
[singleton resources]: ./0156.md
[soft delete]: ./0164.md
[state]: ./0216.md
[stateless protocol]: https://en.wikipedia.org/wiki/Stateless_protocol
[update]: ./0134.md
## Changelog
- **2024-07-08**: Clarify acyclic nature of parent-child relationship.
- **2023-08-24**: Added guidance on consistency guarantees of methods.
- **2023-07-23**: Clarify stateless protocol definition.
- **2023-01-21**: Explicitly require matching schema across standard methods.
- **2022-12-19**: Added a section requiring Get and List.
- **2022-11-02**: Added a section restricting resource references.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0122.md
---
id: 122
state: approved
created: 2019-01-26
placement:
category: resource-design
order: 20
---
# Resource names
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. Additionally, resources are _named_: each
resource has a unique identifier that users use to reference that resource, and
these names are what users should _store_ as the canonical names for the
resources.
## Guidance
All resource names defined by an API **must** be unique within that API. (See
the section on [full resource names](#full-resource-names) below for more
information on referring to resources across APIs.)
Resource names are formatted according to the [URI path schema][], but without
the leading slash:
publishers/123/books/les-miserables
users/vhugo1802
- Resource name components **should** usually alternate between collection
identifiers (example: `publishers`, `books`, `users`) and resource IDs
(example: `123`, `les-miserables`, `vhugo1802`).
- Resource names **must** use the `/` character to separate individual segments
of the resource name.
- Non-terminal segments of a resource name **must not** contain a `/`
character.
- The terminal segment of a resource name **should not** contain a `/`
character.
- Resource names **should** only use characters available in DNS names, as
defined by [RFC-1123](https://tools.ietf.org/html/rfc1123).
- Additionally, resource IDs **should not** use upper-case letters.
- If additional characters are necessary, resource names **should not** use
characters that require URL-escaping, or characters outside of ASCII.
- If Unicode characters can not be avoided, resource names **must** be stored
in Normalization Form C (see [AIP-210][]).
- Resources **must** expose a `name` field that contains its resource name.
- Resources **may** provide the resource ID as a separate field (e.g.
`book_id`). This field **must** apply the
[`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification.
- Resources **may** expose a separate, system-generated unique ID field
[(`uid`)](./0148.md#uid). This field **must** apply the
[`OUTPUT_ONLY`](./0203.md#output-only) field behavior classification.
- Resources **must not** expose tuples, self-links, or other forms of
resource identification.
- All ID fields **should** be strings.
**Note:** Resource names as described here are used within the scope of a
single API (or else in situations where the owning API is clear from the
context), and are only required to be unique within that scope. For this
reason, they are sometimes called _relative resource names_ to distinguish them
from _full resource names_ (discussed below).
[aip-210]: ./0210.md
[uri path schema]: https://datatracker.ietf.org/doc/html/rfc3986#appendix-A
### Collection identifiers
The collection identifier segments in a resource name **must** be the plural
form of the noun used for the resource. (For example, a collection of
`Publisher` resources is called `publishers` in the resource name.)
- Collection identifiers **must** be concise American English terms.
- Collection identifiers **must** be in `camelCase`.
- Collection identifiers **must** begin with a lower-cased letter and contain
only ASCII letters and numbers (`/[a-z][a-zA-Z0-9]*/`).
- Collection identifiers **must** be plural.
- In situations where there is no plural word ("info"), or where the singular
and plural terms are the same ("moose"), the non-pluralized (singular) form
is correct. Collection segments **must not** "coin" words by adding "s" in
such cases (e.g, avoid "infos").
- Within any given single resource name, collection identifiers **must** be
unique. (e.g. `people/xyz/people/abc` is invalid)
#### Nested collections
If a resource name contains multiple levels of a hierarchy, and a parent
collection's name is used as a prefix for the child resource's name, the child
collection's name **may** omit the prefix. For example, given a collection of
`UserEvent` resources that would normally be nested underneath `users`:
```
users/vhugo1802/userEvents/birthday-dinner-226
```
An API **should** use the less-redundant form:
```
users/vhugo1802/events/birthday-dinner-226
```
In this situation, the _message_ and _resource type_ are still called
`UserEvent`; only the collection and resource identifiers in the pattern(s) are
shortened. Since the _resource type_ is not shortened, the `singular` and
`plural` are similarly _not shortened_.
```
message UserEvent {
option (google.api.resource) = {
type: "example.googleapis.com/UserEvent"
// Only the collection & resource identfiers in the `pattern` are shortened.
pattern: "projects/{project}/users/{user}/events/{event}"
singular: "userEvent"
plural: "userEvents"
};
string name = 1;
}
```
**Note:** APIs wishing to do this **must** follow this format consistently
throughout all of its `pattern` entries defined and anywhere else the
resource is referenced in the API, or else not at all.
### Resource ID segments
A resource ID segment identifies the resource within its parent collection. In
the resource name `publishers/123/books/les-miserables`, `123` is the resource
ID for the publisher, and `les-miserables` is the resource ID for the book.
- If resource IDs are user-specified, the API **must** document allowed
formats. User-specified resource IDs **should** conform to [RFC-1034][];
which restricts to letters, numbers, and hyphen, with the first character
a letter, the last a letter or a number, and a 63 character maximum.
- Additionally, user-specified resource IDs **should** restrict letters to
lower-case (`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`).
- Characters outside of ASCII **should not** be permitted; however, if
Unicode characters are necessary, APIs **must** follow guidance in
[AIP-210][].
- If resource IDs are not user-settable, the API **should** document the
basic format, and any upper boundaries (for example, "at most 63
characters").
- For more information, see the [create][] standard method.
[create]: ./0133.md#user-specified-ids
[rfc-1034]: https://tools.ietf.org/html/rfc1034
### Resource ID aliases
It is sometimes valuable to provide an alias for common lookup patterns for
resource IDs. For example, an API with `users` at the top of its resource
hierarchy may wish to provide `users/me` as a shortcut for retrieving
information for the authenticated user.
APIs **may** provide programmatic aliases for common lookup patterns. However,
all data returned from the API **must** use the canonical resource name.
### Full resource names
In most cases, resource names are used within a single API only, or else they
are used in contexts where the owning API is clear (for example,
`string pubsub_topic`).
However, sometimes it is necessary for services to refer to resources in an
arbitrary API. In this situation, the service **should** use the _full resource
name_, a schemeless URI with the owning [API's service name](./0009.md#api-service-name),
followed by the relative resource name:
```
//library.googleapis.com/publishers/123/books/les-miserables
//calendar.googleapis.com/users/vhugo1802
```
**Note:** The full resource name **should not** be used for cross-API
references where the owning API is clear; it is only used if a field refers to
resources in multiple APIs where ambiguity is possible.
### Resource URIs
The full resource name is a schemeless URI, but slightly distinct from the full
URIs we use to access a resource. The latter includes the protocol
(HTTPS), the API version, and the specific [service endpoint](./0009.md#api-service-endpoint)
to target:
```
https://library.googleapis.com/v1/publishers/123/books/les-miserables
https://calendar.googleapis.com/v3/users/vhugo1802
```
The version is not included in the full resource name because the full resource
name is expected to persist from version to version. Even though the API
surface may change between major versions, multiple major versions of the same
API are expected to use the same underlying data.
**Note:** The correlation between the full resource name and the service's
endpoint is by convention. In particular, one service is able to have multiple
endpoints (example use cases include regionalization, MTLS, and private access),
and the full resource name does not change between these.
### Fields representing resource names
When defining a resource, the first field **should** be the resource name,
which **must** be of type `string` and **must** be called `name` for the
resource name. The message **should** include a `google.api.resource`
annotation declaring the type (see [AIP-123][] for more on this).
```proto
// A representation of a book in the library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// Other fields...
}
```
When defining a method that retrieves or acts on an already-existing resource
(such as `GetBook` or `ArchiveBook`), the first field of the request message
**should** be the resource name, which **must** be of type `string` and
**must** be called `name` for the resource name. The field **should** also be
annotated with the `google.api.resource_reference` annotation, referencing the
resource type ([AIP-123][]).
```proto
// Request message for ArchiveBook
message ArchiveBookRequest {
// The book to archive.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// Other fields...
}
```
**Note:** Fields **must not** be called `name` except for this purpose. For
other use cases, either use a different term or prepend an adjective (for
example: `display_name`).
[aip-123]: ./0123.md
### Fields representing a resource's parent
When defining a method that retrieves resources from a collection or adds a new
resource to a collection (such as `ListBooks` or `CreateBook`), the first field
of the request message **should** be of type `string` and **should** be called
`parent` for the resource name of the collection. The `parent` field **should**
also be annotated with the `google.api.resource_reference` annotation,
referencing the parent's resource type ([AIP-123][]).
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The publisher to list books from.
// Format: publishers/{publisher_id}
string parent = 1 [(google.api.resource_reference) = {
type: "library.googleapis.com/Publisher"
}];
// Other fields (e.g. page_size, page_token, filter, etc.)...
}
```
If there is more than one possible parent type, the `parent` field **should**
be annotated with the `child_type` key on `google.api.resource_reference`
instead:
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The parent to list books from.
// Format:
// - publishers/{publisher_id}
// - authors/{author_id}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// Other fields (e.g. page_size, page_token, filter, etc.)...
}
```
**Note:** Fields **should not** be called `parent` except for this purpose. For
other use cases, use a synonymous term if possible.
### Fields representing another resource
When a field represents another resource, the field **should** be of type
`string` and accept the resource name of the other resource. The field name
**should** be equivalent to the corresponding message's name in snake case.
- Field names **may** include a leading adjective if appropriate (such as
`string dusty_book`).
- Field names **should not** use the `_name` suffix unless the field would be
ambiguous without it (e.g., `crypto_key_name`)
- Fields representing another resource **should** provide the
`google.api.resource_reference` annotation with the resource type being
referenced.
- If using the resource name is not possible and using the ID component alone is
strictly necessary, the field **should** use an `_id` suffix (e.g. `shelf_id`).
The field **should not** be of type `message` using the `message` that
implements the resource, __except__ for one of following conditions:
- The API is internal-only, has tight lifecycle relationships, and has a
permission model that enables inherited access to embedded resources.
- The embedding of the resource is done as part of the AIP-162 revisions
pattern.
Example of a resource reference:
```proto
// A representation of a book in a library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// Name of the book.
// Format is `publishers/{publisher}/books/{book}`
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The shelf where the book currently sits.
// Format is `shelves/{shelf}`.
string shelf = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Shelf"
}];
// Other fields...
}
```
## Further reading
- For evolving resource names over time, see
[AIP-180](./0180.md#changing-resource-names).
- For resource types, see [AIP-123][].
## Rationale
### Using names instead of IDs
For any large system, there are many kinds of resources. To use simple resource
IDs to identify a resource, we'd actually need use a resource-specific tuple to
reliably identify it, such as `(bucket, object)` or `(user, album, photo)`. This
creates several issues:
* Developers have to understand and remember such anonymous tuples.
* Passing tuples is generally harder than passing strings.
* Centralized infrastructures, such as logging and access control systems,
don't understand specialized tuples.
* Specialized tuples limit API design flexibility, such as providing
reusable API interfaces. For example,
[Long Running Operations][aip-151] can work with many other API interfaces
because they use flexible resource names.
[aip-151]: ./0151.md
### Standardizing on `name`
The concept of resource names is not a new one, and is formalized in Uniform
Resource Names (URN) in conjunction with Uniform Resource Identifiers (URI) and
Uniform Resource Locators (URL). Considering that the term "name" is so heavily
overloaded in general, usage outside of a very well-defined meaning would be
confusing for developers. So, the field name `name` is reserved in the context
of AIP-compliant APIs so as to eliminate any confusion with resource names, and
force other would be "name" fields to use a more specific field name.
### Disallow embedding of resources
Using a resource message directly as the type of a field within another resource
is problematic for a number of reasons, which are as follows:
* Complicates the resource lifecycle: If the dependency resource is
deleted, what happens to the embedded reference in the dependent resource?
Data retention and clean up operations will be significantly complicated.
* Bypasses permissions: If every resource has its own set of permissions, a user
with read permission on the dependent resource that doesn't have the same
permission on the dependency resource suddenly cannot see the full resource.
* Tightly couples resources in all aspects: Changing the requirements in the
schema, permissions, or otherwise for either resource impacts the other,
significantly increasing complexity of roll outs.
Referencing by name, as is recommended, eliminates all of this complexity by
preventing resource data duplication, and forcing the owning service to be
involved in the resolution of the reference (via Standard Methods), guaranteeing
isolation of logical concerns per-resource.
## History
### Disallowing UUIDs in user-specified IDs
As part of an effort to make APIs more declarative-friendly, APIs were required
to allow user-specified resource ID on creation. As part of this, a supporting
field `string uid` containing a UUID was also encouraged. Guidance was then
added to discourage the _user-specified resource ID_ from being a UUID or
even resembling one.
Ostensibly, the fact that `uid` contained a service-generated UUID motivated
this restriction on the user-specified resource ID, forcing the user-specified
resource ID to be a more "meaningful" value. However, we no longer saw value in
this requirement, hence its removal.
## Changelog
- **2025-03-10**: Drop guidance disallowing UUID in user-specified ID.
- **2024-10-15**: Add some rationale we found for use of `name` as a field and
instead of IDs as an identifier.
- **2024-06-14**: Clarify resource annotation shortening rules for nested
collections.
- **2023-09-19**: Prohibit duplicate collection identifiers.
- **2023-09-01**: Add a clause that allows embedding for revision resource
messages.
- **2023-08-10**: Explicitly disallow embedding resource messages in a resource.
- **2023-03-24**: Correction: full resource name contains the service name rather
than the service endpoint
- **2023-03-17**: Add `OUTPUT_ONLY` guidance for resource ID fields.
- **2020-10-06**: Added declarative-friendly guidance, and tightened character
set restrictions.
- **2020-10-05**: Clarified when full resource names are used.
- **2020-05-19**: Clarified that resource _IDs_ avoid capital characters, not
the entire resource _name_.
- **2020-04-27**: Tighten the restriction on valid characters.
- **2019-12-05**: Added guidance for resource annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership. Also changed the final
example from a Pub/Sub example to the usual Book example.
- **2019-07-30**: Changed the nested collection brevity suggestion from "may"
to "should"
>>>> 0123.md
---
id: 123
state: approved
created: 2019-05-12
placement:
category: resource-design
order: 30
---
# Resource types
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. APIs are allowed to name their resource types
reasonably freely (within the requirements of this AIP), and are only required
to ensure uniqueness within that API. This means that it is possible (and often
desirable) for different APIs to use the same type name. For example, a Memcache
and Redis API would both want to use `Instance` as a type name.
When mapping the relationships between APIs and their resources, however, it
becomes important to have a single, globally-unique type name. Additionally,
tools such as Kubernetes or GraphQL interact with APIs from multiple providers.
## Terminology
In the guidance below, we use the following terms:
- **Service Name:** This is the name defined in the [service configuration][].
This usually (but not necessarily) matches the hostname that users use to
call the service. Example: `pubsub.googleapis.com`. This is equivalent to an
[API Group][] in Kubernetes.
- **Type:** This is the name used for the type within the API e.g. the name of
the Protobuf `message`. This is equivalent to an [Object][] in Kubernetes.
## Guidance
APIs **must** define a resource type for each resource in the API, according to
the following pattern: `{Service Name}/{Type}`. The type name **must**:
- Match the containing API type's name.
- Start with an uppercase letter.
- Only contain alphanumeric characters.
- Be of the singular form of the noun.
- Use PascalCase (UpperCamelCase).
### Examples
Examples of resource types include:
- `pubsub.googleapis.com/Topic`
- `pubsub.googleapis.com/Subscription`
- `spanner.googleapis.com/Database`
- `spanner.googleapis.com/Instance`
- `networking.istio.io/Instance`
### Annotating resource types
APIs **should** annotate the resource types for each resource in the API using
the [`google.api.resource`][resource] annotation:
```proto
// A representation of a Pub/Sub topic.
message Topic {
option (google.api.resource) = {
type: "pubsub.googleapis.com/Topic"
pattern: "projects/{project}/topics/{topic}"
singular: "topic"
plural: "topics"
};
// Name and other fields...
}
```
- Patterns **must** correspond to the [resource name][aip-122].
- Pattern variables (the segments within braces) **must** use `snake_case`, and
**must not** use an `_id` suffix.
- Pattern variables **must** conform to the format `[a-z][_a-z0-9]*[a-z0-9]`.
- Pattern variables **must** be unique within any given pattern. (e.g.
`projects/{abc}/topics/{abc}` is invalid; this is usually a natural
corollary of collection identifiers being unique within a pattern.)
- Resources with [multiple patterns][multi-pattern-resources] **must**
preserve ordering: new patterns **must** be added at the end of the list, and
existing patterns **must not** be removed or re-ordered, as this breaks client
library backward compatibility.
- Singular **must** be the lower camel case of the type.
- Pattern variables **must** be the singular form of the resource type e.g.
a pattern variable representing a `Topic` resource ID is named `{topic}`.
- Plural **must** be the lower camel case plural of the singular.
- Pattern collection identifier segments **must** match the plural of the
resources, except in the case of [nested collections][].
#### Pattern uniqueness
When multiple patterns are defined within a resource, these patterns **must**
be mutually unique, where uniqueness is defined as being by-character identical
once all resource ID path segments have been removed, leaving all `/`
separators.
Therefore the following two patterns **must not** be defined within the same
resource:
- `user/{user}`
- `user/{user_part_1}~{user_part_2}`
## Rationale
### Type and message name alignment
In addition to simple schema-resource coherence and alignment, a number of
consumers benefit from the `{Type}` and `message` names matching. Consumers have
simpler lookups, client libraries get the same in addition to aligned user
experience where resource-oriented code has naming aligned with the generated
`message` code, generated reference documentation aligns resources with
`message` docs, etc.
### Singular and Plural
Well-defined singular and plurals of a resource enable clients to determine the
proper name to use in code and documentation.
lowerCamelCase can be translated into other common forms of a resource name
such as UpperCamelCase and snake_case.
<!-- prettier-ignore-start -->
[aip-122]: ./0122.md
[multi-pattern-resources]: https://google.aip.dev/client-libraries/4231#multi-pattern-resources
[API Group]: https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups
[nested collections]: ./0122.md#collection-identifiers
[Object]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds
[resource]: https://github.com/googleapis/googleapis/blob/master/google/api/resource.proto
[service configuration]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto
<!-- prettier-ignore-end -->
## Changelog
- **2025-01-09**: Strongly align resource type and message naming.
- **2024-08-07**: Added multi-pattern ordering compatibility requirements.
- **2023-09-19**: Prohibited duplicate pattern variables.
- **2023-05-06**: Added requirement of singular and plural.
- **2023-01-28**: Clarified guidance for the resource type name.
- **2022-10-28**: Added pattern variable format guidance.
- **2020-05-14**: Added pattern uniqueness.
- **2019-12-05**: Added guidance on patterns.
- **2019-07-17**: Fleshed out the annotation example somewhat.
>>>> 0124.md
---
id: 124
state: approved
created: 2020-03-20
placement:
category: resource-design
order: 40
---
# Resource association
APIs sometimes have resource hierarchies that can not be cleanly expressed in
the usual tree structure. For example, a resource may have a many-to-one
relationship with two other resource types instead of just one. Alternatively,
a resource may have a many-to-many relationship with another resource type.
## Guidance
A resource **must** have at most one canonical parent, and `List` requests
**must not** require two distinct "parents" to work.
### Multiple many-to-one associations
If a resource has a many-to-one relationship with multiple resource types, it
**must** choose at most one of them to be the canonical parent. The resource
**may** be associated with other resources through other fields on the
resource.
```proto
message Book {
// The resource name pattern for Book indicates that Publisher is the
// canonical parent.
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name for the book.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource name for the book's author.
string author = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
}
```
When listing resources with multiple associations in this way, the RPC **must**
treat the `string parent` field as required as discussed in [AIP-132][], and
**must not** add additional required arguments. The RPC **should** include a
`string filter` field that allows users to filter by other resource
associations as discussed in [AIP-160][].
**Note:** Resource reference fields **must** accept the same resource name
format that is used in the `name` field of the referenced resource.
### Many-to-many associations
Many-to-many associations are less common in APIs than they are in relational
databases, in part because they are more difficult to model and present over
network interfaces.
An API **may** contain many-to-many relationships, and **should** use a
repeated field containing a list of resource names, following the principles
described for repeated fields in [AIP-144][].
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource names for the book's authors.
repeated string authors = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
}
```
**Note:** See [AIP-144][] for more information on repeated fields, including
how to handle common issues such as atomic changes.
If the use of a repeated field is too restrictive, or if more metadata is
required along with the association, an API **may** model a many-to-many
relationship using a sub-resource with two one-to-many associations.
```proto
message BookAuthor {
// The resource pattern for BookAuthor indicates that Book is the
// canonical parent.
option (google.api.resource) = {
type: "library.googleapis.com/BookAuthor"
pattern: "publishers/{publisher}/books/{book}/authors/{book_author}"
};
// The resource name for the book-author association.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The resource name for the author.
string author = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Author"
}];
// Other fields...
}
```
**Note:** Using subresources to model an association between resources is only
recommended if additional metadata is required in the relationship, or if the
restrictions around the use of a repeated field preclude the use of that
approach.
[aip-132]: ./0132.md
[aip-144]: ./0144.md
[aip-160]: ./0160.md
## Changelog
- **2021-04-07**: Clarified that resource reference fields accept resource
names with the same format as the `name` field of the resource.
>>>> 0126.md
---
id: 126
state: approved
created: 2019-07-24
placement:
category: resource-design
order: 60
---
# Enumerations
It is common for a field to only accept or provide a discrete and limited set
of values. In these cases, it can be useful to use enumerations (generally
abbreviated "enums") in order to clearly communicate what the set of allowed
values are.
## Guidance
APIs **may** expose enum objects for sets of values that are expected to change
infrequently:
```proto
// A representation of a book.
message Book {
// Other fields...
// Possible formats in which the book may be published.
enum Format {
// Default value. This value is unused.
FORMAT_UNSPECIFIED = 0;
// The printed format, in hardback.
HARDBACK = 1;
// The printed format, in paperback.
PAPERBACK = 2;
// An electronic book format.
EBOOK = 3;
// An audio recording.
AUDIOBOOK = 4;
}
// The format of the book.
Format format = 99;
// Other fields...
}
```
- All enum values **must** use `UPPER_SNAKE_CASE`.
- The first value of the enum **should** be the name of the enum itself
followed by the suffix `_UNSPECIFIED`.
- An exception to this rule is if there is a clearly useful zero value. In
particular, if an enum needs to present an `UNKNOWN`, it is usually clearer
and more useful for it to be a zero value rather than having both.
- Enums which will only be used in a single message **should** be nested within that message.
In this case, the enum **should** be declared immediately before it is used.
- The non-zero values of such a nested enum definition **should not** be prefixed by the name
of the enum itself. This generally requires users to write `MyState.MYSTATE_ACTIVE` in their
code, which is unnecessarily verbose.
- Enums which will be used by multiple messages **should** be defined at the package level and
**should** be defined at the bottom of the proto file (see [AIP-191][]).
- Some languages (including C++) hoist enum values into the parent namespace, which can result
in conflicts for enums with the same values in the same proto package. To avoid sharing values,
APIs **should** prefix package-level enum values with the name of the enum.
- Enums **should** document whether the enum is frozen or they expect to add
values in the future.
### When to use enums
Enums can be more accessible and readable than strings or booleans in many
cases, but they do add overhead when they change. Therefore, enums **should**
receive new values infrequently. While the definition of "infrequently" may
change based on individual use cases, a good rule of thumb is no more than once
a year. For enums that change frequently, the API **should** use a string and
document the format.
Additionally, enums **should not** be used when there is a competing,
widely-adopted standard representation (such as with [language codes][bcp-47]
or [media types][]).
**Note:** If an enumerated value needs to be shared across APIs, an enum
**may** be used, but the assignment between enum values and their corresponding
integers **must** match.
### Alternatives
For enumerated values where the set of allowed values changes frequently, APIs
**should** use a `string` field instead, and **must** document the allowed
values. String fields with enumerated values **should** use `kebab-case` for
their values.
For enumerated values where there is a competing, widely-adopted standard
representation (generally, but not necessarily, a string), that standard
representation **should** be used. This is true even if only a small subset of
values are permitted, because using enums in this situation often leads to
frustrating lookup tables when trying to use multiple APIs together.
Boolean fields **may** be used in situations where it is clear that no further
flexibility will be needed. The default value **must** be `false`.
**Note:** When using protocol buffers, it is impossible to distinguish between
`false` and unset. If this is a requirement, an enum **may** be a better design
choice (although `google.protobuf.BoolValue` is also available).
## Further reading
- For states, a special type of enum, see [AIP-216][].
[aip-191]: ./0191.md
[aip-216]: ./0216.md
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[media types]: https://en.wikipedia.org/wiki/Media_type
>>>> 0127.md
---
id: 127
state: approved
created: 2019-08-22
placement:
category: protobuf
order: 10
---
# HTTP and gRPC Transcoding
APIs that follow [resource-oriented design][aip-121] are defined using
[RPCs][rpc], but the resource-oriented design framework allows them to also be
presented as APIs that largely follow REST/JSON conventions. This is important
in order to help developers use their existing knowledge: over 80% of the
public APIs available follow most REST conventions, and developers are
accustomed to that pattern.
## Guidance
APIs **must** provide HTTP definitions for each RPC that they define, except
for bi-directional streaming RPCs, which can not be natively supported using
HTTP/1.1. When providing a bi-directional streaming method, an API **should**
also offer an alternative method that does not rely on bi-directional
streaming.
### HTTP method and path
When using protocol buffers, each RPC **must** define the HTTP method and path
using the `google.api.http` annotation:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
message CreateBookRequest {
// The publisher who will publish this book.
// When using HTTP/JSON, this field is automatically populated based
// on the URI, because of the `{parent=publishers/*}` syntax.
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The book to create.
// When using HTTP/JSON, this field is populated based on the HTTP body,
// because of the `body: "book"` syntax.
Book book = 2 [(google.api.field_behavior) = REQUIRED];
// The user-specified ID for the book.
// When using HTTP/JSON, this field is populated based on a query string
// argument, such as `?bookId=foo`. This is the fallback for fields that
// are not included in either the URI or the body.
// Note that clients use camelCase format to communicate the field names
// to the service.
string book_id = 3;
}
```
- The first key (`post` in this example) corresponds to the HTTP method. RPCs
**may** use `get`, `post`, `patch`, or `delete`.
- RPCs **must** use the prescribed HTTP verb for each standard method, as
discussed in [AIP-131][], [AIP-132][], [AIP-133][], [AIP-134][], and
[AIP-135][]
- RPCs **should** use the prescribed HTTP verb for custom methods, as
discussed in [AIP-136][].
- RPCs **should not** use `put` or `custom`.
- The corresponding value represents the URI.
- URIs **must** use the `{foo=bar/*}` syntax to represent a variable that
should be populated in the request proto. When extracting a [resource
name][aip-122], the variable **must** include the entire resource name, not
just the ID component.
- URIs **may** use nested fields for their variable names. (Additionally,
AIP-134 mandates this for `Update` requests.)
- URIs **must** use the `*` character to represent ID components, which
matches all URI-safe characters except for `/`. URIs **may** use `**` as
the final segment of a URI if matching `/` is required.
- The `body` key defines which single top-level field in the request will be
sent as the HTTP body. If the body is `*`, then this indicates that the
request object itself is the HTTP body. The request body is encoded as JSON
as defined by protocol buffers' canonical [JSON encoding][].
- RPCs **must not** define a `body` at all for RPCs that use the `GET` or
`DELETE` HTTP verbs.
- RPCs **must** use the prescribed `body` for Create ([AIP-133][]) and Update
([AIP-134][]) requests.
- RPCs **should** use the prescribed `body` for custom methods ([AIP-136][]).
- The `body` **must not** contain a nested field (or use the `.` character),
- The `body` **must not** be the same as a URI parameter.
- The `body` **must not** be a `repeated` field.
- Fields **should not** use the `json_name` annotation to alter the field
name in JSON, unless doing so for backwards-compatibility reasons.
**Note:** Bi-directional streaming RPCs should not include a `google.api.http`
annotation at all. If feasible, the service **should** provide non-streaming
equivalent RPCs.
### Multiple URI bindings
Occasionally, an RPC needs to correspond to more than one URI:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
additional_bindings: {
post: "/v1/{parent=authors/*}/books"
body: "book"
}
additional_bindings: {
post: "/v1/books"
body: "book"
}
};
}
```
- RPCs **may** define any number of additional bindings. The structure is
identical to the `google.api.http` annotation (in fact, it is a recursive
reference).
- RPCs **must not** define an additional binding within an additional binding.
- The `body` clause **must** be identical in the top-level annotation and each
additional binding.
## Changelog
- **2022-08-18**: Added the comment that query string parameter names are
in camelCase.
- **2021-01-06**: Added clarification around `body` and nested fields.
- **2019-09-23**: Added a statement about request body encoding, and guidance
discouraging `json_name`.
<!-- prettier-ignore-start -->
[aip-121]: ./0121.md
[aip-122]: ./0122.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-136]: ./0136.md
[json encoding]: https://developers.google.com/protocol-buffers/docs/proto3#json
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
<!-- prettier-ignore-end -->
>>>> 0128.md
---
id: 128
state: approved
created: 2020-10-06
placement:
category: resource-design
order: 65
---
# Declarative-friendly interfaces
Many services need to interact with common DevOps tools, particularly those
that create and manage network-addressable resources (such as virtual machines,
load balancers, database instances, and so on). These tools revolve around the
principle of "configuration as code": the user specifies the complete intended
landscape, and tooling is responsible for making whatever changes are necessary
to achieve the user's specification.
These tools are **declarative**: rather than specifying specific _actions_ to
take, they specify the desired _outcome_, with the actions being derived based
on the differences between the current landscape and the intended one.
Furthermore, there are numerous popular DevOps tools, with more being
introduced each year. Integrating hundreds of resource types with multiple
tools requires uniformity, so that integration can be automated.
## Guidance
### Resources
Resources that are declarative-friendly **must** use only strongly-consistent
standard methods for managing resource lifecycle, which allows tools to support
these resources generically, as well as conforming to other
declarative-friendly guidance (see [further reading](#further-reading)).
Declarative-friendly resources **should** designate that they follow the
declarative-friendly style:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
style: DECLARATIVE_FRIENDLY
};
// Name and other fields...
}
```
### Reconciliation
If a resource takes time (more than a few seconds) for updates to be realized,
the resource **should** include a `bool reconciling` field to disclose that
changes are in flight. This field **must** be output only.
A resource **must** set the `reconciling` field to `true` if the current state
of the resource does not match the user's intended state, and the system is
working to reconcile them. This is regardless of whether the root cause of
going into reconciliation was user or system action.
**Note:** Services responding to a `GET` request **must** return the resource's
current state (not the intended state).
## Further reading
A significant amount of guidance is more strict for declarative-friendly
interfaces, due to the focus on automation on top of these resources. This list
is a comprehensive reference to declarative-friendly guidance in other AIPs:
- Resources **should not** employ custom methods: see AIP-136.
- Resources **must** use the `Update` method for repeated fields: see AIP-144.
- Resources **must** include certain standard fields: see AIP-148.
- Resources **must** have an `etag` field: see AIP-154.
- Resources **should** provide change validation: see AIP-163.
- Resources **should not** implement soft-delete. If the id cannot be re-used,
the resource **must** implement soft-delete and the undelete RPC: see AIP-164
### Annotations
See [AIP-148 annotations][].
## Changelog
- **2023-07-13**: Move `annotations` from AIP-148.
- **2023-06-17**: Definition of plane was removed and incorporated into AIP-111.
- **2023-05-11**: Removed must on resource_id, which was upstreamed to a general
must.
[AIP-148 annotations]: ./0148.md#annotations
>>>> 0129.md
---
id: 129
state: approved
created: 2023-09-20
updated: 2023-10-31
placement:
category: resource-design
order: 65
---
# Server-Modified Values and Defaults
Services often provide default values for resource fields, and occasionally
normalize the user input before returning it in the response. The guidance
herein describes how services document such behavior for the benefit of
consumers.
## Guidance
### Single Owner Fields
Fields **must** have a single owner, whether that is the client or the server.
Server owned fields **must** be indicated with the `OUTPUT_ONLY` field_behavior.
All other types of fields **must** be considered to be owned by the client. The
server **must** respect the value (or lack thereof) for all client owned fields
and not modify them.
### Effective Values
There are instances where a service will allocate, generate, or calculate a
value if the client chooses not to specify one. For example: a client creates a
virtual machine without specifying a static IP address for the virtual machine
to be available on. Such a scenario is opting into dynamic IP address
allocation.
Some examples of these types of fields are ones that are:
* generated (UUID)
* allocated (dynamic IP address)
* assigned (most recent software package version)
An attribute with an effective value ***must*** be expressed as two fields in
the API:
* a mutable field that can be optionally set by the user and **must not** be
modified by the service
* an `OUTPUT_ONLY` field that records the effective value decided on by the
service
Example:
```proto
message VirtualMachine {
…
string ip_address = 4;
string effective_ip_address = 5 [
(google.api.field_behavior) = OUTPUT_ONLY
];
}
```
#### Naming
Effective values **must** be named by prefixing `effective_` to the mutable
field's name.
### User-Specified Fields
For user-specified fields, the value in response from the service **must** be
the same as provided by the create or update request. For string fields this
means returning the value unchanged, with one exception:
* When a string field has a data type annotation, a normalized string that
represents the given value **may** be returned.
### Normalizations
A field that is normalized by the service **must** be annotated with the
`google.api.field_info` extension. See ([AIP-202][]) for guidance on using
this extension The allowed set of normalizations includes the following formats:
* uuid
* ipv4
* ipv6
* email
Normalizations on fields **must** be described using the `google.api.field_info`
annotation.
## Rationale
Server-modified and default values often make it harder to implement
[declarative clients][]. These clients are often unable to
tell when their desired state matches the current state for these fields, as the
rules by which a server may modify and return values are complex, not public,
and not repeatable.
### Rationale for Single Owner Fields
When fields do not have a single owner they can cause issues for
[declarative clients][]. These clients may attempt to set
values for fields that are overwritten by server set values, leading to the
client entering an infinite loop to correct the change.
### Rationale for Naming
Consistent naming is important for identifying standard behavior across APIs
and fields. Programmatic association between user-specified and effective values
depends on consistent naming.
### Rationale for Normalizations
Normalizations are important to allow services to store and return values in a
standard way while communicating to clients what changes are semantically
identical. Normalizing a value on the service side allows the service to accept
a wider range of semantically identical inputs without needing to maintain every
value as a raw string. Surfacing the normalization that is being applied to
clients allows for client side comparison of sent and retrieved values to check
for differences.
For example, in a resource that accepts an email address on a particular field
a client may specify a given email address in a variety of ways. For the email
`[email protected]` a client may choose to specify `[email protected]`,
`[email protected]`, or `[email protected]`. These are semantically identical and
*should* all be accepted by the service. The service then may choose to
normalize the email address for storage and retrieval through downcasing or
canonicalization. Importantly, the information surfaced to clients on the
normalization of a field will not describe the normalization algorithm itself,
but instead the comparison method used to accurately compute if two values
should be considered equal.
### Rationale for Field Value Handling
For fields not using an allowed normalization,
[Declarative clients][] will not be able to identify which changes are
semantically meaningful. When a [Declarative client][Declarative clients]
sends a particular value it will ensure that the value is being returned by the
service to validate it was set correctly.
## Changelog
- **2023-10-31:** Update to approved.
[Declarative clients]: ./0009.md#declarative-clients
[aip-202]: ./0202.md
<!-- prettier-ignore-start -->
[aip-180]: ./0180.md
<!-- prettier-ignore-end -->
>>>> 0130.md
---
id: 130
state: approved
created: 2023-03-13
updated: 2023-09-05
placement:
category: operations
order: 9
---
# Methods
An API is composed of one or more methods, which represent a specific operation
that a service can perform on behalf of the consumer.
## Guidance
### Categories of Methods
The following enumerates multiple categories of methods that exist, often
grouped up under some object (e.g. collection or resource) that the method
operates upon.
| Category Name | Related AIPs | [Declarative client][] integration | CLI / UI integration | SDK integration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------- | -------------------- | --------------- |
| *Standard Methods* | | | | |
| **Standard collection methods**: operate on a collection of resources (List or Create). | [AIP-121][], [AIP-132][], [AIP-133][] | automatable | automatable | automatable |
| **Standard resource methods**: fetch or mutate a single resource (Get, Update, Delete). | [AIP-121][], [AIP-131][], [AIP-134][], [AIP-135][] | automatable | automatable | automatable |
| **Batch resource methods**: fetch or mutate multiple resources in a collection by name. | [AIP-231][], [AIP-233][], [AIP-234][], [AIP-235][] | may be used to optimize queries | automatable | automatable |
| **Aggregated list methods**: fetch or mutate multiple resources of the same type across multiple collections. | [AIP-159][] | not useful nor automatable | automatable | automatable |
| *Custom Fetch Methods* | | | | |
| **Custom collection fetch methods**: fetch information across a collection that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable |
| **Custom resource fetch methods**: fetch information for a single resource that cannot be expressed via a standard method. | [AIP-136][] | handwritten | automatable | automatable |
| *Custom Mutation Methods* | | | | |
| **Backing up a resource**: storing a copy of a resource at a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable |
| **Restoring a resource**: setting a resource to a version from a particular point in time. | [AIP-162][] | unused or handwritten | automatable | automatable |
| **Renaming a resource**: modify the resource's name or id while preserving configuration and data. | [AIP-136][] | unused or handwritten | automatable | automatable |
| **Custom collection mutation methods**: perform an imperative operation referencing a collection that may mutate one or more resources within that collection in fashion that cannot be easily achieved by standard methods (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable |
| **Custom resource mutation methods**: perform an imperative operation on a resource that may mutate it in a way a standard method cannot (e.g. state transitions). | [AIP-136][] | unused or handwritten | automatable | automatable |
| *Misc Custom Methods* | | | |
| **Stateless Methods**: a method that has no permanent effect on any data within the API (e.g. translating text) | [AIP-136][] | unused or handwritten | automatable | automatable |
| *None of the above* | | | | |
| **Streaming methods**: methods that communicate via client, server, or bi-directional streams. | | handwritten | handwritten | automatable |
### Choosing a method category
While designing a method, API authors should choose from the defined categories
in the following order:
1. Standard methods (on collections and resources)
1. Standard batch or aggregate methods
1. Custom methods (on collections, resources, or stateless)
1. Streaming methods
## Rationale
Resource-oriented standard and custom methods are recommended first, as they can
be expressed in the widest variety of clients (Declarative clients, CLIs, UIs, and so on), and
offer the most uniform experience that allows users to apply their knowledge of
one API to another.
If a standard method is unsuitable, then custom methods (that are mounted to a
resource or collection) offer a lesser, but still valuable level of consistency,
helping the user reason about the scope of the action and the object whose
configuration is read to inform that action. Although mutative custom methods
are not uniform enough to have a automated integration with exclusively
resource-oriented clients such as [Declarative client][]s, they are still a pattern that
can easily recognized by CLIs, UIs, and SDKs.
If one cannot express their APIs in a resource-oriented fashion at all, then the
operation falls in a category where the lack of uniformity makes it difficult
for any client aside from SDKs to model the operation. This category is
preferred last due to the fact that a user cannot rely on their knowledge of
similar APIs, as well as the issue that integration with many clients will
likely have to be hand-written.
[AIP-121]: ./0121.md
[AIP-131]: ./0131.md
[AIP-132]: ./0132.md
[AIP-133]: ./0133.md
[AIP-134]: ./0134.md
[AIP-135]: ./0135.md
[AIP-136]: ./0136.md
[AIP-159]: ./0159.md
[AIP-162]: ./0162.md
[AIP-231]: ./0231.md
[AIP-233]: ./0233.md
[AIP-234]: ./0234.md
[AIP-235]: ./0235.md
[Declarative client]: ./0009.md#declarative-clients
## Changelog
- **2023-09-05**: Update to approved.
>>>> 0131.md
---
id: 131
state: approved
created: 2019-01-22
updated: 2022-06-02
placement:
category: operations
order: 10
---
# Standard methods: Get
In REST APIs, it is customary to make a `GET` request to a resource's URI (for
example, `/v1/publishers/{publisher}/books/{book}`) in order to retrieve that
resource.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Get`
method. These RPCs accept the URI representing that resource and return the
resource.
## Guidance
APIs **must** provide a get method for resources. The purpose of the get method
is to return data from a single resource.
Get methods are specified using the following pattern:
```proto
rpc GetBook(GetBookRequest) returns (Book) {
option (google.api.http) = {
get: "/v1/{name=publishers/*/books/*}"
};
option (google.api.method_signature) = "name";
}
```
- The RPC's name **must** begin with the word `Get`. The remainder of the RPC
name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. (There is no
`GetBookResponse`.)
- The response **should** usually include the fully-populated resource unless
there is a reason to return a partial response (see [AIP-157][]).
- The HTTP verb **must** be `GET`.
- The URI **should** contain a single variable field corresponding to the
resource name.
- This field **should** be called `name`.
- The URI **should** have a variable corresponding to this field.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- There **must not** be a `body` key in the `google.api.http` annotation.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"name"`.
### Request message
Get methods implement a common request message pattern:
```proto
message GetBookRequest {
// The name of the book to retrieve.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A resource name field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The comment for the `name` field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in
another AIP.
**Note:** The `name` field in the request object corresponds to the `name`
variable in the `google.api.http` annotation on the RPC. This causes the `name`
field in the request to be populated based on the value in the URL when the
REST/JSON interface is used.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
[aip-121]: ./0121.md
[aip-123]: ./0123.md
[aip-157]: ./0157.md
[aip-203]: ./0203.md
[errors]: ./0193.md
[permission-denied]: ./0193.md#permission-denied
## Changelog
- **2023-03-17**: Align with AIP-122 and make Get a must.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-12**: Added guidance for error cases.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
>>>> 0132.md
---
id: 132
state: approved
created: 2019-01-21
updated: 2022-06-02
placement:
category: operations
order: 20
---
# Standard methods: List
In many APIs, it is customary to make a `GET` request to a collection's URI
(for example, `/v1/publishers/1/books`) in order to retrieve a list of
resources, each of which lives within that collection.
Resource-oriented design ([AIP-121][]) honors this pattern through the `List`
method. These RPCs accept the parent collection (and potentially some other
parameters), and return a list of responses matching that input.
## Guidance
APIs **must** provide a `List` method for resources unless the resource is a
[singleton][]. The purpose of the `List` method is to return data from a finite
collection (generally singular unless the operation supports [reading across
collections][]).
List methods are specified using the following pattern:
```proto
rpc ListBooks(ListBooksRequest) returns (ListBooksResponse) {
option (google.api.http) = {
get: "/v1/{parent=publishers/*}/books"
};
option (google.api.method_signature) = "parent";
}
```
- The RPC's name **must** begin with the word `List`. The remainder of the RPC
name **should** be the plural form of the resource being listed.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- The HTTP verb **must** be `GET`.
- The collection whose resources are being listed **should** map to the URI
path.
- The collection's parent resource **should** be called `parent`, and
**should** be the only variable in the URI path. All remaining parameters
**should** map to URI query parameters.
- The collection identifier (`books` in the above example) **must** be a
literal string.
- The `body` key in the `google.api.http` annotation **must** be omitted.
- If the resource being listed is not a top-level resource, there **should**
be exactly one `google.api.method_signature` annotation, with a value of
`"parent"`. If the resource being listed is a top-level resource, there
**should** be either no `google.api.method_signature` annotation, or exactly
one `google.api.method_signature` annotation, with a value of `""`.
### Request message
List methods implement a common request message pattern:
```proto
message ListBooksRequest {
// The parent, which owns this collection of books.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The maximum number of books to return. The service may return fewer than
// this value.
// If unspecified, at most 50 books will be returned.
// The maximum value is 1000; values above 1000 will be coerced to 1000.
int32 page_size = 2;
// A page token, received from a previous `ListBooks` call.
// Provide this to retrieve the subsequent page.
//
// When paginating, all other parameters provided to `ListBooks` must match
// the call that provided the page token.
string page_token = 3;
}
```
- A `parent` field **must** be included unless the resource being listed is a
top-level resource. It **should** be called `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] of the resource
being listed.
- The `page_size` and `page_token` fields, which support pagination, **must**
be specified on all list request messages. For more information, see
[AIP-158][].
- The comment above the `page_size` field **should** document the maximum
allowed value, as well as the default value if the field is omitted (or set
to `0`). If preferred, the API **may** state that the server will use a
sensible default. This default **may** change over time.
- If a user provides a value greater than the maximum allowed value, the API
**should** coerce the value to the maximum allowed.
- If a user provides a negative or other invalid value, the API **must** send
an `INVALID_ARGUMENT` error.
- The `page_token` field **must** be included on all list request messages.
- The request message **may** include fields for common design patterns
relevant to list methods, such as `string filter` and `string order_by`.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
**Note:** List methods **should** return the same results for any user that has
permission to make a successful List request on the collection. Search methods
are more relaxed on this.
### Response message
List methods implement a common response message pattern:
```proto
message ListBooksResponse {
// The books from the specified publisher.
repeated Book books = 1;
// A token, which can be sent as `page_token` to retrieve the next page.
// If this field is omitted, there are no subsequent pages.
string next_page_token = 2;
}
```
- The response message **must** include one repeated field corresponding to the
resources being returned, and **should not** include any other repeated
fields unless described in another AIP (for example, AIP-217).
- The response **should** usually include fully-populated resources unless
there is a reason to return a partial response (see AIP-157).
- The `next_page_token` field, which supports pagination, **must** be included
on all list response messages. It **must** be set if there are subsequent
pages, and **must not** be set if the response represents the final page. For
more information, see AIP-158.
- The message **may** include a `int32 total_size` (or `int64 total_size`)
field with the number of items in the collection.
- The value **may** be an estimate (the field **should** clearly document
this if so).
- If filtering is used, the `total_size` field **should** reflect the size of
the collection _after_ the filter is applied.
### Ordering
`List` methods **may** allow clients to specify sorting order; if they do, the
request message **should** contain a `string order_by` field.
- Values **should** be a comma separated list of fields. For example:
`"foo,bar"`.
- The default sorting order is ascending. To specify descending order for a
field, users append a `" desc"` suffix; for example: `"foo desc, bar"`.
- Redundant space characters in the syntax are insignificant.
`"foo, bar desc"`, `" foo , bar desc "`, and `"foo,bar desc"` are all
equivalent.
- Subfields are specified with a `.` character, such as `foo.bar` or
`address.street`.
- The resulting list order **should** be based on the field type's natural
comparator e.g. numerics ordered numerically, strings ordered
lexicographically, etc. However, APIs **may** choose to use a different
ordering; if so, it **must** be documented in the `order_by` definition.
- Furthermore, [well-known][] types, like `Timestamp` and `Duration` are
compared as their representative type; `Timestamp` is compared as time e.g.
before or after, `Duration` is compared as a quantity e.g. more or less.
<!-- TODO(#220): Add a reference to AIP-161 once it is written. -->
**Note:** Only include ordering if there is an established need to do so. It is
always possible to add ordering later, but removing it is a breaking change.
### Filtering
List methods **may** allow clients to specify filters; if they do, the request
message **should** contain a `string filter` field. Filtering is described in
more detail in AIP-160.
**Note:** Only include filtering if there is an established need to do so. It
is always possible to add filtering later, but removing it is a breaking
change.
### Soft-deleted resources
Some APIs need to "[soft delete][]" resources, marking them as deleted or
pending deletion (and optionally purging them later).
APIs that do this **should not** include deleted resources by default in list
requests. APIs with soft deletion of a resource **should** include a
`bool show_deleted` field in the list request that, if set, will cause
soft-deleted resources to be included.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
## Further reading
- For details on pagination, see [AIP-158](./0158.md).
- For listing across multiple parent collections, see [AIP-159](./0159.md).
[aip-121]: ./0121.md
[aip-123]: ./0123.md
[aip-157]: ./0157.md
[aip-158]: ./0158.md
[aip-203]: ./0203.md
[errors]: ./0193.md
[permission-denied]: ./0193.md#permission-denied
[reading across collections]: ./0159.md
[singleton]: ./0156.md
[soft delete]: ./0135.md#soft-delete
[well-known]: https://protobuf.dev/reference/protobuf/google.protobuf
## Changelog
- **2025-02-25**: Require documentation for ordering not matching field type
with clarification on ordering of well-known types.
- **2023-03-22**: Fix guidance wording to mention AIP-159.
- **2023-03-17**: Align with AIP-122 and make Get a must.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-02**: Add link to the filtering AIP.
- **2020-08-14**: Added error guidance for permission denied cases.
- **2020-06-08**: Added guidance on returning the full resource.
- **2020-05-19**: Removed requirement to document ordering behavior.
- **2020-04-15**: Added guidance on List permissions.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-07-30**: Added guidance about documenting the ordering behavior.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
>>>> 0133.md
---
id: 133
state: approved
created: 2019-01-23
updated: 2022-06-02
placement:
category: operations
order: 30
---
# Standard methods: Create
In REST APIs, it is customary to make a `POST` request to a collection's URI
(for example, `/v1/publishers/{publisher}/books`) in order to create a new
resource within that collection.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Create`
method. These RPCs accept the parent collection and the resource to create (and
potentially some other parameters), and return the created resource.
## Guidance
APIs **should** generally provide a create method for resources unless it is
not valuable for users to do so. The purpose of the create method is to create
a new resource in an already-existing collection.
Create methods are specified using the following pattern:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.api.method_signature) = "parent,book";
}
```
- The RPC's name **must** begin with the word `Create`. The remainder of the
RPC name **should** be the singular form of the resource being created.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. There is no
`CreateBookResponse`.
- The response **should** include the fully-populated resource, and **must**
include any fields that were provided unless they are input only (see
[AIP-203][]).
- If the create RPC is [long-running](#long-running-create), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
- The HTTP verb **must** be `POST`.
- The collection where the resource is being added **should** map to the URI
path.
- The collection's parent resource **should** be called `parent`, and
**should** be the only variable in the URI path.
- The collection identifier (`books` in the above example) **must** be
a literal string.
- There **must** be a `body` key in the `google.api.http` annotation, and it
**must** map to the resource field in the request message.
- All remaining fields **should** map to URI query parameters.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"parent,{resource},{resource}_id"`, or "`"parent,{resource}"`
if the resource ID is not required.
- If the API is operating on the [management plane][], the operation should have
[strong consistency][]: the completion of a create operation **must** mean
that all user-settable values and the existence of the resource have reached a
steady-state and reading resource state returns a consistent response.
### Request message
Create methods implement a common request message pattern:
```proto
message CreateBookRequest {
// The parent resource where this book will be created.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The ID to use for the book, which will become the final component of
// the book's resource name.
//
// This value should be 4-63 characters, and valid characters
// are /[a-z][0-9]-/.
string book_id = 2 [(google.api.field_behavior) = REQUIRED];
// The book to create.
Book book = 3 [(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **must** be included unless the resource being created is a
top-level resource. It **should** be called `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] of the resource
being created.
- A `{resource}_id` field **must** be included for management plane resources,
and **should** be included for data plane resources.
- The resource field **must** be included and **must** map to the POST body.
- The request message **must not** contain any other required fields and
**should not** contain other optional fields except those described in this
or another AIP.
### Long-running create
Some resources take longer to create a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation (AIP-151) instead:
```proto
rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
**Important:** Declarative-friendly resources (AIP-128) **should** use
long-running operations. The service **may** return an LRO that is already set
to done if the request is effectively immediate.
### User-specified IDs
An API **must** allow a user to specify the ID component of a resource (the last
segment of the resource name) on creation if the API is operating on the
[management plane][].
On the [data plane][], an API **should** allow a user to specify the ID.
Exceptional cases should have the following behavior:
- The data plane resource allows identical records without a need to
disambiguate between the two (e.g. rows in a table with no primary key).
- The data plane resource will not be exposed in [Declarative clients][].
An API
**may** allow the `{resource}_id` field have the [field_behavior][] `OPTIONAL`,
and generate a system-generated ID if one is not specified.
For example:
```
// Using user-specified IDs.
publishers/lacroix/books/les-miserables
// Using system-generated IDs.
publishers/012345678-abcd-cdef/books/12341234-5678-abcd
```
- The `{resource}_id` field **must** exist on the request message, not the
resource itself.
- The field **may** be required or optional. If it is required, it **should**
include the corresponding annotation.
- The `name` field on the resource **must** be ignored.
- There **should** be exactly one `google.api.method_signature` annotation on
the RPC, with a value of `"parent,{resource},{resource}_id"` if the resource
being created is not a top-level resource, or with a value of
`"{resource},{resource}_id"` if the resource being created is a top-level
resource.
- The documentation **should** explain what the acceptable format is, and the
format **should** follow the guidance for resource name formatting in
[AIP-122][].
- If a user tries to create a resource with an ID that would result in a
duplicate resource name, the service **must** error with `ALREADY_EXISTS`.
- However, if the user making the call does not have permission to see the
duplicate resource, the service **must** error with `PERMISSION_DENIED`
instead.
**Note:** For REST APIs, the user-specified ID field, `{resource}_id`,
is provided as a query parameters on the request URI.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
## Further reading
- For ensuring idempotency in `Create` methods, see [AIP-155][].
- For naming resources involving Unicode, see [AIP-210][].
## Rationale
### Requiring user-specified ids
[Declarative clients][] use the resource ID as a way to identify a resource for
applying updates and for conflict resolution. The lack of a user-specified ID
means a client is unable to find the resource unless they store the identifier
locally, and can result in re-creating the resource. This in turn has a
downstream effect on all resources that reference it, forcing them to update to
the ID of the newly-created resource.
Having a user-specified ID also means the client can precalculate the resource
name and use it in references from other resources.
[aip-121]: ./0121.md
[aip-122]: ./0122.md
[aip-123]: ./0123.md
[aip-155]: ./0155.md
[aip-203]: ./0203.md
[aip-210]: ./0210.md
[data plane]: ./0111.md#data-plane
[management plane]: ./0111.md#management-plane
[errors]: ./0193.md
[field_behavior]: ./0203.md
[Declarative clients]: ./0009.md#declarative-clients
[permission-denied]: ./0193.md#permission-denied
[strong consistency]: ./0121.md#strong-consistency
## Changelog
- **2023-10-20**: Clarify that {resource}_id is only required for management plane resources.
- **2023-08-24**: Adding consistency requirement.
- **2023-05-11**: Changing guidance around resource_id to a must.
- **2022-11-04**: Referencing aggregated error guidance in AIP-193, similar to
other CRUDL AIPs.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-06**: Added declarative-friendly guidance.
- **2020-08-14**: Updated error guidance to use permission denied over
forbidden.
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-11-22**: Added clarification on what error to use if a duplicate name
is sent.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running create.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
>>>> 0134.md
---
id: 134
state: approved
created: 2019-01-24
updated: 2022-06-02
placement:
category: operations
order: 40
---
# Standard methods: Update
In REST APIs, it is customary to make a `PATCH` or `PUT` request to a
resource's URI (for example, `/v1/publishers/{publisher}/books/{book}`) in
order to update that resource.
Resource-oriented design ([AIP-121][]) honors this pattern through the `Update`
method (which mirrors the REST `PATCH` behavior). These RPCs accept the URI
representing that resource and return the resource.
## Guidance
APIs **should** generally provide an update method for resources unless it is
not valuable for users to do so. The purpose of the update method is to make
changes to the resources without causing side effects.
Update methods are specified using the following pattern:
```proto
rpc UpdateBook(UpdateBookRequest) returns (Book) {
option (google.api.http) = {
patch: "/v1/{book.name=publishers/*/books/*}"
body: "book"
};
option (google.api.method_signature) = "book,update_mask";
}
```
- The RPC's name **must** begin with the word `Update`. The remainder of the
RPC name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **must** be the resource itself. (There is no
`UpdateBookResponse`.)
- The response **should** include the fully-populated resource, and **must**
include any fields that were sent and included in the update mask unless
they are input only (see AIP-203).
- If the update RPC is [long-running](#long-running-update), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
- The method **should** support partial resource update, and the HTTP verb
**should** be `PATCH`.
- If the method will only ever support full resource replacement, then the
HTTP verb **may** be `PUT`. However, this is strongly discouraged because
it becomes a backwards-incompatible change to add fields to the resource.
- The resource's `name` field **should** map to the URI path.
- The `{resource}.name` field **should** be the only variable in the URI
path.
- There **must** be a `body` key in the `google.api.http` annotation, and it
**must** map to the resource field in the request message.
- All remaining fields **should** map to URI query parameters.
- There **should** be exactly one `google.api.method_signature` annotation, with
a value of `"{resource},update_mask"`.
- If the API is operating on the [management plane][], the operation should have
[strong consistency][]: the completion of an update operation **must** mean
that all user-settable values and the existence of the resource have reached a
steady-state and reading resource state returns a consistent response.
**Note:** Unlike the other four standard methods, the URI path here references
a nested field (`book.name`) in the example. If the resource field has a word
separator, `snake_case` is used.
### Request message
Update methods implement a common request message pattern:
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to update.
google.protobuf.FieldMask update_mask = 2;
}
```
- The request message **must** contain a field for the resource.
- The field **must** map to the `PATCH` body.
- The field **should** be [annotated as required][aip-203].
- A `name` field **must** be included in the resource message. It **should**
be called `name`.
- The field **must** identify the [resource type][aip-123] of the resource
being updated.
- If partial resource update is supported, a field mask **must** be included.
It **must** be of type `google.protobuf.FieldMask`, and it **must** be
called `update_mask`.
- The fields used in the field mask correspond to the resource being updated
(not the request message).
- The field **must** be optional, and the service **must** treat an omitted
field mask as an implied field mask equivalent to all fields that are
populated (have a non-empty value).
- Update masks **must** support a special value `*`, meaning full replacement
(the equivalent of `PUT`).
- API producers need to be conscious of how adding new, mutable fields to a
resource will be handled when consumers use `*` without knowledge of said
new, mutable fields. Likewise consumers need to use `*` only when the
risks of doing so are acceptable. In general, it is safest to explicitly
specify the fields to update rather than use `*`.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Side effects
In general, update methods are intended to update the data within the resource.
Update methods **should not** trigger other side effects. Instead, side effects
**should** be triggered by custom methods.
In particular, this entails that [state fields][] **must not** be directly
writable in update methods.
### PATCH and PUT
**TL;DR:** Google APIs generally use the `PATCH` HTTP verb only, and do not
support `PUT` requests.
We standardize on `PATCH` because Google updates stable APIs in place with
backwards-compatible improvements. It is often necessary to add a new field to
an existing resource, but this becomes a breaking change when using `PUT`.
To illustrate this, consider a `PUT` request to a `Book` resource:
PUT /v1/publishers/123/books/456
{"title": "Mary Poppins", "author": "P.L. Travers"}
Next consider that the resource is later augmented with a new field (here we
add `rating`):
```proto
message Book {
string title = 1;
string author = 2;
// Subsequently added to v1 in place...
int32 rating = 3;
}
```
If a rating is set on a book and the existing `PUT` request was executed, it
would wipe out the book's rating. In essence, a `PUT` request unintentionally
wiped out data because the previous version did not know about it.
### Long-running update
Some resources take longer to update a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation (AIP-151) instead:
```proto
rpc UpdateBook(UpdateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
patch: "/v1/{book.name=publishers/*/books/*}"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
**Note:** Declarative-friendly resources ([AIP-128][]) **should** use long-running
update.
### Create or update
If the service uses client-assigned resource names, `Update` methods **may**
expose a `bool allow_missing` field, which will cause the method to succeed in
the event that the user attempts to update a resource that is not present (and
will create the resource in the process):
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to be updated.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to be updated.
google.protobuf.FieldMask update_mask = 2;
// If set to true, and the book is not found, a new book will be created.
// In this situation, `update_mask` is ignored.
bool allow_missing = 3;
}
```
More specifically, the `allow_missing` flag triggers the following behavior:
- If the method call is on a resource that does not exist, the resource is
created. All fields are applied regardless of any provided field mask.
- However, if any required fields are missing or fields have invalid values,
an `INVALID_ARGUMENT` error is returned.
- If the method call is on a resource that already exists, and all fields
match, the existing resource is returned unchanged.
- If the method call is on a resource that already exists, only fields declared
in the field mask are updated.
The user **must** have the update permissions to call `Update` even with
`allow_missing` set to `true`. For customers that want to prevent users from
creating resources using the update method, IAM conditions **should** be used.
### Etags
An API may sometimes need to allow users to send update requests which are
guaranteed to be made against the most current data (a common use case for this
is to detect and avoid race conditions). Resources which need to enable this do
so by including a `string etag` field, which contains an opaque,
server-computed value representing the content of the resource.
In this situation, the resource **should** contain a `string etag` field:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The resource name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The title of the book.
// Example: "Mary Poppins"
string title = 2;
// The author of the book.
// Example: "P.L. Travers"
string author = 3;
// The etag for this book.
// If this is provided on update, it must match the server's etag.
string etag = 4;
}
```
The `etag` field **may** be either [required][] or [optional][]. If it is set,
then the request **must** succeed if and only if the provided etag matches the
server-computed value, and **must** fail with an `ABORTED` error otherwise. The
`update_mask` field in the request does not affect the behavior of the `etag`
field, as it is not a field _being_ updated.
### Expensive fields
APIs sometimes encounter situations where some fields on a resource are
expensive or impossible to reliably return.
This can happen in a few situations:
- A resource may have some fields that are very expensive to compute, and that
are generally not useful to the customer on update requests.
- A single resource sometimes represents an amalgamation of data from multiple
underlying (and eventually consistent) data sources. In these situations, it
is impossible to return authoritative information on the fields that were not
changed.
In this situation, an API **may** return back only the fields that were
updated, and omit the rest, and **should** document this behavior if they do
so.
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and
NOT_FOUND errors][permission-denied].
In addition, if the user does have proper permission, but the requested resource
does not exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless
`allow_missing` is set to `true`.
[aip-121]: ./0121.md
[aip-128]: ./0128.md
[aip-203]: ./0203.md
[create]: ./0133.md
[errors]: ./0193.md
[management plane]: ./0111.md#management-plane
[permission-denied]: ./0193.md#permission-denied
[state fields]: ./0216.md
[strong consistency]: ./0121.md#strong-consistency
[required]: ./0203.md#required
[optional]: ./0203.md#optional
## Changelog
- **2024-12-03**: Add caveats to usage of full replacement `update_mask`.
- **2024-03-14**: Make `update_mask` optional field_behaviour guidance a **must**.
- **2023-08-26**: Adding consistency requirement.
- **2023-07-17**: Make `update_mask` name guidance a **must**.
- **2022-11-04**: Aggregated error guidance to AIP-193.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2021-11-04**: Changed the permission check if `allow_missing` is set.
- **2021-07-08**: Added error guidance for resource not found case.
- **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which
becomes HTTP 400) to `ABORTED` (409).
- **2020-10-06**: Added guidance for declarative-friendly resources.
- **2020-10-06**: Added guidance for `allow_missing`.
- **2020-08-14**: Added error guidance for permission denied cases.
- **2020-06-08**: Added guidance on returning the full resource.
- **2019-10-18**: Added guidance on annotations.
- **2019-09-10**: Added a link to the long-running operations AIP
([AIP-151][]).
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running update.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
>>>> 0135.md
---
id: 135
state: approved
created: 2019-01-24
updated: 2022-06-02
placement:
category: operations
order: 50
---
# Standard methods: Delete
In REST APIs, it is customary to make a `DELETE` request to a resource's URI
(for example, `/v1/publishers/{publisher}/books/{book}`) in order to delete
that resource.
Resource-oriented design (AIP-121) honors this pattern through the `Delete`
method. These RPCs accept the URI representing that resource and usually return
an empty response.
## Guidance
APIs **should** generally provide a delete method for resources unless it is
not valuable for users to do so.
Delete methods are specified using the following pattern:
```proto
rpc DeleteBook(DeleteBookRequest) returns (google.protobuf.Empty) {
option (google.api.http) = {
delete: "/v1/{name=publishers/*/books/*}"
};
option (google.api.method_signature) = "name";
}
```
- The RPC's name **must** begin with the word `Delete`. The remainder of the
RPC name **should** be the singular form of the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be `google.protobuf.Empty`.
- If the resource is [soft deleted](#soft-delete), the response message
**should** be the resource itself.
- If the delete RPC is [long-running](#long-running-delete), the response
message **must** be a `google.longrunning.Operation` which resolves to the
correct response.
- The HTTP verb **must** be `DELETE`.
- The request message field receiving the resource name **should** map to the
URI path.
- This field **should** be called `name`.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- There **must not** be a `body` key in the `google.api.http` annotation.
- There **should** be exactly one `google.api.method_signature` annotation,
with a value of `"name"`. If an etag or force field are used, they **may** be
included in the signature.
- If the API is operating on the [Management Plane][], the operation should have
[strong consistency][]: the completion of a delete operation **must** mean
that the existence of the resource has reached a steady-state and reading
resource state returns a consistent response.
- The API **must** fail with a `FAILED_PRECONDITION` error if child resources
are present. See guidance on [Cascading Delete](#cascading-delete) if forcing
deletion of parent and child resources is necessary.
- If the only child resource type is a [Singleton][aip-156], deletion **must**
be allowed, because the lifecycle of a Singleton is tied to that of its
parent resource. This applies even if there are multiple different Singleton
resource types for the same parent resource.
The Delete method **should** succeed if and only if a resource was present and
was successfully deleted. If the resource did not exist, the method **should**
send a `NOT_FOUND` error.
### Request message
Delete methods implement a common request message pattern:
```proto
message DeleteBookRequest {
// The name of the book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `name` field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Soft delete
**Note:** This material was moved into its own document to provide a more
comprehensive treatment: AIP-164.
### Long-running delete
Some resources take longer to delete a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation instead:
```proto
rpc DeleteBook(DeleteBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
delete: "/v1/{name=publishers/*/books/*}"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the appropriate return type if the RPC
was not long-running: `google.protobuf.Empty` for most Delete RPCs, or the
resource itself for soft delete (AIP-164).
- Both the `response_type` and `metadata_type` fields **must** be specified
(even if they are `google.protobuf.Empty`).
### Cascading delete
Sometimes, it may be necessary for users to be able to delete a resource as
well as all applicable child resources. However, since deletion is usually
permanent, it is also important that users not do so accidentally, as
reconstructing wiped-out child resources may be quite difficult.
If an API allows deletion of a resource that may have child resources, the API
**should** provide a `bool force` field on the request, which the user sets to
explicitly opt in to a cascading delete.
```proto
message DeletePublisherRequest {
// The name of the publisher to delete.
// Format: publishers/{publisher}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Publisher"
}];
// If set to true, any books from this publisher will also be deleted.
// (Otherwise, the request will only work if the publisher has no books.)
bool force = 2;
}
```
The API **must** fail with a `FAILED_PRECONDITION` error if the `force` field
is `false` (or unset) and child resources are present.
### Protected delete
Sometimes, it may be necessary for users to ensure that no changes have been
made to a resource that is being deleted. If a resource provides an [etag][],
the delete request **may** accept the etag (as either required or optional):
```proto
message DeleteBookRequest {
// The name of the book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// Optional. The etag of the book.
// If this is provided, it must match the server's etag.
string etag = 2;
}
```
If the etag is provided and does not match the server-computed etag, the
request **must** fail with a `ABORTED` error code.
**Note:** Declarative-friendly resources (AIP-128) **must** provide the `etag`
field for Delete requests.
### Delete if existing
If the service uses client-assigned resource names, `Delete` methods **may**
expose a `bool allow_missing` field, which will cause the method to succeed in
the event that the user attempts to delete a resource that is not present (in
which case the request is a no-op):
```proto
message DeleteBookRequest {
// The book to delete.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
// If set to true, and the book is not found, the request will succeed
// but no action will be taken on the server
bool allow_missing = 2;
}
```
More specifically, the `allow_missing` flag triggers the following behavior:
- If the method call is on a resource that does not exist, the request is a
no-op.
- The `etag` field is ignored.
- If the method call is on a resource that already exists, the resource is
deleted (subject to other checks).
**Note:** Declarative-friendly resources (AIP-128) **should** expose the
`bool allow_missing` field.
### Errors
If the user does not have permission to access the resource, regardless of
whether or not it exists, the service **must** error with `PERMISSION_DENIED`
(HTTP 403). Permission **must** be checked prior to checking if the resource
exists.
If the user does have proper permission, but the requested resource does not
exist, the service **must** error with `NOT_FOUND` (HTTP 404) unless
`allow_missing` is set to `true`.
[aip-123]: ./0123.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-136]: ./0136.md
[aip-154]: ./0154.md
[aip-156]: ./0156.md
[aip-203]: ./0203.md
[aip-214]: ./0214.md
[aip-216]: ./0216.md
[management plane]: ./0111.md#management-plane
[strong consistency]: ./0121.md#strong-consistency
[etag]: ./0134.md#etags
## Further reading
- For soft delete and undelete, see AIP-164.
- For bulk deleting large numbers of resources based on a filter, see AIP-165.
## Changelog
- **2024-06-11**: Add deletion behavior for parent resource deletion requests without a `force` field.
- **2023-08-24**: Adding consistency requirement.
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2022-02-02**: Changed eTag error from `FAILED_PRECONDITION` to `ABORTED` making it consistent with change to [AIP-154][] & [AIP-134][etag] on 2021-03-05.
- **2020-10-06**: Added guidance for declarative-friendly resources.
- **2020-10-06**: Added guidance for allowing no-op delete for missing
resources.
- **2020-10-06**: Moved soft delete and undelete guidance into a new AIP-164.
- **2020-06-08**: Added guidance for `Get` of soft-deleted resources.
- **2020-02-03**: Added guidance for error cases.
- **2019-10-18**: Added guidance on annotations.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-06-10**: Added guidance for long-running delete.
- **2019-05-29**: Added an explicit prohibition on arbitrary fields in standard
methods.
>>>> 0136.md
---
id: 136
state: approved
created: 2019-01-25
updated: 2023-03-02
placement:
category: operations
order: 100
---
# Custom methods
Resource-oriented design (AIP-121) uses custom methods to provide a means to
express arbitrary actions that are difficult to model using only the standard
methods. Custom methods are important because they provide a means for an API's
vocabulary to adhere to user intent.
## Guidance
Custom methods **should** only be used for functionality that can not be easily
expressed via standard methods; prefer standard methods if possible, due to
their consistent semantics. (Of course, this only applies if the functionality
in question actually conforms to the normal semantics; it is _not_ a good idea
to contort things to endeavor to make the standard methods "sort of work".)
While custom methods vary widely in how they are designed, many principles
apply consistently:
```proto
// Archives the given book.
rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:archive"
body: "*"
};
}
```
**Note:** The pattern above shows a custom method that operates on a specific
resource. Custom methods can be associated with resources, collections, or
services. The bullets below apply in all three cases.
- The name of the method **should** be a verb followed by a noun.
- The name **must not** contain prepositions ("for", "with", etc.).
- The verb in the name **should not** contain any of the standard method verbs ([Get][],
[List][], [Create][], [Update][], [Delete][]).
- The name **must not** include the term `Async`. Instead, if the intention is
to differentiate between immediate and long-running RPCs, the suffix `LongRunning`
**may** be used for this purpose. For example, to create a long-running book creation
RPC (if the standard `CreateBook` method was designed before long-running aspects were
considered), a custom `CreateBookLongRunning` method could be introduced.
- The HTTP method **must** be `GET` or `POST`:
- `GET` **must** be used for methods retrieving data or resource state.
- `POST` **must** be used if the method has side effects or mutates resources
or data.
- The HTTP URI **must** use a `:` character followed by the custom verb
(`:archive` in the above example), and the verb in the URI **must** match the
verb in the name of the RPC.
- If word separation is required, `camelCase` **must** be used.
- The `body` clause in the `google.api.http` annotation **should** be `"*"`.
- See [HTTP and gRPC Transcoding][transcoding] for more information.
- Custom methods **should** take a request message matching the RPC
name, with a `Request` suffix.
- Custom methods **should** return a response message matching the RPC
name, with a `Response` suffix.
- When operating on a specific resource, a custom method **may** return the
resource itself.
### Resource-based custom methods
Custom methods **must** operate on a resource if the API can be modeled
as such:
```proto
// Archives the given book.
rpc ArchiveBook(ArchiveBookRequest) returns (ArchiveBookResponse) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:archive"
body: "*"
};
}
```
- The parameter for the resource's name **must** be called `name`, and
be the only variable in the URI path.
### Collection-based custom methods
While most custom methods operate on a single resource, some custom methods
**may** operate on a collection instead:
```proto
// Sorts the books from this publisher.
rpc SortBooks(SortBooksRequest) returns (SortBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:sort"
body: "*"
};
}
```
- The collection's parent resource **must** be called `parent`, and
be the only variable in the URI path.
- The collection key (`books` in the above example) **must** be literal.
### Stateless methods
Some custom methods are not attached to resources at all. These methods are
generally _stateless_: they accept a request and return a response, and have no
permanent effect on data within the API.
```proto
// Translates the provided text from one language to another.
rpc TranslateText(TranslateTextRequest) returns (TranslateTextResponse) {
option (google.api.http) = {
post: "/v1/{project=projects/*}:translateText"
body: "*"
};
}
```
- If the method runs in a particular scope (such as a project, as in the above
example), the field name in the request message **should** be the name of the
scope resource. If word separators are necessary, `snake_case` **must** be
used.
- The URI **should** place both the verb and noun after the `:` separator
(avoid a "faux collection key" in the URI in this case, as there is no
collection). For example, `:translateText` is preferable to `text:translate`.
- Stateless methods **must** use `POST` if they involve billing.
### Declarative-friendly resources
Declarative-friendly resources usually **should not** employ custom methods
(except specific declarative-friendly custom methods discussed in other AIPs),
because declarative-friendly tools are unable to automatically determine what
to do with them.
An exception to this is for rarely-used, fundamentally imperative operations,
such as a `Move` or `Rename` operation, for which there would not be an
expectation of declarative support.
## Rationale
### HTTP path
Similar to standard methods, a custom method that operates on a resource or
collection needs a `name` or `parent` parameter to indicate the resource that it
operates on. This convention allows clients to map custom methods to the
appropriate resource.
### HTTP methods
Allowing both `GET` and `POST` HTTP verbs allows a clear distinction for
which methods do not mutate data, and which ones do. Methods that only
read data have first-class concepts in some clients (DataSources in
Terraform) and clearly indicate to a user which methods can be called
without risk of runtime impact.
### Disallowing prepositions
Generally, method names with prepositions indicate that a new method is being
used where a field should instead be added to an existing method, or the method
should use a distinct verb. For example, if a `CreateBook` message already
exists and you are considering adding `CreateBookFromDictation`, consider a `TranscribeBook` method instead. Similarly, if there is desire for a
property-specific look-up method, instead of `GetBookByAuthor` consider a
`SearchBooks` with an `author` field as a search dimension.
[get]: ./0131.md
[list]: ./0132.md
[create]: ./0133.md
[update]: ./0134.md
[delete]: ./0135.md
[transcoding]: ./0127.md
### RPC name
The term "async" is commonly used in programming languages to indicate whether
a specific method call is synchronous or asynchronous, including for making RPCs.
That sync/async aspect is at a different abstraction level to whether the RPC
itself is intended to start a long-running operation. Using "async" within the
RPC name itself causes confusion, and can even cause issues for client libraries
which generate both synchronous and asynchronous methods to call the RPC in some
languages.
## Changelog
- **2025-01-09:** Add original rationale for disallowing prepositions in names.
- **2023-11-16:** Included link to AIP-127 "HTTP and gRPC Transcoding" for guidance on body definition.
- **2023-05-16:** Added prohibition of the term "async" within RPC names.
- **2023-05-09:** Adding guidance for POST and GET, require parent instead of
the resource singular.
- **2023-03-02:** Explicitly discourage use of standard method verbs.
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-06:** Added declarative-friendly guidance.
- **2019-08-01:** Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0140.md
---
id: 140
state: approved
created: 2019-07-22
placement:
category: fields
order: 0
---
# Field names
Naming fields in a way that is intuitive to users can often be one of the most
challenging aspects of designing an API. This is true for many reasons; often a
field name that seems entirely intuitive to the author can baffle a reader.
Additionally, users rarely use only one API; they use many APIs together. As a
result, a single company using the same name to mean different things (or
different names to mean the same thing) can often cause unnecessary confusion,
because users can no longer take what they've already learned from one API and
apply that to another.
In short, APIs are easiest to understand when field names are simple,
intuitive, and consistent with one another.
## Guidance
Field names **should** be in correct American English.
Field names **should** clearly and precisely communicate the concept being
presented and avoid overly general names that are ambiguous. That said, field
names **should** avoid including unnecessary words. In particular, avoid
including adjectives that always apply and add little cognitive value. For
example, a `proxy_settings` field might be as helpful as
`shared_proxy_settings` if there is no unshared variant.
**Important:** Field names often appear in generated client surfaces. Ensure they
are appropriately descriptive and of suitable length.
### Case
Field definitions in protobuf files **must** use `lower_snake_case` names.
These names are mapped to an appropriate naming convention in JSON and in
generated code.
Additionally, each word in the field **must not** begin with a number, because
it creates ambiguity when converting between snake case and camel case.
Similarly, fields **must not** contain leading, trailing, or adjacent
underscores.
### Uniformity
APIs **should** endeavor to use the same name for the same concept and
different names for different concepts wherever possible. This includes names
across multiple APIs, in particular if those APIs are likely to be used
together.
### Repeated fields
Repeated fields **must** use the proper plural form, such as `books` or
`authors`. On the other hand, non-repeated fields **should** use the singular
form such as `book` or `author`. This implies that resource names **should**
use the singular form as well, since the field name should follow the resource
name (e.g., use `repeated Book books`, **not** `Books books = 1`).
### Prepositions
Field names **should not** include prepositions (such as "with", "for", "at",
"by", etc). For example:
- `error_reason` (**not** `reason_for_error`)
- `author` (**not** `written_by`)
It is easier for field names to match more often when following this
convention. Additionally, prepositions in field names may also indicate a
design concern, such as an overly-restrictive field or a sub-optimal data type.
This is particularly true regarding "with": a field named `book_with_publisher`
likely indicates that the book resource may be improperly structured and worth
redesigning.
**Note:** The word "per" is an exception to this rule, particularly in two
cases. Often "per" is part of a unit (e.g. "miles per hour"), in which case the
preposition must be present to accurately convey the unit. Additionally, "per"
is often appropriate in reporting scenarios (e.g. "nodes per instance" or
"failures per hour").
### Abbreviations
For well known name abbreviations among software developers, such as "config"
and "spec", the abbreviations **should** be used in API definitions instead of
the full spelling. This will make the source code easy to read and write.
Examples:
- `config` (**not** `configuration`)
- `id` (**not** `identifier`)
- `info` (**not** `information`)
- `spec` (**not** `specification`)
- `stats` (**not** `statistics`)
### Adjectives
For uniformity, field names that contain both a noun and an adjective
**should** place the adjective _before_ the noun. For example:
- `collected_items` (**not** `items_collected`)
- `imported_objects` (**not** `objects_imported`)
### Verbs
Field names **must not** be named to reflect an intent or action. They
**must not** be verbs. Rather, because the field defines the *desired
value* for mutations, e.g. Create and Update, and the *current value*
for reads, e.g. Get and List, the name **must** be a noun. It defines
what is so, not what to do.
- `collected_items` (**not** `collect_items`)
- `disabled` (**not** `disable`)
In contrast, method names, whether standard or custom, change facets of
resources and are named as verbs.
### Booleans
Boolean fields **should** omit the prefix "is". For example:
- `disabled` (**not** `is_disabled`)
- `required` (**not** `is_required`)
**Note:** Field names that would otherwise be [reserved words](#reserved-words)
are an exception to this rule. For example, `is_new` (**not** `new`).
### String vs. bytes
When using `bytes`, the contents of the field are base64-encoded when using
JSON on the wire. Services **should** use `bytes` when there is a need to send
binary contents over the wire, and **should not** ask the user to manually
base64-encode a field into a `string` field. The one exception is when the
API is designed to handle data that is meant to be base64-encoded at rest and
the double base64-encoding as a result of using `bytes` is undesirable - in
this case, services **may** use a `string`.
### URIs
Field names representing URLs or URIs **should** always use `uri` rather than
`url`. This is because while all URLs are URIs, not all URIs are URLs. Field
names **may** use a prefix in front of `uri` as appropriate.
### Reserved words
Field names **should** avoid using names that are likely to conflict with
keywords in common programming languages, such as `new`, `class`, `function`,
`import`, etc. Reserved keywords can cause hardship for developers using the
API in that language.
### Conflicts
Messages **should not** include a field with the same name as the enclosing
message (ignoring case transformations). This causes conflicts when generating
code in some languages.
### Display names
Many resources have a human-readable name, often used for display in UI. This
field **should** be called `display_name`, and **should not** have a uniqueness
requirement.
If an entity has an official, formal name (such as a company name or the title
of a book), an API **may** use `title` as the field name instead. The `title`
field **should not** have a uniqueness requirement.
## Further reading
- For naming resource fields, see [AIP-122][].
- For naming fields representing quantities, see [AIP-141][].
- For naming fields representing time, see [AIP-142][].
## Rationale
### URI vs URL
The guidance itself mentions that all URLs are URIs, but not all URIs are URLs.
The goal of aligning on URI is to enable a more generalizable field, that can
handle a variety of use cases, as well as drive standardization across APIs,
creating a more coherent surface. At the same time, the requirement being a
**should** allows for more specific terms when it truly merits it.
## Changelog
- **2025-03-10**: Add rationale for URI guidance.
- **2024-12-20**: Copy over abbreviations guidance from old design site.
- **2024-08-26**: Codify exception to string and base64 guidance
- **2024-05-18**: Documented the effect of field names on client surfaces.
- **2023-04-25**: Field names **must not** be expressed as verbs.
- **2021-07-12**: Normalized display name guidance to "should".
- **2021-04-07**: Added base64 and bytes guidance.
- **2021-03-05**: Added prohibition on leading, trailing, or adjacent
underscores.
- **2020-06-10**: Added prohibition on starting any word with a number.
- **2020-05-29**: Added guidance around URIs.
- **2020-03-24**: Added guidance around conflicting field and message names.
- **2020-01-30**: Added guidance around `display_name` and `title`.
[aip-122]: ./0122.md
[aip-141]: ./0141.md
[aip-142]: ./0142.md
>>>> 0141.md
---
id: 141
state: approved
created: 2019-07-18
placement:
category: fields
order: 20
---
# Quantities
Many services need to represent a discrete quantity of items (number of bytes,
number of miles, number of nodes, etc.).
## Guidance
Quantities with a clear unit of measurement (such as bytes, miles, and so on)
**must** include the unit of measurement as the suffix. When appropriate, units
**should** use generally accepted abbreviations (for example, `distance_km`
rather than `distance_kilometers`).
```proto
// A representation of a non-stop air route.
message Route {
// The airport where the route begins.
string origin = 1;
// The destination airport.
string destination = 2;
// The distance between the origin and destination airports.
// This value is also used to determine the credited frequent flyer miles.
int32 distance_miles = 3;
}
```
If the quantity is a number of items (for example, the number of nodes in a
cluster), then the field **should** use the suffix `_count` (**not** the prefix
`num_`):
```proto
// A cluster of individual nodes.
message Cluster {
// The number of nodes in the cluster.
int32 node_count = 1;
}
```
**Note:** Fields **must not** use unsigned integer types, because many
programming languages and systems do not support them well.
### Specialized messages
It is sometimes useful to create a message that represents a particular
quantity. This is particularly valuable in two situations:
- Grouping two or more individual quantities together. Example:
[`google.protobuf.Duration`][duration]
- Representing a common concept where the unit of measurement may itself vary.
Example: [`google.type.Money`][money]
APIs **may** create messages to represent quantities when appropriate. When
using these messages as fields, APIs **should** use the name of the message as
the suffix for the field name if it makes intuitive sense to do so.
## Changelog
- **2019-09-13**: Added the prohibition on uint and fixed types.
<!-- prettier-ignore-start -->
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
<!-- prettier-ignore-end -->
>>>> 0142.md
---
id: 142
state: approved
created: 2019-07-16
placement:
category: fields
order: 30
---
# Time and duration
Many services need to represent the concepts surrounding time. Representing
time can be challenging due to the intricacies of calendars and time zones, as
well as the fact that common exchange formats (such as JSON) lack a native
concept of time.
## Guidance
Fields representing time **should** use the common, generally used components
(such as [`google.protobuf.Timestamp`][timestamp] or
[`google.type.Date`][date]) for representing time or duration types. These
types are [common components][aip-213], and using them consistently allows
infrastructure and tooling to provide a better experience when interacting with
time values.
### Timestamps
Fields that represent an absolute point in time (independent of any time zone
or calendar) **should** use the [`google.protobuf.Timestamp`][timestamp] type,
(which uses UNIX timestamps under the hood and hold nanosecond precision).
These fields **should** have names ending in `_time`, such as `create_time` or
`update_time`. For repeated fields, the names **should** end in `_times`
instead.
Many timestamp fields refer to an activity (for example, `create_time` refers
to when the applicable resource was created). For these, the field **should**
be named with the `{imperative}_time` form. For example, if a book is being
published, the field storing the time when this happens would use the
imperative form of the verb "to publish" ("publish") resulting in a field
called `publish_time`. Fields **should not** be named using the past tense
(such as `published_time`, `created_time` or `last_updated_time`).
### Durations
Fields that represent a span between two points in time (independent of any
time zone or calendar) **should** use the
[`google.protobuf.Duration`][duration] type.
To illustrate the distinction between timestamps and durations, consider a
flight record:
```proto
// A representation of a (very incomplete) flight log.
message FlightRecord {
// The absolute point in time when the plane took off.
google.protobuf.Timestamp takeoff_time = 1;
// The length (duration) of the flight, from takeoff to landing.
google.protobuf.Duration flight_duration = 2;
}
```
**Note:** Observant readers may notice that the [timestamp][] and [duration][]
messages have the same structure (`int64 seconds` and `int32 nanos`). However,
the distinction between these is important, because they have different
semantic meaning. Additionally, tooling is able to base behavior off of which
message is used. For example, a Python-based tool could convert timestamps to
[datetime][py_datetime] objects and durations to [timedelta][py_timedelta]
objects.
### Relative time segments
In some cases, it may be necessary to represent a time segment inside a stream.
In these cases, the [`google.protobuf.Duration`][duration] type **should** be
used, and the field name **should** end with `_offset`. To ensure that the
meaning is clear, the field **must** have a comment noting the point that the
offset is relative to.
To illustrate this, consider a resource representing a segment of an audio
stream:
```proto
message AudioSegment {
// The duration relative to the start of the stream representing the
// beginning of the segment.
google.protobuf.Duration start_offset = 1;
// The total length of the segment.
google.protobuf.Duration segment_duration = 2;
}
```
### Civil dates and times
Fields that represent a calendar date or wall-clock time **should** use the
appropriate common components:
- Civil date: [`google.type.Date`][date]
- Wall-clock time: [`google.type.TimeOfDay`][time_of_day]
Fields representing civil dates **should** have names ending in `_date`, while
fields representing civil times or datetimes **should** have names ending in
`_time`.
**Note:** Both the `Date` and `TimeOfDay` components are timezone-naïve. Fields
that require timezone-awareness **should** use `DateTime` (see below).
### Civil timestamps
Fields that represent a civil timestamp (date and time, optionally with a time
zone) **should** use the [`google.type.DateTime`][datetime] component, and the
field name **should** end in `_time`.
### Compatibility
Occasionally, APIs are unable to use the common structures for legacy or
compatibility reasons. For example, an API may conform to a separate
specification that mandates that timestamps be integers or [ISO-8601][]
strings.
In these situations, fields **may** use other types. If possible, the following
naming conventions apply:
- For integers, include the meaning (examples: `time`, `duration`, `delay`,
`latency`) **and** the unit of measurement (valid values: `seconds`,
`millis`, `micros`, `nanos`) as a final suffix. For example,
`send_time_millis`.
- For strings, include the meaning (examples: `time`, `duration`, `delay`,
`latency`) but no unit suffix.
In all cases, clearly document the expected format, and the rationale for its
use.
<!-- prettier-ignore-start -->
[aip-213]: ./0213.md
[date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto
[datetime]: https://github.com/googleapis/googleapis/blob/master/google/type/datetime.proto
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[iso-8601]: https://www.iso.org/iso-8601-date-and-time-format.html
[py_datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime
[py_timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta
[time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
>>>> 0143.md
---
id: 143
state: approved
created: 2019-07-24
placement:
category: fields
order: 40
---
# Standardized codes
Many common concepts, such as spoken languages, countries, currency, and so on,
have common codes (usually formalized by the [International Organization for
Standardization][iso]) that are used in data communication and processing.
These codes address the issue that there are often different ways to express
the same concept in written language (for example, "United States" and "USA",
or "Español" and "Spanish").
## Guidance
For concepts where a standardized code exists and is in common use, fields
representing these concepts **should** use the standardized code for both input
and output.
```proto
// A message representing a book.
message Book {
// Other fields...
// The IETF BCP-47 language code representing the language in which
// the book was originally written.
// https://en.wikipedia.org/wiki/IETF_language_tag
string language_code = 99;
}
```
- Fields representing standardized concepts **must** use the appropriate data
type for the standard code (usually `string`).
- Fields representing standardized concepts **should not** use enums, even if
they only allow a small subset of possible values. Using enums in this
situation often leads to frustrating lookup tables when using multiple APIs
together.
- Fields representing standardized concepts **must** indicate which standard
they follow, preferably with a link (either to the standard itself, the
Wikipedia description, or something similar).
- The field name **should** end in `_code` or `_type` unless the concept has an
obviously clearer suffix.
- When accepting values provided by users, validation **should** be
case-insensitive unless this would introduce ambiguity (for example, accept
both `en-gb` and `en-GB`). When providing values to users, APIs **should**
use the canonical case (in the example above, `en-GB`).
### Content types
Fields representing a content or media type **must** use [IANA media types][].
For legacy reasons, the field **should** be called `mime_type`.
### Countries and regions
Fields representing individual countries or nations **must** use the [Unicode
CLDR region codes][cldr] ([list][]), such as `US` or `CH`, and the field
**must** be called `region_code`.
**Important:** Please read the [rationale](#countryregion-field-naming) for this
requirement.
### Currency
Fields representing currency **must** use [ISO-4217 currency codes][iso-4217],
such as `USD` or `CHF`, and the field **must** be called `currency_code`.
**Note:** For representing an amount of money in a particular currency, rather
than the currency code itself, use [`google.type.Money`][money].
### Language
Fields representing spoken languages **must** use [IETF BCP-47 language
codes][bcp-47] ([list][]), such as `en-US` or `de-CH`, and the field **must**
be called `language_code`.
### Time zones
Fields representing a time zone **should** use the [IANA TZ][] codes, and the
field **must** be called `time_zone`.
Fields also **may** represent a UTC offset rather than a time zone (note that
these are subtly different). In this case, the field **must** use the [ISO-8601
format][] to represent this, and the field **must** be named `utc_offset`.
## Rationale
### Country/region field naming
The use of `region_code` instead of `country_code` is critical to being able to
convey regions that are distinct from any country and to avoid any political
disputes associated with said region regarding their soverignty or affiliation.
Google and many other companies are supporters of Unicode CLDR and standardize
their product internationalization efforts on Unicode CLDR, and APIs are no
different here. Furthermore, many of the values supported by Unicode CLDR are
not countries on their own, so using a more generic name is actually more
compatible with the specification.
## Changelog
- **2024-12-03**: Strengthen rationale of country/region field naming
- **2024-11-12**: Change country/region code list to CLDR list from IANA list
- **2020-05-12**: Replaced `country_code` guidance with `region_code`,
correcting an original error.
<!-- prettier-ignore-start -->
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[cldr]: http://cldr.unicode.org/
[iana media types]: https://www.iana.org/assignments/media-types/media-types.xhtml
[iana tz]: http://www.iana.org/time-zones
[iso]: https://www.iso.org/
[iso-4217]: https://en.wikipedia.org/wiki/ISO_4217
[iso-8601 format]: https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC
[list]: https://www.unicode.org/cldr/charts/46/supplemental/territory_information.html
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
<!-- prettier-ignore-end -->
>>>> 0144.md
---
id: 144
state: approved
created: 2020-03-19
updated: 2022-06-02
placement:
category: fields
order: 50
---
# Repeated fields
Representing lists of data in an API is trickier than it often appears. Users
often need to modify lists in place, and longer data series within a single
resource pose a challenge for pagination.
## Guidance
Resources **may** use repeated fields where appropriate.
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
repeated string authors = 2;
}
```
- Repeated fields **must** use a plural field name.
- If the English singular and plural words are identical ("moose", "info"),
the dictionary word **must** be used rather than attempting to coin a new
plural form.
- Repeated fields **should** have an enforced upper bound that will not cause a
single resource payload to become too large. A good rule of thumb is 100
elements.
- If repeated data has the chance of being too large, the API **should** use
a sub-resource instead.
- Repeated fields **must not** represent the body of another resource inline.
Instead, the message **should** provide the resource names of the associated
resources.
### Scalars and messages
Repeated fields **should** use a scalar type (such as `string`) if they are
certain that additional data will not be needed in the future, as using a
message type adds significant cognitive overhead and leads to more complicated
code.
However, if additional data is likely to be needed in the future, repeated
fields **should** use a message instead of a scalar proactively, to avoid
parallel repeated fields.
### Update strategies
A resource **may** use two strategies to enable updating a repeated field:
direct update using the [standard `Update`][aip-134] method, or custom `Add`
and `Remove` methods.
A standard `Update` method has one key limitation: the user is only able to
update _the entire_ list. Field masks are unable to address individual entries
in a repeated field. This means that the user must read the resource, make
modifications to the repeated field value as needed, and send it back. This is
fine for many situations, particularly when the repeated field is expected to
have a small size (fewer than 10 or so) and race conditions are not an issue,
or can be guarded against with [ETags][aip-154].
**Note:** Declarative-friendly resources **must** use the standard `Update`
method, and not introduce `Add` and `Remove` methods. If declarative tools need
to reason about particular relationships while ignoring others, consider using
a subresource instead.
If atomic modifications are required, the API **should** define custom methods
using the verbs `Add` and `Remove`:
**Note:** If both of these strategies are too restrictive, consider using a
subresource instead.
```proto
rpc AddAuthor(AddAuthorRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:addAuthor"
body: "*"
};
}
rpc RemoveAuthor(RemoveAuthorRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:removeAuthor"
body: "*"
};
}
```
- The data being added or removed **should** be a primitive (usually a
`string`).
- For more complex data structures with a primary key, the API **should** use
a map with the `Update` method instead.
- The RPC's name **must** begin with the word `Add` or `Remove`. The remainder
of the RPC name **should** be the singular form of the field being added.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be the resource itself, unless there is useful
context to provide in the response, in which case the response message must
match the RPC name, with a `Response` suffix.
- When the response is the resource itself, it **should** include the
fully-populated resource.
- The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136].
- The HTTP URI **must** end with `:add*` or `:remove*`, where `*` is the
snake-case singular name of the field being added or removed.
- The request message field receiving the resource name **should** map to the
URI path.
- The HTTP variable **should** be the name of the resource (such as `book`)
rather than `name` or `parent`.
- That variable **should** be the only variable in the URI path.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
- If the data being added in an `Add` RPC is already present, the method
**must** error with `ALREADY_EXISTS`.
- If the data being removed in a `Remove` RPC is not present, the method
**must** error with `NOT_FOUND`.
#### Request Message
```proto
message AddAuthorRequest {
// The name of the book to add an author to.
string book = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
string author = 2 [(google.api.field_behavior) = REQUIRED];
}
message RemoveAuthorRequest {
// The name of the book to remove an author from.
string book = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"
];
string author = 2 [(google.api.field_behavior) = REQUIRED];
}
```
- A resource field **must** be included. It **should** be the name of the
resource (such as `book`) rather than `name` or `parent`.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- A field for the value being added or removed **must** be included. It
**should** be the singular name of the field.
- The field **should** be [annotated as required][aip-203].
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-17**: Recommended returning the resource itself in Add and Remove
RPCs over separate response types.
- **2020-10-17**: Added guidance for Add and Remove RPCs and requests.
[aip-123]: ./0123.md
[aip-134]: ./0134.md
[aip-136]: ./0136.md
[aip-154]: ./0154.md
[aip-203]: ./0203.md
>>>> 0145.md
---
id: 145
state: approved
created: 2020-05-28
placement:
category: fields
order: 60
---
# Ranges
Services often need to represent ranges of discrete or continuous values. These
have wide differences in meaning, and come in many types: integers, floats, and
timestamps, just to name a few, and the expected meaning of a range can vary in
subtle ways depending on the type of range being discussed.
## Guidance
A resource or message representing a range **should** ordinarily use two
separate fields of the same type, with prefixes `start_` and `end_`:
```proto
// A representation of a chapter in a book.
message Chapter {
string title = 1;
// The page where this chapter begins.
int32 start_page = 2;
// The page where the next chapter or section begins.
int32 end_page = 3;
}
```
### Inclusive or exclusive ranges
Fields representing ranges **should** use inclusive start values and exclusive
end values (half-closed intervals) in most situations; in interval notation:
`[start_xxx, end_xxx)`.
Exclusive end values are preferable for the following reasons:
- It conforms to user expectations, particularly for continuous values such as
timestamps, and avoids the need to express imprecise "limit values" (e.g.
`2012-04-20T23:59:59`).
- It is consistent with most common programming languages, including C++, Java,
Python, and Go.
- It is easier to reason about abutting ranges: `[0, x), [x, y), [y, z)`, where
values are chainable from one range to the next.
### Timestamp intervals
The following section describes the use of the [google.type.Interval][interval]
type, found amongst the common protos that are described in [AIP-213][].
This type represents a range between two timestamps, with an inclusive start
value and exclusive end value.
Ranges between two timestamps which conform to the expectations of the
`Interval` message **should** use this rather than having separate start and
end fields. This allows client code to be written against the `Interval`
message (such as checking whether a given timestamp occurs within the interval)
and reused across multiple intervals in the same API, or even across multiple
APIs.
APIs **may** use start and end timestamp fields instead. In particular, if a
message within an API is inherently describing an interval with extra
information about that interval, the additional level of nesting introduced
by using the `Interval` message may be undesirable.
### Exceptions
In some cases, there is significant colloquial precedent for inclusive start
and end values (closed intervals), to the point that using an exclusive end
value would be confusing even for people accustomed to them.
For example, when discussing dates (not to be confused with timestamps), most
people use inclusive end: a conference with dates "April 21-23" is expected to
run for three days: April 21, April 22, and April 23. This is also true for
days of the week: a business that is open "Monday through Friday" is open, not
closed, on Fridays.
In this situation, the prefixes `first` and `last` **should** be used instead:
```proto
// A representation of a chapter in a book.
message Chapter {
string title = 1;
// The first page of the chapter.
int32 first_page = 2;
// The last page of the chapter.
int32 last_page = 3;
}
```
Fields representing ranges with significant colloquial precedent for inclusive
start and end values **should** use inclusive end values with `first_` and
`last_` prefixes for those ranges only. The service **should** still use
exclusive end values for other ranges where this does not apply, and **must**
clearly document each range as inclusive or exclusive.
[aip-213]: ./0213.md
[interval]: https://github.com/googleapis/googleapis/blob/master/google/type/interval.proto
>>>> 0146.md
---
id: 146
state: approved
created: 2019-05-28
placement:
category: fields
order: 70
---
# Generic fields
Most fields in any API, whether in a request, a resource, or a custom response,
have a specific type or schema. This schema is part of the contract that
developers write their code against.
However, occasionally it is appropriate to have a generic or polymorphic field
of some kind that can conform to multiple schemata, or even be entirely
free-form.
## Guidance
While generic fields are generally rare, a service **may** introduce generic
field where necessary. There are several approaches to this depending on how
generic the field needs to be; in general, services **should** attempt to
introduce the "least generic" approach that is able to satisfy the use case.
### Oneof
A `oneof` **may** be used to introduce a type union: the user or service is
able to specify one of the fields inside the `oneof`. Additionally, a `oneof`
**may** be used with the same type (usually strings) to represent a semantic
difference between the options.
Because the individual fields in the `oneof` have different keys, a developer
can programmatically determine which (if any) of the fields is populated.
A `oneof` preserves the largest degree of type safety and semantic meaning for
each option, and services **should** generally prefer them over other generic
or polymorphic options when feasible. However, the `oneof` construct is
ill-suited when there is a large (or unlimited) number of potential options, or
when there is a large resource structure that would require a long series of
"cascading oneofs".
**Note:** Adding additional possible fields to an existing `oneof` is a
non-breaking change, but moving existing fields into or out of a `oneof` is
breaking (it creates a backwards-incompatible change in Go protobuf stubs).
### Maps
Maps **may** be used in situations where many values _of the
same type_ are needed, but the keys are unknown or user-determined.
Maps are usually not appropriate for generic fields because the map values all
share a type, but occasionally they are useful. In particular, a map can
sometimes be suited to a situation where many objects of the same type are
needed, with different behavior based on the names of their keys (for example,
using keys as environment names).
### Struct
The [`google.protobuf.Struct`][struct] object **may** be used to represent arbitrary
nested JSON. Keys can be strings, and values can be floats, strings, booleans,
arrays, or additional nested structs, allowing for an arbitrarily nested
structure that can be represented as JSON (and is automatically represented as
JSON when using REST/JSON).
A `Struct` is most useful when the service does not know the schema in advance,
or when a service needs to store and retrieve arbitrary but structured user
data. Using a `Struct` is convenient for users in this case because they can
easily get JSON objects that can be natively manipulated in their environment
of choice.
If a service needs to reason about the _schema_ of a `Struct`, it **should**
use [JSONSchema][] for this purpose. Because JSONSchema is itself JSON, a valid
JSONSchema document can itself be stored in a `Struct`.
### Any
The [`google.protobuf.Any`][any] object can be used to send an arbitrary
serialized protocol buffer and a type definition.
However, this introduces complexity, because an `Any` becomes useless for any
task other than blind data propagation if the consumer does not have access to
the proto. Additionally, even if the consumer _does_ have the proto, the
consumer has to ensure the type is registered and then deserialize manually,
which is an often-unfamiliar process.
Because of this, `Any` **should not** be used unless other options are
infeasible.
<!-- prettier-ignore-start -->
[any]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/any.proto
[struct]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/struct.proto
[JSONSchema]: https://json-schema.org/
<!-- prettier-ignore-end -->
>>>> 0147.md
---
id: 147
state: approved
created: 2020-07-24
placement:
category: fields
order: 80
---
# Sensitive fields
Sometimes APIs need to collect sensitive information such as private encryption
keys meant to be _stored_ by the underlying service but not intended to be
_read_ after writing due to the sensitive nature of the data. For this type of
data, extra consideration is required for the representation of the sensitive
data in API requests and responses.
## Guidance
If the sensitive information is _required_ for the resource as a whole to
exist, the data **should** be accepted as an [input-only field][input-only]
with no corresponding output field. Because the sensitive data must be present
for the resource to exist, users of the API may assume that existence of the
resource implies storage of the sensitive data. For example:
```proto
message SelfManagedKeypair {
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The public key data in PEM-encoded form.
bytes public_key = 2;
// The private key data in PEM-encoded form.
bytes private_key = 3 [
(google.api.field_behavior) = INPUT_ONLY];
}
```
If the sensitive information is _optional_ within the containing resource, an
[output-only][] boolean field with a postfix of `_set` **should** be used to
indicate whether or not the sensitive information is present. For example:
```proto
message Integration {
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
string uri = 2;
// A secret to be passed in the `Authorization` header of the webhook.
string shared_secret = 3 [
(google.api.field_behavior) = INPUT_ONLY];
// True if a `shared_secret` has been set for this Integration.
bool shared_secret_set = 4 [
(google.api.field_behavior) = OUTPUT_ONLY];
}
```
If it is important to be able to identify the sensitive information without
allowing it to be read back entirely, a field of the same type with an
`obfuscated_` prefix **may** be used instead of the boolean `_set` field to
provide contextual information about the sensitive information. The specific
nature of the obfuscation is outside the scope of this AIP. For example:
```proto
message AccountRecoverySettings {
// An email to use for account recovery.
string email = 1 [
(google.api.field_behavior) = INPUT_ONLY];
// An obfuscated representation of the recovery email. For example,
// `[email protected]` might be represented as `a**@e*****e.com`.
string obfuscated_email = 2 [
(google.api.field_behavior) = OUTPUT_ONLY];
}
```
<!-- prettier-ignore-start -->
[input-only]: ./0203.md#input-only
[output-only]: ./0203.md#output-only
<!-- prettier-ignore-end -->
>>>> 0148.md
---
id: 148
state: approved
created: 2020-10-06
placement:
category: fields
order: 90
---
# Standard fields
Certain concepts are common throughout any corpus of APIs. In these situations,
it is useful to have a standard field name and behavior that is used
consistently to communicate that concept.
## Guidance
Standard fields **should** be used to describe their corresponding concept, and
**should not** be used for any other purpose.
### Resource names and IDs
#### name
Every resource **must** have a `string name` field, used for the resource name
(AIP-122), which **should** be the first field in the resource.
**Note:** The `_name` suffix **should not** be used to describe other types of
names unless otherwise covered in this AIP.
#### parent
The `string parent` field refers to the resource name of the parent of a
collection, and **should** be used in most `List` (AIP-132) and `Create`
(AIP-133) requests.
### Other names
#### display_name
The `string display_name` field **must** be a mutable, user-settable field
where the user can provide a human-readable name to be used in user interfaces.
[Declarative-friendly resources][] **should** include this field.
Display names **should not** have uniqueness requirements, and **should** be
limited to <= 63 characters.
#### title
The `string title` field **should** be the official name of an entity, such as
a company's name. This is a more formal variant of `string display_name`.
#### given_name
The `string given_name` field **must** refer to a human or animal's given name.
Resources **must not** use `first_name` for this concept, because the given
name is not placed first in many cultures.
#### family_name
The `string family_name` field **must** refer to a human or animal's family
name. Resources **must not** use `last_name` for this concept, because the
family name is not placed last in many cultures.
### Timestamps
#### create_time
The output only `google.protobuf.Timestamp create_time` field **must**
represent the timestamp when the resource was created. This **may** be either
the time creation was initiated or the time it was completed.
[Declarative-friendly resources][] **should** include this field.
#### update_time
The output only `google.protobuf.Timestamp update_time` field **must**
represent the timestamp when the resource was most recently updated. Any change
to the resource made by users **must** refresh this value; changes to a
resource made internally by the service **may** refresh this value.
[Declarative-friendly resources][] **should** include this field.
#### delete_time
The output only `google.protobuf.Timestamp delete_time` field **must** represent
the timestamp that a resource was soft deleted. This **may** correspond to either
the time when the user requested deletion, or when the service successfully
soft deleted the resource. If a resource is not soft deleted, the `delete_time`
field **must** be empty.
Resources that support soft delete (AIP-164) **should** provide this field.
#### expire_time
The `google.protobuf.Timestamp expire_time` field **should** represent the time
that a given resource or resource attribute is no longer useful or valid (e.g. a
rotating security key). It **may** be used for similar forms of expiration as
described in AIP-214.
Services **may** provide an `expire_time` value that is inexact, but the
resource **must not** expire before that time.
#### purge_time
The `google.protobuf.Timestamp purge_time` field **should** represent the time
when a soft deleted resource will be purged from the system (see AIP-164).
It **may** be used for similar forms of expiration as described in AIP-214.
Resources that support soft delete **should** include this field.
Services **may** provide a `purge_time` value that is inexact, but the resource
**must not** be purged from the system before that time.
### Annotations
To store small amounts of arbitrary data, a `map<string, string> annotations`
field **may** be added.
The `annotations` field **must** use the [Kubernetes limits][] to maintain wire
compatibility, and **should** require dot-namespaced annotation keys to prevent
tools from trampling over one another.
Examples of information that might be valuable to store in annotations include:
- For CI/CD, an identifier of the pipeline run or version control identifier
used to propagate.
**Note:** Annotations are distinct from various forms of labels. Labels can be
used by server-side policies, such as IAM conditions. Annotations exist to
allow client tools to store their own state information without requiring a
database.
### Well known string fields
#### IP address
A field that represents an IP address **must** comply with the following:
* use type `string`
* use the name `ip_address` or end with the suffix `_ip_address` e.g.
`resolved_ip_address`
* specify the IP address version format via one of the supported formats `IPV4`,
`IPV6`, or if it can be either, `IPV4_OR_IPV6` (see [AIP-202][aip-202])
#### uid
The output only `string uid` field refers to a system-assigned unique
identifier for a resource. When provided, this field **must** be a [UUID4][]
and **must** specify this format via the `UUID4` format extension (see
[AIP-202][aip-202]). [Declarative-friendly resources][] **should** include this
field.
## Further reading
- For standardized codes, see AIP-143.
- For the `etag` field, see AIP-154.
- For the `request_id` field, see AIP-155.
- For the `filter` field, see AIP-160.
- For fields related to resource revisions, see AIP-162.
- For the `validate_only` field, see AIP-163.
- For fields related to soft delete and undelete, see AIP-164.
## Rationale
### Well known string fields
Some fields represent very well defined concepts or artifacts that sometimes
also have strict governance of their semantics. For such fields, presenting an
equally standardized API surface is important. This enables development of
improved API consumer tools and documentation, as well as a more unified user
experience across the platform.
## History
Before 2023-07, `purge_time` for soft-deleted resources was also called
`expire_time`. `purge_time` was introduced to reduce user confusion.
## Changelog
- **2023-10-05**: Introduce well known string fields with IP Address and `uid`.
- **2023-08-14**: Introduce the term `annotations` from AIP-128.
- **2023-07-13**: Introduce the term `purge_time`.
- **2021-04-06**: Require output only field behavior for `uid` and `delete_time`
fields.
<!-- prettier-ignore -->
[aip-202]: ./0202.md
[declarative-friendly resources]: ./0128.md#resources
[kubernetes limits]: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set
[uuid4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
>>>> 0149.md
---
id: 149
state: approved
created: 2021-02-12
placement:
category: fields
order: 100
---
# Unset field values
In many messages, many fields are optional: the user is not required to provide
them, or for output fields, the service might not populate the field.
In most cases, there is no meaningful difference between setting it to a
default value (such as `0`) as opposed to not setting it at all; however,
occasionally this distinction is meaningful.
## Guidance
Services defined in protocol buffers **should** use the `optional` keyword for
primitives if and only if it is necessary to distinguish setting the field to
its default value (`0`, `false`, or empty string) from not setting it at all:
```proto
// A representation of a book in a library.
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The name of the book.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The rating for the book, from 0 to 5.
// 0 is distinct from no rating.
optional int32 rating = 2;
}
```
**Important:** Services **should not** need to distinguish between the default
value and unset most of the time; if an alternative design does not require
such a distinction, it is usually preferred. In practice, this means `optional`
**should** only ever be used for integers and floats.
**Important:** Tracking field presence is *not* the same as documenting API
field behavior as defined in [AIP-203][]. For example, a field labeled with
`optional` for presence tracking **may** also be annotated as
`google.api.field_behavior = REQUIRED` if the field must be set. If you only
want to document the server perceived behavior of a field, read [AIP-203][].
### Backwards compatibility
It is a backwards incompatible change to add or remove the `optional` qualifier
to an existing field. This is because the compiled src API is changed (in some
languages). For example, in Golang, adding `optional` changes the field type of
primitives to be the pointer variant of their original type, e.g. a field
formerly of type `string` becomes `*string`, etc. Accordingly, this change
requires that both clients and servers update their usage of the changed field
in unison, which is risky and error prone. Additional information is
[documented by Protobuf][proto docs].
## Rationale
### field behavior and `optional`
The field behavior annotation and `optional` label are not mutually exclusive,
because they address different problems. The former,
`google.api.field_behavior`, focuses on communicating the server's perception of
a field within the API e.g. if it is required or not, if it is immutable, etc.
The latter, proto3's `optional`, is a wire format and code generation option
that is strictly for toggling field presence tracking. While it might be
confusing for a field to be simultaneously annotated with
`google.api.field_behavior = REQUIRED` and labeled as `optional`, they are
unrelated in practice and can reasonably be used together.
## Changelog
- **2024-06-05**: Add backwards compatibility considerations
- **2023-06-20**: Differentiate from field behavior documentation
[AIP-203]: ./0203.md
[proto docs]: https://protobuf.dev/programming-guides/field_presence/#considerations-for-change-compatibility
>>>> 0151.md
---
id: 151
state: approved
created: 2019-07-25
placement:
category: operations
order: 110
---
# Long-running operations
Occasionally, an API may need to expose a method that takes a significant
amount of time to complete. In these situations, it is often a poor user
experience to simply block while the task runs; rather, it is better to return
some kind of promise to the user and allow the user to check back in later.
The long-running operations pattern is roughly analogous to a [Python
Future][], or a [Node.js Promise][]. Essentially, the user is given a token
that can be used to track progress and retrieve the result.
## Guidance
Individual API methods that might take a significant amount of time to complete
**should** return a [`google.longrunning.Operation`][lro] object instead of the
ultimate response message.
```proto
// Create a book.
rpc CreateBook(CreateBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be `google.longrunning.Operation`. The `Operation`
proto definition **must not** be copied into individual APIs.
- The response **must not** be a streaming response.
- The method **must** include a `google.longrunning.operation_info` annotation,
which **must** define both response and metadata types.
- The response and metadata types **must** be defined in the file where the
RPC appears, or a file imported by that file.
- If the response and metadata types are defined in another package, the
fully-qualified message name **must** be used.
- The response type **should not** be `google.protobuf.Empty` (except for
[`Delete`][aip-135] methods), unless it is certain that response data will
_never_ be needed. If response data might be added in the future, define an
empty message for the RPC response and use that.
- The metadata type is used to provide information such as progress, partial
failures, and similar information on each `GetOperation` call. The metadata
type **should not** be `google.protobuf.Empty`, unless it is certain that
metadata will _never_ be needed. If metadata might be added in the future,
define an empty message for the RPC metadata and use that.
- APIs with messages that return `Operation` **must** implement the
[`Operations`][lro] service. Individual APIs **must not** define their own
interfaces for long-running operations to avoid non-uniformity.
- If an RPC supports a [validate-only mode][aip-163], the response to a
validation request **must** be one of the following:
- A successful response with an `Operation` which is already complete, with
the `done` field set to `true`, and a valid (but potentially empty) response
message in the `response` field, wrapped in a `google.protobuf.Any` message.
The `name` field **may** be empty, to avoid the service having to maintain
state for successful validation.
- An immediate error response (typically "bad request")
- An `Operation` with the `done` field set to `false`, to indicate
long-running validation. In this case, the `name` field **must** be set,
to allow clients to poll the long-running validation operation until it
has completed. Successful validation **must** eventually be represented by
an operation with `done=true` and a valid (but potentially empty) wrapped
response message in the `response` field. Unsuccessful validation **must**
eventually be represented by an operation with `done=true` and the error
details provided in the `error` field.
**Note:** User expectations can vary on what is considered "a significant
amount of time" depending on what work is being done. A good rule of thumb is
10 seconds.
### Standard methods
APIs **may** return an `Operation` from the [`Create`][aip-133],
[`Update`][aip-134], or [`Delete`][aip-135] standard methods if appropriate. In
this case, the response type in the `operation_info` annotation **must** be the
standard and expected response type for that standard method.
When creating or deleting a resource with a long-running operation, the
resource **should** be included in [`List`][aip-132] and [`Get`][aip-131]
calls; however, the resource **should** indicate that it is not usable,
generally with a [state enum][aip-216].
### Parallel operations
A resource **may** accept multiple operations that will work on it in parallel,
but is not obligated to do so:
- Resources that accept multiple parallel operations **may** place them in a
queue rather than work on the operations simultaneously.
- Resources that do not permit multiple operations in parallel (denying any
new operation until the one that is in progress finishes) **must** return
`ABORTED` if a user attempts a parallel operation, and include an error
message explaining the situation.
- Resources with [declarative-friendly APIs][aip-128] **may** allow subsequent
updates to preempt existing operations. In this case, the latest update
begins processing and previous operations are marked as `ABORTED` with an
error message explaining the situation.
### Expiration
APIs **may** allow their operation resources to expire after sufficient time
has elapsed after the operation completed.
**Note:** A good rule of thumb for operation expiry is 30 days.
### Errors
Errors that prevent a long-running operation from _starting_ **must** return an
error response ([AIP-193][]), similar to any other method.
Operations that fail during their execution phase **must** return an
error response ([AIP-193][]), placed in the `Operation.error` [google.rpc.Status][]
field.
Non-terminal errors that occur over the course of an operation **may** be placed
in the metadata message and the field(s) **must** be [AIP-193][] compliant
[google.rpc.Status][].
### Backwards compatibility
Changing either the `response_type` or `metadata_type` of a long-running operation
is a breaking change.
## Rationale
### Validate-only behavior
The guidance for validate-only responses comes from a tension
between clients, which benefit from "fully formed" operations that can
be treated uniformly, and servers, which don't wish to maintain
additional state for trivial operations. It seems counterintuitive
that just validating a request should generate more state, but a
full operation response that can be fetched later would either
require that or "special" singleton operation IDs. The guidance
provided is a compromise: by returning a "done" operation, clients
can use existing logic to check that the operation has completed
successfully (and therefore doesn't need to be fetched for an
updated status) but server don't need to maintain any additional state.
<!-- prettier-ignore-start -->
[aip-128]: ./0128.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-163]: ./0163.md
[aip-193]: ./0193.md
[aip-216]: ./0216.md
[google.rpc.Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
[lro]: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto
[node.js promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises
[python future]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
<!-- prettier-ignore-end -->
## Changelog
- **2025-02-04**: Clarified error propagation behavior for failures
that occur during long-running operations.
- **2024-04-23**: Provided pattern for validation on RPCs returning
long-running operations.
- **2022-05-31**: Added compatibility section.
- **2020-08-24**: Clarified that responses are not streaming responses.
- **2020-06-24**: Added guidance for parallel operations.
- **2020-03-20**: Clarified that both `response_type` and `metadata_type` are
required.
- **2019-11-22**: Added a short explanation of what `metadata_type` is for.
- **2019-09-23**: Added guidance on errors.
- **2019-08-23**: Added guidance about fully-qualified message names when the
message name is in another package.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0152.md
---
id: 152
state: approved
created: 2020-04-27
updated: 2022-06-02
placement:
category: design-patterns
order: 10
---
# Jobs
Occasionally, APIs may need to expose a task that takes significant time to
complete, and where a transient [long-running operation][aip-151] is not
appropriate. For example, a task could need to run repeatedly, or have separate
permissions for configuring the task as opposed to running it.
## Guidance
An API **may** define a `Job` resource to represent a particular task with
distinct setup, configuration, and execution:
```proto
message WriteBookJob {
option (google.api.resource) = {
type: "library.googleapis.com/WriteBookJob"
pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}"
};
// Name and other fields...
}
```
- The name of the resource **must** end with the word "Job".
- The prefix **should** be a valid RPC name, with a verb and a noun.
- The service **should** define all five of the standard methods (AIP-131,
AIP-132, AIP-133, AIP-134, AIP-135), and use them as the
primary way to configure the job.
### Run method
The service **should** define a `Run` custom method that executes the job
immediately:
```proto
rpc RunWriteBookJob(RunWriteBookJobRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/writeBookJobs/*}:run"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "RunWriteBookJobResponse"
metadata_type: "RunWriteBookJobMetadata"
};
}
```
- The RPC's name **must** begin with the word `Run`. The remainder of the
RPC name **should** be the singular form of the job resource being run.
- The request message **must** match the RPC name, with a `Request` suffix.
- The method **should** return a [long-running operation][aip-151], which
**must** resolve to a response message that includes the result of running
the job.
- The response message name must match the RPC name, with a `Response`
suffix.
- The method **may** use any metadata message it wishes.
- The HTTP verb **must** be `POST`, as is usual for [custom methods][aip-136].
- The body clause in the `google.api.http` annotation **should** be `"*"`.
- The URI path **should** contain a single `name` variable corresponding to the
name of the job resource being run.
- The URI path **must** end with `:run`.
- Errors that prevent execution of the job from _starting_ **must** return an
error response (AIP-193), similar to any other method. Errors that occur
over the course of the job execution **may** be placed in the metadata
message. The errors themselves **must** still be represented with a
[`google.rpc.Status`][status] object.
### Run request message
Run methods implement a common request message pattern:
```proto
message RunWriteBookJobRequest {
// The name of the job to run.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/WriteBookJob"
}];
}
```
- A singular `string name` field **must** be included.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
### Executions and results
Ordinarily, the API **should** provide results to the user as the final
response of the `Run` method. However, this is sometimes insufficient; for
example, a job that runs on a recurring schedule in the background can not
deliver results to the user in this way.
The service **may** store resources representing individual executions along
with their result as a sub-collection of resources under the job, which allows
the user to list *past* job executions. A service that does this **should**
define the `Get`, `List`, and `Delete` methods for the execution resources:
```proto
message WriteBookJobExecution {
option (google.api.resource) = {
type: "library.googleapis.com/WriteBookJobExecution"
pattern: "publishers/{publisher}/writeBookJobs/{write_book_job}/executions/{execution}"
};
// Name and other information about the execution, such as metadata, the
// result, error information, etc.
}
```
In this case, the operation returned by job's `Run` method **should** refer to
the child resource.
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-11-02**: Expanded guidance on HTTP, field behavior, and resource
reference annotations and request format.
<!-- prettier-ignore-start -->
[aip-123]: ./0123.md
[aip-136]: ./0136.md
[aip-151]: ./0151.md
[aip-203]: ./0203.md
[status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
<!-- prettier-ignore-end -->
>>>> 0153.md
---
id: 153
state: approved
created: 2019-12-16
placement:
category: design-patterns
order: 20
---
# Import and export
Many users want to be able to load data into an API, or get their existing data
out of an API. This is particularly important for enterprise users, who are
often concerned about vendor lock-in.
## Guidance
APIs **may** support import and export operations, which **may** create
multiple new resources, or they **may** populate data into a single resource.
### Multiple resources
Services **may** support importing and exporting multiple resources into or out
of an API, and **should** implement a common pattern to do so:
```proto
rpc ImportBooks(ImportBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:import"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ImportBooksResponse"
metadata_type: "ImportBooksMetadata"
};
}
rpc ExportBooks(ExportBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:export"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ExportBooksResponse"
metadata_type: "ExportBooksMetadata"
};
}
```
- The method **must** return a long-running operation (see [AIP-151][]) unless
the service can guarantee that it will _never_ need more than a few seconds
to complete.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- A `parent` field **should** be included as part of the URI.
- If importing into or exporting from multiple resources is required, the API
**should** keep the `parent` field and allow the user to use the `-`
character to indicate multiple parents (see [AIP-159][]).
- On import, if the user provides a specific parent, the API **must** reject
any imported resources that would be added to a different parent.
- The URI suffix **should** be `:import` or `:export`.
### Data for a single resource
Services **may** support importing and exporting data into or out of a single
resource, and **should** implement a common pattern to do so:
```proto
rpc ImportPages(ImportPagesRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:importPages"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ImportPagesResponse"
metadata_type: "ImportPagesMetadata"
};
}
rpc ExportPages(ExportPagesRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{book=publishers/*/books/*}:exportPages"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "ExportPagesResponse"
metadata_type: "ExportPagesMetadata"
};
}
```
- The method **must** return a long-running operation (see [AIP-151][]) unless
the service can guarantee that it will _never_ need more than a few seconds
to complete.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- A field representing the resource that data is being imported into **should**
be included as part of the URI. The field **should** be named after the
resource (and **should not** be called `name`).
- The URI suffix should include both the verb and a noun for the data itself,
such as `:importPages` or `:exportPages`.
### Request object
Imports and exports often require two fundamentally different types of
configuration:
1. Configuration specific to the source or destination.
2. Configuration regarding the imported or exported data itself.
Source or destination configuration should be grouped into a single message and
placed inside a oneof:
```proto
message ImportBooksRequest {
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
oneof source {
AuthorSource author_source = 2;
TranslatorSource translator_source = 3;
}
string isbn_prefix = 4;
}
message ExportBooksRequest {
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
oneof destination {
PrinterDestination printer_destination = 2;
TranslatorDestination translator_destination = 3;
}
string filter = 4;
}
```
- The source configuration messages **must** be placed within a `oneof source`
(for import) or `oneof destination` (for export), even if there is only one.
(This maintains flexibility to add more later.)
- Configuration related to the data itself (and therefore common across all
sources) **must** be placed at the top-level of the request message.
**Note:** The configuration for import and export **may** be different from one
another. (For example, it would be sensible to import from a file but export to
a directory.)
### Inline sources
APIs **may** also permit import and export "inline", where the contents to be
imported or exported are provided in the request or response.
```proto
message InlineSource {
repeated Book books = 1;
}
```
- The source or destination **should** be named `InlineSource` or
`InlineDestination`.
- The message **should** include a repeated field representing the resource.
However, if the resource structure is complex, the API **may** use a separate
inline representation. In this situation, the same format **must** be used
for both import and export.
### Partial failures
While partial failures are normally discouraged, import and export RPCs
**should** include partial failure information in the metadata object. Each
individual error **should** be a `google.rpc.Status` object describing the
error. For more on errors, see [AIP-193][].
[aip-151]: ./0151.md
[aip-159]: ./0159.md
[aip-193]: ./0193.md
[google bigquery]: https://cloud.google.com/bigquery
[google storage]: https://cloud.google.com/storage
>>>> 0154.md
---
id: 154
state: approved
created: 2019-07-24
placement:
category: design-patterns
order: 30
---
# Resource freshness validation
APIs often need to validate that a client and server agree on the current state
of a resource before taking some kind of action on that resource. For example,
two processes updating the same resource in parallel could create a race
condition, where the latter process "stomps over" the effort of the former one.
ETags provide a way to deal with this, by allowing the server to send a
checksum based on the current content of a resource; when the client sends that
checksum back, the server can ensure that the checksums match before acting on
the request.
## Guidance
A resource **may** include an `etag` field on any resource where it is
important to ensure that the client has an up to date resource before acting on
certain requests:
```proto
// A representation of a book.
message Book {
// Other fields...
// This checksum is computed by the server based on the value of other
// fields, and may be sent on update and delete requests to ensure the
// client has an up-to-date value before proceeding.
string etag = 99;
}
```
- The etag field **must** be a string, and **must** be named `etag`.
- The etag field on the _resource_ **should not** be given any behavior
annotations.
- The etag field **must** be provided by the server on output, and values
**should** conform to [RFC 7232][].
- If a user sends back an etag which matches the current etag value, the
service **must** permit the request (unless there is some other reason for
failure).
- If a user sends back an etag which does not match the current etag value, the
service **must** send an `ABORTED` error response (unless another error takes
precedence, such as `PERMISSION_DENIED` if the user is not authorized).
- If the user does not send an etag value at all, the service **should** permit
the request. However, services with strong consistency or parallelism
requirements **may** require users to send etags all the time and reject the
request with an `INVALID_ARGUMENT` error in this case.
**Note:** ETag values **should** include quotes as described in [RFC 7232][].
For example, a valid etag is `"foo"`, not `foo`.
### Declarative-friendly resources
A resource that is declarative-friendly (AIP-128) **must** include an `etag`
field.
### Etags on request methods
In some situations, the etag needs to belong on a request message rather than
the resource itself. For example, an `Update` standard method can "piggyback"
off the `etag` field on the resource, but the `Delete` standard method can not:
```proto
message DeleteBookRequest {
// The name of the book.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
// The current etag of the book.
// If an etag is provided and does not match the current etag of the book,
// deletion will be blocked and an ABORTED error will be returned.
string etag = 2 [(google.api.field_behavior) = OPTIONAL];
}
```
On a request message, the `etag` field **should** be given a behavior annotation
- either `REQUIRED` or `OPTIONAL`. See AIP-203 for more information.
An `etag` field **may** also be used on custom methods, similar to the example
above.
### Strong and weak etags
ETags can be either "strongly validated" or "weakly validated":
- A strongly validated etag means that two resources bearing the same etag are
byte-for-byte identical.
- A weakly validated etag means that two resources bearing the same etag are
equivalent, but may differ in ways that the service does not consider to be
important.
Resources **may** use either strong or weak etags, as it sees fit, but
**should** document the behavior. Additionally, weak etags **must** have a `W/`
prefix as mandated by [RFC 7232][].
[rfc 7232]: https://tools.ietf.org/html/rfc7232#section-2.3
## Further reading
- For how to retry on errors in client libraries, see AIP-194.
## Changelog
- **2021-04-01**: Updated an additional reference to `FAILED_PRECONDITION`
to `ABORTED`.
- **2021-03-05**: Changed the etag error from `FAILED_PRECONDITION` (which
becomes HTTP 400) to `ABORTED` (409).
- **2020-10-06**: Added declarative-friendly resource requirement.
- **2020-09-02**: Clarified that other errors may take precedence over
`FAILED_PRECONDITION` for etag mismatches.
- **2020-09-02**: Add guidance for etags on request messages.
- **2019-09-23**: Changed the title to "resource freshness validation".
>>>> 0155.md
---
id: 155
state: approved
created: 2019-05-06
placement:
category: design-patterns
order: 40
---
# Request identification
It is sometimes useful for an API to have a unique, customer-provided
identifier for particular requests. This can be useful for several purposes,
such as de-duplicating requests from parallel processes, ensuring the safety of
retries, or auditing.
The most important purpose for request IDs is to provide idempotency
guarantees: allowing the same request to be issued more than once without
subsequent calls having any effect. In the event of a network failure, the
client can retry the request, and the server can detect duplication and ensure
that the request is only processed once.
## Guidance
APIs **may** add a `string request_id` parameter to request messages (including
those of standard methods) in order to uniquely identify particular requests.
```proto
message CreateBookRequest {
// The parent resource where this book will be created.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The ID to use for the book, which will become the final component of
// the book's resource name.
//
// This value should be 4-63 characters, and valid characters
// are /[a-z][0-9]-/.
string book_id = 2 [(google.api.field_behavior) = REQUIRED];
// The book to create.
Book book = 3 [(google.api.field_behavior) = REQUIRED];
// A unique identifier for this request. Restricted to 36 ASCII characters.
// A random UUID is recommended.
// This request is only idempotent if a `request_id` is provided.
string request_id = 4 [(google.api.field_info).format = UUID4];
}
```
- Providing a request ID **must** guarantee idempotency.
- If a duplicate request is detected, the server **should** return the
response for the previously successful request, because the client most
likely did not receive the previous response.
- APIs **may** choose any reasonable timeframe for honoring request IDs.
- The `request_id` field **must** be provided on the request message to which
it applies (and it **must not** be a field on resources themselves).
- Request IDs **should** be optional.
- Request IDs **should** be able to be UUIDs, and **may** allow UUIDs to be the
only valid format. The format restrictions for request IDs **must** be
documented.
- Request IDs that are UUIDs **must** be annotated with the
`google.api.FieldInfo.Format` value `UUID4` using the extension
`(google.api.field_info).format = UUID4`. See [AIP-202](./0202.md) for more.
### Stale success responses
In some unusual situations, it may not be possible to return an identical
success response. For example, a duplicate request to create a resource may
arrive after the resource has not only been created, but subsequently updated;
because the service has no other need to retain the historical data, it is no
longer feasible to return an identical success response.
In this situation, the method **may** return the current state of the resource
instead. In other words, it is permissible to substitute the historical success
response with a similar response that reflects more current data.
## Further reading
- For which codes to retry, see [AIP-194](https://aip.dev/194).
- For how to retry errors in client libraries, see
[AIP-4221](https://aip.dev/4221).
## Rationale
### Using UUIDs for request identification
When a value is required to be unique, leaving the format open-ended can lead to
API consumers incorrectly providing a duplicate identifier. As such,
standardizing on a universally unique identifier drastically reduces the chance
for collisions when done correctly.
## Changelog
- **2024-01-08**: Add book_id to request message.
- **2023-10-02**: Add UUID format extension guidance.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0156.md
---
id: 156
state: approved
created: 2019-05-12
placement:
category: resource-design
order: 70
---
# Singleton resources
APIs sometimes need to represent a resource where exactly one instance of the
resource always exists within any given parent. A common use case for this is
for a config object.
## Guidance
An API **may** define _singleton resources_. A singleton resource **must**
always exist by virtue of the existence of its parent, with one and exactly one
per parent.
For example:
```proto
message Config {
option (google.api.resource) = {
type: "api.googleapis.com/Config"
pattern: "users/{user}/config"
singular: "config"
plural: "configs"
};
// additional fields including name
}
```
The `Config` singleton would have the following RPCs:
```proto
rpc GetConfig(GetConfigRequest) returns (Config) {
option (google.api.http) = {
get: "/v1/{name=users/*/config}"
};
}
rpc UpdateConfig(UpdateConfigRequest) returns (Config) {
option (google.api.http) = {
patch: "/v1/{config.name=users/*/config}"
body: "config"
};
}
```
- Singleton resources **must not** have a user-provided or system-generated ID;
their [resource name][aip-122] includes the name of their parent followed by
one static-segment.
- Example: `users/1234/config`
- Singleton resources are always singular.
- Example: `users/1234/thing`
- Singleton resource definitions **must** provide both the `singular` and
`plural` fields (see above example).
- Singleton resources **may** parent other resources.
- Singleton resources **must not** define the [`Create`][aip-133] or
[`Delete`][aip-135] standard methods. The singleton is implicitly created or
deleted when its parent is created or deleted.
- Singleton resources **should** define the [`Get`][aip-131] and
[`Update`][aip-134] methods, and **may** define custom methods as
appropriate.
- However, singleton resources **must not** define the [`Update`][aip-134]
method if all fields on the resource are [output only][aip-203].
- Singleton resources **may** define the [`List`][aip-132] method, but **must**
implement it according to [AIP-159][aip-159]. See the example below.
- The trailing segment in the path pattern that typically represents the
collection **should** be the `plural` form of the Singleton resource e.g.
`/v1/{parent=users/*}/configs`.
- If a parent resource ID is provided instead of the hyphen `-` as per
AIP-159, then the service **should** return a collection of one Singleton
resource corresponding to the specified parent resource.
```proto
rpc ListConfigs(ListConfigsRequest) returns (ListConfigsResponse) {
option (google.api.http) = {
get: "/v1/{parent=users/*}/configs"
};
}
message ListConfigsRequest {
// To list all configs, use `-` as the user id.
// Formats:
// * `users/-`
// * `users/{user}`
//
// Note: Specifying an actual user id will return a collection of one config.
// Use GetConfig instead.
string parent = 1 [
(google.api.resource_reference).child_type = "api.googleapis.com/Config"];
// other standard pagination fields...
}
```
## Rationale
### Support for Standard List
While Singleton resources are not directly part of a collection themselves, they
can be viewed as part of their parent's collection. The one-to-one relationship
of parent-to-singleton means that for every one parent there is one singleton
instance, naturally enabling some collection-based methods when combined with
the pattern of [Reading Across Collections][aip-159]. The Singleton can present
as a collection to the API consumer as it is indirectly one based on its parent.
Furthermore, presenting the Singleton resource as a pseudo-collection in such
methods enables future expansion to a real collection, should a Singleton be
found lacking.
### Including `plural` definition
While a Singleton is by definition singular, there are certain cases where
a Singleton resource may appear in a plural form e.g., if the service supports
Standard List (as defined here). As such, it is better to forward declare the
plural form of the Singleton resource type than to not have it when needed.
## Changelog
- **2024-04-15:** Singletons must specify `singular` and `plural` in resource.
- **2023-08-10:** Add Standard `List` support.
- **2023-07-26:** Clarified that read-only singletons should not have `Update`.
- **2021-11-02:** Added an example message and state parent eligibility.
- **2021-01-14:** Changed example from `settings` to `config` for clarity.
[aip-122]: ./0122.md
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-135]: ./0135.md
[aip-159]: ./0159.md
[aip-203]: ./0203.md#output-only
>>>> 0157.md
---
id: 157
state: approved
created: 2019-01-26
placement:
category: design-patterns
order: 50
---
# Partial responses
Sometimes, a resource can be either large or expensive to compute, and the API
needs to give the user control over which fields it sends back.
## Guidance
APIs **may** support partial responses in one of two ways:
### Field masks parameter
Field masks (`google.protobuf.FieldMask`) can be used for granting the user
fine-grained control over what fields are returned. An API **should** support the mask in a side channel.
For example, the parameter can be specified either using an HTTP query
parameter, an HTTP header, or a [gRPC metadata entry][1]. Google Cloud APIs specify field masks as a [system parameter][0].
Field masks **should not** be specified in the [request](./0157.md#read-masks-as-a-request-field).
- The value of the field mask parameter **must** be a `google.protobuf.FieldMask`.
- The field mask parameter **must** be optional:
- An explicit value of `"*"` **should** be supported, and **must** return all
fields.
- If the field mask parameter is omitted, it **must** default to `"*"`, unless otherwise documented.
- An API **may** allow read masks with non-terminal repeated fields (unlike
update masks), but is not obligated to do so.
**Note:** Changing the default value of the field mask parameter is a [breaking change](../0180.md#semantic-changes).
### View enumeration
Alternatively, an API **may** support partial responses with view enums.
View enums are useful for situations where an API only wants to expose a small
number of permutations to the user:
```proto
enum BookView {
// The default / unset value.
// The API will default to the BASIC view.
BOOK_VIEW_UNSPECIFIED = 0;
// Include basic metadata about the book, but not the full contents.
// This is the default value (for both ListBooks and GetBook).
BOOK_VIEW_BASIC = 1;
// Include everything.
BOOK_VIEW_FULL = 2;
}
```
- The enum **should** be specified as a `view` field on the request message.
- The enum **should** be named something ending in `-View`
- The enum **should** at minimum have values named `BASIC` and `FULL` (although
it **may** have values other than these).
- The `UNSPECIFIED` value **must** be valid (not an error), and the API
**must** document what the unspecified value will do).
- For List RPCs, the effective default value **should** be `BASIC`.
- For Get RPCs, the effective default value **should** be either `BASIC` or
`FULL`.
- The enum **should** be defined at the top level of the proto file (as it is
likely to be needed in multiple requests, e.g. both `Get` and `List`). See
[AIP-126][] for more guidance on top-level enumerations.
- APIs **may** add fields to a given view over time. APIs **must not** remove a
field from a given view (this is a breaking change).
**Note:** If a service requires (or might require) multiple views with
overlapping but distinct values, there is a potential for a namespace
conflict. In this situation, the service **should** nest the view enum within
the individual resource.
### Read masks as a request field
**Warning:** Read masks as a single field on the request message, for example: `google.protobuf.FieldMask read_mask` are **DEPRECATED**.
## Changelog
- **2023-05-09**: Fix top-level enum example and link to AIP-126.
- **2022-03-14:** Updated guidance on default value and how to specify a read mask.
- **2021-10-06:** Updated the guidance with system parameters.
- **2021-03-04:** Added guidance for conflicting view enums.
[0]: https://cloud.google.com/apis/docs/system-parameters
[1]: https://grpc.io/docs/what-is-grpc/core-concepts/#metadata
[AIP-126]: ./0126.md
>>>> 0158.md
---
id: 158
state: approved
created: 2019-02-18
placement:
category: design-patterns
order: 60
---
# Pagination
APIs often need to provide collections of data, most commonly in the [List][]
standard method. However, collections can often be arbitrarily sized, and also
often grow over time, increasing lookup time as well as the size of the
responses being sent over the wire. Therefore, it is important that collections
be paginated.
## Guidance
RPCs returning collections of data **must** provide pagination _at the outset_,
as it is a [backwards-incompatible change](#backwards-compatibility) to add
pagination to an existing method.
```proto
// The request structure for listing books.
message ListBooksRequest {
// The parent, which owns this collection of books.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The maximum number of books to return. The service may return fewer than
// this value.
// If unspecified, at most 50 books will be returned.
// The maximum value is 1000; values above 1000 will be coerced to 1000.
int32 page_size = 2;
// A page token, received from a previous `ListBooks` call.
// Provide this to retrieve the subsequent page.
//
// When paginating, all other parameters provided to `ListBooks` must match
// the call that provided the page token.
string page_token = 3;
}
// The response structure from listing books.
message ListBooksResponse {
// The books from the specified publisher.
repeated Book books = 1;
// A token that can be sent as `page_token` to retrieve the next page.
// If this field is omitted, there are no subsequent pages.
string next_page_token = 2;
}
```
- Request messages for collections **should** define an `int32 page_size`
field, allowing users to specify the maximum number of results to return.
- The `page_size` field **must not** be required.
- If the user does not specify `page_size` (or specifies `0`), the API
chooses an appropriate default, which the API **should** document. The API
**must not** return an error.
- If the user specifies `page_size` greater than the maximum permitted by the
API, the API **should** coerce down to the maximum permitted page size.
- If the user specifies a negative value for `page_size`, the API **must**
send an `INVALID_ARGUMENT` error.
- The API **may** return fewer results than the number requested (including
zero results), even if not at the end of the collection.
- Request messages for collections **should** define a `string page_token`
field, allowing users to advance to the next page in the collection.
- The `page_token` field **must not** be required.
- If the user changes the `page_size` in a request for subsequent pages, the
service **must** honor the new page size.
- The user is expected to keep all other arguments to the RPC the same; if
any arguments are different, the API **should** send an `INVALID_ARGUMENT`
error.
- The response **must not** be a streaming response.
- Response messages for collections **should** define a
`string next_page_token` field, providing the user with a page token that may
be used to retrieve the next page.
- The field containing pagination results **should** be the first field in
the message and have a field number of `1`. It **should** be a repeated
field containing a list of resources constituting a single page of results.
- If the end of the collection has been reached, the `next_page_token` field
**must** be empty. This is the _only_ way to communicate
"end-of-collection" to users.
- If the end of the collection has not been reached (or if the API can not
determine in time), the API **must** provide a `next_page_token`.
- Response messages for collections **may** provide an `int32 total_size`
field, providing the user with the total number of items in the list.
- This total **may** be an estimate (but the API **should** explicitly
document that).
### Skipping results
The request definition for a paginated operation **may** define an `int32 skip`
field to allow the user to skip results.
The `skip` value **must** refer to the number of individual resources to skip,
not the number of pages.
For example:
- A request with no page token and a `skip` value of `30` returns a single page
of results starting with the 31st result.
- A request with a page token corresponding to the 51st result (because the
first 50 results were returned on the first page) and a `skip` value of `30`
returns a single page of results starting with the 81st result.
If a `skip` value is provided that causes the cursor to move past the end of
the collection of results, the response **must** be `200 OK` with an empty
result set, and not provide a `next_page_token`.
### Opacity
Page tokens provided by APIs **must** be opaque (but URL-safe) strings, and
**must not** be user-parseable. This is because if users are able to
deconstruct these, _they will do so_. This effectively makes the implementation
details of your API's pagination become part of the API surface, and it becomes
impossible to update those details without breaking users.
**Warning:** Base-64 encoding an otherwise-transparent page token is **not** a
sufficient obfuscation mechanism.
For page tokens which do not need to be stored in a database, and which do not
contain sensitive data, an API **may** obfuscate the page token by defining an
internal protocol buffer message with any data needed, and send the serialized
proto, base-64 encoded.
Page tokens **must** be limited to providing an indication of where to continue
the pagination process only. They **must not** provide any form of
authorization to the underlying resources, and authorization **must** be
performed on the request as with any other regardless of the presence of a page
token.
### Expiring page tokens
Many APIs store page tokens in a database internally. In this situation, APIs
**may** expire page tokens a reasonable time after they have been sent, in
order not to needlessly store large amounts of data that is unlikely to be
used. It is not necessary to document this behavior.
**Note:** While a reasonable time may vary between APIs, a good rule of thumb
is three days.
## Backwards compatibility
Adding pagination to an existing RPC is a backwards-incompatible change. This
may seem strange; adding fields to proto messages is generally backwards
compatible. However, this change is _behaviorally_ incompatible.
Consider a user whose collection has 75 resources, and who has already written
and deployed code. If the API later adds pagination fields, and sets the
default to 50, then that user's code breaks; it was getting all resources, and
now is only getting the first 50 (and does not know to advance pagination).
Even if the API set a higher default limit, such as 100, the user's collection
could grow, and _then_ the code would break.
Additionally, [client libraries implement automatic
pagination](https://google.aip.dev/client-libraries/4233), typically representing paginated
RPCs using different method signatures to unpaginated ones. This means that
adding pagination to a previously-unpaginated method causes a breaking change
in those libraries.
For this reason, it is important to always add pagination to RPCs returning
collections _up front_; they are consistently important, and they can not be
added later without causing problems for existing users.
**Warning:** This also entails that, in addition to presenting the pagination
fields, they **must** be _actually implemented_ with a non-infinite default
value. Implementing an in-memory version (which might fetch everything then
paginate) is reasonable for initially-small collections.
[list]: ./0132.md
## Changelog
- **2020-05-24**: Clarified that adding pagination breaks client
libraries.
- **2020-05-13**: Added guidance for skipping results.
- **2020-08-24**: Clarified that responses are not streaming responses.
- **2020-06-24**: Clarified that page size is always optional for users.
- **2019-02-12**: Added guidance on the field being paginated over.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
- **2019-07-19**: Update the opacity requirement from "should" to "must".
>>>> 0159.md
---
id: 159
state: approved
created: 2019-07-26
placement:
category: design-patterns
order: 70
---
# Reading across collections
Sometimes, it is useful for a user to be able to retrieve resources across
multiple collections, or retrieve a single resource without needing to know
what collection it is in.
## Guidance
APIs **may** support reading resources across multiple collections by allowing
users to specify a `-` (the hyphen or dash character) as a wildcard character
in a standard [`List`][aip-132] method:
```
GET /v1/publishers/-/books?filter=...
```
- The URI pattern **must** still be specified with `*` and permit the
collection to be specified; a URI pattern **must not** hard-code the `-`
character.
- The method **must** explicitly document that this behavior is supported.
- The resources provided in the response **must** use the canonical name of the
resource, with the actual parent collection identifiers (instead of `-`).
- Services **may** support reading across collections on `List` requests
regardless of whether the identifiers of the child resources are guaranteed
to be unique. However, services **must not** support reading across
collections on `Get` requests if the child resources might have a collision.
- Cross-parent requests **should not** support `order_by`. If they do, the
field **must** document that it is best effort. This is because cross-parent
requests introduce ambiguity around ordering, especially if there is
difficulty reaching a parent (see [AIP-217][]).
**Important:** If listing across multiple collections introduces the
possibility of partial failures due to unreachable parents (such as when
listing across locations), the method **must** indicate this following the
guidance in [AIP-217][].
### Unique resource lookup
Sometimes, a resource within a sub-collection has an identifier that is unique
across parent collections. In this case, it may be useful to allow a
[`Get`][aip-131] method to retrieve that resource without knowing which parent
collection contains it. In such cases, APIs **may** allow users to specify the
wildcard collection ID `-` (the hyphen or dash character) to represent any
parent collection:
```
GET https://example.googleapis.com/v1/publishers/-/books/{book}
```
- The URI pattern **must** still be specified with `*` and permit the
collection to be specified; a URI pattern **must not** hard-code the `-`
character.
- The method **must** explicitly document that this behavior is supported.
- The resource name in the response **must** use the canonical name of the
resource, with actual parent collection identifiers (instead of `-`). For
example, the request above returns a resource with a name like
`publishers/123/books/456`, _not_ `publishers/-/books/456`.
- The resource ID **must** be unique within parent collections.
## Further reading
- For partial failures due to unreachable resources, see [AIP-217][].
[aip-131]: ./0131.md
[aip-132]: ./0132.md
[aip-217]: ./0217.md
## Changelog
- **2019-08-26**: Added a reference to guidance for unreachable resources.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0160.md
---
id: 160
state: approved
created: 2020-02-24
placement:
category: design-patterns
order: 80
---
# Filtering
Often, when listing resources (using a list method as defined in [AIP-132][] or
something reasonably similar), it is desirable to filter over the collection
and only return results that the user is interested in.
It is tempting to define a structure to handle the precise filtering needs for
each API. However, filtering requirements evolve frequently, and therefore it
is prudent to use a string field with a structured syntax accessible to a
non-technical audience. This allows updates to be able to be made
transparently, without waiting for UI or client updates.
**Note:** Because list filters are intended for a potentially non-technical
audience, they sometimes borrow from patterns of colloquial speech rather than
common patterns found in code.
## Guidance
APIs **may** provide filtering to users on `List` methods (or similar methods
to query a collection, such as `Search`). If they choose to do so, they
**should** follow the common specification for filters discussed here. The
syntax is formally defined in the [EBNF grammar][].
When employing filtering, a request message **should** have exactly one
filtering field, `string filter`. Filtering of related objects is handled
through [traversal](#traversal-operator) or [functions](#functions).
**Note:** List Filters have fuzzy matching characteristics with support for
result ranking and scoring. For developers interested in deterministic
evaluation of list filters, see [CEL][].
### Literals
A bare literal value (examples: "42", "Hugo") is a value to be matched against.
Literals appearing alone (with no specified field) **should** usually be
matched anywhere it may appear in an object's field values.
However, a service **may** choose to only consider certain fields; if so, it
**must** document which fields it considers. A service **may** include new
fields over time, but **should** do so judiciously and consider impact on
existing users.
**Note:** Literals separated by whitespace are considered to have a fuzzy
variant of `AND`. Therefore, `Victor Hugo` is roughly equivalent to
`Victor AND Hugo`.
### Logical Operators
Filtering implementations **should** provide the binary operators:
| Operator | Example | Meaning |
| -------- | ------------- | -------------------------------------- |
| `AND` | `a AND b` | True if `a` and `b` are true. |
| `OR` | `a OR b OR c` | True if any of `a`, `b`, `c` are true. |
**Note:** To match common patterns of speech, the `OR` operator has higher
precedence than `AND`, unlike what is found in most programming languages. The
expression `a AND b OR c` evaluates: `a AND (b OR c)`. API documentation and
examples **should** encourage the use of explicit parentheses to avoid
confusion, but **should not** require explicit parentheses.
### Negation Operators
Filtering implementations **should** provide the unary operators `NOT` and `-`.
These are used interchangeably, and a service that supports negation **must**
support both formats.
| Operator | Example | Meaning |
| -------- | ------- | ------------------------ |
| `NOT` | `NOT a` | True if `a` is not true. |
| `-` | `-a` | True if `a` is not true. |
### Comparison Operators
Filtering implementations **should** provide the binary comparison operators
`=`, `!=`, `<`, `>`, `<=`, and `>=` for string, numeric, timestamp, and
duration fields (but **should not** provide them for booleans or enums).
| Operator | Example | Meaning |
| -------- | ------------ | ----------------------------------------------- |
| `=` | `a = true` | True if `a` is true. |
| `!=` | `a != 42` | True unless `a` equals 42. |
| `<` | `a < 42` | True if `a` is a numeric value below 42. |
| `>` | `a > "foo"` | True if `a` is lexically ordered after "foo". |
| `<=` | `a <= "foo"` | True if `a` is "foo" or lexically before it. |
| `>=` | `a >= 42` | True if `a` is a numeric value of 42 or higher. |
**Note:** Unlike in most programming languages, field names **must** appear on
the left-hand side of a comparison operator; the right-hand side only accepts
literals and logical operators.
Because filters are accepted as query strings, type conversion takes place to
translate the string to the appropriate strongly-typed value:
- Enums expect the enum's string representation (case-sensitive).
- Booleans expect `true` and `false` literal values.
- Numbers expect the standard integer or float representations. For floats,
exponents are supported (e.g. `2.997e9`).
- [Durations][] expect a numeric representation followed by an `s` suffix (for
seconds). Examples: `20s`, `1.2s`.
- [Timestamps][] expect an [RFC-3339][] formatted string (e.g.
`2012-04-21T11:30:00-04:00`). UTC offsets are supported.
**Warning:** The identifiers `true`, `false`, and `null` only carry intrinsic
meaning when used in the context of a typed field reference.
Additionally, when comparing strings for equality, services **should** support
wildcards using the `*` character; for example, `a = "*.foo"` is true if `a`
_ends with_ ".foo".
### Traversal operator
Filtering implementations **should** provide the `.` operator, which indicates
traversal through a message, map, or struct.
| Example | Meaning |
| --------------- | ----------------------------------------------------- |
| `a.b = true` | True if `a` has a boolean `b` field that is true. |
| `a.b > 42` | True if `a` has a numeric `b` field that is above 42. |
| `a.b.c = "foo"` | True if `a.b` has a string `c` field that is "foo". |
Traversal **must** be written using the field names from the resource. If a
service wishes to support "implicit fields" of some kind, they **must** do so
through well-documented [functions](#functions). A service **may** specify a
subset of fields that are supported for traversal.
If a user attempts to traverse to a field that is not defined on the message,
the service **should** return an error with `INVALID_ARGUMENT`. A service
**may** permit traversal to undefined keys on maps and structs, and **should**
document how it behaves in this situation.
When evaluating an expression involving a traversal, if any non-primitive field
in the chain is not set on the entry being evaluated, the entry **should** be
skipped i.e. not match the filter expression. This applies even when the
comparison is a `!=`, which would imply matching on empty values. In the
examples above, if resource field `a` is not set on the resource instance, that
instance is skipped as a non-match.
**Important:** The `.` operator **must not** be used to traverse through a
repeated field or list, except for specific use with the `:` operator.
### Has Operator
Filtering implementations **must** provide the `:` operator, which means "has".
It is usable with collections (repeated fields or maps) as well as messages,
and behaves slightly differently in each case.
Repeated fields query to see if the repeated structure contains a matching
element:
| Example | Meaning |
| ---------- | ----------------------------------------------------------- |
| `r:42` | True if `r` contains 42. |
| `r.foo:42` | True if `r` contains an element `e` such that `e.foo = 42`. |
**Important:** Filters can not query a _specific_ element on a repeated field
for a value. For example, `e.0.foo = 42` and `e[0].foo = 42` are **not** valid
filters.
Maps, structs, messages can query either for the presence of a field in the map
or a specific value:
| Example | Meaning |
| ---------- | ----------------------------------- |
| `m:foo` | True if `m` contains the key "foo". |
| `m.foo:*` | True if `m` contains the key "foo". |
| `m.foo:42` | True if `m.foo` is 42. |
There are two slight distinctions when parsing messages:
- When traversing messages, a field is only considered to be present if it has
a non-default value.
- When traversing messages, field names are snake case, although
implementations **may** choose to support automatic conversion between camel
case and snake case.
### Functions
The filtering language supports a function call syntax in order to support
API-specific extensions. An API **may** define a function using the
`call(arg...)` syntax, and **must** document any specific functions it
supports.
### Limitations
A service **may** specify further structure or limitations for filter queries,
above what is defined here. For example, a service may support the logical
operators but only permit a certain number of them (to avoid "queries of death"
or other performance concerns).
Further structure or limitations **must** be clearly documented, and
**must not** violate requirements set forth in this document.
### Validation
If a non-compliant or schematically invalid `filter` string is specified,
the API **should** error with `INVALID_ARGUMENT`. Wherever validation is relaxed
for `filter`, the API **must** document the difference.
Schematic validation refers, but is not limited to, the following:
- Fields referenced in the `filter` **must** exist on the filtered schema
- Field values provided in the `filter` **must** align to the type of the field
- For example, for a field `int32 age` a `filter` like `"age=hello"` is
invalid
- Field values for bounded data types e.g. `enum` provided in the `filter`
**must** be a valid value in the set
- Field values for standardized types e.g. `Timestamp` **must** conform to the
documented standard (see [Comparison Operators](#comparison-operators) for a
list of such types)
<!-- prettier-ignore-start -->
[aip-132]: ./0132.md
[cel]: https://github.com/google/cel-spec
[cel-cpp]: https://github.com/google/cel-cpp
[cel-go]: https://github.com/google/cel-go
[durations]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[ebnf grammar]: /assets/misc/ebnf-filtering.txt
[rfc-3339]: https://tools.ietf.org/html/rfc3339
[timestamps]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
## Changelog
- **2025-01-07**: Clarify behavior for unset field in traversal operator chain.
- **2024-12-11**: Move non-compliant filter guidance to Validation section.
>>>> 0161.md
---
id: 161
state: approved
created: 2021-03-01
placement:
category: design-patterns
order: 85
---
# Field masks
Often, when updating resources (using an update method as defined in AIP-134 or
something reasonably similar), it is desirable to specify exactly which fields
are being updated, so that the service can ignore the rest, even if the user
sends new values.
It is tempting to define a mask format to handle the precise needs for each
API. However, masking requirements evolve, and therefore it is prudent to use a
structured syntax. This allows updates to be able to be made transparently,
without waiting for UI or client updates.
## Guidance
These masks of field names are called "field masks". Fields representing a field
mask **must** use the `google.protobuf.FieldMask` type. Field masks are most
common on Update requests (AIP-134).
Field masks **must** always be relative to the resource:
**Warning:** Read masks as a single field on the request message, for
example: `google.protobuf.FieldMask read_mask` are **DEPRECATED**.
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(google.api.field_behavior) = REQUIRED];
// The list of fields to update.
// Fields are specified relative to the book
// (e.g. `title`, `rating`; *not* `book.title` or `book.rating`).
google.protobuf.FieldMask update_mask = 2;
}
```
### Read-write consistency
Read and write behavior for field masks **must** be self-consistent if a mask
is present:
- If a user updates a resource with a given mask, and then reads the same
resource with the same mask, the exact same data **must** be returned.
- Exception: [Output only fields](#output-only-fields).
- Similarly, reading a resource with a given mask and then updating the
resource with the returned data and the same mask **must** be a no-op.
**Note:** This implies that any mask that is valid for either read or write
**must** be valid for both.
### Specifying specific fields
Field masks **must** permit the specification of specific fields in a defined
struct, using the `.` character for traversal.
Because field masks are always relative to the resource, direct fields on the
resource require no traversal (examples: `title`, `rating`). Traversal is used
when resources contain messages (example: `author.given_name`).
**Note:** A user **must** be able to specify either a field as a whole, or one
of its subfields: `author` and `author.given_name` are both valid.
### Map fields
Field masks **may** permit the specification of specific fields in a map, if
and only if the map's keys are either strings or integers, using the `.`
character for traversal.
Field masks **should** support string keys that contain characters that are
problematic for the field mask syntax, using the backtick character.
```proto
message Book {
// The name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1;
// Reviews for the back cover. The key is the author of the review,
// and the value is the text of the review.
//
// Valid field masks: reviews, reviews.smith, reviews.`John Smith`
map<string, string> reviews = 2;
}
```
### Wildcards
Field masks **may** permit the use of the `*` character on a repeated field or
map to indicate the specification of particular sub-fields in the collection:
```proto
message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern: "publishers/{publisher}/books/{book}"
};
// The name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// The author or authors of the book.
// Valid field masks: authors, authors.*.given_name, authors.*.family_name
// Invalid field masks: authors.0, authors.0.given_name
repeated Author authors = 2;
}
message Author {
// The author's given name.
string given_name = 1;
// The author's family name.
string family_name = 2;
}
```
**Note:** Field masks **must not** permit accessing a particular element of a
repeated field by index, and **must** return an `INVALID_ARGUMENT` error if
this is attempted.
### Output only fields
If a user includes an output only field in an update mask indirectly (by using
a wildcard or specifying an overall message that includes an output-only
subfield), the service **must** ignore any output only fields provided as
input, even if they are cleared or modified.
If a user directly specifies an output only field in an update mask, the
service **must** ignore the output only fields provided as input, even if
they are cleared or modified, to permit the same field mask to be used for
input and output.
### Invalid field mask entries
When reading data, field masks **may** ignore entries that point to a value
that can not exist (either a field that does not exist, or a map key that the
service considers invalid).
When writing data, field masks **should** return an `INVALID_ARGUMENT` error if
an entry points to a value that can not exist; however, the service **may**
permit deletions.
## Changelog
- **2023-10-18**: Update guidance for presence of output only fields in update
mask.
- **2023-07-17**: Move `update_mask` guidance to AIP-134.
>>>> 0162.md
---
id: 162
state: draft
created: 2019-09-17
updated: 2023-09-01
placement:
category: design-patterns
order: 88
---
# Resource Revisions
Some APIs need to have resources with a revision history, where users can
reason about the state of the resource over time. There are several reasons for
this:
- Users may want to be able to roll back to a previous revision, or diff
against a previous revision.
- An API may create data which is derived in some way from a resource at a
given point in time. In these cases, it may be desirable to snapshot the
resource for reference later.
**Note:** We use the word _revision_ to refer to a historical reference for a
particular resource, and intentionally avoid the term _version_, which refers
to the version of an API as a whole.
## Guidance
APIs **may** store a revision history for a resource. Examples of when it is
useful include:
- When it is valuable to expose older versions of a resource via an API. This
can avoid the overhead of the customers having to write their own API to store
and enable retrieval of revisions.
- Other resources depend on different revisions of a resource.
- There is a need to represent the change of a resource over time.
APIs implementing resources with a revision history **should** abstract resource
revisions as nested collection of the resource. Sometimes, the revisions
collection can be a top level collection, exceptions include:
- If resource revisions are meant to have longer lifespan than the parent
resource. In other words, resource revisions exist after resource deletion.
```proto
message BookRevision {
// The name of the book revision.
string name = 1;
// The snapshot of the book
Book snapshot = 2
[(google.api.field_behavior) = OUTPUT_ONLY];
// The timestamp that the revision was created.
google.protobuf.Timestamp create_time = 3
[(google.api.field_behavior) = OUTPUT_ONLY];
// Other revision IDs that share the same snapshot.
repeated string alternate_ids = 4
[(google.api.field_behavior) = OUTPUT_ONLY];
}
```
- The `message` **must** be annotated as a resource (AIP-123).
- The `message` name **must** be named `{ResourceType}Revision`.
- The resource revision **must** contain a field with a message type of the
parent resource, with a field name of `snapshot`.
- The value of `snapshot` **must** be the configuration of the parent
at the point in time the revision was created.
- The resource revision **must** contain a `create_time` field (see [AIP-142][]).
- The resource revision **may** contain a repeated field `alternate_ids`, which would
contain a list of resource IDs that the revision is also known by (e.g. `latest`)
### Creating Revisions
Depending on the resource, different APIs may have different strategies for
- Create a new revision any time that there is a change to the parent resource
- Create a new revision when important system state changes
- Create a new revision when specifically requested
APIs **may** use any of these strategies. APIs **must**
document their revision creation strategy.
### Resource names for revisions
When referring to specific revision of a resource, the subcollection name
**must** be named `revisions`. Resource revisions have names with the format
`{resource_name}/revisions/{revision_id}`. For example:
```
publishers/123/books/les-miserables/revisions/c7cfa2a8
```
### Server-specified Aliases
Services **may** reserve specific IDs to be [aliases][alias] (e.g.
`latest`). These are read-only and managed by the service.
```
GET /v1/publishers/{publisher}/books/{book}/revisions/{revision_id}
```
- If a `latest` ID exists, it **must** represent the most recently created
revision. The content of `publishers/{publisher}/books/{book}/revisions/latest`
and `publishers/{publisher}/books/{book}` can differ, as the latest revision may
be different from the current state of the resource.
### User-Specified Aliases
APIs **may** provide a mechanism for users to assign an [alias][] ID to an
existing revision with a custom method "alias":
```proto
rpc AliasBookRevision(AliasBookRevisionRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*/revisions/*}:alias"
body: "*"
};
}
```
```proto
message AliasBookRevisionRequest {
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/BookRevision"
}];
// The ID of the revision to alias to, e.g. `CURRENT` or a semantic
// version.
string alias_id = 2 [(google.api.field_behavior) = REQUIRED];
}
```
- The request message **must** have a `name` field:
- The field **must** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
- The request message **must** have a `alias_id` field:
- The field **must** be [annotated as required][aip-203].
- If the user calls the method with an existing `alias_id`, the request **must**
succeed and the alias will be updated to refer to the provided revision. This
allows users to write code against a specific alias (e.g. `published`) and the
revision can change with no code change.
### Rollback
A common use case for a resource with a revision history is the ability to roll
back to a given revision. APIs **should** handle this with a `Rollback` custom
method:
```proto
rpc RollbackBook(RollbackBookRequest) returns (BookRevision) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*/revisions/*}:rollback"
body: "*"
};
}
```
- The method **must** use the `POST` HTTP verb.
- The method **should** return a resource revision.
```proto
message RollbackBookRequest {
// The revision that the book should be rolled back to.
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/BookRevision"
}];
}
```
- The request message **must** have a `name` field, referring to the resource
revision whose configuration the resource should be rolled back to.
- The field **must** be [annotated as required][aip-203].
- The field **must** identify the [resource type][aip-123] that it
references.
### Child resources
Resources with a revision history **may** have child resources. If they do,
there are two potential variants:
- Child resources where each child resource is a child of the parent resource
as a whole.
- Child resources where each child resource is a child of _a single revision
of_ the parent resource.
APIs **should not** include multiple levels of resources with revisions, as
this quickly becomes difficult to reason about.
### Standard methods
Any standard methods **must** implement the corresponding AIPs (AIP-131,
AIP-132, AIP-133, AIP-134, AIP-135), with the following additional behaviors:
- List methods: By default, revisions in the list response **must** be ordered
in reverse chronological order. User can supply `order_by` to override the
default behavior.
- If the revision supports aliasing, a delete method with the resource name
of the alias (e.g. `revisions/1.0.2`) **must** remove the alias instead of
deleting the resource.
As revisions are nested under the resource, also see [cascading delete][].
## Rationale
### Abstract revisions as nested collection
Revisions being resources under nested collection make revisions a first class
citizen.
- Revisions can offer standard get, list, and delete methods.
- It retains the flexibility of extending new fields to revision in addition to
the resource message.
### Tagging to Aliases
Previously, a concept of `tag` existed. This concept was redundant with that of
an [alias][], and the terms were consolidated to reduce complexity in the AIPs.
### Output only resource configuration
Although it was an option to have the revision take in the resource
configuration as part of the create method, doing so would have allowed users to
submit resource configuration for a revision that the resource was never in.
`OUTPUT_ONLY` and requiring that a created revision represents the resource at
current point in time eliminates that issue.
## History
### Switching from a collection extension to a subcollection
In 2023-09, revisions are abstracted as a nested resource collection. Prior to
this, revisions are more like extension of an existing resource by using `@`
symbol. List and delete revisions were custom methods on the resource
collection. A single Get method was used to retrieve either the resource
revision, or the resource.
Its primary advantage was allowing a resource reference to seamlessly refer to
a resource, or its revision.
It also had several disadvantages:
- List revisions is a custom method (:listRevisions) on the resource collection
- Delete revision is a custom method on the resource collection
- Not visible in API discovery doc
- Resource ID cannot use `@`
The guidance was modified ultimately to enable revisions to behave like a
resource, which reduces the users cognitive load and allows resource-oriented
clients to easily list, get, create, and update revisions.
### Using resource ID instead of tag
In the previous design, revisions had a separate identifer for a revision known
as a `tag`, that would live in a revision.
Tags were effectively a shadow resource ID, requiring methods to create, get and
filter revisions based on the value of the tag.
By consolidating the concept of a tag into the revision ID, the user no longer
needs to be familiar with a second set of retrieval and identifier methods.
## Changelog
- **2023-09-01**: AIP was updated to be a sub-collection.
- **2021-04-27**: Added guidance on returning the resource from Delete Revision.
[aip-142]: ./0142.md
[alias]: ./0122.md#resource-id-aliases
[cascading delete]: ./0135.md#cascading-delete
[UUID4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
>>>> 0163.md
---
id: 163
state: approved
created: 2019-12-16
placement:
category: design-patterns
order: 90
---
# Change validation
Occasionally, a user wants to validate an intended change to see what the
result will be before actually making the change. For example, a request to
provision new servers in a fleet will have an impact on the overall fleet size
and cost, and could potentially have unexpected downstream effects.
## Guidance
APIs **may** provide an option to validate, but not actually execute, a
request, and provide the same response (status code, headers, and response
body) that it would have provided if the request was actually executed.
To provide this option, the method **should** include a `bool validate_only`
field in the request message:
```proto
message ReviewBookRequest {
string name = 1 [(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
int32 rating = 2;
string comment = 3;
// If set, validate the request and preview the review, but do not actually
// post it.
bool validate_only = 4;
}
```
The API **must** perform permission checks and any other validation that would
be performed on a "live" request; a request using `validate_only` **must** fail
if it determines that the actual request would fail.
**Note:** It may occasionally be infeasible to provide the full output. For
example, if creating a resource would create an auto-generated ID, it does not
make sense to do this on validation. APIs **should** omit such fields on
validation requests in this situation.
### Declarative-friendly resources
A resource that is declarative-friendly (AIP-128) **must** include a
`validate_only` field on methods that mutate the resource.
## Changelog
- **2020-10-06:** Added declarative-friendly resource requirement.
>>>> 0164.md
---
id: 164
state: approved
created: 2020-10-06
placement:
category: design-patterns
order: 95
---
# Soft delete
There are several reasons why a client could desire soft delete and undelete
functionality, but one over-arching reason stands out: recovery from mistakes.
A service that supports undelete makes it possible for users to recover
resources that were deleted by accident.
## Guidance
APIs **may** support the ability to "undelete", to allow for situations where
users mistakenly delete resources and need the ability to recover.
If a resource needs to support undelete, the `Delete` method **must** simply
mark the resource as having been deleted, but not completely remove it from the
system. If the method behaves this way, it **should** return the updated
resource instead of `google.protobuf.Empty`.
Resources that support soft delete **should** have both a `delete_time` and
`purge_time` field as described in AIP-148. Additionally, resources **should**
include a `DELETED` state value if the resource includes a `state` field
(AIP-216).
### Undelete
A resource that supports soft delete **should** provide an `Undelete` method:
```proto
rpc UndeleteBook(UndeleteBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:undelete"
body: "*"
};
}
```
- The HTTP verb **must** be `POST`.
- The `body` clause **must** be `"*"`.
- The response message **must** be the resource itself. There is no
`UndeleteBookResponse`.
- The response **should** include the fully-populated resource unless it is
infeasible to do so.
- If the undelete RPC is [long-running](#long-running-undelete), the response
message **must** be a `google.longrunning.Operation` which resolves to the
resource itself.
### Undelete request message
Undelete methods implement a common request message pattern:
```proto
message UndeleteBookRequest {
// The name of the deleted book.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference).type = "library.googleapis.com/Book"];
}
```
- A `name` field **must** be included. It **should** be called `name`.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
### Long-running undelete
Some resources take longer to undelete a resource than is reasonable for a
regular API request. In this situation, the API **should** use a long-running
operation (AIP-151) instead:
```proto
rpc UndeleteBook(UndeleteBookRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:undelete"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "Book"
metadata_type: "OperationMetadata"
};
}
```
- The response type **must** be set to the resource (what the return type would
be if the RPC was not long-running).
- Both the `response_type` and `metadata_type` fields **must** be specified.
### List and Get
Soft-deleted resources **should not** be returned in `List` (AIP-132) responses
by default (unless `bool show_deleted` is true). `Get` (AIP-131) requests for
soft-deleted resources **should** return the resource (rather than a
`NOT_FOUND` error).
APIs that soft delete resources **may** choose a reasonable strategy for
purging those resources, including automatic purging after a reasonable time
(such as 30 days), allowing users to set an expiry time (AIP-214), or retaining
the resources indefinitely. Regardless of what strategy is selected, the API
**should** document when soft deleted resources will be completely removed.
### Declarative-friendly resources
Soft-deletable resources have a poorer experience than hard-deleted resources in
declarative clients: since an ID on a soft-deleted resource is not re-usable
unless a custom method (undelete) is called, an imperative client must be
introduced or hand-written code is required to incorporate the usage of the
custom method.
### Errors
If the user does not have permission to access the resource, regardless of
whether or not it exists, the service **must** error with `PERMISSION_DENIED`
(HTTP 403). Permission **must** be checked prior to checking if the resource
exists.
If the user does have proper permission, but the requested resource does not
exist (either it was never created or already expunged), the service **must**
error with `NOT_FOUND` (HTTP 404).
If the user calling a soft `Delete` has proper permission, but the requested
resource is already deleted, the service **must** succeed if `allow_missing` is
`true`, and **should** error with `NOT_FOUND` (HTTP 404) if `allow_missing` is
`false`.
If the user calling `Undelete` has proper permission, but the requested
resource is not deleted, the service **must** respond with `ALREADY_EXISTS`
(HTTP 409).
## Further reading
- For the `Delete` standard method, see AIP-135.
- For long-running operations, see AIP-151.
- For resource freshness validation (`etag`), see AIP-154.
- For change validation (`validate_only`), see AIP-163.
## Changelog
- **2024-09-24**: Included missing requirement for `delete_time`.
- **2023-07-13**: Renamed overloaded `expire_time` to `purge_time`.
- **2021-07-12**: Added error behavior when soft deleting a deleted resource.
- **2021-07-12**: Clarified that `ALREADY_EXISTS` errors apply to `Undelete`.
- **2021-07-12**: Changed the `expire_time` field to "should" for consistency
with AIP-148.
- **2020-09-23**: Soft delete material in AIP-135 migrated to this AIP.
[aip-123]: ./0123.md
[aip-203]: ./0203.md
>>>> 0165.md
---
id: 165
state: approved
created: 2019-12-18
updated: 2022-06-02
placement:
category: design-patterns
order: 100
---
# Criteria-based delete
Occasionally, an API may need to provide a mechanism to delete a large number
of resources based on some set of filter parameters, rather than requiring the
individual resource name of the resources to be deleted.
This is a rare case, reserved for situations where users need to delete
thousands or more resources at once, in which case the normal Batch Delete
pattern ([AIP-235][]) becomes unwieldy and inconvenient.
## Guidance
**Important:** Most APIs **should** use only Delete ([AIP-135][]) or Batch
Delete ([AIP-235][]) for deleting resources, and **should not** implement
deleting based on criteria. This is because deleting is generally irreversible
and this type of operation makes it easy for a user to accidentally lose
significant amounts of data.
An API **may** implement a Purge method to permit deleting a large number of
resources based on a filter string; however, this **should** only be done if
the Batch Delete ([AIP-235][]) pattern is insufficient to accomplish the
desired goal:
```proto
rpc PurgeBooks(PurgeBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:purge"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PurgeBooksResponse"
metadata_type: "PurgeBooksMetadata"
};
}
```
- The RPC's name **must** begin with the word `Purge`. The remainder of the
RPC name **should** be the plural form of the resource being purged.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response type **must** be a `google.longrunning.Operation` (see
AIP-151) that resolves to a message whose name matches the RPC name, with
a `Response` suffix.
- The HTTP verb **must** be `POST`, and the `body` **must** be `"*"`.
- The URI path **should** represent the collection for the resource.
- The `parent` field **should** be included in the URI. If the API wishes to
support deletion across multiple parents, it **should** accept the `-`
character consistent with [AIP-159][].
### Request message
Purge methods implement a common request message pattern:
```proto
message PurgeBooksRequest {
// The publisher to purge books from.
// To purge books across publishers, send "publishers/-".
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// A filter matching the books to be purged.
string filter = 2 [(google.api.field_behavior) = REQUIRED];
// Actually perform the purge.
// If `force` is set to false, the method will return a sample of
// resource names that would be deleted.
bool force = 3;
}
```
- A singular `string parent` field **should** be included, unless the resource
is top-level.
- The field **should** be [annotated as required][aip-203].
- The field **should** identify the [resource type][aip-123] that it
references.
- A singular `string filter` field **must** be included and **must** follow the
same semantics as in List methods (AIP-160).
- It **should** be [annotated as required][aip-203].
- A singular `bool force` field **must** be included. If it is not set, the API
**must** return a count of the resources that would be deleted as well as a
sample of those resources, without actually performing the deletion.
### Response message
Purge methods implement a common response message pattern:
```proto
message PurgeBooksResponse {
// The number of books that this request deleted (or, if `force` is false,
// the number of books that will be deleted).
int32 purge_count = 1;
// A sample of the resource names of books that will be deleted.
// Only populated if `force` is set to false.
repeated string purge_sample = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A singular `int32 purge_count` field **should** be included, and provide the
number of resources that were deleted (or would be deleted). This count
**may** be an estimate similar to `total_size` in AIP-158 (but the service
**should** document this if so).
- A `repeated string purge_sample` field **should** be included: If `force` is
`false`, it **should** provide a sample of resource names that will be
deleted. If `force` is true, this field **should not** be populated.
- The sample **should** be a sufficient size to catch clearly obvious
mistakes: A good rule of thumb is 100. The API **should** document the
size, and **should** document that it is a maximum (it is possible to send
fewer).
- The sample **may** be random or **may** be deterministic (such as the first
matched resource names). The API **should** document which approach is
used.
- The field **should** identify the [resource type][aip-123] that it
references.
**Note:** Even if `purge_count` and `purge_sample` are not included, the
`force` field **must** still be included in the request.
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-29**: Expanded guidance on HTTP, field behavior, and resource
reference annotations.
[aip-123]: ./0123.md
[aip-135]: ./0135.md
[aip-159]: ./0159.md
[aip-203]: ./0203.md
[aip-235]: ./0235.md
>>>> 0180.md
---
id: 180
state: approved
created: 2019-07-23
placement:
category: compatibility
order: 10
---
# Backwards compatibility
APIs are fundamentally contracts with users, and users often write code against
APIs that is then launched into a production service with the expectation that
it continues to work (unless the API has a [stability level][aip-181] that
indicates otherwise). Therefore, it is important to understand what constitutes
a backwards compatible change and what constitutes a backwards incompatible
change.
## Guidance
Existing client code **must not** be broken by a service updating to a new
minor or patch release. Old clients **must** be able to work against newer
servers (with the same major version number).
**Important:** It is not always clear whether a change is compatible or not.
The guidance here **should** be treated as indicative, rather than as a
comprehensive list of every possible change.
There are three distinct types of compatibility to consider:
1. Source compatibility: Code written against a previous version **must**
compile against a newer version, and successfully run with a newer version
of the client library.
2. Wire compatibility: Code written against a previous version **must** be able
to communicate correctly with a newer server. In other words, not only are
inputs and outputs compatible, but the serialization and deserialization
expectations continue to match.
3. Semantic compatibility: Code written against a previous version **must**
continue to receive what most reasonable developers would expect. (This can
be tricky in practice, however, and sometimes determining what users will
expect can involve a judgment call.)
**Note:** In general, the specific guidance here assumes use of protocol
buffers and JSON as transport formats. Other transport formats may have
slightly different rules.
**Note:** This guidance assumes that APIs are intended to be called from a
range of consumers, written in multiple languages and with no control over
how and when consumers update. Any API which has a more limited scope (for
example, an API which is only called by client code written by the same team
as the API producer, or deployed in a way which can enforce updates) should
carefully consider its own compatibility requirements.
### Adding components
In general, new components (interfaces, methods, messages, fields, enums, or
enum values) **may** be added to existing APIs in the same major version.
However, keep the following guidelines in mind when doing this:
- Code written against the previous surface (and thus is unaware of the new
components) **must** continue to be treated the same way as before.
- New required fields **must not** be added to existing request messages or
resources.
- Any field being populated by clients **must** have a default behavior
matching the behavior before the field was introduced.
- Any field previously populated by the server **must** continue to be
populated, even if it introduces redundancy.
- For enum values specifically, be aware that it is possible that user code
does not handle new values gracefully.
- Enum values **may** be freely added to enums which are only used in request
messages.
- Enums that are used in response messages or resources and which are
expected to receive new values **should** document this. Enum values still
**may** be added in this situation; however, appropriate caution **should**
be used.
**Note:** It is possible when adding a component closely related to an existing
component (for example, `string foo_value` when `string foo` already exists) to
enter a situation where generated code will conflict. Service owners **should**
be aware of subtleties in the tooling they or their users are likely to use
(and tool authors **should** endeavor to avoid such subtleties if possible).
### Removing or renaming components
Existing components (interfaces, methods, messages, fields, enums, or enum
values) **must not** be removed from existing APIs in the same major version.
Removing a component is a backwards incompatible change.
**Important:** Renaming a component is semantically equivalent to "remove and
add". In cases where these sorts of changes are desirable, a service **may**
add the new component, but **must not** remove the existing one. In situations
where this can allow users to specify conflicting values for the same semantic
idea, the behavior **must** be clearly specified.
### Moving components between files
Existing components **must not** be moved between files.
Moving a component from one proto file to another within the same package is
wire compatible, however, the code generated for languages like C++ or Python
will result in breaking change since `import` and `#include` will no longer
point to the correct code location.
### Moving into oneofs
Existing fields **must not** be moved into or out of a oneof. This is a
backwards-incompatible change in the Go protobuf stubs.
### Changing the type of fields
Existing fields and messages **must not** have their type changed, even if the
new type is wire-compatible, because type changes alter generated code in a
breaking way.
### Changing resource names
A resource **must not** change its [name][aip-122].
Unlike most breaking changes, this affects major versions as well: in order for
a client to expect to use v2.0 access a resource that was created in v1.0 or
vice versa, the same resource name **must** be used in both versions.
More subtly, the set of valid resource names **should not** change either, for
the following reasons:
- If resource name formats become more restrictive, a request that would
previously have succeeded will now fail.
- If resource name formats become less restrictive than previously documented,
then code making assumptions based on the previous documentation could break.
Users are very likely to store resource names elsewhere, in ways that may be
sensitive to the set of permitted characters and the length of the name.
Alternatively, users might perform their own resource name validation to
follow the documentation.
- For example, Amazon gave customers [a lot of warning][ec2] and had a
migration period when they started allowing longer EC2 resource IDs.
### Semantic changes
Code will often depend on API behavior and semantics, _even when such behavior
is not explicitly supported or documented_. Therefore, APIs **must not** change
visible behavior or semantics in ways that are likely to break reasonable user
code, as such changes will be seen as breaking by those users.
**Note:** This does involve some level of judgment; it is not always clear
whether a proposed change is likely to break users, and an expansive reading of
this guidance could ostensibly prevent _any_ change (which is not the intent).
#### Default values must not change
Default values are the values set by servers for resources when they are not
specified by the client. This section only applies to static default values within
fields on resources and does not apply to dynamic defaults such as the default IP
address of a resource.
Changing the default value is considered breaking and **must not** be done. The
default behavior for a resource is determined by its default values, and this
**must not** change across minor versions.
For example:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of FICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
Changing to:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of NONFICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
would constitute a breaking change.
#### Serializing defaults
APIs **must not** change the way a field with a default value is serialized. For
example if a field does not appear in the response if the value is equal to the
default, the serialization **must not** change to include the field with the
default. Clients may depend on the presence or absence of a field in a resource
as semantically meaningful, so a change to how serialization is done for absent
values **must not** occur in a minor version.
Consider the following proto, where the default value of `wheels` is `2`:
```proto
// A representation of an automobile
message Automobile {
// google.api.resource and other annotations and fields
// The number of wheels on the automobile.
// The default value is 2, when no value is sent by the client.
int wheels = 2;
}
```
First the proto serializes to JSON when the value of `wheels` is `2` as follows:
```json
{
"name": "my-car"
}
```
Then, the API service changes the serialization to include `wheel` even if the
value is equal to the default value, `2` as follows:
```json
{
"name": "my-car",
"wheels": 2
}
```
This constitutes a change that is not backwards compatible within a major
version.
## Further reading
- For compatibility around field behavior, see [AIP-203][].
- For compatibility around pagination, see [AIP-158][].
- For compatibility around long-running operations, see [AIP-151][].
- For understanding stability levels and expectations, see [AIP-181][].
- For compatibility with client library resource name parsing, see [AIP-4231][]
- For compatibility with client library method signatures, see [AIP-4232][]
- For compatibility around field presence changes, see [AIP-149][].
- For compatibility around resource types, see [AIP-123][].
## Changelog
- **2024-08-07**: Added reference to resource type compatibility.
- **2024-06-05**: Added reference to field presence compatibility.
- **2023-07-26**: Added reference to field behavior compatibility.
- **2023-07-26**: Added note on APIs which have limited clients.
- **2022-08-11**: Added "Moving components between files" section.
- **2022-06-01**: Added more links to other AIPs with compatibility concerns
- **2019-12-16**: Clarified that moving existing fields into oneofs is
breaking.
<!-- prettier-ignore-start -->
[aip-122]: ./0122.md
[aip-123]: ./0123.md
[aip-149]: ./0149.md
[aip-151]: ./0151.md
[aip-158]: ./0158.md
[aip-181]: ./0181.md
[aip-203]: ./0203.md
[aip-4231]: https://google.aip.dev/client-libraries/4231
[aip-4232]: https://google.aip.dev/client-libraries/4232
[ec2]: https://aws.amazon.com/blogs/aws/theyre-here-longer-ec2-resource-ids-now-available/
<!-- prettier-ignore-end -->
>>>> 0181.md
---
id: 181
state: approved
created: 2019-02-18
placement:
category: compatibility
order: 20
---
# Stability levels
While different organizations (both inside Google and outside) have different
product life cycles, AIPs refer to the _stability_ of an API component using
the following terms.
**Note:** These stability levels roughly correspond to the product launch
stages (alpha, beta, GA) in Google Cloud, but are not identical. GCP imposes
its own additional expectations and commitments on top of what is outlined
here.
## Alpha
An _alpha_ component undergoes rapid iteration with a known set of users who
**must** be tolerant of change. The number of users **should** be a
curated, manageable set, such that it is feasible to communicate with all
of them individually.
Breaking changes **must** be both allowed and expected in alpha components, and
users **must** have no expectation of stability.
## Beta
A _beta_ component **must** be considered complete and ready to be declared
stable, subject to public testing. Beta components **should** be exposed to an
unknown and potentially large set of users. In other words, beta components
**should not** be behind an allowlist; instead, they **should** be available to
the public.
Because users of beta components tend to have a lower tolerance of change, beta
components **should** be as stable as possible; however, the beta component
**must** be permitted to change over time. These changes **should** be minimal
but **may** include backwards-incompatible changes to beta components.
Backwards-incompatible changes **must** be made only after a reasonable
deprecation period to provide users with an opportunity to migrate their code.
This deprecation period **must** be defined at the time of being marked beta.
Beta components **should** be time-boxed and promoted to stable if no issues
are found in the specified timeframe, which **should** be specified at the time
of being marked beta. A reasonable time period **may** vary, but a good rule of
thumb is 90 days.
## Stable
A _stable_ component **must** be fully-supported over the lifetime of the major
API version. Because users expect such stability from components marked stable,
there **must** be no breaking changes to these components, subject to the
caveats described below.
### Major versions
When breaking changes become necessary, the API producer **should** create the
next major version of the API, and start a deprecation clock on the existing
version.
Turn-down of any version containing stable components **must** have a formal
process defined at the time of being marked stable. This process **must**
specify a deprecation period for users which provides them with reasonable
advance warning.
### Isolated changes
On very rare occasions, it could be preferable to make a small, isolated
breaking change, if this will only cause inconvenience to a small subset of
users. (Creating a new major version is an inconvenience to all users.) In this
case, the API producer **may** deprecate the component, but **must** continue
to support the component for the normal turndown period for a stable component.
**Important:** Making an in-place breaking change in a stable API is considered
an extreme course of action, and should be treated with equal or greater
gravity as creating a new major version. For example, at Google, this requires
the approval of the API Governance team.
### Emergency changes
In certain exceptional cases, such as security concerns or regulatory
requirements, any API component **may** be changed in a breaking manner
regardless of its stability level, and a deprecation is not promised in these
situations.
>>>> 0182.md
---
id: 182
state: reviewing
created: 2021-08-13
placement:
category: compatibility
order: 30
---
# External software dependencies
Some services have a particular type of dependency on external software: they
allow users to create resources that run on or expose the external software in
some way. For example:
- A database admin service can allow users to create databases running on a
particular version of a particular database engine (for example, PostgreSQL
13.4).
- A virtual machine service can allow users to create VMs running a particular
operating system (for example, Ubuntu 20.04).
- An application or function platform service can allow users to write code
that runs against a particular version of a programming language (for
example, Node.js 16.6).
Services that provide external software to users in this way will eventually
need to address the fact that all of these types of software have release
lifecycles, and the versions they currently expose will eventually reach
end-of-life.
## Guidance
Services that expose external software dependencies **should** allow users to
create resources using any currently-supported LTS (long-term support) version
of the supported software, and **may** allow users to create resources using
non-LTS versions.
Services **should not** indefinitely allow users to create new resources using
versions that have reached end-of-life, although they **may** have a transition
period between when the software version reaches end-of-life and when support
for creating new resources with that version is removed.
**Note:** Restricting or removing the ability to create resources using
end-of-life versions of software is **not** considered a breaking change for
the service for the purpose of AIP-181, even though it actually is one.
However, because the change can break existing users' workflows, services
**must** notify users who are using resources approaching end-of-life.
If possible, services **should** allow previously-created resources to remain,
and **may** warn users of the risks associated with continuing to use
end-of-life software. Services **should not** proactively remove resources
using end-of-life software, or impose other restrictions on existing resources,
unless critical security concerns require the service to do so.
### Continued support
If supporting a version that has reached end-of-life is necessary for business
reasons (usually because the end-of-life software still has significant
adoption), the service **may** choose to officially support the end-of-life
version, but **must** take on the responsibility of patching and maintaining
the software if it does so.
>>>> 0185.md
---
id: 185
state: approved
created: 2024-10-22
placement:
category: compatibility
order: 40
---
# API Versioning
This topic describes the versioning strategies used by Google APIs. In
general, these strategies apply to all Google-managed services.
## Guidance
All Google API interfaces **must** provide a _major version number_, which is
encoded at the end of the protobuf package, and included as the first part of
the URI path for REST APIs. In the event an API needs to make an incompatible
change, consult [AIP-180][] and [AIP-181][] for necessary steps based on the
stability level of the surface in question.
Note: The use of the term "major version number" above is taken from
[semantic versioning][]. However, unlike in traditional semantic versioning,
Google APIs **must not** expose minor or patch version numbers. For
example, Google APIs use `v1`, not `v1.0`, `v1.1`, or `v1.4.2`. From a user's
perspective, major versions are updated in place, and users receive new
functionality without migration.
A new major version of an API **must not** depend on a previous major version of
the same API. An API surface **must not** depend on other APIs, except for in
the cases outlined in [AIP-213][] and [AIP-215][].
Different versions of the same API **must** be able to work at the same time
within a single client application for a reasonable transition period. This time
period allows the client to transition smoothly to the newer version. An older
version **must** go through a reasonable, well-communicated deprecation period
before being shut down.
For releases which have [alpha or beta stability][AIP-181], APIs **must** append
the stability level after the major version number in the protobuf package and
URI path using one of these strategies:
- Channel-based versioning (recommended)
- Release-based versioning
- Visibility-based versioning
### Channel-based versioning
A *stability channel* is a long-lived release at a given stability level that
receives in-place updates. There is no more than one channel per stability level
for a major version. Under this strategy, there are up to three channels
available: alpha, beta, and stable.
The alpha and beta channel **must** have their stability level appended to the
version, but the stable channel **must not** have the stability level appended.
For example, `v1` is an acceptable version for the stable channel, but `v1beta`
or `v1alpha` are not. Similarly, `v1beta` or `v1alpha` are acceptable versions
for the respective beta and alpha channel, but `v1` is not acceptable for
either. Each of these channels receives new features and updates "in-place".
The beta channel's functionality **must** be a superset of the stable channel's
functionality, and the alpha channel's functionality **must** be a superset of
the beta channel's functionality.
#### Deprecating API functionality
API elements (fields, messages, RPCs) **may** be marked deprecated in any
channel to indicate that they should no longer be used:
```proto
// Represents a scroll. Books are preferred over scrolls.
message Scroll {
option deprecated = true;
// ...
}
```
Deprecated API functionality **must not** graduate from alpha to beta, nor beta
to stable. In other words, functionality **must not** arrive "pre-deprecated"
in any channel.
The beta channel's functionality **may** be removed after it has been deprecated
for a sufficient period; we recommend 180 days. For functionality that exists
only in the alpha channel, deprecation is optional, and functionality **may** be
removed without notice. If functionality is deprecated in an API's
alpha channel before removal, the API **should** apply the same annotation, and
**may** use any timeframe it wishes.
### Release-based versioning
**Important:** This pattern is not commonly used for new services. There are
existing services that follow it, but Channel-based Versioning is the preferred
mechanism.
An *individual release* is an alpha or beta release that is expected to be
available for a limited time period before its functionality is incorporated
into the stable channel, after which the individual release will be shut down.
When using release-based versioning strategy, an API may have any number of
individual releases at each stability level.
Note: Both the channel-based and release-based strategies update the _stable_
version in-place. There is a single stable channel, rather than individual
stable releases, even when using the release-based strategy.
Alpha and beta releases **must** have their stability level appended to the
version, followed by an incrementing release number. For example, `v1beta1` or
`v1alpha5`. APIs **should** document the chronological order of these versions
in their documentation (such as comments).
Each alpha or beta release **may** be updated in place with backwards-compatible
changes. For beta releases, backwards-incompatible updates **should** be made by
incrementing the release number and publishing a new release with the change.
For example, if the current version is `v1beta1`, then `v1beta2` is released
next.
Alpha and beta releases **should** be shut down after their functionality
reaches the stable channel. An alpha release **may** be shut down at any time,
while a beta release **should** allow users a reasonable transition period; we
recommend 180 days.
### Visibility-based versioning
[API visibility][] is an advanced feature provided by Google API infrastructure.
It allows API producers to expose multiple external API views from one internal
API surface, and each view is associated with an API _visibility label_, such
as:
```proto
import "google/api/visibility.proto";
message Resource {
string name = 1;
// Preview. Do not use this feature for production.
string display_name = 2
[(google.api.field_visibility).restriction = "PREVIEW"];
}
```
A visibility label is a case-sensitive string that can be used to tag any API
element. By convention, visibility labels should always use UPPER case.
An implicit `PUBLIC` label is applied to all API elements unless an explicit
visibility label is applied as in the example above.
Each visibility label is an allow-list. API producers need to grant visibility
labels to API consumers for them to use API features associated with the labels.
In other words, an API visibility label is like an ACL'ed API version.
Multiple visibility labels **may** be applied to an element by using a
comma-separated string (e.g. `"PREVIEW,TRUSTED_TESTER"`). When multiple
visibility labels are used, then the client needs only _one_ of the visibility
labels (logical `OR`).
By default, the visibility labels granted to the API consumer are used to verify
incoming requests. However, a client can send requests with an explicit
visibility label as follows:
```
GET /v1/projects/my-project/topics HTTP/1.1
Host: pubsub.googleapis.com
Authorization: Bearer y29....
X-Goog-Visibilities: PREVIEW
```
A single API request can specify at most one visibility label.
API producers can use API visibility for API versioning, such as
`INTERNAL` and `PREVIEW`. A new API feature starts with the `INTERNAL` label,
then moves to the `PREVIEW` label. When the feature is stable and becomes
generally available, all API visibility labels are removed from the API
definition.
In general, API visibility is easier to implement than API versioning for
incremental changes, but it depends on sophisticated API infrastructure support.
Google Cloud APIs often use API visibility for Preview features.
[AIP-180]: https://aip.dev/180
[AIP-181]: https://aip.dev/181
[AIP-213]: https://aip.dev/213
[AIP-215]: https://aip.dev/215
[api visibility]: https://github.com/googleapis/googleapis/blob/master/google/api/visibility.proto
[semantic versioning]: https://semver.org/
>>>> 0191.md
---
id: 191
state: approved
created: 2019-07-25
placement:
category: polish
order: 10
---
# File and directory structure
A consistent file and directory structure, while making minimal difference
technically, makes API surface definitions easier for users and reviewers to
read.
## Guidance
**Note:** The following guidance applies to APIs defined in protocol buffers,
such as those used throughout Google. While the spirit of this guidance applies
to APIs defined using other specification languages or formats, some of the
particular recommendations might be irrelevant.
### Syntax
APIs defined in protocol buffers **must** use `proto3` syntax.
### Single package
APIs defined in protocol buffers **must** define each individual API in a
single package, which **must** end in a version component. For example:
```proto
syntax = "proto3";
package google.cloud.translation.v3;
```
Google APIs **must** reside in a directory that matches the protocol buffer
`package` directive. For example, the package above dictates that the directory
be `google/cloud/translation/v3`.
### File names
It is often useful to divide API definitions into multiple files. File names
**must** use `snake_case`.
APIs **should** have an obvious "entry" file, generally named after the API
itself. An API with a small number of discrete services (Google Cloud Pub/Sub's
`Publisher` and `Subscriber` is a good example) **may** have a separate entry
file per service.
APIs with only one file **should** use a filename corresponding to the name of
the API.
API `service` definitions and associated RPC request and response `message`
definitions **should** be defined in the same file.
Bear in mind that the file names often become module names in client libraries,
and customers use them in `import` or `use` statements. Therefore, choosing a
descriptive and language keyword-free filename does matter. For example, a file
called `import.proto` may be problematic in Python.
**Note:** The version **must not** be used as a filename, because this creates
bizarre imports in client libraries. Filenames such as `v3.proto` or
`v1beta1.proto` are prohibited.
### File layout
Individual files **should** place higher level and more important definitions
before lower level and less important definitions.
In a proto file, components **should** be in the following order, and each of
these **should** be separated by a blank line:
- Copyright and license notice (if applicable).
- The proto `syntax` statement.
- The proto `package` statement.
- Any `import` statements, in alphabetical order.
- Any file-level `option` statements.
- Any `service` definitions.
- Methods **should** be grouped by the resource they impact, and standard
methods **should** precede custom methods.
- Resource `message` definitions. A parent resource **must** be defined before
its child resources.
- The RPC request and response `message` definitions, in the same order of the
corresponding methods. Each request message **must** precede its
corresponding response message (if any).
- Any remaining `message` definitions.
- Any top-level `enum` definitions.
### Packaging annotations
Protocol buffers ships with annotations to declare the package or namespace
(depending on the vocabulary of the target language) of the generated files.
For example, setting `go_package` or `csharp_namespace` will override the
inferred package name.
When defining APIs, the following rules apply:
- Java
- The `java_package` annotation **must** be set. The correct value is usually
the proto package with the appropriate TLD prefixed. Example:
`com.google.example.v1`.
- The `java_multiple_files` annotation **must** be set to `true`.
- The `java_outer_classname` annotation **must** be set, and **should** be
set to the name of the proto filename, in `PascalCase`, with `Proto`
appended. Example: `LibraryProto`.
- Other languages
- Package or namespace directives for other languages **must** be set either
in every file in the proto package, or none of them. If they are set, the
values **must** be identical in every file.
- If any part of the protobuf package is a compound name (such as
accessapproval), C#, Ruby and PHP options **must** be specified in order
to take account of the word breaks using PascalCase (UpperCamelCase).
Example:
```proto
option csharp_namespace = "Google.Cloud.AccessApproval.V1";
option php_namespace = "Google\\Cloud\\AccessApproval\\V1";
option ruby_package = "Google::Cloud::AccessApproval::V1";
```
- The `go_package` value depends directly on how the Go code is managed i.e.
if the module name is based on the VCS provider or using a remote import
path, but often has a consistent structure.
- The module **may** differ based on product area e.g.
`google.cloud.accessapproval.v1` would be in module
`cloud.google.com/go/accessapproval`.
- The package import path **should** be derived from the proto package.
- An API version in the proto package **should** be prefixed with `api` e.g.
the proto package segment `v1` becomes `apiv1`.
- The terminal import path segment **should** be based on the product name
found within the proto package and **must** be suffixed with `pb` e.g.
`accessapproval` becomes `accessapprovalpb`.
- This value **should** be left to the team owning the generated code to
decide on.
All packaging annotations **should** be specified in alphabetical order of
name. Refer to the [Protobuf documentation][package docs] for more about
language package options.
**Important:** While languages other than Java have sensible defaults for APIs
which don't include compound names, be aware that _adding_ this annotation
(with a value not equivalent to the default) constitutes a breaking change in
that language. When releasing protos, be sure that omissions are intentional.
## Rationale
### Java packaging options
Set the option, `java_multiple_files`, to true to get a cleaner file structure.
Doing so instructs `protoc` to create one output file per Protobuf type, which
allows for more fine-grained imports. The option, `java_outer_classname`, is
required in combination with `java_multiple_files`. It instructs `protoc` to
wrap each compiled Protobuf type in a Java class whose name is the value of the
option. This prevents potential naming collisions between generated types.
### Go packaging option
The Go packaging option needs to be decided by the team that owns the generated
code, because it is directly tied to the source code management practices of the
team. Allowing every proto package to decide on their own Go package creates
inconsistencies and friction in management of the code. Within that owning team,
having a consistent structure in the Go package naming is critical to a
consistent end user experience.
## Changelog
- **2024-06-13**: Added guidance for Go packaging annotation.
- **2024-06-05**: Added rationale for Java packaging options.
- **2023-02-24**: Added guidance on protobuf syntax.
- **2022-10-18**: Added guidance on Ruby/PHP/C# options.
- **2019-11-18**: Added guidance on the packaging annotations.
[package docs]: https://protobuf.dev/programming-guides/proto3/#packages
>>>> 0192.md
---
id: 192
state: approved
created: 2019-07-25
placement:
category: polish
order: 20
---
# Documentation
Documentation is one of the most critical aspects of API design. Users of your
API are unable to dig into the implementation to understand the API better;
often, the API surface definition and its corresponding documentation will be
the only things a user has. Therefore, it is important that documentation be as
clear, complete, and unambiguous as possible.
## Guidance
In APIs defined in protocol buffers, public comments **must** be included over
every component (service, method, message, field, enum, and enum value) using
the protocol buffers comment format. This is important even in cases where the
comment is terse and uninteresting, as numerous tools read these comments and
use them.
Services, in particular, **should** have descriptive comments that explain what
the service is and what users are able to do with it.
**Note:** Many readers will not be native English speakers. Comments **should**
avoid jargon, slang, complex metaphors, pop culture references, or anything
else that will not easily translate. Additionally, many readers will have
different backgrounds and viewpoints; if writing examples involving people,
comments **should** use people who are non-controversial and no longer alive.
### Style
Comments **should** be in grammatically correct American English. However, the
first sentence of each comment **should** omit the subject and be in the
third-person present tense:
```proto
// Creates a book under the given publisher.
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
```
### Descriptions
Descriptions of messages and fields **should** be brief but complete. Sometimes
comments are necessarily perfunctory because there is little to be said;
however, before jumping to that conclusion, consider whether some of the
following questions are relevant:
- What is it?
- How do you use it?
- What does it do if it succeeds? What does it do if it fails?
- Is it idempotent?
- What are the units? (Examples: meters, degrees, pixels)
- What are the side effects?
- What are common errors that may break it?
- What is the expected input format?
- What range of values does it accept? (Examples: `[0.0, 1.0)`, `[1, 10]`)
- Is the range inclusive or exclusive?
- For strings, what is the minimum and maximum length, and what characters
are allowed?
- If a value is above the maximum length, do you truncate or send an error?
- Is it always present? (Example: "Container for voting information. Present
only when voting information is recorded.")
- Does it have a default setting? (Example: "If `page_size` is omitted, the
default is 50.")
### Formatting
Any formatting in comments **must** be in [CommonMark][]. Headings and tables
**must not** be used, as these cause problems for several tools, and are
unsuitable for client library reference documentation.
Comments **should** use `code font` for field or method names and for literals
(such as `true`).
Raw HTML **must not** be used.
"ASCII art" attempts to present a diagram within the protos **must not** be
used. The Markdown within the protos is consumed by a large number of renderers,
and any ASCII art is very unlikely to be well-presented by all of them. If
a diagram is useful in order to understand the API, include a link to a
documentation page containing the diagram as an image.
### Cross-references
A comment can "link" to another component (service, method, message, field,
enum, or enum value) as a Markdown reference link. The reference **must** be one
of the following forms:
- The fully-qualified name of the element e.g. `[Book][google.example.v1.Book]`
- A scope-relative reference qualified e.g. `[Sci-Fi genre][Genre.GENRE_SCI_FI]`
- An implied reference e.g. `[Book][]` which equates to `[Book][Book]`
These references are resolved as per [name resolution][] rules.
Containing fields names **must not** be used in references. They will not
resolve. The original definition **must** be referenced instead. For example,
`[author][Book.author.family_name]` where `author` is a field of `Book`, will
not resolve, but `[author][Author.family_name]` will.
[name resolution]: https://protobuf.dev/programming-guides/proto3/#name-resolution
### External links
Comments **may** link to external pages to provide background information
beyond what is described in the public comments themselves. External links
**must** use absolute (rather than relative) URLs, including the protocol
(usually `https`), and **should not** assume the documentation is located on
any particular host. For example:
`[Spanner Documentation](https://cloud.google.com/spanner/docs)`
### Trademarked names
When referring to the proper, trademarked names of companies or products in
comments, acronyms **should not** be used, unless the acronym is such dominant
colloquial use that avoiding it would obscure the reference (example: [IBM][]).
Comments **should** spell and capitalize trademarked names consistent with the
trademark owner's current branding.
### Deprecations
To deprecate a component (service, method, message, field, enum, or enum value),
the `deprecated` [option](https://developers.google.com/protocol-buffers/docs/proto#options)
**must** be set to `true`, and the first line of the respective comment
**must** start with `"Deprecated: "` and provide alternative solutions for
developers. If there is no alternative solution, a deprecation reason **must**
be given.
### Internal comments
<!-- TODO: This does not work outside of Google.
We should probably try to get that fixed. -->
Comments **may** be explicitly marked as internal by wrapping internal content
in `(--` and `--)`.
Non-public links, internal implementation notes (such as `TODO` and `FIXME`
directives), and other such material **must** be marked as internal.
**Note:** Comments **should** use only leading comments (not trailing comments
or detached comments). In particular, comments **must not** use both a leading
and trailing comment to describe any component, because this is a common source
of inadvertent omissions of the internal content annotation.
[commonmark]: https://commonmark.org/
[ibm]: https://en.wikipedia.org/wiki/IBM
## Changelog
- **2024-10-29**: Include cross-reference resolution rules.
- **2023-08-11**: Expand deprecated comment requirement to all components.
- **2021-04-20**: Added guidance for deprecated services and RPCs.
- **2020-04-01**: Added guidance requiring absolute URLs for external links.
- **2020-02-14**: Added guidance around the use of trademarked names.
- **2019-09-23**: Added guidance about not using both leading and trailing
comments.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0193.md
---
id: 193
state: approved
created: 2019-07-26
placement:
category: polish
order: 30
---
# Errors
Effective error communication is an important part of designing simple
and intuitive APIs. Services returning standardized error responses
enable API clients to construct centralized common error handling logic.
This common logic simplifies API client applications and eliminates the
need for cumbersome custom error handling code.
## Guidance
Services **must** return a [`google.rpc.Status`][Status] message when an
API error occurs, and **must** use the canonical error codes defined in
[`google.rpc.Code`][Code]. More information about the particular codes
is available in the [gRPC status code documentation][].
Error messages **should** help a reasonably technical user *understand*
and *resolve* the issue, and **should not** assume that the user is an
expert in your particular API. Additionally, error messages **must not**
assume that the user will know anything about its underlying
implementation.
Error messages **should** be brief but actionable. Any extra information
**should** be provided in the `details` field. If even more information
is necessary, you **should** provide a link where a reader can get more
information or ask questions to help resolve the issue. It is also
important to [set the right tone][writing-tone] when writing messages.
The following sections describe the fields of `google.rpc.Status`.
### Status.message
The `message` field is a developer-facing, human-readable "debug message"
which **should** be in English. (Localized messages are expressed using
a `LocalizedMessage` within the `details` field. See
[`LocalizedMessage`](#localizedmessage) for more details.) Any dynamic aspects of
the message **must** be included as metadata within the `ErrorInfo` that appears
in [`details`](#status-details).
The message is considered a problem description. It is intended for
developers to understand the problem and is more detailed than
[`ErrorInfo.reason`][ErrorInfo-reason], discussed [later](#errorinfo).
Messages **should** use simple descriptive language that is easy to understand
(without technical jargon) to clearly state the problem that results in an
error, and offer an actionable resolution to it.
For pre-existing (brownfield) APIs which have previously returned errors
without machine-readable identifiers, the value of `message` **must**
remain the same for any given error. For more information, see
[Changing Error Messages](#changing-error-messages).
### Status.code
The `code` field is the status code, which **must** be the numeric value of
one of the elements of the [`google.rpc.Code`][Code] enum.
For example, the value `5` is the numeric value of the `NOT_FOUND`
enum element.
### Status.details
The `details` field allows messages with additional error information to
be included in the error response, each packed in a `google.protobuf.Any`
message.
Google defines a set of [standard detail payloads][details] for error
details, which cover most common needs for API errors.
Services **should** use these standard detail payloads when feasible.
Each type of detail payload **must** be included at most once. For
example, there **must not** be more than one [`BadRequest`][BadRequest]
message in the `details`, but there **may** be a `BadRequest` and a
[`PreconditionFailure`][PreconditionFailure].
All error responses **must** include an `ErrorInfo` within `details`. This
provides machine-readable identifiers so that users can write code against
specific aspects of the error.
The following sections describe the most common standard detail payloads.
#### ErrorInfo
The [`ErrorInfo`][ErrorInfo] message is the primary way to send a
machine-readable identifier. Contextual information **should** be
included in `metadata` in `ErrorInfo` and **must** be included if it
appears within an error message.
The `reason` field is a short snake_case description of the cause of the
error. Error reasons are unique within a particular domain of errors.
The reason **must** be at most 63 characters and match a regular expression of
`[A-Z][A-Z0-9_]+[A-Z0-9]`. (This is UPPER_SNAKE_CASE, without leading
or trailing underscores, and without leading digits.)
The reason **should** be terse, but meaningful enough for a human reader to
understand what the reason refers to.
Good examples:
- `CPU_AVAILABILITY`
- `NO_STOCK`
- `CHECKED_OUT`
- `AVAILABILITY_ERROR`
Bad examples:
- `THE_BOOK_YOU_WANT_IS_NOT_AVAILABLE` (overly verbose)
- `ERROR` (too general)
The `domain` field is the logical grouping to which the `reason` belongs.
The domain **must** be a globally unique value, and is typically the name of the service
that generated the error, e.g. `pubsub.googleapis.com`.
The (reason, domain) pair form a machine-readable way of identifying a particular error.
Services **must** use the same (reason, domain) pair for the same error, and
**must not** use the same (reason, domain) pair for logically different errors.
The decision about whether two errors are "the same" or not is not always clear, but
**should** generally be considered in terms of the expected action a client might take
to resolve them.
The `metadata` field is a map of key/value pairs providing additional
dynamic information as context. Each key within `metadata` **must** be at most
64 characters long, and conform to the regular expression `[a-z][a-zA-Z0-9-_]+`.
Any request-specific information which contributes to the `Status.message` or
`LocalizedMessage.message` messages **must** be represented within `metadata`.
This practice is critical so that machine actors do not need to parse error
messages to extract information.
For example consider the following message:
> An &lt;e2-medium&gt; VM instance with &lt;local-ssd=3,nvidia-t4=2&gt; is currently unavailable
> in the &lt;us-east1-a&gt; zone. Consider trying your request in the &lt;us-central1-f,us-central1-c&gt;
> zone(s), which currently has/have capacity to accommodate your request. Alternatively,
> you can try your request again with a different VM hardware configuration
> or at a later time. For more information, see the troubleshooting documentation.
The `ErrorInfo.metadata` map for the same error could be:
- `"zone": "us-east1-a"`
- `"vmType": "e2-medium"`
- `"attachment": "local-ssd=3,nvidia-t4=2"`
- `"zonesWithCapacity": "us-central1-f,us-central1-c"`
Additional contextual information that does not appear in an error message
**may** also be included in `metadata` to allow programmatic use by the client.
The metadata included for any given (reason,domain) pair can evolve over time:
- New keys **may** be included
- All keys that have been included **must** continue to be included (but may have empty values)
In other words, once a user has observed a given key for a (reason, domain) pair, the
service **must** allow them to rely on it continuing to be present in the future.
The set of keys provided in each (reason, domain) pair is independent from other pairs,
but services **should** aim for consistent key naming. For example, two error reasons
within the same domain should not use metadata keys of `vmType` and `virtualMachineType`.
#### LocalizedMessage
[`google.rpc.LocalizedMessage`][LocalizedMessage] is used to provide an error
message which **should** be localized to a user-specified locale where
possible.
If the [`Status.message`](#status-message) field has a sub-optimal value
which cannot be changed due to the constraints in the
[Changing Error Messages](#changing-error-messages) section, `LocalizedMessage`
**may** be used to provide a better error message even when no user-specified
locale is available.
Regardless of how the locale for the message was determined, both the `locale`
and `message` fields **must** be populated.
The `locale` field specifies the locale of the message,
following [IETF bcp47](https://www.rfc-editor.org/rfc/bcp/bcp47.txt) (Tags for
Identifying Languages). Example values: `"en-US"`, `"fr-CH"`, `"es-MX"`.
The `message` field contains the localized text itself. This
**should** include a brief description of the error and a call to action
to resolve the error. The message **should** include contextual information
to make the message as specific as possible. Any contextual information
in the message **must** be included in `ErrorInfo.metadata`. See
[`ErrorInfo`](#errorinfo) for more details of how contextual information
may be included in a message and the corresponding metadata.
The `LocalizedMessage` payload **should** contain the complete resolution
to the error. If more information is needed than can reasonably fit in this
payload, then additional resolution information **must** be provided in
a `Help` payload. See the [Help](#help) section for guidance.
#### Help
When other textual error messages (in `Status.message` or
`LocalizedMessage.message`) don't provide the user sufficient
context or actionable next steps, or if there are multiple points of
failure that need to be considered in troubleshooting, a link to
supplemental troubleshooting documentation **must** be provided in the
`Help` payload.
Provide this information in addition to a clear problem definition and
actionable resolution, not as an alternative to them. The linked
documentation **must** clearly relate to the error. If a single page
contains information about multiple errors, the
[`ErrorInfo.reason`](#errorinfo) value **must** be used to narrow down
the relevant information.
The `description` field is a textual description of the linked information.
This **must** be suitable to display to a user as text for a hyperlink.
This **must** be plain text (not HTML, Markdown etc).
Example `description` value: `"Troubleshooting documentation for STOCKOUT errors"`
The `url` field is the URL to link to. This **must** be an absolute URL,
including scheme.
Example `url` value:
`"https://cloud.google.com/compute/docs/resource-error"`
For publicly-documented services, even those with access controls on actual
usage, the linked content **must** be accessible without authentication.
For privately-documented services, the linked content **may** require
authentication.
### Error messages
Textual error messages can be present in both `Status.message` and
`LocalizedMessage.message` fields. Messages **should** be succinct but
actionable, with request-specific information (such as a resource name
or region) providing precise details where appropriate. Any request-specific
details **must** be present in [`ErrorInfo.metadata`](#errorinfo).
#### Changing error messages
Changing the content of `Status.message` over time must be done carefully,
to avoid breaking clients who have previously had to rely on the message
for all information. See [the rationale section](#updating-statusmessage)
for more details.
For a given RPC:
- If the RPC has *always* returned `ErrorInfo` with machine-readable
information, the content of `Status.message` **may** change over time.
(For example, the API producer may provide a clearer explanation,
or more request-specific information.)
- Otherwise, the content of `Status.message` **must** be stable,
providing the same text with the same request-specific information.
Instead of changing `Status.message`, the API **should** include a
[`LocalizedMessage`](#localizedmessage) within `Status.details`.
Even if an RPC has always returned `ErrorInfo`, the API **may** keep
the existing `Status.message` stable and add a
[`LocalizedMessage`](#localizedmessage) within `Status.details`.
The content of `LocalizedMessage.details` **may** change over time.
### Partial errors
APIs **should not** support partial errors. Partial errors add
significant complexity for users, because they usually sidestep the use
of error codes, or move those error codes into the response message,
where the user **must** write specialized error handling logic to
address the problem.
However, occasionally partial errors are necessary, particularly in bulk
operations where it would be hostile to users to fail an entire large
request because of a problem with a single entry.
Methods that require partial errors **should** use [long-running
operations][], and the method **should** put partial failure information
in the metadata message. The errors themselves **must** still be
represented with a [`google.rpc.Status`][Status] object.
### Permission Denied
If the user does not have permission to access the resource or parent,
regardless of whether or not it exists, the service **must** error with
`PERMISSION_DENIED` (HTTP 403). Permission **must** be checked prior to
checking if the resource or parent exists.
If the user does have proper permission, but the requested resource or
parent does not exist, the service **must** error with `NOT_FOUND` (HTTP
404).
## HTTP/1.1+JSON representation
When clients use HTTP/1.1 as per [AIP-127](./0127.md), the error information
is returned in the body of the response, as a JSON object. For backward
compatibility reasons, this does not map precisely to `google.rpc.Status`,
but contains the same core information. The schema is defined in the following proto:
```proto
message Error {
message Status {
// The HTTP status code that corresponds to `google.rpc.Status.code`.
int32 code = 1;
// This corresponds to `google.rpc.Status.message`.
string message = 2;
// This is the enum version for `google.rpc.Status.code`.
google.rpc.Code status = 4;
// This corresponds to `google.rpc.Status.details`.
repeated google.protobuf.Any details = 5;
}
Status error = 1;
}
```
The most important difference is that the `code` field in the JSON is an HTTP status code,
*not* the direct value of `google.rpc.Status.code`. For example, a `google.rpc.Status`
message with a `code` value of 5 would be mapped to an object including the following
code-related fields (as well as the message, details etc):
```json
{
"error": {
"code": 404, // The HTTP status code for "not found"
"status": "NOT_FOUND" // The name in google.rpc.Code for value 5
}
}
```
The following JSON shows a fully populated HTTP/1.1+JSON representation of an error response.
```json
{
"error": {
"code": 429,
"message": "The zone 'us-east1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.",
"status": "RESOURCE_EXHAUSTED",
"details": [
{
"@type": "type.googleapis.com/google.rpc.ErrorInfo",
"reason": "RESOURCE_AVAILABILITY",
"domain": "compute.googleapis.com",
"metadata": {
"zone": "us-east1-a",
"vmType": "e2-medium",
"attachment": "local-ssd=3,nvidia-t4=2",
"zonesWithCapacity": "us-central1-f,us-central1-c"
}
},
{
"@type": "type.googleapis.com/google.rpc.LocalizedMessage",
"locale": "en-US",
"message": "An <e2-medium> VM instance with <local-ssd=3,nvidia-t4=2> is currently unavailable in the <us-east1-a> zone. Consider trying your request in the <us-central1-f,us-central1-c> zone(s), which currently has/have capacity to accommodate your request. Alternatively, you can try your request again with a different VM hardware configuration or at a later time. For more information, see the troubleshooting documentation."
},
{
"@type": "type.googleapis.com/google.rpc.Help",
"links": [
{
"description": "Additional information on this error",
"url": "https://cloud.google.com/compute/docs/resource-error"
}
]
}
]
}
}
```
## Rationale
### Requiring ErrorInfo
`ErrorInfo` is required because it further identifies an error. With
only approximately twenty [available values][Code] for `Status.status`,
it is difficult to disambiguate one error from another across an entire
[API Service][API Service].
Also, error messages often contain dynamic segments that express
variable information, so there needs to be machine-readable component of
*every* error response that enables clients to use such information
programmatically.
### Including LocalizedMessage
`LocalizedMessage` was selected as the location to present alternate
error messages. While `LocalizedMessage` **may** use a locale specified
in the request, a service **may** provide a `LocalizedMessage` even without
a user-specified locale, typically to provide a better error message in
[situations where `Status.message` cannot be changed](#updating-statusmessage).
Where the locale is not specified by the user, it **should** be `en-US`
(US English).
A service **may** include `LocalizedMessage` even when the same message is
provided in `Status.message` and when localization into a user-specified locale
is not supported. Reasons for this include:
- An intention to support user-specified localization in the near future, allowing
clients to consistently use `LocalizedMessage` and not change their error-reporting
code when the functionality is introduced.
- Consistency across all RPCs within a service: if some RPCs include
`LocalizedMessage` and some only use `Status.message` for error messages, clients
have to be aware of which RPCs will do what, or implement a fall-back mechanism.
Providing `LocalizedMessage` on all RPCs allows simple and consistent client code
to be written.
### Updating Status.message
If a client has ever observed an error with `Status.message` populated
(which it always will be) but without `ErrorInfo`, the developer of that client
may well have had to resort to parsing `Status.message` in order to find out
information beyond just what `Status.code` conveys. That information may be
found by matching specific text (e.g. "Connection closed with unknown cause")
or by parsing the message to find out metadata values (e.g. a region with
insufficient resources). At that point, `Status.message` is implicitly part
of the API contract, so **must not** be updated - that would be a breaking
change. This is one reason for introducing `LocalizedMessage` into the
`Status.details`.
RPCs which have **always** included `ErrorInfo` are in a better position:
the contract is then more about the stability of `ErrorInfo` for any given
error. The reason and domain need to be consistent over time, and the
metadata provided for any given (reason,domain) can only be expanded.
It's still possible that clients could be parsing `Status.message` instead of
using `ErrorInfo`, but they will always have had a more robust option
available to them.
## Further reading
- For which error codes to retry, see [AIP-194][aip-194].
- For how to retry errors in client libraries, see
[AIP-4221][aip-4221].
## Changelog
- **2024-10-18**: Rewrite/restructure for clarity.
- **2024-01-10**: Incorporate guidance for writing effective messages.
- **2023-05-17**: Change the recommended language for `Status.message` to be
the service's native language rather than English.
- **2023-05-17**: Specify requirements for changing error messages.
- **2023-05-10**: Require [`ErrorInfo`][ErrorInfo] for all error responses.
- **2023-05-04**: Require uniqueness by message type for error details.
- **2022-11-04**: Added guidance around PERMISSION_DENIED errors previously
found in other AIPs.
- **2022-08-12**: Reworded/Simplified intro to add clarity to the intent.
- **2020-01-22**: Added a reference to the [`ErrorInfo`][ErrorInfo] message.
- **2019-10-14**: Added guidance restricting error message mutability to if
there is a machine-readable identifier present.
- **2019-09-23**: Added guidance about error message strings being able to
change.
<!-- prettier-ignore-start -->
[aip-4221]: https://google.aip.dev/client-libraries/4221
[details]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto
[ErrorInfo]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L27-L76
[ErrorInfo-reason]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L57
[PreconditionFailure]: https://github.com/googleapis/googleapis/blob/6f3fcc058ff29989f6d3a71557a44b5e81b897bd/google/rpc/error_details.proto#L139-L166
[BadRequest]: https://github.com/googleapis/googleapis/blob/477a59d764428136ba1d857a9633c0d231de6efa/google/rpc/error_details.proto#L168-L218
[LocalizedMessage]: https://github.com/googleapis/googleapis/blob/e9897ed945336e2dc967b439ac7b4be6d2c62640/google/rpc/error_details.proto#L275-L285
[grpc status code documentation]: https://github.com/grpc/grpc/blob/master/doc/statuscodes.md
[Code]: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
[Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
[Status details]: https://github.com/googleapis/googleapis/blob/aeae5ea2b01ece6c0cee046ae84b881cdc62b95d/google/rpc/status.proto#L46-L48
[long-running operations]: ./0151.md
[API Service]: https://cloud.google.com/apis/design/glossary#api_service
[writing-tone]: https://developers.google.com/tech-writing/error-messages/set-tone
[Error model]: https://cloud.google.com/apis/design/errors#error_model
[Help]: https://github.com/googleapis/googleapis/blob/master/google/rpc/error_details.proto#L256
[resource-error]: https://cloud.google.com/compute/docs/resource-error
[InvalidArgument]: https://github.com/googleapis/googleapis/blob/d4acb64370d333024a167551e7da854506109ba2/google/rpc/code.proto#L52-L58
<!-- prettier-ignore-end -->
>>>> 0194.md
---
id: 194
state: approved
created: 2019-08-16
placement:
category: polish
order: 40
---
# Automatic retry configuration
RPCs sometimes fail. When one does, the client performing the RPC needs to know
whether it is safe to retry the operation. When status codes are used
consistently across multiple APIs, clients can respond to failures
appropriately.
## Guidance
Clients **should** automatically retry requests for which repeated runs would
not cause unintended state changes, which are non-transactional, and which are
[unary](https://grpc.io/docs/guides/concepts/).
Clients **should not** automatically retry transactional requests; instead
these requests **should** have application-level retry logic that retries the
entire transaction block from the start.
Clients **should not** automatically retry requests in which repeated runs
would cause unintended state changes.
**Note:** This AIP does not cover client streaming or bi-directional streaming.
**Note:** For client side retry behavior in the client libraries: see
[AIP-4221][].
### Retryable codes
For methods listed as retryable above, clients **should** retry the following
error codes:
- `UNAVAILABLE`: This code generally results from network hiccups, and is
generally transient. It is retryable under the expectation that the
connection will become available (soon).
### Non-retryable codes
The following codes **should not** be automatically retried for any request:
- `OK`: The request succeeded.
- `CANCELLED`: An application can cancel a request, which **must** be honored.
- `DEADLINE_EXCEEDED`: An application can set a deadline, which **must** be
honored.
- `INVALID_ARGUMENT`: Retrying a request with an invalid argument will never
succeed.
- `DATA_LOSS`: This is an unrecoverable error and **must** immediately be
surfaced to the application.
### Generally non-retryable codes:
The following codes generally **should not** be automatically retried for any
request:
- `RESOURCE_EXHAUSTED`: This code may be a signal that quota is exhausted.
Retries therefore may not be expected to work for several hours; meanwhile
the retries may have billing implications. If `RESOURCE_EXHAUSTED` is used
for other reasons than quota and the expected time for the resource to become
available is much shorter, it may be retryable.
- `INTERNAL`: This code generally means that some internal part of the system
has failed, and usually means a bug should be filed against the system. These
**should** immediately be surfaced to the application.
- `UNKNOWN`: Unlike `INTERNAL`, this code is reserved for truly
unknown-to-the-system errors, and therefore may not be safe to retry. These
**should** immediately be surfaced to the application.
- `ABORTED`: This code typically means that the request failed due to a
sequencer check failure or transaction abort. These **should not** be retried
for an individual request; they **should** be retried at a level higher (the
entire transaction, for example).
Some codes **may** be automatically retried if a system is designed without
synchronization or signaling between various components. For example, client
might retry `NOT_FOUND` on a read operation, which is designed to hang forever
until the resource is created. However, these types of systems are generally
discouraged.
Therefore, the following codes **should not** be automatically retried for any
request:
- `NOT_FOUND`: A client **should not** retry until a resource is created.
- `ALREADY_EXISTS`: A client **should not** retry until a resource is deleted.
- `PERMISSION_DENIED`: A client **should not** retry until it has permission.
- `UNAUTHORIZED`: A client **should not** retry until it is authorized.
- `UNAUTHENTICATED`: A client **should not** retry until it is authenticated.
- `FAILED_PRECONDITION`: A client **should not** retry until system state
changes.
- `OUT_OF_RANGE`: A client **should not** retry until the range is extended.
- `UNIMPLEMENTED`: A client **should not** retry until the RPC is implemented.
## Further reading
- For parallel or retried request disambiguation, see [AIP-154](./0154.md).
>>>> 0200.md
---
id: 200
state: approved
created: 2018-06-28
placement:
category: meta
order: 30
redirect_from: /not-precedent
---
# Precedent
Many times, APIs are written in ways that do not match new guidance that is
added to these standards after those APIs have already been released.
Additionally, sometimes it can make sense to intentionally violate standards
for particular reasons, such as maintaining consistency with established
systems, meeting stringent performance requirements, or other practical
concerns. Finally, as carefully as everyone reviews APIs before they are
released, sometimes mistakes can slip through.
Since it often is not feasible to fix past mistakes or make the standards serve
every use case, APIs may be stuck with these exceptions for quite some time.
Further, since new APIs often base their designs (names, types, structures,
etc) on existing APIs, it is possible that a standards violation in one API
could spill over into other APIs, even if original reason for the exception is
not applicable to the other APIs.
As a result of this problem, it is important to "stop the bleeding" of these
standards exceptions into new APIs, and additionally document the reasons for
each exception so that historical wisdom is not lost.
## Guidance
If an API violates the AIP standards for any reason, there **must** be an
internal comment linking to this document using its descriptive link
([aip.dev/not-precedent]()) to ensure others do not copy the violations or cite
the errors as precedent of a "previously approved API".
The comment should also include an explanation of what violates standards and
why it is necessary. For example:
```proto
message DailyMaintenanceWindow {
// Time within the maintenance window to start the maintenance operations.
// It must use the format "HH MM", where HH : [00-23] and MM : [00-59] GMT.
// (-- aip.dev/not-precedent: This was designed for consistency with crontab,
// and preceded the AIP standards.
// Ordinarily, this type should be `google.type.TimeOfDay`. --)
string start_time = 2;
// Output only. Duration of the time window, automatically chosen to be
// smallest possible in the given scenario.
// (-- aip.dev/not-precedent: This preceded the AIP standards.
// Ordinarily, this type should be `google.protobuf.Duration`. --)
string duration = 3;
}
```
**Important:** APIs should only be considered to be precedent-setting if they
are in beta or GA.
### Local consistency
If an API violates a standard throughout, it would be jarring and frustrating
to users to break the existing pattern only for the sake of adhering to the
global standard.
For example, if all of an API's resources use `creation_time` (instead of the
standard field `create_time` described in [AIP-142][]), a new resource in that
API should continue to follow the local pattern.
However, others who might otherwise copy that API should be made aware that
this is contra-standard and not something to cite as precedent when launching
new APIs.
```proto
// ...
message Book {
// (-- aip.dev/not-precedent: This field was present before there was a
// standard field.
// Ordinarily, it should be spelled `create_time`. --)
google.protobuf.Timestamp creation_time = 1;
}
// ...
message Author {
// (-- aip.dev/not-precedent: `Book` had `creation_time` before there was
// a standard field, so we match that here for consistency. Ordinarily,
// this would be spelled `create_time`. --)
google.protobuf.Timestamp creation_time = 1;
}
```
### Pre-existing functionality
Standards violations are sometimes overlooked before launching, resulting in
APIs that become stable and therefore can not easily be modified. Additionally,
a stable API may pre-date a standards requirement.
In these scenarios, it is difficult to make the API fit the standard. However,
the API should still cite that the functionality is contra-standard so that
other APIs do not copy the mistake and cite the existing API as a reason why
their design should be approved.
### Adherence to external spec
Occasionally, APIs must violate standards because specific requests are
implementations of an external specification (for example, OAuth), and their
specification may be at odds with AIP guidelines. In this case, it is likely to
be appropriate to follow the external specification.
### Adherence to existing systems
Similar to the example of an external specification above, it may be proper for
an API to violate AIP guidelines to fit in with an existing system in some way.
This is a fundamentally similar case where it is wise to meet the customer
where they are. A potential example of this might be integration with or
similarity to a partner API.
### Expediency
Sometimes there are users who need an API surface by a very hard deadline or
money walks away. Since most APIs serve a business purpose, there will be times
when an API could be better but cannot get it that way and into users' hands
before the deadline. In those cases, API review councils **may** grant
exceptions to ship APIs that violate guidelines due to time and business
constraints.
### Technical concerns
Internal systems sometimes have very specific implementation needs (e.g., they
rely on operation transforms that speak UTF-16, not UTF-8) and adhering to AIP
guidelines would require extra work that does not add significant value to API
consumers. Future systems which are likely to expose an API at some point
should bear this in mind to avoid building underlying infrastructure which
makes it difficult to follow AIP guidelines.
## Changelog
- **2020-03-27**: Reworded much of this AIP to follow [AIP-8][], and remove
first and second person. No semantic changes.
- **2019-05-04**: Changed to a public link ([aip.dev/not-precedent]()), and
changed references to "the style guide" to use the more generic term
"standards" (to account for a general shift to AIPs).
[aip-8]: ./0008.md
[aip-142]: ./0142.md
>>>> 0202.md
---
id: 202
state: approved
created: 2023-09-22
placement:
category: fields
order: 0
---
# Fields
The [`google.api.FieldInfo`][field info proto] type, through its accompanying
extension `google.api.field_info`, enriches a field's schema beyond the basic
name and type information.
## Guidance
Decorating a field with `google.api.field_info` is only necessary when
explicitly stated in this AIP or another that leverages `google.api.FieldInfo`
information. As such, the guidance herein applies to those scenarios as well.
### Format
Fields with a primitive type can still have a specific format. To convey that
type format, the `FieldInfo.Format` enumeration is used via the
`(google.api.field_info).format` extension field. The following guidance conveys
the meaning of and requirements for use of each `FieldInfo.Format` value.
#### UUID4
The `UUID4` format represents a UUID version 4 value as governed by
[RFC 4122][]. It **must** only be used on a field of type `string`.
Such a value **may** be normalized by the service to entirely lowercase letters.
For example, the value `F47AC10B-58CC-0372-8567-0E02B2C3D479` would be
normalized to `f47ac10b-58cc-0372-8567-0e02b2c3d479`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 4122][] compliant implementation **must** be used.
#### IPv4
The `IPV4` format represents an IP v4 address as governed by [RFC 791][]. It
**must** only be used on a field of type `string`.
Such a value **may** be condensed by the service, with leading zeros in each
octet stripped. For example, `001.022.233.040` would be condensed to
`1.22.233.40`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 791][] compliant implementation **must** be used.
#### IPv6
The `IPV6` format represents an IP v6 address as governed by [RFC 4291][]. It
**must** only be used on a field of type `string`.
Such a value **may** be normalized by the service to entirely lowercase letters
with zeros compressed, following [RFC 5952][]. For example, the value
`2001:0DB8:0::0` would be normalized to `2001:db8::`.
As such, equivalence comparison **must not** be done via primitive text
comparison. Instead, an [RFC 4291][] compliant implementation **must** be used.
#### IPv4 or IPv6
The `IPV4_OR_IPV6` value indicates that the field can be either an IP v4 or v6
address, as described in the [IPv4](#ipv4) and [IPv6](#ipv6) sections.
#### Format Compatibility
Adding a format specifier to an existing, unspecified field **is not** backwards
compatible, *unless* the field in question has always conformed to the format
being specified.
Changing an existing format specifier to a different one in all cases **is not**
backwards compatible.
#### Extending Format
Any new `FieldInfo.Format` value **must** be governed by an
[IETF-approved RFC][ietf rfc] or a [Google-approved AIP](./0001.md).
## Rationale
#### Why add a format specifier?
The format of a primitive-typed field can be critical to its usability. Some
programming languages may convey a specific type format as a standalone type,
as Java does with [UUID][java uuid]. Most have specific structural requirements
that are validated by the service, so conveying the format to the user ahead of
time is critical to their experience.
#### Why discourage primitive equality comparisons?
The text representations of the supported formats have many nuances and
transforming the value into a canonical representation is non-trivial. As such,
aligning implementations between each consumer and each service without any
issue is infeasiable.
#### Why document value normalizations?
While primitive comparison is not recommended for any of the supported formats,
uniform normalization of values is important to set consumer expectations, and
create a user-friendly surface.
#### Why require an RFC or AIP for new formats?
Those formats which are sufficiently standardized to merit an RFC or AIP are
stable enough and widely enough known to be incorporated as a supported value
and see usage in Google APIs. Requiring such extra guidance means that governing
the format specification is not the responsibility of the `FieldInfo.Format`
enumeration itself.
[field info proto]: https://github.com/googleapis/googleapis/blob/master/google/api/field_info.proto
[rfc 4122]: https://datatracker.ietf.org/doc/html/rfc4122
[rfc 791]: https://datatracker.ietf.org/doc/html/rfc791
[rfc 4291]: https://datatracker.ietf.org/doc/html/rfc4291#section-2.2
[rfc 5952]: https://datatracker.ietf.org/doc/html/rfc5952
[ietf rfc]: https://www.ietf.org/standards/rfcs
[java uuid]: https://docs.oracle.com/javase/8/docs/api/java/util/UUID.html
>>>> 0203.md
---
id: 203
state: approved
created: 2018-07-17
placement:
category: fields
order: 15
---
# Field behavior documentation
When defining fields in protocol buffers, it is customary to explain to users
certain aspects of the field's behavior (such as whether it is required or
optional). Additionally, it can be useful for other tools to understand this
behavior (for example, to optimize client library signatures).
## Guidance
APIs use the `google.api.field_behavior` annotation to describe well-understood
field behavior, such as a field being required or immutable.
```proto
// The audio data to be recognized.
RecognitionAudio audio = 2 [(google.api.field_behavior) = REQUIRED];
```
- APIs **must** apply the `google.api.field_behavior` annotation on every field
on a message or sub-message used in a request.
- The annotation **must** include any [google.api.FieldBehavior][] values that
accurately describe the behavior of the field.
- `FIELD_BEHAVIOR_UNSPECIFIED` **must not** be used.
- APIs **must** at minimum use one of `REQUIRED`, `OPTIONAL`, or `OUTPUT_ONLY`.
**Warning:** Although `field_behavior` does not impact proto-level behavior,
many clients (e.g. CLIs and SDKs) rely on them to generate code. Thoroughly
review and consider which values are relevant when adding a new field.
Fields with no annotation are interpreted as `OPTIONAL` for
backwards-compatility. Nontheless, this annotation **must not** be omitted.
**Note:** The vocabulary given in this document is for _descriptive_ purposes
only, and does not itself add any validation. The purpose is to consistently
document this behavior for clients.
### field behavior of nested messages
`google.api.field_behavior` annotations on a nested message are independent of
the annotations of the parent.
For example, a nested message can have a field behavior of `REQUIRED` while the
parent field can be `OPTIONAL`:
```proto
message Title {
string text = 1 [(google.api.field_behavior) = REQUIRED];
}
message Slide {
Title title = 1 [(google.api.field_behavior) = OPTIONAL];
}
```
In the case above, if a `title` is specified, the `text` field is required.
## Vocabulary
### Identifier
The use of `IDENTIFIER` indicates that a field within a resource message is used
to identify the resource. It **must** be attached to the `name` field and **must
not** be attached to any other field (see [fields representing resource names]).
The `IDENTIFIER` value conveys that the field is not accepted as input (i.e.
`OUTPUT_ONLY`) in the context of a create method, while also being considered
`IMMUTABLE` and accepted as input for mutation methods that accept the
resource as the primary input e.g. [Standard Update][aip-134].
This annotation **must not** be applied to references to other resources within
a message.
### Immutable
The use of `IMMUTABLE` indicates that a field on a resource cannot be changed
after its creation. This can apply to either fields that are input or outputs,
required or optional.
When a service receives an immutable field in an update request (or similar),
even if included in the update mask, the service **should** ignore the field if
the value matches, but **should** error with `INVALID_ARGUMENT` if a change is
requested.
Potential use cases for immutable fields (this is not an exhaustive list) are:
- Attributes of resources that are not modifiable for the lifetime of the
application (e.g. a disk type).
**Note:** Fields which are "conditionally immutable" **must not** be given the
immutable annotation.
### Input only
The use of `INPUT_ONLY` indicates that the field is provided in requests and
that the corresponding field will not be included in output.
Additionally, a field **should** only be described as input only if it is a
field in a resource message or a field of a message included within a resource
message. Notably, fields in request messages (a message which only ever acts as
an argument to an RPC, with a name usually ending in `Request`) **should not**
be described as input only because this is already implied.
Potential use cases for input only fields (this is not an exhaustive list) are:
- The `ttl` field as described in AIP-214.
**Warning:** Input only fields are rare and should be considered carefully
before use.
### Optional
The use of `OPTIONAL` indicates that a field is not required.
A field **may** be described as optional if it is a field on a request message
(a message that is an argument to an RPC, usually ending in `Request`), or a
field on a submessage.
### Output only
The use of `OUTPUT_ONLY` indicates that the field is provided in responses, but
that including the field in a message in a request does nothing (the server
**must** clear out any value in this field and **must not** throw an error as a
result of the presence of a value in this field on input). Similarly, services
**must** ignore the presence of output only fields in update field masks (see:
AIP-161).
Additionally, a field **should** only be described as output only if it is a
field in a resource message, or a field of a message farther down the tree.
Notably, fields in response messages (a message which only ever acts as a
return value to an RPC, usually ending in `Response`) **should not** be
described as output only because this is already implied.
Output only fields **may** be set to empty values if appropriate to the API.
Potential use cases for output only fields (this is not an exhaustive list)
are:
- Create or update timestamps.
- Derived or structured information based on original user input.
- Properties of a resource assigned by the service which can not be altered.
### Required
The use of `REQUIRED` indicates that the field **must** be present (and set to
a non-empty value) on the request or resource.
A field **should** only be described as required if _either_:
- It is a field on a resource that a user provides somewhere as input. In this
case, the resource is only valid if a "truthy" value is _stored_.
- When [creating][aip-133] the resource, a value **must** be provided for the
field on the create request.
- When [updating][aip-134] the resource, the user **may** omit the field
provided that the field is also absent from the field mask, indicating no
change to the field (otherwise it **must** be provided).
- It is a field on a request message (a message that is an argument to an RPC,
with a name usually ending in `Request`). In this case, a value **must** be
provided as part of the request, and failure to do so **must** cause an error
(usually `INVALID_ARGUMENT`).
We define the term "truthy" above as follows:
- For primitives, values other than `0`, `0.0`, empty string/bytes, and `false`
- For repeated fields maps, values with at least one entry
- For messages, any message with at least one "truthy" field.
Fields **should not** be described as required in order to signify:
- A field which will always be present in a response.
- A field which is conditionally required in some situations.
- A field on any message (including messages that are resources) which is never
used as user input.
**Note:** In most cases, empty values (such as `false` for booleans, `0` for
integers, or the unspecified value for enums) are indistinguishable from unset
values, and therefore setting a required field to a falsy value yields an
error. A corollary to this is that a required boolean must be set to `true`.
### Unordered List
The use of `UNORDERED_LIST` on a repeated field of a resource indicates that
the service does not guarantee the order of the items in the list.
A field **should** be described as an unordered list if the service does not
guarantee that the order of the elements in the list will match the order that
the user sent, including a situation where the service will sort the list on
the user's behalf.
A resource with an unordered list **may** return the list in a stable order, or
**may** return the list in a randomized, unstable order.
## Backwards compatibility
Adding or changing `google.api.field_behavior` values can represent a semantic
change in the API that is perceived as incompatible for existing clients. The
following are examples of backwards incompatible changes with
`google.api.field_behavior`:
* Adding `REQUIRED` to an existing field previously considered `OPTIONAL`
(implicitly or otherwise)
* Adding a new field annotated as `REQUIRED` to an existing request message
* Adding `OUTPUT_ONLY` to an existing field previously accepted as input
* Adding `INPUT_ONLY` to an existing field previously emitted as output
* Adding `IMMUTABLE` to an existing field previously considered mutable
* Removing `OUTPUT_ONLY` from an existing field previously ignored as input
* Removing `IDENTIFIER` from an existing field.
There are some changes that *are* backwards compatible, which are as follows:
* Adding `OPTIONAL` to an existing field
* Adding `IDENTIFIER` to an existing `name` field
* Changing from `REQUIRED` to `OPTIONAL` on an existing field
* Changing from `OUTPUT_ONLY` and/or `IMMUTABLE` to `IDENTIFIER` on an existing
field
* Removing `REQUIRED` from an existing field
* Removing `INPUT_ONLY` from an existing field previously excluded in responses
* Removing `IMMUTABLE` from an existing field previously considered immutable
## Rationale
### Identifier field behavior
Resource names, the primary identifiers for any compliant resource, are never
fully constructed by the user on create. Such fields are typically assigned
`OUTPUT_ONLY` field behavior. They are, however, also often consumed as
the primary identifier in scenarios where the resource itself is the primary
request payload. Such fields could *not* be considered `OUTPUT_ONLY`.
Furthermore, in mutation requests, like Standard Update, the resource name as
the primary identifier cannot be changed in place. Such fields are typically
assigned `IMMUTABLE` field behavior. These conflicting and context-dependent
field behaviors meant that a new value was necessary to single out and convey
the behavior of the resource name field.
### Required set of annotations
A field used in a request message must be either an input or an output.
In the case of an output, the `OUTPUT_ONLY` annotation is sufficient.
In the case of an input, a field is either required or optional, and therefore
should have at least the `REQUIRED` or `OPTIONAL` annotation, respectively.
Only providing `INPUT_ONLY` does not convey the necessity of the field, so
specifying either `REQUIRED` or `OPTIONAL` is still necessary.
### Requiring field behavior
By including the field behavior annotation for each field, the overall behavior
that the resource exhibits is more clearly defined. Clearly defined field
behavior improves programmatic clients and user understanding.
Requiring the annotation also forces the API author to explicitly consider the
behavior when initially authoring of the API.
Modifying field behavior after initial authoring can result in
backwards-incompatible changes in clients. For example, making an optional field
required results in backwards-incompatible changes in the method signature of an
RPC or a resource in a [Declarative client][]. See the
[Backwards compatibility](#backwards-compatibility) section for more detailed
compatibility guidance.
## History
In 2023-05 field_behavior was made mandatory. Prior to this change, the
annotation was often omitted. Its values are relied upon to produce high quality
clients. Furthermore, adding or changing some of the field_behavior values after
the fact within a major version can be backwards-incompatible. See the
[Backwards compatibility](#backwards-compatibility) section for more detailed
compatibility guidance.
The benefits of requiring field_behavior at the time that the API is authored
surpass the costs to clients and API users of not doing so.
[aip-133]: ./0133.md
[aip-134]: ./0134.md
[aip-180]: ./0180.md
[google.api.FieldBehavior]: https://github.com/googleapis/googleapis/blob/master/google/api/field_behavior.proto#L49
[Declarative client]: ./0009.md#declarative-clients
[fields representing resource names]: ./0122.md#fields-representing-resource-names
## Changelog
- **2023-09-14**: Clarify that nested behavior and parent behavior are
independent.
- **2023-08-25**: Add guidance on `IDENTIFIER`.
- **2023-07-20**: Describe compatibility guidance with new section.
- **2023-05-24**: Clarify that `IMMUTABLE` does not imply input nor required.
- **2023-05-10**: Added guidance to require the annotation.
- **2020-12-15**: Added guidance for `UNORDERED_LIST`.
- **2020-05-27**: Clarify behavior when receiving an immutable field in an
update.
- **2019-12-05**: Added guidance on output only fields in field masks.
- **2019-06-18**: Use the machine-readable annotation, not comments.
>>>> 0205.md
---
id: 205
state: approved
created: 2019-02-18
placement:
category: process
order: 20
redirect_from: /beta-blocker
---
# Beta-blocking changes
APIs often release an Alpha version of their API in order to get early feedback
from customers. This API is provisional and can change many times before the
important feedback is incorporated and the API is made stable for Beta.
Since the purpose of Alpha is to gather feedback, the API does not need to be
perfect yet, and it's not strictly necessary for API authors to address every
usability concern or address every point in the API standards. Often, API
authors and API reviewers will not agree on the best design, and the best way
to find out is by having users try out the API.
However, once the feedback has been collected and the API is going to be
promoted to Beta, usability concerns and style issues do need to be addressed.
In order to ensure that these issues are not forgotten, they should be
explicitly documented in the API.
## Guidance
If an API has usability concerns or violates API standards, and the present
design should receive additional scrutiny before being carried through to the
Beta version, there **must** be an internal comment linking to this document
using its descriptive link ([aip.dev/beta-blocker]()) to ensure that the design
is corrected before the API is released to Beta.
The comment **must** also indicate what kind of change should be made for Beta.
For example:
```proto
message InputConfig {
// Parameters for input.
// (-- aip.dev/beta-blocker: Convert well-known parameters into explicit
// fields before the Beta launch. --)
map<string, string> parameters = 1;
}
```
If an exception to API standards _does_ need to be carried through to Beta and
GA, see [AIP-200][].
[aip-200]: ./0200.md
>>>> 0210.md
---
id: 210
state: approved
created: 2018-08-20
placement:
category: design-patterns
order: 110
---
# Unicode
APIs should be consistent on how they explain, limit, and bill for string
values and their encodings. This ranges from little ambiguities (like fields
"limited to 1024 characters") all the way to billing confusion (are names and
values of properties in Datastore billed based on characters or bytes?).
In general, if we talk about limits measured in bytes, we are discriminating
against non-ASCII text since it takes up more space. On the other hand, if we
talk about "characters", we are ambiguous about whether those are Unicode "code
points", "code units" for a particular encoding (e.g. UTF-8 or UTF-16),
"graphemes", or "grapheme clusters".
## Unicode primer
Character encoding tends to be an area we often gloss over, so a quick primer:
- Strings are just bytes that represent numbers according to some encoding
format.
- When we talk about **characters**, we sometimes mean Unicode **code points**,
which are numbers in the Unicode spec (up to 21 bits).
- Other times we might mean **graphemes** or **grapheme clusters**, which may
have multiple numeric representations and may be represented by more than one
code point. For example, `á` may be represented as a composition of
`U+0061 + U+0301` (the `a` + the accent combining mark) or as a single code
point, `U+00E1`.
- Protocol buffers uses **UTF-8** ("Unicode Transformation Format") which is a
variable-length encoding scheme using up to 4 **code units** (8-bit bytes)
per code point.
## Guidance
### Character definition
**TL;DR:** In our APIs, "characters" means "Unicode code points".
In API documentation (e.g., API reference documents, blog posts, marketing
documentation, billing explanations, etc), "character" **must** be defined as a
Unicode code point.
### Length units
**TL;DR:** Set size limits in "characters" (as defined above).
All string field length limits defined in API comments **must** be measured and
enforced in characters as defined above. This means that there is an underlying
maximum limit of (`4 * characters`) bytes, though this limit will only be hit
when using exclusively characters that consist of 4 UTF-8 code units (32 bits).
If you use a database system (e.g. Spanner) which allows you to define a limit
in characters, it is safe to assume that this byte-defined requirement is
handled by the underlying storage system.
### Billing units
APIs **may** use either code points or bytes (using the UTF-8 encoding) as the
unit for billing or quota measurement (e.g., Cloud Translation chooses to use
characters). If an API does not define this, the assumption is that the unit of
billing is characters (e.g., $0.01 _per character_, not $0.01 _per byte_).
### Unique identifiers
**TL;DR:** Unique identifiers **should** limit to ASCII, generally only
letters, numbers, hyphens, and underscores, and **should not** start with a
number.
Strings used as unique identifiers **should** limit inputs to ASCII characters,
typically letters, numbers, hyphens, and underscores
(`[a-zA-Z][a-zA-Z0-9_-]*`). This ensures that there are never accidental
collisions due to normalization. If an API decides to allow all valid Unicode
characters in unique identifiers, the API **must** reject any inputs that are
not in Normalization Form C. Generally, unique identifiers **should not** start
with a number as that prefix is reserved for Google-generated identifiers and
gives us an easy way to check whether we generated a unique numeric ID for or
whether the ID was chosen by a user.
Unique identifiers **should** use a maximum length of 64 characters, though
this limit may be expanded as necessary. 64 characters should be sufficient for
most purposes as even UUIDs only require 36 characters.
**Note:** See AIP-122 for recommendations about resource ID segments.
### Normalization
**TL;DR:** Unicode values **should** be stored in [Normalization Form C][].
Values **should** always be normalized into Normalization Form C. Unique
identifiers **must** always be stored in Normalization Form C (see the next
section).
Imagine we're dealing with Spanish input "estar<b>é</b>" (the accented part
will be bolded throughout). This text has what we might visualize as 6
"characters" (in this case, they are grapheme clusters). It has two possible
Unicode representations:
- Using 6 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072`
**`U+00E9`**
- Using 7 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072` **`U+0065`
`U+0301`**
Further, when encoding to UTF-8, these code points have two different
serialized representations:
- Using 7 code-units (7 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0xC3`
`0xA9`**
- Using 8 code-units (8 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0x65`
`0xCC` `0x81`**
To avoid this discrepancy in size (both code units and code points), use
[Normalization Form C][] which provides a canonical representation for strings.
[normalization form c]: https://unicode.org/reports/tr15/
### Uniqueness
**TL;DR:** Unicode values **must** be normalized to [Normalization Form C][]
before checking uniqueness.
For the purposes of unique identification (e.g., `name`, `id`, or `parent`),
the value **must** be normalized into [Normalization Form C][] (which happens
to be the most compact). Otherwise we may have what is essentially "the same
string" used to identify two entirely different resources.
In our example above, there are two ways of representing what is essentially
the same text. This raises the question about whether the two representations
should be treated as equivalent or not. In other words, if someone were to use
both of those byte sequences in a string field that acts as a unique
identifier, would it violate a uniqueness constraint?
The W3C recommends using Normalization Form C for all content moving across the
internet. It is the most compact normalized form on Unicode text, and avoids
most interoperability problems. If we were to treat two Unicode byte sequences
as different when they have the same representation in NFC, we'd be required to
reply to possible "Get" requests with content that is **not** in normalized
form. Since that is definitely unacceptable, we **must** treat the two as
identical by transforming any incoming string data into Normalized Form C or
rejecting identifiers not in the normalized form.
There is some debate about whether we should view strings as sequences of code
points represented as bytes (leading to uniqueness determined based on the
byte-representation of said string) or to interpret strings as a higher level
abstraction having many different possible byte-representations. The stance
taken here is that we already have a field type for handling that: `bytes`.
Fields of type `string` already express an opinion of the validity of an input
(it must be valid UTF-8). As a result, treating two inputs that have identical
normalized forms as different due to their underlying byte representation seems
to go against the original intent of the `string` type. This distinction
typically doesn't matter for strings that are opaque to our services (e.g.,
`description` or `display_name`), however when we rely on strings to uniquely
identify resources, we are forced to take a stance.
Put differently, our goal is to allow someone with text in any encoding (ASCII,
UTF-16, UTF-32, etc) to interact with our APIs without a lot of "gotchas".
## References
- [Unicode normalization forms](https://unicode.org/reports/tr15/)
- [Datastore pricing "name and value of each property"](https://cloud.google.com/datastore/pricing)
doesn't clarify this.
- [Natural Language pricing](https://cloud.google.com/natural-language/pricing)
uses charges based on UTF-8 code points rather than code units.
- [Text matching and normalization](https://sites.google.com/a/google.com/intl-eng/apis/matching?pli=1)
>>>> 0211.md
---
id: 211
state: approved
created: 2021-02-24
placement:
category: design-patterns
order: 115
---
# Authorization checks
The majority of operations, whether reads or writes, require authorization:
permission to do the thing the user is asking to do. Additionally, it is
important to be careful how much information is provided to _unauthorized_
users, since leaking information can be a security concern.
## Guidance
Services **must** check authorization before validating any request, to ensure
both a secure API surface and a consistent user experience. An operation
**may** require multiple permissions or preconditions in order to grant
authorization.
If a request can not pass the authorization check for any reason, the service
**must** error with `PERMISSION_DENIED`, and the corresponding error message
**should** look like: "Permission '`{p}`' denied on resource '`{r}`' (or it
might not exist)." This avoids leaking resource existence.
If it is not possible to determine authorization for a resource because the
resource does not exist, the service **should** check authorization to read
children on the parent resource, and return `NOT_FOUND` if the authorization
check passes.
### Multiple operations
A service could encounter a situation where it has two different operations
with two different permissions, either of which would reveal the existence of a
resource if called, but a user only has permission to call one of them.
In this situation, the service **should** still only check for authorization
applicable to the operation being called, and **should not** try to "help out"
by checking for related authorization that would provide permission to reveal
existence, because such algorithms are complicated to implement correctly and
prone to accidental leaks.
For example, posit a scenario where:
- A resource exists within a given collection that a user is unable to read.
- The user _does_ have the ability to create other resources, and the
collection uses user-specified IDs (meaning that a failure because of a
duplicate ID would reveal existence).
In this situation, the get or create methods **should** still only check
_their_ permissions when determining what error to return, and not one
another's.
## Rationale
[RFC 7231 §6.5.3][] states that services are permitted to use `404 Not Found`
in lieu of `403 Forbidden` in situations where the service does not want to
divulge existence, whereas this AIP argues for the use of `PERMISSION_DENIED`
(which corresponds to `403 Forbidden` in HTTP) instead. We take this position
for the following reasons:
- The practice of "getting `404 Not Found` until you have enough permission to
get `403 Forbidden`" is counter-intuitive and increases the difficulty of
troubleshooting.
- A service _could_ ameliorate this by sending information about missing
permissions while still using the `404 Not Found` status code, but this
constitutes a mixed message.
- While `403 Forbidden` is essentially always an error requiring manual action,
`404 Not Found` is often a valid response that the application can handle
(e.g. "get or create"); overloading it for permission errors deprives
applications of this benefit.
- RFC 7231 §6.5.4 states that `404 Not Found` results are cacheable, but
permission errors are not generally cacheable. Sending explicit cache
controls on a conditional basis could ameliorate this, but would defeat the
purpose.
- The guidance here is more consistent with most other real-world authorization
systems.
[rfc 7231 §6.5.3]: https://tools.ietf.org/html/rfc7231#section-6.5.3
## Changelog
- **2021-05-11:** Added a more detailed "Rationale" section.
>>>> 0213.md
---
id: 213
state: approved
created: 2018-08-17
placement:
category: protobuf
order: 20
---
# Common components
As specified in [AIP-215][], APIs must be self-contained except for the use
of "common component" packages which are intended for use by multiple APIs.
There are two kinds of common component packages:
- Organization-specific common components, covering organization-specific
concepts such as a "viewport" in Maps.
- Global common components which are generic (i.e. not domain-specific),
such as "timestamp" or "postal address".
Where it is safe to share a single representation across multiple APIs,
common components can make it easier for clients to interact with those APIs.
Concept-specific client code can be written once, and messages can be used from
the response of one API in the request of another without clunky copying, for
example.
This benefit comes with significant restrictions and limitations, however,
and should not be attempted lightly.
Note that even if the *domain* of a component is common, the requirements of
a component may be organization-specific. For example, some organizations may
have particular requirements of how financial values are represented, leading
to multiple finance-oriented organization-specific common components - because
any global common component would either not meet the organization-specific
requirements, or be too complex for general use.
## Guidance
- Organization-wide common component packages **must** end with `.type`, e.g.
`google.geo.type` or `google.shopping.type`.
- Organizations **must** consult the API design team before creating a new
organization-wide common component package.
- Organization-wide common component packages **must** be published in [the
`googleapis` repository](https://github.com/googleapis/googleapis).
- Organizations creating and publishing a new organization-wide common component
package **must** update this AIP to include it in the
[list below](#existing-organization-specific-common-component-packages).
- Organizations **must not** define generic components in organization-specific
common component packages, instead preferring global common components.
- Common components **must not** be "moved" (that is, deleted from one common
component package and added to a different one) from an organization-specific
common component package to a global common component package or vice versa.
- A common component **may** be copied from an organization-specific common
component package to a global common component package (without deleting the
original component) if it is found to be more widely-applicable than
originally expected.
- Fields **should not** be added to existing messages.
- Values **should not** be added to existing enums.
- Fields **must not** be removed from existing messages.
- Values **must not** be removed from existing enums.
- While documentation **may** be clarified, it **should not** change the
meanings of existing values, including the validity of any given message
or set of messages.
- New proto messages and enums **may** be added to common component packages.
- API teams **should** allow sufficient time for propagation to clients
before using the new messages and enums in their APIs. Fields may take some
time for any changes to propagate through publication to client libraries
and other surfaces.
- API teams **should** consult widely within their organization, and ideally
with the API design team, before adding a new message or enum, due to the
limitations listed above.
## Existing global common components
The global common components, which public-facing protos for an API **may** safely
import, are as follows:
- [`google.api.*`](https://github.com/googleapis/googleapis/blob/master/google/api) (but *not* subpackages of `google.api`)
- [`google.longrunning.Operation`](https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto)
- [`google.protobuf.*`](https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf)
- [`google.rpc.*`](https://github.com/googleapis/googleapis/blob/master/google/rpc/)
- [`google.type.*`][type]
Note that some common components may have internal-only fields. APIs **should**
generally only rely on fields which have been
[released into open source](https://github.com/googleapis/googleapis).
Google APIs **may** also import [`google.iam.v1.*`][iam], which provides the
IAM messages used throughout Google.
<!-- prettier-ignore -->
[iam]: https://github.com/googleapis/googleapis/tree/master/google/iam/v1
**Note:** Many APIs also import components from other packages for internal-only
use (e.g. to apply visibility labels or provide instructions to internal
infrastructure). This is acceptable provided that the _public_ components do not
contain such references.
### Protobuf types
The `google.protobuf` package is somewhat special in that it is shipped with
protocol buffers itself, rather than with API tooling. (For most API designers,
this should be an implementation detail).
This package includes a small library of types useful for representing common
programming language constructs:
- [`google.protobuf.Duration`][duration]: Durations, with nanosecond-level
precision. The protobuf runtime provides helper functions to convert to and
from language-native duration objects where applicable (such as Python's
[`timedelta`][timedelta]).
- [`google.protobuf.Struct`][struct]: JSON-like structures (a dictionary of
primitives, lists, and other dictionaries). The protobuf runtime provides
helper functions in most languages to convert struct objects to and from
JSON.
- [`google.protobuf.Timestamp`][timestamp]: Timestamps, with nanosecond-level
precision. The protobuf runtime provides helper functions in most languages
to convert to and from language-native timestamp objects (such as Python's
[`datetime`][datetime]).
<!-- prettier-ignore-start -->
[datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[struct]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/struct.proto
[timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
### API Types
The `google.type` package provides a "standard library" of types useful for
representing common concepts in APIs. While types are added from time to time
and the definitive list is always [the code][type], several types deserve note:
- [`google.type.Color`][color]: RGB or RGBA colors.
- [`google.type.Date`][date]: Calendar dates, with no time or time zone
component.
- [`google.type.DayOfWeek`][day_of_week]: The day of the week, with no other
date, time, or time zone component.
- [`google.type.LatLng`][lat_lng]: Geographic coordinates.
- [`google.type.Money`][money]: Currency.
- [`google.type.PostalAddress`][postal_address]: Postal addresses in most
countries.
- [`google.type.TimeOfDay`][time_of_day]: Wall-clock time, with no date or time
zone component.
<!-- prettier-ignore-start -->
[type]: https://github.com/googleapis/googleapis/tree/master/google/type
[color]: https://github.com/googleapis/googleapis/blob/master/google/type/color.proto
[date]: https://github.com/googleapis/googleapis/blob/master/google/type/date.proto
[day_of_week]: https://github.com/googleapis/googleapis/blob/master/google/type/dayofweek.proto
[lat_lng]: https://github.com/googleapis/googleapis/blob/master/google/type/latlng.proto
[money]: https://github.com/googleapis/googleapis/blob/master/google/type/money.proto
[postal_address]: https://github.com/googleapis/googleapis/blob/master/google/type/postal_address.proto
[time_of_day]: https://github.com/googleapis/googleapis/blob/master/google/type/timeofday.proto
<!-- prettier-ignore-end -->
### Adding to common protos
Occasionally, it may be useful to add protos to these packages or to add to the
list of commonly-available protos. In order to do this, [open an issue][] on
the AIP repository in GitHub, noting the guidelines above.
## Existing organization-specific common component packages
The following organization-specific common component packages exist and conform with the above guidance:
- [`google.apps.script.type`](https://github.com/googleapis/googleapis/tree/master/google/apps/script/type)
Common component package for Google Apps Script.
- [`google.geo.type`](https://github.com/googleapis/googleapis/tree/master/google/geo/type)
Common component package for Google Maps and the Geo organization.
- [`google.actions.type`](https://github.com/googleapis/googleapis/tree/master/google/actions/type)
Common component package for Actions on Google APIs.
## Non-conformant common component packages
The following common component packages exist, but do not conform with the above guidance,
and do not form a precedent for further such packages.
- [`google.cloud.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/common):
This does not conform to the requirement for the package name to end in `.type`. (This would otherwise
be acceptable, and this package should be considered as the Cloud common component package.)
- [`google.logging.type`](https://github.com/googleapis/googleapis/tree/master/google/logging/type):
This appears to be API-specific, although it's used from multiple APIs; some aspects should probably
be global or in a Cloud common component package.
- [`google.cloud.workflows.type`](https://github.com/googleapis/googleapis/tree/master/google/cloud/workflows/type):
API-specific types.
- [`google.cloud.oslogin.common`](https://github.com/googleapis/googleapis/tree/master/google/cloud/oslogin/common):
API-specific types, and a non-conformant name.
- [`google.identity.accesscontextmanager.type`](https://github.com/googleapis/googleapis/tree/master/google/identity/accesscontextmanager/type):
API-specific types.
- [`google.networking.trafficdirector.type`](https://github.com/googleapis/googleapis/tree/master/google/networking/trafficdirector/type):
API-specific types.
## Rationale
Common components are effectively unversioned: APIs evolve independently of
each other, both in terms of definition and implementation. A change such as
adding a field is backward-compatible and predictable in specific APIs, and the
API team can ensure that the server implementation is available before the API
definition is published. By contrast, a change in a common component would
effectively be universally available even if most API implementations did not
take it into account.
Adding a new message or enum is backward-compatible, as it does not affect
existing APIs that may import other messages or enums from the same common
component package.
Consultation with the API design team is required for global common components
and suggested for organization-specific common components as the border between
"generic" and "organization-specific" is a gray area; some generic *concepts*
have organization-specific use cases which surface through the components.
## Changelog
- **2023-06-27**: Restructured AIPs 215 and 213 for clarity, and introduced the
concept of organization-wide common protos more formally.
- **2018-08-17**: Initial AIP written.
[open an issue]: https://github.com/googleapis/aip/issues
[aip-215]: ./0215.md
>>>> 0214.md
---
id: 214
state: approved
created: 2018-06-19
placement:
category: design-patterns
order: 120
---
# Resource expiration
Customers often want to provide the time that a given resource or resource
attribute is no longer useful or valid (e.g. a rotating security key). Currently
we recommend that customers do this by specifying an exact "expiration time"
into a `google.protobuf.Timestamp expire_time` field; however, this adds
additional strain on the user when they want to specify a relative time offset
until expiration rather than a specific time until expiration.
Furthermore, the world understands the concept of a "time-to-live", often
abbreviated to TTL, but the typical format of this field (an integer, measured
in seconds) results in a sub-par experience when using an auto-generated client
library.
## Guidance
1. APIs wishing to convey an expiration **must** rely on a
[`google.protobuf.Timestamp`][timestamp] field called `expire_time`.
2. APIs wishing to allow a relative expiration time **must** define a `oneof`
called `expiration` (or `{something}_expiration`) containing both the
`expire_time` field and a separate [`google.protobuf.Duration`][duration]
field called `ttl`, the latter marked as input only.
3. APIs **must** always return the expiration time in the `expire_time` field
and leave the `ttl` field blank when retrieving the resource.
4. APIs that rely on the specific semantics of a "time to live" (e.g., DNS
which must represent the TTL as an integer) **may** use an `int64 ttl`
field (and **should** provide an [aip.dev/not-precedent](./0200.md) comment
in this case).
### Example
```proto
message ExpiringResource {
// google.api.resource and other annotations and fields
oneof expiration {
// Timestamp in UTC of when this resource is considered expired.
// This is *always* provided on output, regardless of what was sent
// on input.
google.protobuf.Timestamp expire_time = 2;
// Input only. The TTL for this resource.
google.protobuf.Duration ttl = 3 [(google.api.field_behavior) = INPUT_ONLY];
}
}
```
## Rationale
### Alternatives considered
#### A new standard field called `ttl`
We considered allowing a standard field called `ttl` as an alternative way of
defining the expiration, however doing so would require that API services
continually update the field, like a clock counting down. This could
potentially cause problems with the read-modify-write lifecycle where a
resource is being processed for some time, and effectively has its life
extended as a result of that processing time.
#### Always use `expire_time`
This is the current state of the world with a few exceptions. In this scenario,
we could potentially push the computation of `now + ttl = expire_time` into
client libraries; however, this leads to a somewhat frustrating experience in
the command-line and using REST/JSON. Leaving things as they are is typically
the default, but it seems many customers want the ability to define relative
expiration times as it is quite a bit easier and removes questions of time
zones, stale clocks, and other silly mistakes.
<!-- prettier-ignore-start -->
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
<!-- prettier-ignore-end -->
>>>> 0215.md
---
id: 215
state: approved
created: 2018-10-01
placement:
category: protobuf
order: 30
---
# API-specific protos
APIs are mostly defined in terms of protos which are API-specific, with
occasional dependencies on common components. Keeping APIs isolated from each
other avoids versioning problems and client library packaging problems.
## Guidance
- All protos specific to an API **must** be within a package with a major version
(e.g., `google.library.v1`).
- References to resources in other APIs **must** be expressed in terms of
resource names ([AIP-122][]), rather than using the resource messages.
- When two versions of an API use effectively the same (API-specific) proto
that proto **must** be duplicated in each version. (In other words, APIs
**must not** create their own "API-specific common component" packages.)
- Organization-specific common components **may** be placed in a common package,
as described in [AIP-213][], but **must not** be used by any API outside
that organization.
- Global common components (also described in AIP-213) **may** be freely used by any API.
## Rationale
When one API depends on protos defined by another API, this introduces uncertainty
in terms of customer-expected behavior and client library dependency management.
Suppose `google.cloud.library.v1` depends on the protos (rather than abstract resources)
in `google.cloud.movies.v2`. Any change to `google.cloud.movies.v2` can cause problems.
For example:
- If a field is added to a message in `google.cloud.movies.v2`, should customers using
`google.cloud.library.v1` expect to see it? If so, how soon after the field has
been added? What about other API changes?
- If the whole major version `google.cloud.movies.v2` is deprecated
(typically after v3 has been released), does that mean `google.cloud.library.v1` has
to change to use `google.cloud.movies.v3`, and if so, does that require a new major version
for the library API as well?
- How should client library versioning reflect changes to dependent APIs?
Keeping APIs isolated from each other, with a limited set of common components which are
maintained in a highly disciplined way, reduces a lot of the issues with dependencies.
API-specific common components shared across versions add complexity for client
library generation and packaging, and are inflexible in terms of versioning.
When protos are duplicated because they *start* off the same in multiple versions,
they can still diverge over time as they are isolated from each other.
## Changelog
- **2023-06-27**: Restructured AIPs 215 and 213 for clarity.
- **2023-05-11**: Changed "PA" to "organization".
- **2018-10-01**: Initial AIP written.
[aip-213]: ./0213.md
[aip-122]: ./0122.md
>>>> 0216.md
---
id: 216
state: approved
created: 2018-10-01
updated: 2022-06-02
placement:
category: fields
order: 110
---
# States
Many API resources carry a concept of "state": ordinarily, the resource's place
in its life cycle. For example, a virtual machine may be being provisioned,
available for use, being spun down, or potentially be in one of several other
situations. A job or query may be preparing to run, be actively running, have
completed, and so on.
## Guidance
Resources needing to communicate their state **should** use an enum, which
**should** be called `State` (or, if more specificity is required, end in the
word `State`). This enum **should** be nested within the message it describes
when only used as a field within that message.
**Important:** We use the term `State`, and _not_ `Status` (which is reserved
for the HTTP and gRPC statuses).
### Enum values
Ideally, Google APIs use the same terminology throughout when expressing the
same semantic concepts. There are usually many words available to express a
given state, but our customers often use multiple APIs together, and it is
easier for them when our terms are consistent.
At a high level:
- Resources that are available for use are `ACTIVE` (preferred over terms such
as "ready" or "available").
- Resources that have completed a (usually terminal) requested action use past
participles (usually ending in `-ED`), such as `SUCCEEDED` (not
"successful"), `FAILED` (not "failure"), `DELETED`, `SUSPENDED`, and so on.
- Resources that are currently undergoing a state change use present
participles (usually ending in `-ING`), such as `RUNNING`, `CREATING`,
`DELETING`, and so on. In this case, it is expected that the state is
temporary and will resolve to another state on its own, with no further user
action.
**Note:** Remember to only add states that are useful to customers. Exposing a
large number of states simply because they exist in your internal system is
unnecessary and adds confusion for customers. Each state must come with a use
case for why it is necessary.
### Output only
The field referencing the `State` enum in a resource **should** behave and be
documented as "Output only", in accordance with [AIP-203](./0203.md).
APIs **should not** allow a `State` enum to be directly updated through an
"update" method (or directly set through the "create" method), and **should**
instead use custom state transition methods.
This is because update methods are generally not expected to have side effects,
and also because updating state directly implies that it is possible to set the
state to any available value, whereas states generally reflect a resource's
progression through a lifecycle.
### State transition methods
State transition methods are a special type of custom method that are
responsible for transitioning a state field from one enum value to another. As
part of the transition, other fields may also change, e.g. an `update_time`
field. The method definition should look like the following:
```proto
// Publishes a book.
// The `state` of the book after publishing is `PUBLISHED`.
// `PublishBook` can be called on Books in the state `DRAFT`; Books in a
// different state (including `PUBLISHED`) returns an error.
rpc PublishBook(PublishBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{name=publishers/*/books/*}:publish"
body: "*"
};
}
```
- The name of the method **should** be a verb followed by the singular form of
the resource's message name.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be the resource itself.
- If the RPC is [long-running][], the response message **should** be a
`google.longrunning.Operation` which resolves to the resource itself.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** use a `:` character followed by the custom verb
(`:publish` in the above example), and the verb in the URI **must** match the
verb in the name of the RPC.
- If word separation is required, `camelCase` **must** be used.
- The `body` clause in the `google.api.http` annotation **must** be `"*"`.
- The request message field receiving the resource name **should** map to the
URI path.
- This field **should** be called `name`.
- The `name` field **should** be the only variable in the URI path. All
remaining parameters **should** map to URI query parameters.
- If the state transition is not allowed, the service **must** error with
`FAILED_PRECONDITION` (HTTP 400).
The request message should look like this:
```proto
message PublishBookRequest {
// The name of the book to publish.
// Format: publishers/{publisher}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A resource name field **must** be included. It **should** be called `name`.
- The comment for the field **should** document the resource pattern.
- Other fields **may** be included.
## Additional Guidance
### Default value
The zero value of each state enum **should** adhere to the following
convention:
```proto
enum State {
// The default value. This value is used if the state is omitted.
STATE_UNSPECIFIED = 0;
// Other values...
}
```
Resources **should not** provide an unspecified state to users, and this value
**should not** actually be used.
### Value uniqueness
Multiple top-level enums within the same package **must** not share the same
values. This is because the C++ protoc code generator flattens top-level enum
values into a single namespace.
State enums **should** live inside the resource definition.
### Prefixes
Using a `STATE_` prefix on every enum value is unnecessary. State enum values
**should not** be prefixed with the enum name, except for the default value
`STATE_UNSPECIFIED`.
### Breaking changes
**TL;DR:** Clearly communicate to users that state enums may receive new values
in the future, and be conscientious about adding states to an existing enum.
Even though adding states to an existing states enum _can_ break existing user
code, adding states is not considered a breaking change. Consider a state with
only two values: `ACTIVE` and `DELETED`. A user may add code that checks
`if state == ACTIVE`, and in the else cases simply assumes the resource is
deleted. If the API later adds a new state for another purpose, that code will
break.
We ultimately can not control this behavior, but API documentation **should**
actively encourage users to code against state enums with the expectation that
they may receive new values in the future.
APIs **may** add new states to an existing State enum when appropriate, and
adding a new state is _not_ considered a breaking change.
### When to avoid states
Sometimes, a `State` enum may not be what is best for your API, particularly in
situations where a state has a very small number of potential values, or when
states are not mutually exclusive.
Consider the example of a state with only `ACTIVE` and `DELETED`, as discussed
above. In this situation, the API may be better off exposing a
`google.protobuf.Timestamp delete_time`, and instructing users to rely on
whether it is set to determine deletion.
### Common states
The following is a list of states in common use. APIs **should** consider prior
art when determining state names, and **should** value local consistency above
global consistency in the case of conflicting precedent.
#### Resting states
"Resting states" are lifecycle states that, absent user action, are expected to
remain indefinitely. However, the user can initiate an action to move a
resource in a resting state into certain other states (resting or active).
- `ACCEPTED`
- `ACTIVE`
- `CANCELLED`
- `DELETED`
- `FAILED`
- `SUCCEEDED`
- `SUSPENDED`
- `VERIFIED`
#### Active states
"Active states" are lifecycle states that typically resolve on their own into a
single expected resting state.
**Note:** Remember only to expose states that are useful to customers. Active
states are valuable only if the resource will be in that state for a sufficient
period of time. If state changes are immediate, active states are not
necessary.
- `CREATING` (usually becomes `ACTIVE`)
- `DELETING` (usually becomes `DELETED`)
- `PENDING` (usually becomes `RUNNING`)
- `REPAIRING` (usually becomes `ACTIVE`)
- `RUNNING` (usually becomes `SUCCEEDED`)
- `SUSPENDING` (usually becomes `SUSPENDED`)
## Further reading
- For information on enums generally, see [AIP-126][].
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-10-20**: Added guidance on prefixing enum values with enum name.
- **2020-09-02**: Clarified that states are not directly set on create either.
- **2019-12-05**: Changed guidance on state transition methods, downgrading
**must** to **should** on the response type.
- **2019-08-16**: Added guidance for state transition methods.
- **2019-07-18**: Added explicit guidance on the unspecified value.
[aip-126]: ./0126.md
[long-running]: ./0151.md
>>>> 0217.md
---
id: 217
state: approved
created: 2019-08-26
placement:
category: design-patterns
order: 130
---
# Unreachable resources
Occasionally, a user may ask for a list of resources, and some set of resources
in the list are temporarily unavailable. The most typical use case is while
supporting [Reading Across Collections][aip-159]. For example, a user may ask to
list resources across multiple parent locations, but one of those locations is
temporarily unreachable. In this situation, it is still desirable to provide
the user with all the available resources, while indicating that something is
missing.
## Guidance
If a method to retrieve data is capable of partially failing due to one or more
resources being temporarily unreachable, the response message **must** include
a field to indicate this:
```proto
message ListBooksResponse {
// The books matching the request.
repeated Book books = 1;
// The next page token, if there are more books matching the
// request.
string next_page_token = 2;
// Unreachable resources.
repeated string unreachable = 3 [
(google.api.field_behavior) = UNORDERED_LIST
];
}
```
- The field **must** be a repeated string, and **should** be named
`unreachable`.
- The field **must** contain the resource names of the resources that are
unreachable or those that impede reaching the requested collection, such as
the parent resource of the collection that could not be reached.
- For example, if an entire location is unreachable, preventing access to the
localized collection of resources requested, the location resource is
included.
- The field **must** contain _service-relative_ resource names, and **must not**
contain full resource names, resource URIs, or simple resource IDs. See
[AIP-122][aip-122] for definitions.
- For example, if a `Book` resource is unreachable, the _service-relative_
resource name `"shelves/scifi1/books/starwars4"` is included in
`unreachable`, as opposed to the _full_ resource name
`"//library.googleapis.com/shelves/scifi1/books/starwars4"`, the
_parent-relative_ resource `"books/starwars4"`, the resource ID
`"starwars4"`, or the resource URI.
- The response **must not** provide any other information about the issue(s)
that made the listed resources unreachable.
- For example, the response cannot contain an extra field with error reasons
for each `unreachable` entry.
- The service **must** provide a way for the user to make a more specific
request and receive an error with additional information e.g. via a Standard
Get or a Standard List targeted at the unreachable collection parent.
- The service **must** also allow the user to repeat the original call with
more restrictive parameters.
- The resource names that appear in `unreachable` **may** be heterogeneous.
- The `unreachable` field definition **should** document what potential
resources could be provided in this field, and note that it might expand
later.
- For example, if both an entire location and a specific resource in a
different location are unreachable, the unreachable location's name
e.g. `"projects/example123/locations/us-east1"` and the unreachable
resource's name e.g.
`"projects/example123/locations/europe-west2/instances/example456"` will
both appear in `unreachable`.
- The `unreachable` field **must not** have semantically meaningful ordering or
structure within the list. Put differently, `unreachable` **must** be an
unordered list.
- As such, the `unreachable` field **must** be annotated with `UNORDERED_LIST`
field behavior (see [AIP-203][unordered]).
[aip-122]: ./0122.md
[aip-160]: ./0160.md
[unordered]: ./0203.md#unordered-list
**Important:** If a single unreachable location or resource prevents returning
any data by definition (for example, a list request for a single publisher
where that publisher is unreachable), the service **must** fail the entire
request with an error.
### Pagination
While preparing a page of results to fulfill a page fetch RPC e.g. an
[AIP-132][aip-132] Standard List call, if the service encounters any unreachable
resources or collections they **must** do the following:
- Include the resource name for the unreachable resource in the `unreachable`
response field.
- The resource name **must** be the most appropriately scoped for the
unreachable resource or collection.
- For example, if a specific zone within a region is unreachable, the
unreachable resource name would be a zonal Location e.g.
`projects/example/locations/us-west1-a`, but if an entire region is
unreachable, the resource name would be a regional Location e.g.
`projects/example/locations/us-west1`.
- The resource name **must** be included, regardless of restrictive paging
parameters e.g. `order_by`, when it is identified as unreachable.
- Populate results that were previously considered unreachable on a following
page if their availability is restored and the paging parameters allow for
their inclusion.
- Determining inclusion eligibility based on paging parameters also includes
any documented default ordering behavior in the absence of user-specified
ordering in the request.
- For example, if region `projects/example/locations/us-west1` was unavailable
in the first page of an ordered paging call, and including its resources
would violate the ordering, those out-of-order resources are not included in
the following page.
- Similarly, if the same exact request is made, and resources previously
considered unreachable are available again, they **must** be populated,
within the constraints of the paging parameters.
- Limit the number of unreachable resource names returned in a given response
if, even after up-scoping the unreachable resource name, the number of
unreachable resource names exceeds a documented maximum.
- This maximum **must** be documented in the `unreachable` field comments
directly.
- This is independent of the `page_size` set by the caller.
#### Retaining previous behavior
Services **may** continue with previously implemented `unreachable` pagination
behavior where changing it would induce an incompatible change as per
[AIP-180][aip-180], but **must** document said behavior on the `unreachable`
field(s) directly.
[aip-180]: ./0180.md
### Adopting partial success
In order for an existing API that has a default behavior *differing* from the
aforementioned guidance i.e. the API call returns an error status instead of a
partial result, to adopt the `unreachable` pattern the API **must** do the
following:
- The default behavior **must** be retained to avoid incompatible behavioral
changes
- For example, if the default behavior is to return an error if any location
is unreachable, that default behavior **must** be retained.
- The request message **must** have a `bool return_partial_success` field
- The response message **must** have the standard
`repeated string unreachable` field
- The two aforementioned fields **must** be added simultaneously
When the `bool return_partial_success` field is set to `true` in a request, the
API **must** behave as described in the aforementioned guidance with regards to
populating the `repeated string unreachable` response field.
```proto
message ListBooksRequest {
// Standard List request fields...
// Setting this field to `true` will opt the request into returning the
// resources that are reachable, and into including the names of those that
// were unreachable in the [ListBooksResponse.unreachable] field. This can
// only be `true` when reading across collections e.g. when `parent` is set to
// `"projects/example/locations/-"`.
bool return_partial_success = 4;
}
message ListBooksResponse {
// Standard List Response fields...
// Unreachable resources. Populated when the request opts into
// `return_partial_success` and reading across collections e.g. when
// attempting to list all resources across all supported locations.
repeated string unreachable = 3 [
(google.api.field_behavior) = UNORDERED_LIST
];
}
```
#### Partial success granularity
If the `bool return_partial_success` field is set to `true` in a request that is
scoped beyond the supported granualirty of the API's ability to reasonably
report unreachable resources, the API **should** return an `INVALID_ARGUMENT`
error with details explaining the issue. For example, if the API only supports
`return_partial_success` when [Reading Across Collections][aip159], it returns
an `INVALID_ARGUMENT` error when given a request scoped to a specific parent
resource collection. The supported granularity **must** be documented on the
`return_partial_success` field.
## Rationale
### Using service-relative resource names
In general, relative resource names, as defined in AIP-122, are the best
practice for referring to resources by name _within_ a service and in other
services when that other service is obvious. The full resource name format is
strictly less consumable (e.g., requires extra parsing client side), and
over-specified for the uses of `unreachable`. Resource URIs are not transport
agnostic, as they are unusable in standard methods for gRPC users, and simple
resource IDs do not provide enough information about exactly which resource
was unreachable in a heterogenous list of resources.
### Minimizing extra error details in response
The context in which an unreachable resource is discovered may be sensitive and
the state of the system fluid between calls. As such, it is preferred to defer
to the service by making a more specific RPC to get more details about a
specific resource or parent. This allows the parent to handle all necessary RPC
checks and system state resolution on at time of request, rather than by
shoehorning potentially privileged or stale information into the broader list
call it was unreachable for.
### Unordered `unreachable` contents
It is important for broad API consistency that the contents of `unreachable` not
have a specific or order semantic structure. If each API baked a specific
ordering into a standard field, no single implementation, client or server side,
would be correct.
### Per page `unreachable` resources
Populating `unreachable` resources on a per page basis allows end users to
identify immediately when a page is incomplete, rather than _after_ paging
through all results. Paging to completion is not guaranteed, so it is important
to communicate as soon as possible when there are unreachable resource missing
from a given page. Furthermore, it allows users to identify when there is a
potential issue that they need to account for in subsequent calls. Finally,
retaining unreachable resources until the end of paging results requires
services to retain the state for what should be indepedent and fully isolated
API calls.
### Using request field to opt-in
Introducing a new request field as means of opting into the partial success
behavior is the best way to communicate user intent while keeping the
default behavior backwards compatible. The alternative, changing the default
behavior with the introduction of the `unreachable` response field, presents
a backwards incompatible change. Users that previously expected failure when any
resource was unreachable, assume the successful response means all resources
are accounted for in the response.
### Introducing fields simultaneously
Introducing the request and response fields simultaneously is to prevent an
invalid intermediate state that is presented by only adding one or the other. If
only `unreachable` is added, then it could be assumed that it being empty means
all resources were returned when that may not be true. If only
`return_partial_success` is added, then the user wouldn't have a means of
knowing which resources were unreachable.
### Partial success granularity limitations
At a certain level of request scope granularity, an API is simply unable to
enumerate the resources that are unreachable. For example, global-only APIs may
be unable to provide granularity at a localized collection level. In such a
case, preemptively returning an error when `return_partial_success=true`
protects the user from the risks of the alternative - expecting unreachable
resources if there was an issue, but not getting any, thus falsely assuming
everything was retrieved. This aligns with guidance herein that suggests failing
requests that cannot be fulfilled preemptively.
## History
### Pagination guidance
The original guidance for how to populate the `unreachable` field revolved
around consuming the contents as if they were the paged results. This meant that
paged resources and unreachable resources couldn't be returned in the same
response i.e. page, and users needed to completely page through all results
in order to see if any were unreachable. See the Rationale section for the
reasoning around the changes.
## Further reading
- For listing across collections, see [AIP-159][].
## Changelog
- **2024-07-29**: Reformat guidance, add explicit resource name format
- **2024-07-26**: Change pagination guidance.
requirement.
- **2024-07-19**: Add guidance for brownfield adoption of partial success.
[aip-159]: ./0159.md
>>>> 0231.md
---
id: 231
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 200
---
# Batch methods: Get
Some APIs need to allow users to get a specific set of resources at a
consistent time point (e.g. using a read transaction). A batch get method
provides this functionality.
## Guidance
APIs **may** support Batch Get using the following pattern:
```proto
rpc BatchGetBooks(BatchGetBooksRequest) returns (BatchGetBooksResponse) {
option (google.api.http) = {
get: "/v1/{parent=publishers/*}/books:batchGet"
};
}
```
- The RPC's name **must** begin with `BatchGet`. The remainder of the RPC name
**should** be the plural form of the resource being retrieved.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- The HTTP verb **must** be `GET`.
- The HTTP URI **must** end with `:batchGet`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- There **must not** be a body key in the `google.api.http` annotation.
- The operation **must** be atomic: it **must** fail for all resources or
succeed for all resources (no partial success). For situations requiring
partial failures, `List` ([AIP-132][]) methods **should** be used.
- If the operation covers multiple locations and at least one location is
down, the operation **must** fail.
### Request message
The request for a batch get method **should** be specified with the following
pattern:
```proto
message BatchGetBooksRequest {
// The parent resource shared by all books being retrieved.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in `names`
// must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The names of the books to retrieve.
// A maximum of 1000 books can be retrieved in a batch.
// Format: publishers/{publisher}/books/{book}
repeated string names = 2 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `parent` field **should** be included, unless the resource being retrieved
is a top-level resource, to facilitate inclusion in the URI as
well to permit a single permissions check. If a caller sets this field, and
the parent collection in the name of any resource being retrieved does not
match, the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
resource names specifying the resources to retrieve. The field **should** be
named `names`.
- If no resource names are provided, the API **should** error with
`INVALID_ARGUMENT`.
- The field **should** be required.
- The field **should** identify the [resource type][aip-122-names] that it
references.
- The comment for the field **should** document the resource pattern.
- Other fields besides `name` **may** be "hoisted" from the [standard Get
request][request-message]. There is no way to allow for these fields to
accept different values for different resources; if this is needed, use the
[alternative request message form](#nested-request-objects).
- Batch get **should not** support pagination because transactionality across
API calls would be extremely difficult to implement or enforce, and the
request defines the exact scope of the response anyway.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `names` field **should** document the maximum number of
requests allowed.
### Response message
The response for a batch get method **should** be specified with the following
pattern:
```proto
message BatchGetBooksResponse {
// Books requested.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources being retrieved.
- The order of books in the response **must** be the same as the names in the
request.
[aip-122-names]: ./0122.md#fields-representing-resource-names
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0131.md#request-message
### Nested request objects
If the [standard Get request message][request-message] contains a field besides
the resource name that needs to be different between different resources being
requested, the batch message **may** alternatively hold a `repeated` field of
the [standard Get request message][request-message]. This is generally
discouraged unless your use case really requires it.
The request for a batch get method using this approach **should** be specified
with the following pattern:
```proto
message BatchGetBooksRequest {
// The parent resource shared by all books being retrieved.
// Format: publishers/{publisher}
// If this is set, the parent field in the GetBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The requests specifying the books to retrieve.
// A maximum of 1000 books can be retrieved in a batch.
repeated GetBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included. If a caller sets this field, and the
parent collection in the name of any resource being retrieved does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to retrieve, as specified for
[standard Get methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Get
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Batch get **should not** support pagination because transactionality across
API calls would be extremely difficult to implement or enforce, and the
request defines the exact scope of the response anyway.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
[aip-132]: https://aip.dev/132
## Changelog
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2020-03-24**: Clarified behavior if no resource names are sent.
- **2019-09-11**: Changed the primary recommendation to specify a repeated
string instead of a repeated standard Get request message. Moved the original
recommendation into its own section.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0233.md
---
id: 233
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 210
---
# Batch methods: Create
Some APIs need to allow users to create multiple resources in a single
transaction. A batch create method provides this functionality.
## Guidance
APIs **may** support Batch Create using the following two patterns:
Returning the response synchronously
```proto
rpc BatchCreateBooks(BatchCreateBooksRequest) returns (BatchCreateBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchCreate"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchCreateBooks(BatchCreateBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchCreate"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "BatchCreateBooksResponse"
metadata_type: "BatchCreateBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchCreate`. The remainder of the RPC
name **should** be the plural form of the resource being created.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** end with `:batchCreate`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch create method **may** support atomic (all resources created or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch create **must** be atomic.
- Asynchronous batch create **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch create method **should** be specified with the
following pattern:
```proto
message BatchCreateBooksRequest {
// The parent resource shared by all books being created.
// Format: publishers/{publisher}
// If this is set, the parent field in the CreateBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The request message specifying the resources to create.
// A maximum of 1000 books can be created in a batch.
repeated CreateBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included, unless the resource being created is
a top-level resource. If a caller sets this field, and the
`parent` field of any child request message does not match, the request
**must** fail. The `parent` field of child request messages can be omitted if
the `parent` field in this request is set.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to create, as specified for
[standard Create methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Create
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Fields which must be unique cannot be hoisted (e.g. Customer-provided id
fields).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
### Response message
The response for a batch create method **should** be specified with the
following pattern:
```proto
message BatchCreateBooksResponse {
// Books created.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were created.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch create method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field in
the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Create method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchCreateBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources created.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
create request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0133.md#request-message
## Changelog
- **2025-03-06**: Added detailed guidance for partial success behavior, and
decision framework for choosing between atomic and partial success
- **2023-04-18**: Changed the recommendation to allow returning partial
successes.
- **2022-06-02**: Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent` and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0234.md
---
id: 234
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 220
---
# Batch methods: Update
Some APIs need to allow users to modify a set of resources in a single
transaction. A batch update method provides this functionality.
## Guidance
APIs **may** support Batch Update using the following two patterns:
Returning the response synchronously
```proto
rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (BatchUpdateBooksResponse) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchUpdate"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchUpdateBooks(BatchUpdateBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchUpdate"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "BatchUpdateBooksResponse"
metadata_type: "BatchUpdateBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchUpdate`. The remainder of the RPC
name **should** be the plural form of the resource being updated.
- The request and response messages **must** match the RPC name, with
`Request` and `Response` suffixes.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- The HTTP verb **must** be `POST`.
- The HTTP URI **must** end with `:batchUpdate`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch update method **may** support atomic (all resources updated or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch update **must** be atomic.
- Asynchronous batch update **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch update method **should** be specified with the
following pattern:
```proto
message BatchUpdateBooksRequest {
// The parent resource shared by all books being updated.
// Format: publishers/{publisher}
// If this is set, the parent field in the UpdateBookRequest messages
// must either be empty or match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The request message specifying the resources to update.
// A maximum of 1000 books can be modified in a batch.
repeated UpdateBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included, unless the resource being updated is
a top-level resource. If a caller sets this field, and the
parent collection in the name of any resource being updated does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to update, as specified for
[standard Update methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Update
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- The `update_mask` field is a good candidate for hoisting.
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number of
requests allowed.
### Response message
The response for a batch update method **should** be specified with the
following pattern:
```proto
message BatchUpdateBooksResponse {
// Books updated.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were updated.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch update method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field
in the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Update method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchUpdateBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources updated.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
update request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[request-message]: ./0134.md#request-message
## Changelog
- **2025-03-06**: Changed recommendation to allow partial success, along with
detailed guidance
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent` and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2019-09-11**: Fixed the wording about which child field the `parent` field
should match.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0235.md
---
id: 235
state: approved
created: 2019-06-18
updated: 2022-06-02
placement:
category: operations
order: 230
---
# Batch methods: Delete
Some APIs need to allow users to delete a set of resources in a single
transaction. A batch delete method provides this functionality.
## Guidance
APIs **may** support Batch Delete using the following two patterns:
Returning the response synchronously
```proto
rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.protobuf.Empty) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchDelete"
body: "*"
};
}
```
Returning an Operation which resolves to the response asynchronously
```proto
rpc BatchDeleteBooks(BatchDeleteBooksRequest) returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books:batchDelete"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "BatchDeleteBooksOperationMetadata"
};
}
```
- The RPC's name **must** begin with `BatchDelete`. The remainder of the RPC
name **should** be the plural form of the resource being deleted.
- The request message **must** match the RPC name, with a `Request` suffix.
- The response message **should** be `google.protobuf.Empty`.
- If the resource is [soft deleted][soft-delete], the response message
**should** be a response message containing the updated resources.
- If the batch method returns an `google.longrunning.Operation`, both the
`response_type` and `metadata_type` fields **must** be specified.
- If the resource is [soft deleted][soft-delete], the `response_type`
**should** be a response message containing the updated resources.
- The HTTP verb **must** be `POST` (not `DELETE`).
- The HTTP URI **must** end with `:batchDelete`.
- The URI path **should** represent the collection for the resource, matching
the collection used for simple CRUD operations. If the operation spans
parents, a dash (`-`) **may** be accepted as a wildcard.
- The body clause in the `google.api.http` annotation **should** be `"*"`.
### Atomic vs. Partial Success
- The batch delete method **may** support atomic (all resources deleted or none
are) or partial success behavior. To make a choice, consider the following
factors:
- **Complexity of Ensuring Atomicity:** Operations that are simple
passthrough database transactions **should** use an atomic operation,
while operations that manage complex resources **should** use partial
success operations.
- **End-User Experience:** Consider the perspective of the API consumer.
Would atomic behavior be preferable for the given use case, even if it
means that a large batch could fail due to issues with a single or a few
entries?
- Synchronous batch delete **must** be atomic.
- Asynchronous batch delete **may** support atomic or partial success.
- If supporting partial success, see
[Operation metadata message](#operation-metadata-message) requirements.
### Request message
The request for a batch delete method **should** be specified with the
following pattern:
```proto
message BatchDeleteBooksRequest {
// The parent resource shared by all books being deleted.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in `names`
// must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The names of the books to delete.
// A maximum of 1000 books can be deleted in a batch.
// format: publishers/{publisher}/books/{book}
repeated string names = 2 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
- A `parent` field **should** be included, unless the resource being deleted is
a top-level resource. If a caller sets this field, and the
parent collection in the name of any resource being deleted does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
resource names specifying the resources to delete. The field **should** be
named `names`.
- The field **should** be required.
- The field **should** identify the [resource type][aip-122-names] that it
references.
- The comment for the field **should** document the resource pattern.
- Other fields besides `name` **may** be "hoisted" from the [standard Delete
request][request-message]. There is no way to allow for these fields to
accept different values for different resources; if this is needed, use the
[alternative request message form](#request-message-containing-standard-delete-request-messages).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `names` field **should** document the maximum number of
requests allowed.
- Filter-based matching **must not** be supported.
### Request message containing standard delete request messages
If the [standard Delete request message][request-message] contains a field
besides the resource name that needs to be different between different
resources being requested, the batch message **may** alternatively hold a
`repeated` field of the [standard Delete request message][request-message].
This is generally discouraged unless your use case really requires it.
The request for a batch delete method **should** be specified with the
following pattern:
```proto
message BatchDeleteBooksRequest {
// The parent resource shared by all books being deleted.
// Format: publishers/{publisher}
// If this is set, the parent of all of the books specified in the
// DeleteBookRequest messages must match this field.
string parent = 1 [
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The requests specifying the books to delete.
// A maximum of 1000 books can be deleted in a batch.
repeated DeleteBookRequest requests = 2
[(google.api.field_behavior) = REQUIRED];
}
```
- A `parent` field **should** be included. If a caller sets this field, and the
parent collection in the name of any resource being deleted does not match,
the request **must** fail.
- This field **should** be required if only 1 parent per request is allowed.
- The field **should** identify the [resource type][aip-122-parent] that it
references.
- The comment for the field **should** document the resource pattern.
- The request message **must** include a repeated field which accepts the
request messages specifying the resources to delete, as specified for
[standard Delete methods][request-message]. The field **should** be named
`requests`.
- The field **should** be required.
- Other fields **may** be "hoisted" from the [standard Delete
request][request-message], which means that the field can be set at either
the batch level or child request level. Similar to `parent`, if both the
batch level and child request level are set for the same field, the values
**must** match.
- Fields which must be unique cannot be hoisted (e.g. `etag`).
- The request message **must not** contain any other required fields, and
**should not** contain other optional fields except those described in this
or another AIP.
- The comment above the `requests` field **should** document the maximum number
of requests allowed.
- Filter-based matching **must not** be supported unless it is infeasible to
support critical use cases without it, because it makes it too easy for users
to accidentally delete important data. If it is unavoidable, see [AIP-165][].
### Response message (soft-delete only)
In the case where a response message is necessary because the resource is
soft-deleted, the response **should** be specified with the following pattern:
```proto
message BatchDeleteBooksResponse {
// Books deleted.
repeated Book books = 1;
}
```
- The response message **must** include one repeated field corresponding to the
resources that were soft-deleted.
### Operation metadata message
- The `metadata_type` message **must** either match the RPC name with
`OperationMetadata` suffix, or be named with `Batch` prefix and
`OperationMetadata` suffix if the type is shared by multiple Batch methods.
- If batch delete method supports partial success, the metadata message **must**
include a `map<int32, google.rpc.Status> failed_requests` field to communicate
the partial failures.
- The key in this map is the index of the request in the `requests` field in
the batch request.
- The value in each map entry **must** mirror the error(s) that would normally
be returned by the singular Standard Delete method.
- If a failed request can eventually succeed due to server side retries, such
transient errors **must not** be communicated using `failed_requests`.
- When all requests in the batch fail, `Operation.error` **must** be set with
`code = google.rpc.Code.Aborted` and `message = "None of the requests
succeeded, refer to the BatchDeleteBooksOperationMetadata.failed_requests
for individual error details"`
- The metadata message **may** include other fields to communicate the
operation progress.
### Adopting Partial Success
In order for an existing Batch API to adopt the partial success pattern, the API
must do the following:
- The default behavior must be retained to avoid incompatible behavioral
changes.
- If the API returns an Operation:
- The request message **must** have a `bool return_partial_success` field.
- The Operation `metadata_type` **must** include a
`map<int32, google.rpc.Status> failed_requests` field.
- When the `bool return_partial_success` field is set to true in a request,
the API should allow partial success behavior, otherwise it should continue
with atomic behavior as default.
- If the API returns a direct response synchronously:
- Since the existing clients will treat a success response as an atomic
operation, the existing version of the API **must not** adopt the partial
success pattern.
- A new version **must** be created instead that returns an Operation and
follows the partial success pattern described in this AIP.
## Rationale
### Restricting synchronous batch methods to be atomic
The restriction that synchronous batch methods must be atomic is a result of
the following considerations.
The previous iteration of this AIP recommended batch methods must be atomic.
There is no clear way to convey partial failure in a sync response status code
because an OK implies it all worked. Therefore, adding a new field to the
response to indicate partial failure would be a breaking change because the
existing clients would interpret an OK response as all resources created.
On the other hand, as described in [AIP-193](https://aip.dev/193), Operations
are more capable of presenting partial states. The response status code for an
Operation does not convey anything about the outcome of the underlying operation
and a client has to check the response body to determine if the operation was
successful.
### Communicating partial failures
The AIP recommends using a `map<int32, google.rpc.Status> failed_requests` field
to communicate partial failures, where the key is the index of the failed
request in the original batch request. The other options considered were:
- A `repeated google.rpc.Status` field. This was rejected because it is not
clear which entry corresponds to which request.
- A `map<string, google.rpc.Status>` field, where the key is the request id of
the failed request. This was rejected because:
- Client will need to maintain a map of request_id -> request in order to use
the partial success response.
- Populating a request id for the purpose of communicating errors could
conflict with [AIP-155](https://aip.dev/155) if the service can not
guarantee idempotency for an individual request across multiple batch
requests.
- A `repeated FailedRequest` field, where FailedRequest contains the individual
create request and the `google.rpc.Status`. This was rejected because echoing
the request payload back in response is discouraged due to additional
challenges around user data sensitivity.
[aip-122-names]: ./0122.md#fields-representing-resource-names
[aip-122-parent]: ./0122.md#fields-representing-a-resources-parent
[aip-165]: ./0165.md
[request-message]: ./0135.md#request-message
[soft-delete]: ./0135.md#soft-delete
## Changelog
- **2025-03-06**: Changed recommendation to allow partial success, along with
detailed guidance
- **2022-06-02:** Changed suffix descriptions to eliminate superfluous "-".
- **2020-09-16**: Suggested annotating `parent`, `names`, and `requests` fields.
- **2020-08-27**: Removed parent recommendations for top-level resources.
- **2020-03-27**: Added reference to AIP-165 for criteria-based deletion.
- **2019-10-11**: Changed the primary recommendation to specify a repeated
string instead of a repeated standard Delete message. Moved the original
recommendation into its own section.
- **2019-09-11**: Fixed the wording about which child field the `parent` field
should match.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 0236.md
---
id: 236
state: approved
created: 2023-03-30
updated: 2023-03-30
placement:
category: resource-design
order: 240
---
# Policy preview
A policy is a resource that provides rules that admit or deny access to other
resources. Generally, the outcome of a policy can be evaluated to a specific set
of outcomes.
Changes to policies without proper validation may have unintended consequences
that can severely impact a customer’s overall infrastructure setup. To safely
update resources, it is beneficial to test these changes via policy rollout
APIs.
Preview is a rollout safety mechanism for policy resources, which gives the
customer the ability to validate the effect of their proposed changes against
production traffic prior to the changes going live. The result of the policy
evaluation against traffic is logged in order to give the customer the data
required to test the correctness of the change.
Firewall policies exemplify a case that is suitable for previewing. A new
configuration can be evaluated against traffic to observe which IPs would be
allowed or denied. This gives the customer the data to guide a decision on
whether to promote the proposed changes to live.
The expected flow for previewing a policy is as follows:
1. The user creates an experiment containing a new policy configuration
intended to replace the live policy.
2. The user uses the "startPreview" method to start generating logs which compare
the live and experiment policy evaluations against live traffic.
3. The user inspects the logs to determine whether the experiment has the
intended result.
4. The user uses the "commit" method to promote the experiment to live.
## Guidance
### Non-goals
This proposal is for a safety mechanism for policy rollouts only. Safe rollouts
for non-policy resources are not in scope.
### Experiments
A new configuration of a policy to be previewed is stored as a nested collection
under the policy. These nested collections are known as experiments.
A hypothetical policy resource called, `Policy`, is used throughout. It has the
following resource name pattern:
`projects/{project}/locations/{location}/policies/{policy}`
The experimental versions of the resource used for previewing or other safe
rollout practices are represented as a nested collection under `Policy` using a
new resource type. The resource type **must** follow the naming convention
*RegularResourceType*`Experiment`.
The following pattern is used for the experiment collection:
`projects/{project}/locations/{location}/policies/{policy}/experiments/{experiment}`
A proto used to represent an experiment **must** contain the following:
1. The required top-level fields for a resource, like `name` and `etag`
2. The policy message that is being tested itself
3. The field, `preview_metadata`, which contains metadata specific to
previewing the experiment of a specific resource type.
```proto
message PolicyExperiment {
// google.api.resource, name, and other annotations and fields
// The policy experiment. This Policy will be used to preview the effects of
// the change but will not affect live traffic.
Policy policy = 2;
// The metadata associated with this policy experiment.
PolicyPreviewMetadata preview_metadata = 3
[(google.api.field_behavior) = OUTPUT_ONLY];
// Allows clients to store small amounts of arbitrary data.
map<string, string> annotations = 4;
}
```
- The experiment proto **must** have a top-level field with the same type as the
live policy.
- It **must** be named as the live resource type. For example, if the
experiment is for FirewallPolicy, then this field **must** be named
`firewall_policy`.
- The name inside the embedded `policy` message **must** be the name of the
live policy.
- When the user is ready to promote an experiment, they **must** copy the
`policy` message into the live policy and delete the experiment. This can be
done manually or via a "commit" custom method.
- A product **may** support multiple experiments concurrently being previewed
for a single live policy.
- Each experiment must generate logs having each entry preceded by log_prefix
so that the user can compare the results of the experiment with the behavior
of the live policy.
- The number of experimental configurations for a given live policy **may** be
capped at a certain number and the cap **must** be documented.
- Cascading deletes **must** occur: if the live policy is deleted, all
experiments **must** also be deleted.
- `map<string,string>` [annotations][aip-128-annotations] **must** allow clients
to store small amounts of arbitrary data.
### Metadata
`preview_metadata` tracks all metadata of previewing the experiment. The
messages **must** follow the convention: *RegularResourceType*`PreviewMetadata`.
This is so the proto can be defined uniquely for each resource type in the
same service with experiments.
```proto
message PolicyPreviewMetadata {
// Possible values of the state of previewing the experiment.
enum State {
// Default value. This value is unused.
STATE_UNDEFINED = 0;
// The experiment is actively previewing.
ACTIVE = 1;
// The previewing of the experiment has been stopped.
SUSPENDED = 2;
}
// The state of previewing the experiment.
State state = 1;
// An identifying string common to all logs generated when previewing the
// experiment. Searching all logs for this string will isolate the results.
string log_prefix = 2;
// The most recent time at which this experiment started previewing.
google.protobuf.Timestamp start_time = 3;
// The most recent time at which this experiment stopped previewing.
google.protobuf.Timestamp stop_time = 4;
}
```
- `PolicyPreviewMetadata` **must** have the fields defined in the proto above.
- It **may** have additional fields if the service or resource requires it.
- When an experiment is first previewed, `preview_metadata` **must** be
absent.
- It is present on the experiment once the "startPreview" method is used.
- All `preview_metadata` fields **must** be output only.
- `state` changes between `ACTIVE` and `SUSPENDED` when previewing is started
or stopped. This happens when the "startPreview" or "stopPreview custom methods
are invoked, respectively.
- The first time the "startPreview" custom method is used, the system **must**
create `preview_metadata` and do the following:
- It **must** set the `state` to `ACTIVE`
- It **must** populate `start_time` with the current time.
- `start_time` **must** be updated every time `state` is changed to
`ACTIVE`.
- It **must** set a system generated `log_prefix` string, which is a
predefined constant hard coded by the system developers.
- The same value is used for previewing experiments for the given resource
type. For example, "FirewallPolicyPreviewLog" for FirewallPolicy.
- When the "stopPreview" custom method is used, the system **must** do the
following:
- It **must** set the `state` to `SUSPENDED`
- It **must** populate the `stop_time` with the current time.
### Methods
#### create
- The resource **must** be created using long-running
[Create][aip-133-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
- Creating a new experiment to preview **must** support the following use
cases:
- Preview a new policy.
- Preview an update to an already live policy.
- Preview a deletion of a current policy.
- For the update and delete use cases, the `policy` field in the experiment
**must** have the full payload of the live policy copied into it, including
the name.
- The user **must** set the rules to the new intended state to preview an
update.
- The user **must** set set the rules to represent a no-op to preview a
delete.
- To preview a new policy, the system must do the following:
- If the system does not support a nested collection without a live policy,
the user **must** create a live policy and set the rules to represent a
no-op. For example, the rules of a no-op policy **may** be empty.
- An experiment is created as a child of the no-op policy.
- If the system supports previewing multiple experiments for a live policy,
calling "create" more than once **must** create multiple experiments.
#### update
- The resource **must** be updated using long-running
[Update][aip-134-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
- The name inside `policy` **must not** change but the other fields can in
order to change the experiment being previewed because this `policy` is
intended to replace the live policy, and the name of the live policy
**must not** change.
- The system **must** set the `state` to `SUSPENDED` if the `state` was `ACTIVE`
at the time of an update.
- This is so the user can easily distinguish between different versions of
the experiment being previewed.
#### get
- The standard method, [Get][aip-131], **must** be included for
`PolicyExperiment` resource types.
#### list
- The standard method, [List][aip-132], **must** be included for
`PolicyExperiment` resource types.
- Filtering on `PolicyPreviewMetadata` indicates which experiments are actively
previewed.
- For example, the following filter string returns a List response with
experiments being previewed: preview_metadata.state = ACTIVE.
#### delete
- The resource **must** be deleted using long-running
[Delete][aip-135-long-running] and
`google.longrunning.operation_info.response_type` **must** be
`PolicyExperiment`.
#### startPreview
```proto
// Starts previewing a PolicyExperiment. This triggers the system to start
// generating logs to evaluate the PolicyExperiment.
rpc StartPreviewPolicyExperiment(StartPreviewPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:startPreview"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PolicyExperiment"
metadata_type: "StartPreviewPolicyExperimentMetadata"
};
}
// The request message for the startPreview custom method.
message StartPreviewPolicyExperimentRequest {
// The name of the PolicyExperiment.
string name = 1;
}
```
- This custom method is required.
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- This method **must** trigger the system to start generating logs to preview
the experiment.
- Whenever the method is called successfully, the system **must** set the
following values in the `PolicyPreviewMetadata`:
- `log_prefix` to the predefined constant.
- `start_time` to the current time
- `state` to `ACTIVE`.
- If the method is called on an experiment with the rules representing a no-op,
then the system **must** preview the deletion of the live policy.
#### stopPreview
```proto
// Stops previewing a PolicyExperiment. This triggers the system to stop
// generating logs to evaluate the PolicyExperiment.
rpc StopPreviewPolicyExperiment(StopPreviewPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:stopPreview"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "PolicyExperiment"
metadata_type: "StopPreviewPolicyExperimentMetadata"
};
}
// The request message for the stopPreview custom method.
message StopPreviewPolicyExperimentRequest {
// The name of the PolicyExperiment.
string name = 1;
}
```
- This custom method is required.
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- This method **must** trigger the system to stop generating logs to preview the
experiment.
- Whenever the method is called successfully, the system **must** set the
following values in the `PolicyPreviewMetadata`:
- `stop_time` to the current time
- `state` to `SUSPENDED`
#### commit
The resource **may** expose a new custom method called "commit" to promote an
experiment. The system copies `policy` from the experiment into the live policy
and then deletes the experiment.
Declarative clients **may** manually copy fields from an experiment into the
live policy and then delete the experiment rather than calling "commit" if
preferable.
```proto
// Commits a PolicyExperiment. This copies the PolicyExperiment's policy message
// to the live policy then deletes the PolicyExperiment.
rpc CommitPolicyExperiment(CommitPolicyExperimentRequest)
returns (google.longrunning.Operation) {
option (google.api.http) = {
post: "/v1/{name=policies/*/experiments/*}:commit"
body: "*"
};
option (google.longrunning.operation_info) = {
response_type: "google.protobuf.Empty"
metadata_type: "CommitPolicyExperimentMetadata"
};
}
// The request message for the commit custom method.
message CommitPolicyExperimentRequest {
string name = 1;
string etag = 2;
string parent_etag = 3;
}
```
- `google.longrunning.Operation.metadata_type` **must** follow guidance on
[Long-running operations][aip-151]
- The method **must** atomically copy `policy` from the experiment into the live
policy, and then delete the experiment.
- If any experiment fails "commit", previewing it **must not** stop, and the
live policy **must not** be updated.
- The method can be called on an experiment in any state.
- The `etag` **must** match that of the experiment in order for commit to be
successful. This is so the user does not commit an unintended version of the
experiment.
- If no `etag` is provided, the API **must not** succeed to prevent the user
from unintentionally committing a different version of the experiment as
intended.
- A `parent_etag` **may** be provided to guarantee that the experiment
overwrites a specific version of the live policy.
- The method is not idempotent and calling it twice on the same experiment
**must** return a 404 NOT_FOUND as the experiment is deleted as part of the
first call.
### Changes to live policy API methods
#### delete
- A delete of the live policy **must** delete all experiments.
- To maintain the experiments while negating the effect of the live policy, the
live policy **must** be changed to a no-op policy instead of using this
method.
### Logging
Logging is crucial for the user to evaluate whether an experiment should be
promoted to live.
Logs **must** contain the results of the evaluated experiment, the `etag`
associated with that experiment alongside that of the live policy, and be
preceded by the value of `log_prefix`.
- The `etag` fields help the user identify which
configurations of the live and experiment are evaluated in the log.
- `log_prefix` helps the user separate logs specifically generated for
previewing the experiment from other use cases.
Overall, these logs help the user make a decision about whether to promote the
experiment to live.
## Changelog
- **2023-04-27:** Methods for start and stop renamed. State to enum. Annotations
added.
- **2023-03-30:** Initial AIP written.
[aip-128-annotations]: https://aip.dev/128#annotations
[aip-131]: https://aip.dev/131
[aip-132]: https://aip.dev/132
[aip-133-long-running]: https://aip.dev/133#long-running-create
[aip-134-long-running]: https://aip.dev/134#long-running-update
[aip-135-long-running]: https://aip.dev/135#long-running-delete
[aip-151]: https://google.aip.dev/151
>>>> 2020-03.md
# Welcome to the AIP Newsletter
Welcome to the first (hopefully monthly) edition of the **AIP newsletter**,
which is designed to keep you up to date about the AIP program, and particular
proposals making their way through the system.
In this edition, we will discuss five pending proposals. This is intended as a
"public comment" period: the AIP editors are happy with these proposals, but we
want to ensure that you are too. Assuming feedback is sufficiently positive, we
intend to formally approve these proposals on Friday, March 27, 2020.
## API linter
As many readers know, we have published an [API linter][], which is able to
check much of the guidance provided in API Improvement Proposals for APIs
defined in protocol buffers.
The linter now has over 150 rules, and is running on many new and existing APIs
at Google, hopefully pushing us toward greater consistency.
[api linter]: https://github.com/api-linter
## AIPs under review
### AIP-153: Import and export
> Many users want to be able to load data into an API, or get their existing
> data out of an API. This is particularly important for enterprise users, who
> are often concerned about vendor lock.
[Import and export][aip-153] are particularly important concepts for enterprise
APIs, although they are increasing in importance for consumer and SaaS APIs as
well.
We have actually had a reasonably consistent import and export story in Google
Cloud for a long time, but it has largely been passed along by tribal knowledge
("go look at X and copy what they did"). Therefore, this AIP attempts to codify
what we believe is a long-standing practice.
**Summary:** [AIP-153][] mandates that both import and export methods return
[long-running operations][aip-151]. Unlike most API methods, they do support
partial failures, which are handled through the LRO metadata. Finally, they
have a specified place for per-source and per-destination configuration,
distinct from configuration for the data itself.
We are aiming to approve AIP-153 on March 27. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/422).
[aip-151]: ../151
[aip-153]: ../153
### AIP-162: Resource revisions
> Some APIs need to have resources with a revision history, where users can
> reason about the state of the resource over time.
[Resource revisions][aip-162] is a concept we are seeing crop up more and more,
and we are seeing an increasing number of situations where audit logging is a
necessary but insufficient solution. Several APIs have struggled with how to
model a resource's history over time, and handle common use cases like commit
and rollback, which is what prompted this AIP.
This is new guidance, put together by the AIP team as well as several API teams
who needed this functionality near the end of 2019. It has been positively
received by the teams that needed it, and so we would like to codify it.
**Summary:** [AIP-162][] mandates the use of revision IDs that are appended to
a resource name with the `@` character for signifying revisions, and sets out
method structures for most common operations (listing revisions, rollback,
etc.). It also proposes a tagging mechanism for user-friendly revision names.
We are aiming to approve AIP-162 on March 27. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/436).
[aip-162]: ../162
### AIP-163: Change validation
> Occasionally, a user wants to validate an intended change to see what the
> result will be before actually making the change. For example, a request to
> provision new servers in a fleet will have an impact on the overall fleet
> size and cost, and could potentially have unexpected downstream effects.
[Change validation][aip-163] is the concept of performing a "dry run" on an API
request, and returning back the result without making any changes.
This is effectively a migration of long-standing guidance that was not very
fleshed out. The previous API design guide mentioned `validate_only` as an
entry in the standard fields table with minimal explanation. This proposal
fleshes that out with an example, and addresses some questions we have received
over time.
**Summary:** [AIP-163][] offers guidance for "dry run" requests using a
`bool validate_only` field in the request message, and addresses some common
situations around permissions and incomplete output.
We are aiming to approve AIP-163 on March 27. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/437).
[aip-163]: ../163
### AIP-165: Criteria-based delete
> Occasionally, an API may need to provide a mechanism to delete a large number
> of resources based on some set of filter parameters, rather than requiring
> the individual resource name of the resources to be deleted.
[Criteria-based delete][aip-165] is designed to address problems where a user
needs to delete so many items that [batch delete][aip-235] simply becomes
infeasible.
This proposal represents a concession of sorts: we have long tried to avoid
delete-by-filter because of concerns that users would accidentally delete
important records and be unable to recover them. However, several APIs have had
legitimate reasons to need to do this, because the number of records the user
would want to delete was so high.
**Summary:** [AIP-165][] offers guidance for criteria-based delete, with a
common name (`Purge`) and structure, and mandates a dry-run by default with a
`force` flag to actually perform the purge.
We are aiming to approve AIP-165 on March 27. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/438).
[aip-165]: ../165
[aip-235]: ../235
### AIP-194: Retries for gRPC clients
> RPCs sometimes fail. When one does, the client performing the RPC needs to
> know whether it is safe to retry the operation. When status codes are used
> consistently across multiple APIs, clients can respond to failures
> appropriately.
[Retries for gRPC clients][aip-194] handles how to deal with errors, and in
particular when a client can transparently retry those errors.
This proposal came about because of a need for first-party client libraries to
be able to transparently retry some server errors. gRPC offers a configuration
strategy for this, which is defined by API producers.
**Summary:** [AIP-194][] offers guidance for configuring what error codes from
an API are safe to retry. It recommends usually retrying `UNAVAILABLE` and
nothing else.
We are aiming to approve AIP-194 on March 27. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/439).
[aip-194]: ../194
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-8](../8): AIPs have reverse-chronological changelogs
([#425](https://github.com/aip-dev/google.aip.dev/pull/425))
- [AIP-135](../135): Delete methods send 403 errors if the user lacks access
([#409](https://github.com/aip-dev/google.aip.dev/pull/409))
- [AIP-140](../140): `display_name` and `title` standard fields
([#405](https://github.com/aip-dev/google.aip.dev/pull/405))
- [AIP-158](../158): Field order for pagination response messages
([#419](https://github.com/aip-dev/google.aip.dev/pull/419))
- [AIP-193](../193): Added reference to new `ErrorInfo` message
([#396](https://github.com/aip-dev/google.aip.dev/pull/396))
## What is coming?
We have more coming over the course of the next few months, including
long-awaited public guidance for filtering and versioning.
You also may notice some pull requests around making AIPs more generic. We are
exploring the possibility of bringing AIPs to even more API producers across
multiple companies, so that others can publish their standards under a common
information architecture. This effort is still nascent, but we are excited and
hopeful. Stay tuned!
>>>> 2020-04.md
# Data Series
Welcome to the second edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system.
This month's new AIPs (unintentionally) share a common theme: handling lists
and series of data, whether in fields, resource relationships, or actual list
methods. These new AIPs are long overdue: they codify knowledge that has long
been reasonably consistent, but tribal.
The AIP newsletter kicks off what is effectively a "public comment" period: the
AIP editors are happy with these proposals, but we want to ensure that you are
too. Assuming feedback is sufficiently positive, we intend to formally approve
these proposals on Friday, April 24, 2020.
## AIPs under review
### AIP-124: Resource associations
> APIs sometimes have resource hierarchies that can not be cleanly expressed in
> the usual tree structure.
[Resource association][aip-124] comes into play when a resource needs something
other than a direct parent-child relationship. This might mean a many-to-many
relationship, or it might be a one-to-many association that is distinct from
the parent-child relationship.
We have had several APIs struggle with unusual associations over the past year
or so. In general, this has been handled reasonably consistently, but we never
codified the guidance into an official AIP.
**Summary:** [AIP-124][] provides patterns for the two most common
associations: a resource with multiple different one-to-many relationships, and
a resource with a many-to-many association with another resource.
We are aiming to approve AIP-124 on April 24. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/475).
[aip-124]: ../124
### AIP-144: Repeated fields
> Representing lists of data in an API is trickier than it often appears. Users
> often need to modify lists in place, and longer data series within a single
> resource pose a challenge for pagination.
[Repeated fields][aip-144] are somewhat common in APIs, and a common question
that we get is how to handle adding or removing a single item from them
atomically. Additionally, repeated fields can become a trap if they are
unbounded in size and become too large.
This is codification of guidance that we have been giving on a case-by-case
basis for some time. In particular, this documents the various restrictions on
the two update strategies.
**Summary:** [AIP-144][] lays out guidance for repeated fields, and details two
distinct strategies for updating repeated fields.
We are aiming to approve AIP-144 on April 24. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/476).
[aip-144]: ../144
### AIP-160: Filtering
> Often, when listing resources, it is desirable to filter over the collection
> and only return results that the user is interested in.
[Filtering][aip-160] involves taking a list of resources (usually) and only
providing the results that the user wants, specified using a familiar syntax
similar to that of Google search.
This is likely the most-requested AIP of all time. We have had internal
filtering documents for ages, but we have not had a public version, and one has
been sorely needed for years. Also, this work would likely still be unfinished
if not for the efforts of [Tristan Swadell][]; we are grateful for his efforts.
**Summary:** [AIP-160][] offers a detailed breakdown of the filtering syntax
and how to provide filtering results in list and search methods.
We are aiming to approve AIP-160 on April 24. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/473).
[aip-160]: ../160
[tristan swadell]: https://github.com/tristonianjones
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-133](../133): Clarify which fields can be required
([#460](https://github.com/aip-dev/google.aip.dev/pull/460))
- [AIP-140](../140): Add guidance about message and field name conflicts
([#470](https://github.com/aip-dev/google.aip.dev/pull/470))
- [AIP-151](../151): Clarify that both `response_type` and `metadata_type` are
expected ([#469](https://github.com/aip-dev/google.aip.dev/pull/469))
- [AIP-231](../231): Clarify Batch Get behavior if no resource names are sent
([#474](https://github.com/aip-dev/google.aip.dev/pull/474))
>>>> 2020-05.md
# Jobs and filters
Welcome to the third edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system.
This month, we have one new AIP, which is around _jobs_: resources created for
managing a task that may be repeated. We also have one AIP from April that we
have delayed finalizing.
As always, the AIP newsletter kicks off what is effectively a "public comment"
period: the AIP editors are happy with these proposals, but we want to ensure
that you are too. Assuming feedback is sufficiently positive, we intend to
formally approve these proposals on Friday, May 29, 2020.
## AIPs under review
### AIP-152: Jobs
> Occasionally, APIs may need to expose a task that takes significant time to
> complete, and where a transient long-running operation is not appropriate.
> For example, a task could need to run repeatedly, or have separate
> permissions for configuring the task as opposed to running it.
[Jobs][aip-152] are resources that are created that represent a (usually)
repeated task in an API. The idea is that the job is configured once, and
exists as an independent resource, which can either be executed by a user, or
by the system on a schedule.
Jobs are useful when more structure or more permanence is needed than a simple
RPC with a long-running operation can provide. They can also be useful if a
service requires a permission model that distinguishes between who can
_configure_ a job as opposed to who can _run_ it.
**Summary:** [AIP-152][] provides patterns for jobs: first-class resources that
represent tasks that are run repeatedly, whether by a user or by the system on
a schedule.
We are aiming to approve AIP-152 on May 29. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/504).
[aip-152]: ../152
### AIP-160: Filtering
> Often, when listing resources, it is desirable to filter over the collection
> and only return results that the user is interested in.
[Filtering][aip-160] involves taking a list of resources (usually) and only
providing the results that the user wants, specified using a familiar syntax
similar to that of Google search.
This AIP is a holdover from last month's newsletter: we intended to approve it
in April, but several teams brought forth issues and clarifications that
necessitated further iteration. (Thanks!!) These are now represented in the
approval pull request ([#473][]), and we hope to approve this AIP in May.
**Summary:** [AIP-160][] offers a detailed breakdown of the filtering syntax
and how to provide filtering results in list and search methods.
We are aiming to approve AIP-160 on May 29. If you have feedback, please [leave
a comment][#473].
[#473]: https://github.com/aip-dev/google.aip.dev/pull/473
[aip-160]: ../160
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-122](../122): Tighten restrictions on resource names
([#496](https://github.com/aip-dev/google.aip.dev/pull/496))
- [AIP-192](../192): Mandate use of absolute links in comments
([#480](https://github.com/aip-dev/google.aip.dev/pull/480))
>>>> 2020-06.md
# Field patterns
Welcome to the fourth edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system.
This month, we have two new AIPs, both around patterns for specific kinds of
fields.
As always, the AIP newsletter kicks off what is effectively a "public comment"
period: the AIP editors are happy with these proposals, but we want to ensure
that you are too. Assuming feedback is sufficiently positive, we intend to
formally approve these proposals on Friday, June 26, 2020.
## AIPs under review
### AIP-145: Ranges
> Services often need to represent ranges of discrete or continuous values.
> These have wide differences in meaning, and come in many types: integers,
> floats, and timestamps, just to name a few, and the expected meaning of a
> range can vary in subtle ways depending on the type of range being discussed.
[Ranges][aip-145] are fields that represent a range of discrete or continuous
values. These are often integers or floats, but can be other values where
ranges make sense, such as dates.
The primary challenge for ranges is whether their values (particularly their
end values) should be inclusive or exclusive. Most developers are used to
exclusive end values, but with some notable and precise exceptions (example:
dates are an exception, but not timestamps), and the AIP attempts to encode
this using the idea of "colloquial precedent".
**Summary:** [AIP-145][] provides patterns for ranges of values: two fields
that represent a start and end value.
We are aiming to approve AIP-145 on June 26. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/523).
[aip-145]: ../145
### AIP-146: Generic fields
> Occasionally it is appropriate to have a generic or polymorphic field of some
> kind that can conform to multiple schemata, or even be entirely free-form.
[Generic fields][aip-146] discusses the various ways to approach a polymorphic
or entirely generic field in the API. These provide implementation flexibility,
but the tradeoff is a less clear API contract. There are several ways to
provide a generic field, and the AIP enumerates them and discussed the
tradeoffs.
**Summary:** [AIP-146][] offers four patterns for generic fields, and describes
the tradeoffs of each.
We are aiming to approve AIP-146 on June 26. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/524).
[aip-146]: ../146
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-122](../122): Corrected a mistaken piece of guidance about
capitalization ([#513](https://github.com/aip-dev/google.aip.dev/pull/513))
- [AIP-132](../132): Removed mandate to document default ordering
([#512](https://github.com/aip-dev/google.aip.dev/pull/512))
- [AIP-140](../140): Add guidance for URI fields
([#519](https://github.com/aip-dev/google.aip.dev/pull/519))
- [AIP-143](../143): Change guidance for countries to `region_code`
([#507](https://github.com/aip-dev/google.aip.dev/pull/507))
- [AIP-203](../203): Clarified behavior for updating immutable fields
([#516](https://github.com/aip-dev/google.aip.dev/pull/516))
>>>> 2020-08.md
# Field patterns, redux
Welcome to the fifth edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system. (Our apologies for not publishing a newsletter in
July.)
We have one new AIP this month, on sensitive fields. As always, the AIP
newsletter kicks off what is effectively a "public comment" period: the AIP
editors are happy with this proposal, but we want to ensure that you are too.
Assuming feedback is sufficiently positive, we intend to formally approve these
proposals on Friday, August 28, 2020.
## AIPs under review
### AIP-147: Sensitive fields
> Sometimes APIs need to collect sensitive information such as private
> encryption keys meant to be _stored_ by the underlying service but not
> intended to be _read_ after writing due to the sensitive nature of the data.
[Sensitive fields][aip-147] are fields that are stored by a service but, once
set, not retrievable by the user, such as passwords or private keys. These are
not particularly common in APIs, and often it is sufficient to set them as
`INPUT_ONLY` (as described in AIP-203).
However, there is a challenge in situations where the service needs to provide
some kind of indication that a value has been provided, or provide an
obfuscated value that the original user ought to recognize. This AIP provides a
standard for each of those situations.
A special thanks to [Michael Bleigh][] for writing this AIP.
**Summary:** [AIP-147][] provides patterns for indicators around sensitive
fields.
We are aiming to approve AIP-147 on August 28. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/583).
[michael bleigh]: https://github.com/mbleigh
[aip-147]: ../147
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-135](../135): Add "get" guidance for soft delete.
([#526](https://github.com/aip-dev/google.aip.dev/pull/526))
- [AIP-140](../140): Word segments in field names must not begin with a number.
([#528](https://github.com/aip-dev/google.aip.dev/pull/528))
- [AIP-151](../151): Add guidance for parallel operations.
([#535](https://github.com/aip-dev/google.aip.dev/pull/535))
- [AIP-158](../158): Clarify that `page_size` is always an optional field.
([#537](https://github.com/aip-dev/google.aip.dev/pull/537))
>>>> 2020-10.md
# Declarative-friendly APIs
Welcome to the sixth edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system.
We have an exciting slate of AIPs entering review this cycle, all around the
theme of "declarative-friendly APIs". The linchpin here is AIP-128, which
introduces and explains the topic.
The idea behind declarative-friendly APIs addresses an important need: the
ability to integrate with popular, declarative tools, such as Kubernetes or
Terraform. These tools and others are used by numerous developers, and the
follow a straightforward paradigm: the user provides the tool with the desired
state, and the tool brings that state into fruition.
Integration with these tools is critical for cloud providers, but with the
number of services _and_ the number of tools continuously increasing, support
is difficult without a reliance on automation. Automation, in turn, requires
strong consistency.
While ordinarily AIPs stay in review for about a month, we intend for this
review cycle to be longer, to allow time for APIs to be released that follow
this guidance. Assuming the feedback is sufficiently positive, we will probably
be looking at formal approval around the end of the year.
## AIPs under review
### AIP-128: Declarative-friendly interfaces
> Many services need to interact with common DevOps tools, particularly those
> that create and manage network-addressible resources (such as virtual
> machines, load balancers, database instances, and so on). These tools revolve
> around the principle of "configuration as code": the user specifies the
> complete intended landscape, and tooling is responsible for making whatever
> changes are necessary to achieve the user's specification.
>
> These tools are **declarative**: rather than specifying specific actions to
> take, they specify the desired _outcome_, with the actions being derived
> based on the differences between the current landscape and the intended one.
[Declarative-friendly interfaces][aip-128] describes patterns for making
interfaces that are friendly to automated tooling. The basic idea is that by
being more strict and consistent, it becomes possible to write tools that
"translate" the interface to a declarative form without having to know anything
about the specific API.
This reduces feature latency for users. Essentially, the adapter or plugin can
be automatically generated, and so an automated pipeline becomes possible: as
soon as the service launches new features, they become available in the
adapters also.
Because declarative-friendliness is a wide-ranging concept, impacting the
guidance in numerous other AIPs as well, it was difficult to determine how to
structure the information. Therefore, this AIP also has a robust "further
reading" section that points to declarative-friendly guidance in other AIPs.
**Summary:** [AIP-128][] provides an explanation of what it means for a service
to be declarative-friendly, and why a service might want to adopt these
patterns. It also indexes declarative-friendly guidance found in other AIPs.
[aip-128]: /128
### AIP-148: Standard fields
> Certain concepts are common throughout any corpus of APIs. In these
> situations, it is useful to have a standard field name that is used
> consistently to communicate that concept.
[Standard fields][aip-148] is, in some ways, an overdue AIP. It describes
fields common throughout our corpus, such as `name`, `parent`, and
`create_time`, that were on the list of standard fields in the original API
style guide and did not get a full treatment in other AIPs. Common fields that
are covered in other AIPs are not discussed in this one. (As an example,
`language_code` is given extensive treatment in [AIP-143][], so not repeated
here.)
This takes on new importance with our focus on declarative-friendly APIs
because tools written to marshal between a declarative tool and the service can
see those tools and know what they mean.
**Summary:** [AIP-148][] describes common fields that are already in common use
throughout our API corpus. It also describes certain fields that are expected
on declarative-friendly resources.
[aip-143]: /143
[aip-148]: /148
### AIP-164: Soft delete
> There are several reasons why a client could desire soft delete and undelete
> functionality, but one over-arching reason stands out: recovery from
> mistakes. A service that supports undelete makes it possible for users to
> recover resources that were deleted by accident.
[Soft delete][aip-164] is increasingly critical for APIs that need to cater to
enterprise users, due to the need to be able to recover from mistakes without
data loss. While [AIP-135][] previously discussed soft delete, it was clear
that a more thorough treatment is necessary.
This AIP goes through the request patterns in more detail, and discusses topics
such as LROs and errata, that are relevant to soft delete and undelete.
**Summary:** [AIP-164][] expands the guidance on soft delete and undelete and
provides a more detailed look at both.
[aip-135]: /135
[aip-164]: /164
## AIP sections under review
Because of the need to add declarative-friendly guidance into existing AIPs,
certain sections needed to be introduced into existing, approved AIPs in a
reviewing state. These are all in use for declarative-friendly guidance and
would be approved alongside AIP-128:
- AIP-134: [Create or update](/134#create-or-update)
- AIP-135: [Delete if existing](/135#delete-if-existing)
- AIP-136: [Custom method restrictions](/136#declarative-friendly-resources)
Additionally, new sections in [AIP-154](/154#declarative-friendly-resources)
and [AIP-163](/163#declarative-friendly-resources) mandate the use of these
concepts for declarative-friendly resources.
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-122](/122): Clarified when to use full resource names
([#629](https://github.com/aip-dev/google.aip.dev/pull/629))
- [AIP-132](/132): Loosened guidance around top-level resources
([#602](https://github.com/aip-dev/google.aip.dev/pull/602))
- [AIP-132](/132): Added `PERMISSION_DENIED` guidance
([#582](https://github.com/aip-dev/google.aip.dev/pull/582))
- [AIP-151](/151): Clarified that `stream` and LROs are mutually incompatible
([#597](https://github.com/aip-dev/google.aip.dev/pull/597))
- [AIP-216](/216): Clarified that states are not directly set on Create
([#606](https://github.com/aip-dev/google.aip.dev/pull/606))
Special thanks to Andrew Paseltiner and Peter Novotney for some of these
changes, and to several others who send minor corrections not noted here.
>>>> 2021-03.md
# Auth and Field Masks
Welcome to the seventh edition of the **AIP newsletter**, which is designed to
keep you up to date about the AIP program, and particular proposals making
their way through the system.
We have three new AIPs this month. As always, the AIP newsletter kicks off what
is effectively a "public comment" period: the AIP editors are happy with this
proposal, but we want to ensure that you are too. Assuming feedback is
sufficiently positive, we intend to formally approve these proposals on Friday,
March 26, 2021.
## AIPs under review
### AIP-149: Unset field values
> In many messages, many fields are optional: the user is not required to
> provide them, or for output fields, the service might not populate the field.
[Unset field values][aip-149] refer to primitive fields where the service makes
a meaningful distinction between an empty value (such as `0` or `false`) and
not setting the value at all. While this is generally discouraged if an
alternative design is feasible, there are certainly occasions where this is the
best approach.
**Summary:** [AIP-149][] provides guidance around use of the `optional` keyword
for primitives.
We are aiming to approve AIP-149 on March 26. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/711).
[aip-149]: ../149
### AIP-161: Field masks
> Often, when updating resources, it is desirable to specify exactly which
> fields are being updated, so that the service can ignore the rest, even if
> the user sends new values.
[Field masks][aip-161] are the way that users are able to specify exactly which
fields to read or update. They are necessary to ensure that adding fields to
existing messages remains backwards-compatible.
While we have had an internal specification for field masks for some time, this
AIP publicizes the specification.
**Summary:** [AIP-161][] provides a public specification for field masks.
We are aiming to approve AIP-161 on March 26. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/718).
[aip-161]: ../161
### AIP-211: Authorization checks
> The majority of operations, whether reads or writes, require authorization:
> permission to do the thing the user is asking to do. Additionally, it is
> important to be careful how much information is provided to _unauthorized_
> users, since leaking information can be a security concern.
[Authorization checks][aip-211] discusses when in the logical process of a
request the authorization check ought to occur, and what error to send when an
auth check fails.
In particular, if an authorization check fails, regardless of whether the
resource exists, the service consistently returns `PERMISSION_DENIED` with an
error message that also indicates that the resource might not exist.
**Summary:** [AIP-211][] provides guidance about authorization checks: when to
perform them, and what error to send.
We are aiming to approve AIP-211 on March 26. If you have feedback, please
[leave a comment](https://github.com/aip-dev/google.aip.dev/pull/719).
[aip-211]: ../211
## Additional approvals
We also intend to approve the AIPs that were placed under review in the October
2020 newsletter. This includes [AIP-128][] ([#708][]), [AIP-148][] ([#710][]),
and [AIP-164][] ([#709][]). If you have feedback, please leave a comment in the
referenced pull requests.
[aip-128]: ../128
[aip-148]: ../148
[aip-164]: ../164
[#708]: https://github.com/aip-dev/google.aip.dev/pull/708
[#709]: https://github.com/aip-dev/google.aip.dev/pull/709
[#710]: https://github.com/aip-dev/google.aip.dev/pull/710
## Recent updates
In addition to the new AIPs under review, we have added the following guidance
to existing AIPs:
- [AIP-144](../144): Expand Add and Remove guidance.
([#648](https://github.com/aip-dev/google.aip.dev/pull/648))
- [AIP-154](../154): Etag failures should use `ABORTED`.
([#705](https://github.com/aip-dev/google.aip.dev/pull/705))
- [AIP-162](../162): Forbid deleting the only remaining revision.
([#692](https://github.com/aip-dev/google.aip.dev/pull/692))
- [AIP-162](../162): Restrict revision tags to lower-case.
([#707](https://github.com/aip-dev/google.aip.dev/pull/707))
- [AIP-165](../165): Make implicit recommendations more explicit.
([#662](https://github.com/aip-dev/google.aip.dev/pull/662))
- [AIP-203](../203): Add `UNORDERED_LIST` to field behaviors.
([#671](https://github.com/aip-dev/google.aip.dev/pull/671))
>>>> 2510.md
---
id: 2510
state: approved
created: 2018-12-26
placement:
category: gcp
---
# Project identifiers
Historically, Google had two kinds of projects: API projects and App Engine
projects. API projects used project numbers (e.g. `12345`) as identifiers, and
App Engine projects used project IDs (e.g. `happy-armadillo-789`) as
identifiers. Later, Google converged API projects and App Engine projects, so
now each project has _both_ unique and immutable identifiers.
The two types of identifiers are used differently in different contexts, and
create a lot of complexity for application development. One critical issue is
that applications cannot reliably join data from different services, because
different services use different project identifiers.
## Guidance
**TL;DR:** The project number is the canonical identifier, and the project ID
is an [alias][]; however, unlike normal aliases, it **should** be returned if it
is what the user sent. Additionally, third-party services are unable to
_accept_ project IDs.
The rationale for this is:
- Each resource should always have one canonical identifier.
- Because Google's privacy policy restricts the use of project IDs, both
internally and with partners, only the project number can be the canonical
identifier.
- However, even though the project number is the canonical identifier, a policy
of returning it even if the user sent a project ID has proven to be
unfriendly to both humans and declarative tools.
### Google APIs
Externally-facing Google APIs **should** accept both project IDs and project
numbers for incoming API requests.
However, even though the project number is the canonical identifier as
described in AIP-122, services **should** return whichever ID the user sent.
The reason for this is because automatic translation between user-friendly
project IDs and user-unfriendly project numbers has proven to cause real-world
difficulty for users, and also for declarative tools (see AIP-128 for more on
declarative-friendliness).
Two additional points:
- Error responses **must** return the originally-provided value without
modification. Error responses **must not** perform any translation between
project IDs and project numbers.
- If a service receives a resource name for a resource that the service does
not own, it **should not** perform any translation between project IDs and
project numbers for those resource names.
### Internal Google services
Internal Google services **must** use project numbers for internal data storage
and for output. Project identifiers are widely used as storage keys, which
often appear in logs and metrics. Project IDs are user-settable and thus
considered PII and user data, but project numbers are not.
Therefore, when an internal service calls an external Google APIs, it
**should** use project numbers for making API requests.
### Resource References
Project identifiers also appear in [resource names][]. These resource
names are used both to identify the resource itself and can refer to
other resources ([example][]).
When project identifiers are provided, the response **should**
include the identifier as it occurred in the request: if the project ID
was provided it should be returned, and if the project number was
provided, that is what should be in the response.
For example, consider a `Book` resource,
```proto
message Book {
option (google.api.resource) = {
type: "pubsub.googleapis.com/Book"
pattern: "projects/{project}/books/{book}"
};
// The resource name of the Book.
string name = 1 [(google.api.field_behavior) = IDENTIFIER];
// A reference to another resource, a Shelf.
string shelf = 2 [(google.api.resource_reference) = {
type: "library.googleapis.com/Shelf"
}];
// Other fields...
}
```
with the following book submitted from a previous create request (represented as
JSON in this example),
```json
{
"name": "projects/my-project/books/les-miserables",
"shelf": "projects/12345/shelves/top-shelf"
}
```
and a `GetBookRequest`,
```proto
message GetBookRequest {
// The name of the Book to retrieve.
// Format: projects/{project}/books/{book}
string name = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
type: "library.googleapis.com/Book"
}];
}
```
and if the value of `name` on such a request is,
```
projects/my-project/books/les-miserables
```
then the value of the field, `name`, returned for the `Book`, **should** match:
the project ID and not the number should be returned. But, the value for the
field, `shelf` should use the project number, as the create request had
submitted a shelf with the project number in the resource name.
In other words, the following values should be returned:
```
book: projects/my-project/books/les-miserables
shelf: projects/12345/shelves/top-shelf
```
This guidance is to ensure that there is a minimal difference between user
input and output.
### Third-party services
[Third-party services](https://cloud.google.com/marketplace) that are
integrated with Google Cloud Platform **must** only store or provide project
numbers. Google's privacy policy prohibits sharing project IDs with third-party
services, or providing a service for third-party services to translate between
project IDs and project numbers at runtime.
### Project identifier format
Services **must** use project resource names as defined by the [Resource
Manager API][] to refer to projects, such as `projects/123456`. This allows the
same API to work with other resources similar to projects, such as
organizations and folders.
[alias]: ../0122.md#resource-id-aliases
[resource manager api]: https://cloud.google.com/resource-manager/
[resource names]: ../0122.md
[example]: ../0122.md#fields-representing-another-resource
## Changelog
- **2022-10-19**: Clarified guidance for project identifiers in resource
references.
- **2021-07-29**: Reversed previous guidance on returning project IDs; this AIP
now advocates returning what the user sent.
- **2019-08-11**: Add an exception for resources that a service does not own.
- **2019-06-19**: Clarify how error messages should be treated
- **2019-06-10**: Minor language and organization tweaks
>>>> 2602.md
---
id: 2602
state: approved
created: 2019-06-12
placement:
category: sdk
---
# Parameter-dependent arguments
When designing CLI commands, a common scenario is that a command corresponding
to some conceptual action may take different, largely non-overlapping arguments
depending on the value of a particular parameter.
For instance, suppose we're designing a new gcloud surface for managing images,
and we need a command that formats an image as JPEG, GIF, or PNG. Regardless of
the image format, the command will need arguments for the source and
destination files, but otherwise depending on the format, an entirely different
set of options will apply. JPEG, for example, might take arguments for
smoothing, subsampling, and DCT method, which are only relevant for JPEG
images. GIF, on the other hand, might take arguments for controlling animated
gifs e.g. whether to loop forever, and a delay time in between frames. Finally,
PNG might take arguments for color type and bits per channel.
## Guidance
When the parameter-specific arguments are numerous relative to the other
arguments, create a command group for the action, with separate subcommands
named after each of the parameter values. In the example above, we would thus
create a `gcloud images format` command group, with subcommands called `jpeg`,
`gif`, and `png`. These subcommands can then take their own format-specific
flags.
Example usage:
```
$ gcloud images format jpeg --help
$ gcloud images format jpeg --source-file=foo --destination-file=bar \
--dct-method=integer --smoothing=0.1 --subsampling=4:4:4
$ gcloud images format png --source-file=foo --destination-file=bar \
--color-type=0 --bits-per-channel=16
$ gcloud images format gif --source-file=foo --destination-file=bar \
--loop-forever --frame-delay=1ms
```
It's possible that over time more and more arguments are added that are common
to all parameters. In that case, it may make sense to instead use a single
command that takes an argument for the parameter. This can be done without
breaking backward compatibility by making the parameter a positional argument.
In the example above, this would involve changing `format` from a command group
to a command, and having it take a positional image format argument which can
be one of: `jpeg`, `gif`, or `png`. Note that all the example commands above
would still function identically (except for the first one, but since it only
affects help text it's not considered a breaking change.)
## Alternatives considered
There are several other possibilities for the design of such a command,
outlined below:
### Single command with an explicit flag corresponding to the parameter
In the example, this would involve a `--type` flag to specify the image format:
```
$ gcloud images format --source-file=source --destination-file=dest \
--type=JPEG --dct-method=integer --smoothing=0.1 --subsampling=4:4:4
$ gcloud images format --source-file=source --destination-file=dest \
--type=GIF --loop-forever --frame-delay=1ms
$ gcloud images format --source-file=source --destination-file=dest \
--type=PNG --color-type=0 --bits-per-channel=16
```
Conceptually it makes the most sense to just have a single format command.
However, this approach has several drawbacks:
- Unnecessary help text. The user will see all of the format-specific options,
most of which will be irrelevant since they apply to different formats. In
graphical image editing programs such as Photoshop or GIMP, the UI can
selectively show these format-specific options once the user chooses the
desired format from a dropdown. On the CLI, however, we have no such
capability because the help text is statically generated.
- Additional logic needed for validation. Since some arguments will be invalid
depending on the format, the command author needs to ensure specifying
invalid combinations returns an appropriate error. While this can be
accomplished with appropriately nested mutex groups, the nesting has the
potential to become overly deep and complex.
### Multiple commands named after the action hyphenated with the parameter value
In the example, this would look like:
```
$ gcloud images format-jpeg ...
$ gcloud images format-gif ...
$ gcloud images format-png ...
```
This is similar to the recommended design in that each parameter value gets its
own command. However, there are disadvantages:
- Backward compatibility. If in the future it becomes desirable to make the
parameter value an argument to a single command, this would necessitate a
breaking change.
- It's less elegant from a command tree layout perspective. Grouping the
parameter-specific commands into a command group allows for a natural
decomposition of the command space, in keeping with gcloud's CLI design
philosophy, and allows for
[progressive disclosure](https://en.wikipedia.org/wiki/Progressive_disclosure)
in the help text and in autocompletion.
>>>> 2603.md
---
id: 2603
scope: cloud
state: approved
created: 2019-06-13
placement:
category: sdk
---
# List command arguments
Some list requests take argument(s) for the parent collection in which to list
resources. For example, suppose an API has book resources belonging to publisher
resources, and consider a request to list books belonging to a publisher:
```proto
message ListBooksRequest {
// The parent, which owns this collection of books.
// Format: publishers/{publisher}
string parent = 1 [
(google.api.field_behavior) = REQUIRED,
(google.api.resource_reference) = {
child_type: "library.googleapis.com/Book"
}];
// The maximum number of items to return.
int32 page_size = 2;
// The next_page_token value returned from a previous List request, if any.
string page_token = 3;
}
```
For the corresponding gcloud list command, we have the choice between
`gcloud publishers books list PUBLISHER` (positional) and
`gcloud publishers books list --publisher=PUBLISHER` (flag).
## Guidance
All list command arguments should be flags, not positionals. In gcloud's
resource model, command groups generally correspond to resources, and the
positional arguments for commands in a group are reserved for those resources.
In the case of a list command that takes an argument, the argument will refer
to the parent resource and not the command group's resource; therefore, it
should be a flag instead of a positional.
In the example above, the list command takes an argument for a publisher. Commands
in the `books` command group should reserve positional arguments for book
resources. Thus, the publisher argument for the list command should be a flag:
```
gcloud publishers books list --publisher=PUBLISHER
```
## Changelog
- **2020-09-23**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 2604.md
---
id: 2604
state: approved
created: 2019-06-13
placement:
category: sdk
---
# Numeric arguments
Some API fields refer to either a percentage or a fixed number. For example, in
GCE a group can be created with a configurable limit for either the percentage
or number of instances in the group that can undergo maintenance
simultaneously:
```proto
message ConcurrencyControl {
enum LimitType {
INVALID = 0;
PERCENT = 1;
FIXED = 2;
}
optional int32 concurrency_limit = 1;
optional LimitType limit_type = 2;
}
```
## Guidance
### Flag layout
In gcloud, such API fields should correspond to a mutually exclusive group
consisting of two flags: one for specifying a number, and the other for
specifying a percentage. The API field in the example above would thus
correspond to the following surface specification in gcloud:
```yaml
- group:
mutex: true
arguments:
- name: concurrency-limit
help_text: |
Maximum number of instances in the group that can undergo maintenance
simultaneously.
- name: concurrency-limit-percent
help_text: |
Integer from 0 to 100 representing the maximum percentage of instances
in the group that can undergo maintenance simultaneously.
```
### Flag naming
Any flag taking a percentage should end with `-percent` (not `-percentage`).
### Flag type
Any flag taking a percentage should take an integer from 0 to 100. If more
precision is required, it is acceptable to take a float from 0 to 100.
## Alternatives considered
### One flag, where percentage is specified if the flag value ends with '%', and number is assumed otherwise
In other words, to specify 10% one would use `--concurrency-limit=10%`, and to
specify 10 instances one would use `--concurrency-limit=10`.
This would be a cleaner design since it naturally maps a single concept (the
limit) to a single flag (`--concurrency-limit`), avoiding the need for
additional flags that clutter the help text.
However, the main reason not to prefer this approach is that it significantly
increases the risk of user error. For instance, suppose a group currently
contains 20 instances with a concurrency limit of 10%, and a user wishes to
update the limit to 20%. If the user issues a describe command (or performs a
GET request) to see the existing limit, the API response will contain something
like:
```yaml
concurrencyControl:
concurrencyLimit: 10
limitType: PERCENT
```
It's easy for the user to gloss over limitType and just assume that
`--concurrency-limit=20` will update the limit to 20%. However, in reality this
would set the limit to 20 instances (100% of the group), potentially leading to
catastrophic consequences.
The recommend approach avoids this ambiguity at the expense of some elegance,
but we deem this tradeoff necessary.
### One flag, where percentage is specified if the flag value ends with '%', and number is specified if the flag value ends with 'n'
In other words, to specify 10% one would use `--concurrency-limit=10%`, and to
specify 10 instances one would use `--concurrency-limit=10n`. It would be an
error to provide a value that doesn't end in '%' or 'n'.
While this avoids the potential for a user to confuse percentages and numbers,
requiring 'n' at the end of a number is inelegant UX. A number should obviously
resemble a number, and the choice of 'n' is also rather arbitrary.
## Exceptions
This CIP does not apply for API fields like timestamps, where we allow the user
to enter either absolute times or durations in the same flag. This is because
absolute times and durations have natural formats that differ, and thus there's
no potential for a user to confuse the two. In general if this is the case, one
flag that accepts both formats should be preferred for the sake of UX
simplicity.
>>>> 2712.md
---
id: 2712
state: approved
created: 2018-07-13
---
# API completeness
Our customers expect that they can be as productive in their Apps programs as
they can be sitting at the UI. When they can not, this is surprising and
disappointing. Historically, Google Workspace has not insisted on this kind of
completeness. However, we want to move towards that goal, hence this
requirement for a plan, while we only advise completeness itself.
For major changes, however, the desire for completeness will be amplified.
## Guidance
Each action that can be performed in the UI **should** be possible to do via a
public API. For example, if there is a button in the UI to add a widget, there
**should** be some way to add a widget using the API. This simplifies the
user's understanding of the API, and makes it easier to record UI actions to be
replayed via the API.
The question of _when_ the completeness ought to be achieved is not fixed. Each
team **must** have a plan for API completeness, even if it is a simple
statement, such as, "When we are given resources to do so." (We would prefer
more, but if that is your plan, then so be it.)
>>>> 2713.md
---
id: 2713
state: approved
created: 2018-07-13
---
# One team owns each type
This guidance supplements [AIP-213][].
Sharing types is tempting, and it is possible. However, it is surrounded by
some complicated issues.
We will consider making shared types for classes that are useful across
multiple Apps APIs, we just will not do it lightly. If you feel you have a case
for this, please contact us at [email protected].
## Guidance
Each API owns its own types.
If your API needs to reference another API's objects, your API **must** contain
a reference ([resource name][aip-122] or ID) that can be used to get the
information from that other API. For example, if your API wants to reference a
relevant Gmail message, you **must** store a Gmail message name, and the
message itself will be accessed via the Gmail API, passing in that name. Your
API **must not** contain an actual Gmail message object.
For smaller types, it is also reasonable to make a copy of the class.
[aip-122]: ../0122.md
[aip-213]: ../0213.md
>>>> 2715.md
---
id: 2715
state: reviewing
created: 2018-09-10
---
# Documenting authorization changes
Authorization is critical to manage well. Understanding an API includes
understanding not only what operations do, but when one is allowed to do them.
Authorization changes, either new authorization constructs or new uses of
existing authorization constructs, should be easy to find in a design rather
than scattered through the document in individual sections.
In order to make it easy to see what (if any) authorization constructs are
being modified in your design, we want to be able to find them in a single,
clear place.
**Note:** This is _not_ talking about RPC-level permissions, like those covered
by `RpcSecurityPolicy`, but about authorization constructs used by your API for
its own purposes.
## Definitions
There are a couple of common types of authorization construct:
A **role** refers to an authorization construct which is granted to an
_identity_ or an _identity group_ on a _resource_. The most well known examples
of roles in G Suite are the `commenter`, `reader`, `writer`, and `owner` roles
from Drive.
A **permission** refers to an authorization construct which is used to check
whether a particular _identity_ can execute some _operation_. For example, to
check whether the current user can delete a file, the application may check
whether the user has the `file.delete` permission on the file. Permissions are
most often assigned to roles, so that only roles are granted to identities or
identity groups, but some systems allow permissions to be granted directly.
Applications often only use each permission to authorize a single operation on
a resource, but there are cases where an application may use a permission to
authorize multiple different operations.
Within an application, authorization checks are made using only one of these
authorization constructs (typically permissions, if the application has that
concept). For example, in order to determine if a user can read a document, an
application would _either_ check whether the user has a `doc.read` permission
_or_ has the `reader` role, but not both.
## Guidance
If your design creates new authorization constructs or extends/modifies the use
of any existing authorization constructs, you **must** have a separate section
in your design document that describes this. You **may** additionally discuss
these in any other place in your document.
It is not important whether this is a top-level section or subsection. It is
important that all such changes are listed together in a distinct section so
they are easy to find and analyze.
If the identifier of an authorization construct that is new or modified is
visible to users or developers, the way that it is identified should also be in
your document. Examples of this are:
- If access is granted using roles and a new role is being added, the way that
the role is identified in the role granting RPCs should be in your document.
- If there is an API that allows permissions assigned to roles to be
manipulated, the way that the permission is identified in the role
manipulation RPCs should be in your document.
>>>> 2716.md
---
id: 2716
state: approved
created: 2018-09-10
---
# Standard terms in names
APIs have a lot of identifier names (fields, constants, methods, ...), that
often contain terms for things, such as `url` (`getUrl()`, `doc_url`, ...) or
`id` (`lookupId()`, `student_id`).
It is useful to have a standard set of terms for common concepts in these
various names. This makes it easier for developers to understand new APIs based
on previous experience. It is not (for example) that `id` is a better term than
`uid` or other options, but it also is not worse, and having different choices
in different APIs is confusing and unnecessary.
This AIP defines terms to use for such common concepts. These terms should be
used for the same concept in identifier names.
This is supplemental to the API design guide, which lists [standard field
names][] and has other rules about identifiers.
[standard field names]: https://cloud.google.com/apis/design/standard_fields
## Guidance
Absent other, more specific guidance, APIs **should** use the following
standard terms (as opposed to alternatives) as parts of identifier names to
refer to common concepts, and **must not** use them to refer to other things:
- `uri` for URIs/URLs (as opposed to `url`, `link`, `href`, etc.)
- `id` for a unique identifier, either globally unique or within a specified
context
- `index` to indicate an ordinal number of an item within a collection, such as
`message_index` on a message to say what place it occupies in a stream of
message (unless there is a more specific name to use, such as if the number
also is used as an ID, use `id`).
APIs **should** avoid use of the term `resource` in fields that have resource
names, as this is redundant.
Any more specific guidance overrides these rules. For example, the [AIP-122][]
rule to use `name` for the resource name overrides using the term `id` for
unique identifiers.
[aip-122]: ../0122.md
## Fallback to schema.org
If you are looking for standard terms for things, [schema.org][] is a good
place to look for terms that have been fairly widely vetted as being normal,
customary, and/or clear. You **should** consult it when choosing identifier
names when other strategies fail.
[schema.org]: https://schema.org/
>>>> 2717.md
---
id: 2717
state: reviewing
created: 2018-09-10
---
# Patterns for generic fields
Developers have several options for how to represent generic values in proto
messages. There are reasons to choose one over the other. Understanding them
will lead to better and more consistent APIs.
## Guidance
APIs **should** follow a consistent application of `oneof` vs.
`map<string, Foo>` vs. `Any` vs. `Struct`.
### `oneof`
A `oneof` is used to create a restriction on a set of optional fields,
enforcing that only one of them may be set (these fields are still separate
individual fields). A common pattern is to have a message that contains a
single `oneof` collection of various message types. Such a `oneof` message is
conceptually similar to a C `union`, or C++ `std::variant`. These **should** be
used in most places where a generic message type is needed, in preference to
other approaches.
**Note:** Adding additional possible values to an existing `oneof` is a
non-breaking change, but moving existing fields into or out of a `oneof` is
breaking (it creates a backwards-incompatible change in Go protobuf stubs).
### `map<string, Foo>`
If a more generic structure is needed, a map of strings to objects **may** be
used. Such a map is represented by a normal JSON object, such as
`{"a": "foo", "b": "bar"}`. The downside to such maps is that they are limited
to flat structures, and they can be difficult to work with because many string
constants may be needed.
### `Any`
An [`Any`][any] allows any message to be packed into the field. This is
conceptually similar to a "bytes" field containing a serialized message. The
advantage of an `Any` is that a user-defined proto message can be stored along
with type information. The user must be able to know which kind of object is in
the `Any`, which complicates the design. The disadvantage is that working with
and debugging any protos is much more complicated since the code may or may not
know how to deserialize the packed message. Also, the developer must contend
with unexpected types of messages in the `Any`.
`Any` **should not** be used as a request parameter. Request parameters will
either be a fixed set of types, in which case a `oneof` **should** be used, or
a type descriptor would need to be sent along, in which case a struct
**should** be used, which achieves essentially the same thing with much simpler
semantics.
<!-- prettier-ignore -->
[any]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto
### `Struct`
The [`Struct`][struct] message can be used to represent arbitrary nested JSON.
For example, given the following code:
```java
Struct.Builder builder = Struct.newBuilder();
Value town = Value.newBuilder().setStringValue("Springfield").build();
Value population = Value.newBuilder().setNumberValue(273).build();
builder.putFields("town", town);
builder.putFields("population", population);
Struct survey = builder.build();
```
`survey` would serialize as the actual JSON
`{"town": "Springfield": "population": 273}`. This message type is fairly
uncommon, and **should** only be used rarely.
<!-- prettier-ignore -->
[struct]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/struct.proto
>>>> 2718.md
---
id: 2718
state: draft
created: 2018-11-08
---
# References to objects in other APIs
Your API may need to refer to objects in other APIs. For example, a Sheets cell
may want to refer to the Slides deck that explains its meaning. This kind of
cross reference should be as natural to use as possible with the target API.
## Guidance
When your API refers to an object accessed via another API (the "target" API),
the reference your API provides depends upon whether the reference is compliant
with One Platform or not.
- If a field always holds a compliant reference to a single specific target
API, the reference you provide **must** be a relative URI from the top level
of the target API's naming hierarchy. For example, if the full URL of the
object is `https://koalas.googleapis.com/koalas/123435` the reference you
return to it **must** be the relative URI `koalas/123435`. Further, your
documentation must make clear that the reference is used with
`koalas.googleapis.com`.
This provides the developer with a natural way to use the reference, using
the returned path directly. It also ensures that they know which API
understands that reference.
- If a field always holds a compliant reference, but is not limited to a single
target API, the reference you provide **must** be a relative URI that
includes the path of its specific target API, such as
`koalas.googleapis.com/koalas/123435`. This provides the developer with the
correct target in a formalized, usable way.
- Otherwise, the reference you return **must** be chosen to be as natural to
use as possible with the target API, and the API and its version must be
documented. For example, if the Apiary-based Koala API uses an ID in its
`GetKoalaData` method, you **must** return the ID that can be used as a
parameter, as well as documenting that you are using IDs that work in Koala
API 1.0.
### Naming
For compliant references, you **should** avoid the use of the terms "reference"
and "name" in the field name. That is, `koala` is preferred to
`koala_reference` or `koala_name`
(see [AIP-122][]).
For other target APIs, you **should** echo that API's terminology in the name
for clarity. For example, if the target API's `GetKoalaDataRequest` has a
`koala_num` field, your field name **should** probably be `koala_num`, or it
would include `koala_num` in its name, such as `responsible_koala_num`.
>>>> 3001.md
---
id: 3001
state: approved
created: 2019-10-03
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# Actions on Google AIP Process
This AIP extends [AIP-1][] with details specific to Actions on Google AIPs. Any
details of [AIP-1][] not modified or contradicted by this AIP also apply to
Actions on Google AIPs.
## Stakeholders
As with any process there are many different stakeholders when it comes to
reviewing and working with AIPs. Below is a summary of the escalation path
starting with the API producer.
```graphviz
digraph d_front_back {
rankdir=BT;
ranksep=0.3;
node [ style="filled,solid" shape=box fontname="Roboto" ];
producer [ label="API Producer" ];
editors [ label="AIP Editors" ];
aog_editors [ label="Actions on Google AIP Editors" ];
tl_infra [ label="Infrastructure TL" ];
tl_design [ label="Design TL" ];
tl [ label="TL" ];
producer -> aog_editors;
aog_editors -> editors;
editors -> tl_infra -> tl;
editors -> tl_design -> tl;
}
```
### Actions on Google Editors
The Actions on Google editors are the set of people who make decisions on
Actions on Google AIPs before escalation to the general editors defined in
[AIP-1][].
The list of Actions on Google AIP editors is currently:
- Ali Ibrahim ([@ahahibrahim][])
- Richard Frankel ([@rofrankel][])
- Shuyang Chen ([@Canain][])
The Actions on Google editors have the same responsibilities as the general
editors. They also have the additional responsibility of establishing
correctness of, and leadership support for, the contents of Actions on Google
AIPs.
Actions on Google AIP editorship is by invitation of the current Actions on
Google editors.
[aip-1]: ../0001.md
[@ahahibrahim]: https://github.com/ahahibrahim
[@rofrankel]: https://github.com/rofrankel
[@canain]: https://github.com/Canain
>>>> 3010.md
---
id: 3010
state: approved
created: 2020-01-14
---
# Actions on Google Vertical Integration Webhook Format
If an action is part of a vertical program that requires a webhook for Google
to integrate with the action, the webhook format **must** follow this AIP.
## Guidance
### Custom method name
Custom methods in a vertical program are defined by Google and implemented by
third-party actions. Guidelines related to names of custom methods in
[AIP-136](../0136.md) **should** be applied.
- The name of the method **should** be a verb followed by a noun.
- The name **must not** contain prepositions (“for”, “with”, etc.).
- If word separation is required for the method name, `lowerCamelCase`
**should** be used.
Examples:
- `getAccountBalance`
- `completeTransaction`
### Base URL in Action Package
The base URL configured in Action Package defines the common part of the actual
execution URL of the webhook.
- The base URL **may** include a trailing slash, but **must** work if
`:customMethodName` is appended directly.
- Google Cloud Functions base URLs without a trailing slash do not work,
because the first segment of the path is used to identify the webhook.
- The endpoint **must** be HTTPS and **must** have a valid certificate.
Examples:
- `https://us-central1.cloudfunctions.net/myWebhook/` (**not**
`https://us-central1.cloudfunctions.net/myWebhook`)
- `https://test.com/assistant/api`
### Actual execution URL
The vertical program **must** compute the actual execution URL by appending
`baseUrl` with `:customMethodName`.
Examples:
- `https://us-central1.cloudfunctions.net/myWebhook/:completeTransaction`
- `https://test.com/assistant/api:completeTransaction`
>>>> 3020.md
---
id: 3020
state: approved
created: 2020-03-11
---
# BII Schema Principles
Built-in intents (BIIs) allow third party developers to declare what their
Actions can do. When a developer registers an Action for a BII, Google can then
invoke that action to fulfill user requests matching that built-in intent.
## Terminology
In the general principles below, we use the following terms:
- **Built-in intents (BII)**: Fulfillment schemas that are used to call
compatible APIs to perform Actions in response to a user query. Built-in
intents allow your service to express its fulfillment capabilities to Google.
By registering for built-in intents and mapping intent parameters to the
fulfillment, it becomes possible for the Google Assistant to invoke the
service to perform a task in response to natural language queries.
A BII comprises an operation (full list below) and a Schema.org entity,
otherwise known as a type. For example, `CREATE_RESERVATION` would be a BII
for initiating a new reservation; this intent applies to various domains such
as airline, hotel, restaurant, and other reservations.
- **Operation**: An Action which acts as a wrapper around a type.
- **Type**: The object on which the operation is acting upon.
## General principles
Each BII **should** be treated like an API. While BIIs are technically
fulfillment schemas, they fill the same role as an API, in that they tell third
party developers about the structure of the request that their Action handles.
This AIP details principles for schema creation, with operation-specific and
type-specific information, followed by a list of operations for BIIs.
### Schema creation
- Existing Schema.org types and properties **should** be reused whenever the
semantics needed already exist.
- The formula for naming a BII **must** be:
- **Operation + Type** `:=` `VERB` ["Operation" from list below] + '\_' +
`NOUN` ["Type" from Schema.org].
- Types can be compound nouns which have more than one token. Types with more
than one token have tokens separated by '\_'.
- _Example: `CREATE` + '\_' + `MONEY_TRANSFER` = `CREATE_MONEY_TRANSFER`_
- BII names **must** be in CAPITAL_SNAKE_CASE (e.g. VERB_NOUN_NOUN).
- For more detail, see [here](https://schema.org/docs/documents.html).
#### Operations
- Operations comprise a small catalog of simple, generalizable verbs. These can
be found at the bottom of the AIP under "List of BII Operations."
- There **must not** be overlap between the scope covered by any operations
(mutual exclusivity).
- _Example: `GET` and `CREATE` do not overlap in scope; while both involve
presenting an object, `GET` is for retrieving an existing object, while
`CREATE` assumes no object yet exists and creates one._
- Operation + type combos **should** make semantic sense. If there is no
operation that can work with the type chosen, either:
- Choose a new type that is in semantic scope for the Action to attach to an
existing operation, with a semantically sensible result.
- _Example: Say you want to fulfill a request for checking into a
reservation. `CREATE_CHECKIN` does not seem to make the most semantic
sense, since nothing is being created. Upon further thought, you decide
to choose a different operation and call the BII `UPDATE_RESERVATION`
instead, which makes more sense with the semantics of checking into a
reservation._
- Or, Create a new operation. Take this step only when there is no operation
in the list that can cover the Action requested for the schema.
- _Example: Imagine you only had the operations `START`, `STOP`, and
`RESUME` for controlling an ongoing activity. A fulfillment request comes
in for temporarily stopping an ongoing activity of media playback but not
terminating it. You try to create a BII called `STOP_MEDIA`, but then
realize that this operation means the API must terminate the activity.
You then create a new operation, `PAUSE`, which acts to "temporarily
stop/pause activity", and does not overlap in scope with the existing
operation, `STOP`. Your new BII is `PAUSE_MEDIA`._
#### Types
- Types **must** be defined as Schema.org types.
- The type **must** be the _object_ upon which the operation is carried out.
- Semantic scope of the type **should** be generalizable, specificity **may**
be in properties.
- _Example: [Reservation][] is a type, which has a property named `provider`.
The `provider` property specifies the service provider, service operator,
or service performer. This ensures that any BII created using the type
[Reservation][] is generalizable to any service provider, and that the
service provider information is still passed forward through
[Reservation][]’s property `provider`._
- Different types **should** take distinctly different properties.
- BIIs **must** be specific enough to avoid semantic ambiguity when calling an
API. That is, we shouldn’t inadvertently trigger an Action that the user
isn’t expecting.
- _Example: brokerage account vs. account: If we do not specify brokerage
account, a `GET_ACCOUNT` BII could call an API that fulfills the Action for
getting another sort of account’s information, such as a social media
account. `GET_BROKERAGE_ACCOUNT` ensures the correct scope of financial
account is retrieved by the API._
- The function prototype/set of parameters **should** be bound to the
operation. That is, the operation decides the fulfillment boundaries of the
Action.
- Each BII operation **must** take a Schema.org type.
- In BII schemas, the same Schema.org type, when combined with different verbs,
**may** take different properties.
- _Example: `GET_MESSAGE` and `CREATE_MESSAGE` use a different but
overlapping set of properties of [Message][]._
## Handling User Queries
- Generic/unresolved search criteria at the level of an object **should** be
passed through the Description field.
- If a query argument can be recognized as a property of a type, it **should**
be passed through as a property; if, on the other hand, an argument is
recognized as just a description of the type, it **should** be passed through
Description field.
- _Example: "find funky blues songs on YouTube" ->
get_media_object(description="funky blues")._
## List of BII Operations
| Verb | Definition | Example BII schema |
| ------ | ------------------------------------------------------------- | ------------------ |
| GET | retrieve & present an object; if ambiguous, a list of objects | GET_CALL_HISTORY |
| CREATE | create an object | CREATE_ORDER |
| DELETE | remove or terminate an existing object | DELETE_RESERVATION |
| START | control an ongoing activity: start activity | START_GAME |
| STOP | control an ongoing activity: stop activity | STOP_ALARM |
| PAUSE | control an ongoing activity: temporarily stop/pause activity | PAUSE_SONG |
| RESUME | control an ongoing activity: continue activity | RESUME_TIMER |
<!-- prettier-ignore-start -->
[message]: http://schema.org/Message
[reservation]: http://schema.org/Reservation
<!-- prettier-ignore-end -->
>>>> 3021.md
---
id: 3021
state: approved
created: 2019-10-04
---
# Type protos in Actions on Google APIs
Many Actions on Google fulfillment APIs need to represent typed data (datetime,
money, etc.). In some cases, there are common components for representing these
types.
## Guidance
Fulfillment APIs **must** use the Actions on Google common types when
applicable, and **must not** use alternative type protos. For example, an API
which needs to represent a monetary amount **must** use
[google.type.Money][money], and **must not** use an alternative representation.
APIs **may** wrap existing types in new protos. This may be useful in order to
define compound types, provide additional metadata, provide semantic signals
(if the typed value was obtained from a user query), etc.
## Actions on Google common types
The Actions on Google common types include all the protobuf types and API types
defined in [AIP-213][], as well as all the protos in
[google.actions.type.\*][actions-type].
### Adding new types
If a given API needs a type for which there is no existing common type, and the
type is not conceptually specific to that API, and the type is well understood
enough to be modeled in a permanent/unversioned way, then API author **should**
consider proposing a new Actions on Google common type in
[google.actions.type.\*][actions-type].
The guidance in [AIP-213][] also applies to creating new
Actions on Google common types. Specifically, API authors proposing new types
**should** consider whether a new type should be a Google-wide common type
rather than an Actions on Google common type. One exception to this is that new
Actions on Google common types are not proposed by opening a GitHub ticket.
Instead, Actions on Google API authors who want to create a new type should
email the owners of the //google/actions/type directory directly.
<!-- prettier-ignore-start -->
[aip-213]: ../0213.md
[date]: https://github.com/googleapis/googleapis/tree/master/google/type/date.proto
[timeofday]: https://github.com/googleapis/googleapis/tree/master/google/type/timeofday.proto
[duration]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/duration.proto
[money]: https://github.com/googleapis/googleapis/tree/master/google/type/money.proto
[postaladdress]: https://github.com/googleapis/googleapis/tree/master/google/type/postal_address.proto
[type]: https://github.com/googleapis/googleapis/tree/master/google/type/
[actions-type]: https://github.com/googleapis/googleapis/tree/master/google/actions/type/
<!-- prettier-ignore-end -->
>>>> 3022.md
---
id: 3022
state: approved
created: 2020-01-30
---
# Built-in intent arguments and fulfillment APIs
Actions on Google's [built-in intents][bii] (BIIs) are a _fulfillment schema_,
and use Schema.org as a foreign vocabulary. BIIs are _not_ a fulfillment API,
and do not use the Actions on Google common types defined in [AIP-3021][].
Actions on Google has two types of generic fulfillment APIs:
- URI-based APIs like [App Actions][], which can only support string arguments.
- Google standard APIs, which support JSON primitive types like `string` and
the Actions on Google common types.
These APIs do not support Schema.org JSON-LD objects. These APIs **should**
support BII arguments by mapping them, or their properties, to the argument
type schema they _do_ support. For example, a BII argument of type
[MonetaryAmount][schema-monetaryamount] may be mapped to an API field of type
[Money][].
## BII argument paths
Only certain Schema.org entity types should be mapped to a given common type.
In order to support BII arguments with types that cannot be mapped to an
appropriate type, APIs **may** support configuration via BII argument paths,
strings which allow users to specify a property of a Schema.org entity which
_can_ be mapped to an appropriate type.
APIs which support BII argument paths **should** use the syntax defined in this
section.
BII argument paths are dot-delimited strings that identify a specific
Schema.org entity within the list of arguments to the BII.
For example, take the following hypothetical BII arguments `grant` and `date`:
```json
{
"grant": {
"@context": "http://schema.org",
"@type": "MonetaryGrant",
"funder": {
"@type": "Organization",
"name": "Google"
},
"amount": {
"@type": "MonetaryAmount",
"currency": "USD",
"value": 100
}
},
"date": "2000-01-01"
}
```
There is no common type to which `MonetaryGrant` can be mapped, but the `grant`
argument contains an `amount` property of type
[MonetaryAmount][schema-monetaryamount], which _can_ be mapped to [Money][].
This property can be referenced with the path string `"grant.amount"`.
Similarly, `grant.name` can be mapped to a string field.
A URI-based API which does not support [Money][] would instead need to use the
path strings `"grant.amount.currency"` and `"grant.amount.value"` to map
`grant.amount` to two separate URI parameters.
On the other hand, `date` can be mapped to [DateTime][], and so the path string
`"date"` would be sufficient for either URI-based APIs or string fields in JSON
APIs.
### Properties with multiple values
Since any Schema.org property may have multiple values, values other than the
first may be referenced by indexing with square brackets. For example, take
this hypothetical BII argument:
```json
{
"menuItem": {
"@context": "http://schema.org",
"@type": "MenuItem",
"name": "pizza",
"menuAddOn": {
"@type": "MenuItem",
"name": "pineapple"
},
"menuAddOn": {
"@type": "MenuItem",
"name": "extra cheese"
}
},
"date": "2000-01-01"
}
```
In this example, the path `"menuItem.menuAddOn[1].name"` would evaluate to
`"extra cheese"`.
When there are no square brackets, the first value is assumed; using `[0]` is a
no-op. That is, `"menuItem.menuAddOn.name"` and `"menuItem.menuAddOn[0].name"`
are equivalent, and would _both_ evaluate to `"pineapple"`.
## BII argument to common type mappings
Google standard APIs which support BII argument mapping **should** support at
least these type mappings from Schema.org entity type to Actions on Google
common types:
| Common type | Schema.org entity types |
| ----------------- | ------------------------------------------------------------ |
| [Date][] | [Date][schema-date] |
| [DateTime][] | [DateTime][schema-datetime] |
| [TimeOfDay][] | [Time][schema-time] |
| [Duration][] | [Duration][schema-duration] |
| [Money][] | [MonetaryAmount][schema-monetaryamount] |
| [PostalAddress][] | [Place][schema-place], [PostalAddress][schema-postaladdress] |
## BII argument to JSON type mappings
Google standard APIs which support BII argument mapping **should** support at
least these type mappings from Schema.org entity type to JSON type:
| JSON type | Schema.org entity types |
| --------- | ------------------------- |
| `string` | [Text][schema-text] |
| `number` | [Number][schema-number] |
| `bool` | [Boolean][schema-boolean] |
## BII argument to string mappings
URI-based fulfillment APIs like [App Actions][] can only map BII arguments to
strings (URI parameters). These APIs **should** support mapping at least the
following Schema.org entity types to URI parameters:
- [Text][schema-text]
- [Number][schema-number]
- [Boolean][schema-boolean]
- [Date][schema-date]
- [DateTime][schema-datetime]
- [Time][schema-time]
<!-- prettier-ignore-start -->
[aip-3021]: ./3021.md
[date]: https://github.com/googleapis/googleapis/tree/master/google/type/date.proto
[datetime]: https://github.com/googleapis/googleapis/blob/master/google/type/datetime.proto
[timeofday]: https://github.com/googleapis/googleapis/tree/master/google/type/timeofday.proto
[duration]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/duration.proto
[money]: https://github.com/googleapis/googleapis/tree/master/google/type/money.proto
[postaladdress]: https://github.com/googleapis/googleapis/tree/master/google/type/postal_address.proto
[app actions]: https://developers.google.com/assistant/app/intents#handling_intent_parameters
[schema-date]: https://schema.org/Date
[schema-datetime]: https://schema.org/DateTime
[schema-time]: https://schema.org/Time
[schema-duration]: http://schema.org/Duration
[schema-monetaryamount]: https://schema.org/MonetaryAmount
[schema-place]: https://schema.org/Place
[schema-postaladdress]: https://schema.org/PostalAddress
[schema-text]: https://schema.org/Text
[schema-number]: https://schema.org/Number
[schema-boolean]: https://schema.org/Boolean
[bii]: https://developers.google.com/assistant/app/reference/built-in-intents/
<!-- prettier-ignore-end -->
>>>> 3201.md
---
id: 3201
scope: firebase
state: draft
created: 2020-06-11
---
# Projects and Apps
## Guidance
### Resource Hierarchy
APIs that utilize an App ID as part of a resource name **must** include a
Project identifier in the parent:
```
apps/*/items
```
Must instead be:
```
projects/*/apps/*/items
```
Project identifiers **must** follow [Cloud guidance][aip-2510]. As App IDs are
globally unique, Firebase APIs **should** allow the [unique resource
lookup][aip-159-unique] pattern, allowing a `-` in place of a Project
ID/Number. For example:
```
projects/-/apps/1:111222333444:web:abcdef0123456789/things
```
### Referencing App IDs
When referencing an App ID in an SDK or API field, method, or parameter, the
platform-correct initialism of `app_id` **must** be used (as opposed to e.g.
`app`, `application_id`, or `gmp_app_id`).
[aip-2510]: ../cloud/2510.md
[aip-159-unique]: ../0159.md#unique-resource-lookup
>>>> 3260.md
---
id: 3260
scope: firebase
state: draft
created: 2020-06-10
---
# Firebase CLI
## Guidance
### Command Naming
Commands **must** be named in lowercase using colons to separate namespaces.
The final segment of a command **should** be an action verb, for example
`list`, `deploy`, or `clone`. A "get" of a single resource **may** omit the
action for the sake of brevity (e.g. `apps:sdkconfig` instead of
`apps:sdkconfig:get`).
### Project Selection
The Firebase CLI works with "project directories" that can be associated with
one or more Firebase projects via the `use` command. This ambient project
selection can be overridden by passing the `-P` or `--project` flag.
All Firebase CLI commands **must** determine the intended project for a command
based on common project selection, except when a command involves multiple
projects (e.g. a command that clones configuration from one project to
another).
### JSON Output
All Firebase CLI commands support machine-readable output via the `--json`
flag. If a command correlates to a single API call, the unmodified response of
that API call should be provided as the JSON response (with wrapped CLI
metadata).
If a command does not directly correlate to an API call, the output **should**
include all information a user might reasonably need for programmatic
interpretation of the command. Output **must** be formatted in snake-case and
generally conform to all standard Google API guidance (e.g. [field
names][aip-140]).
### Arguments and Flags
Required inputs for a command **must** be represented as arguments (e.g.
`database:get /path`) unless doing so would be confusing to the user. For
instance, `appdistribution:distribute` requires as input both a file to
distribute and an App ID, so `--app <appId>` as a required flag is more clear
than two unrelated arguments with no clear hierarchy.
Flags **must** be named using kebab case (e.g. `--example-flag`) and **may**
provide a single-letter alias that does not conflict with [defined global
flags][global-flags].
#### Common Flags
- `--app <appId>` - used to refer to a specific app within a Firebase project.
- `--force` - used to indicate that a command should proceed without additional
prompting regarding potentially destructive actions. For example, deploying
Cloud Functions with `--force` will automatically delete functions, while
without the user is prompted.
- `-o, --output <path>` - this flag **may** be used to allow convenient output
of a serialized response to a local file. If the output is JSON, it **must**
output equivalent JSON to `--json` without wrapped CLI metadata. If the
command fetches a deployable resource, it **must** fetch it in a
deploy-compatible format. The string path **may** be optional if a clear
default path exists. If a file already exists at the path, a confirmation
prompt **must** be displayed unless a `--force` flag is also provided.
### Common Command Types
#### List Commands
Commands that return a list of like resources **must** use a `:list` suffix
(e.g. `projects:list`).
List commands **must not** require manual pagination (e.g. a `--page-token`
flag) and will generally fetch all results (via multiple API calls if
necessary). List commands **may** include a `--limit <int>` flag when a
natural ordering exists, such as a reverse chronological list of releases.
A `--limit 0` flag indicates that all results should be fetched and
returned.
The JSON output of a list command **must** be a concatenated array of all
fetched resources in their original wire format.
#### Clone Commands
If a command's purpose is to copy information from one resource to another, it
**must** be suffixed with `:clone` and **must** take a `from` and `to`
argument:
firebase example:clone <fromProject> <toProject>
To avoid confusion and accidental misuse, clone commands **must** explicitly
require both arguments and **must not** rely on project selection to populate
either.
If a clone operation is destructive (replaces existing state), the command
**should** prompt the user for confirmation before proceeding. If a prompt is
added, the command **must** allow a `--force` flag to bypass the prompt.
### Deployment
Behavior that takes local state (e.g. a rules file) and applies it to the live
operation of a Firebase project **must** integrate with the `deploy` command.
#### Scoped Deployments
If a service has multiple deployable resources, it **may** allow specifying
which resources are deployed through colon namespacing, for example:
firebase deploy --only functions:func1,functions:func2,hosting:site3
Colon namespacing **may** be used to identify one of many like resources (as
with functions above) or to identify specific resource type (e.g.
`firestore:indexes` vs. `firestore:rules`).
If the service integrates with [deploy targets][deploy-targets], the targets
**must** be addressable via colon namespacing.
<!-- prettier-ignore-start -->
[global-flags]: https://github.com/firebase/firebase-tools/blob/master/src/index.js
[deploy-targets]: https://firebase.google.com/docs/cli/targets
[aip-140]: ../0140.md
<!-- prettier-ignore-end -->
>>>> 3270.md
---
id: 3270
scope: firebase
state: draft
created: 2020-06-10
---
# JavaScript APIs
## Guidance
## Changelog
>>>> 3271.md
---
id: 3271
scope: firebase
state: draft
created: 2020-06-10
---
# Android APIs
## Guidance
## Changelog
>>>> 3272.md
---
id: 3272
scope: firebase
state: draft
created: 2020-06-10
---
# iOS APIs
## Guidance
## Changelog
>>>> 3273.md
---
id: 3273
scope: firebase
state: draft
created: 2020-06-10
---
# C++ APIs
## Guidance
## Changelog
>>>> 3274.md
---
id: 3274
scope: firebase
state: draft
created: 2020-06-10
---
# .NET APIs
## Guidance
## Changelog
>>>> 3275.md
---
id: 3275
scope: firebase
state: draft
created: 2020-06-10
---
# Python APIs
## Guidance
## Changelog
>>>> 3276.md
---
id: 3276
scope: firebase
state: draft
created: 2020-06-10
---
# Go APIs
## Guidance
## Changelog
>>>> 4110.md
---
id: 4110
scope: auth
state: approved
created: 2018-06-24
placement:
category: auth
order: 10
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# Application Default Credentials
Google auth libraries use a strategy called _Application Default Credentials
(ADC)_ to detect and select credentials based on environment or context. With
ADC, developers should be able to run the code in different environments and the
supporting systems fetch the appropriate credentials based on each environment
in an effortless manner.
Auth libraries following the standards in these AIPs are known as _"Google
Unified Auth Clients"_, or _GUAC_ for short. The resulting libraries are
colloquially called _GUACs_.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
### Credential Types
This section outlines the supported credential types of the ADC.
- **Gcloud Credential**: A credential provided by the [Gcloud tool][0] that
identifies a human user that needs to authenticate to access Google APIs. The
auth libraries **must** support this credential type.
- **Service Account Key**: A credential that identifies a non-human user that
needs to authenticate to access Google APIs. The auth libraries **must** support
this credential type.
- **OAuth Client ID**: A credential that identifies the client application which
allows human users to sign-in through [3-legged OAuth flow][1], which grants the
permissions to the application to access Google APIs on behalf of the human
user. The auth libraries **may** support this credential type.
- **External Account Credential**: A configuration file identifying
[external non-Google credentials][8] that can be exchanged for Google access
tokens to access Google APIs. The auth libraries **must** support this
credential type.
### Environment Variables
The auth libraries **must** support the following environment variables to allow
developers to provide authentication configuration for their application:
- **GOOGLE_APPLICATION_CREDENTIALS**: The specified value will be used as the
full path for ADC to locate the credentials file. The credentials file
**should** be one of the following types:
- Gcloud credentials
- Service account key
- External account credentials
The credentials **may** be the OAuth Client ID if it is supported by the
auth library. Credentials file path specified at the program level (e.g. via
client options) **must** have priority over the value of this environment
variable.
- **GOOGLE_API_USE_CLIENT_CERTIFICATE:** The specified value **must** be
either true or false. The client certificate **must** be ignored if this
variable is set to false. The default value is false if the value is unset.
```
GOOGLE_API_USE_CLIENT_CERTIFICATE=[true|false]
```
- **GOOGLE_CLOUD_QUOTA_PROJECT:** The quota project id to be set on the credential.
The value from the environment variable will override any quota project that is
present in the credential detected by the ADC mechanism.
### Inputs & Outputs
From the input/output perspective, the inputs of _ADC_ **should** be the
credentials as well as the underlying environment such as environment variables
or metadata service that provides these credentials.
For example, the `GOOGLE_APPLICATION_CREDENTIALS` environment variable can provide
the default credential JSON as the input here, or the well-known path that
gCloud uses to store the default user credential JSON. The output is the access
token that application can use to access the Google APIs. This access token
__may__ be a bearer token, a certificate-bound token, or an identity-bound token
depending on the chosen authentication flow.
## Expected Behavior
This section outlines the expected behavior of the ADC. Auth libraries **must**
implement these concepts in order to be considered complete.
```graphviz
digraph d_front_back {
rankdir=TB;
ranksep=0.3;
node [ style="filled,solid" shape=box fontname="Roboto" ];
check_env_var [ label="1. Check Environment Variables" ];
load_credentials [ label="2. Load Credentials" ];
check_metadata [ label="3. Check workload credentials" ];
auth_flows [ label="4. Determine Auth Flows" ];
execute [ label="5. Execute Auth Flows" ];
post_processing [ label="6. Post Processing" ];
check_env_var -> load_credentials -> check_metadata -> auth_flows -> execute -> post_processing;
load_credentials -> auth_flows;
check_metadata -> post_processing;
}
```
1. **Check environment variables**
1. Check GOOGLE_APPLICATION_CREDENTIALS
1. If set, go to step (2.2)
1. If not set, go to step (2)
1. **Load credentials**
1. Check [gcloud default credentials][5] through its default path
1. If found go to step (2.2)
1. Otherwise go to step (3)
1. Check the provided credential type
1. If the credential is gcloud credentials, go to step (4)
1. If the credential is [a service account key][6] JSON, go to step (4)
1. If the credential is [an external account][8] JSON, go to step (4)
1. If the credential is unknown type, return an error saying that _[END]_
1. Credentials not found _[END]_
1. **Check workload credentials (on GCE, GKE, GAE and Serverless)**
1. If true,
1. If identity binding is enabled, by meeting the requirements in
[mTLS Token Binding][9], use the mTLS Token Binding flow to fetch an
identity-bound access token. Go to step(6).
1. If there is an issue when obtaining bound access tokens, return an error
indicating that _[END]_
1. If identity binding is not enabled, use the [virtual machine flow][3] to
fetch an auth token associated with the current environment
1. If target audience is provided by the developer, get an [identity token][7]. Go to step(6).
1. Otherwise, get an access token. Go to step(6).
1. If false, go to step (2.3)
1. **Determine auth flows**
1. If the credential is gcloud credential go to step (5.3)
1. If target audience or scope is provided by the developer go to step (5.1)
1. If the credential is an external account go to step (5.4)
1. Otherwise, go to step (5.2)
1. **Execute auth flows**
1. Use 2LO flow to exchange for an auth token
1. If target audience is provided by the developer, get an [identity token][7]. Go to step(6).
1. Otherwise, get an access token. Go to step(6).
1. If client certificate is presented, the exchanged token will be a certificate bind token. Go to step(6).
1. Use self-signed JWT flow to create an access token locally.
1. If certificate is presented, embed the certificate into the JWT.
1. Use the regular [self-signed JWT flow][4] for an access token. Go to step(6).
1. Use user identity flow to exchange for an access token. Go to step(6).
1. Use [external account][8] flow to exchange for an access token. Go to step(6).
1. **Post Processing**
1. Update Quota Project
1. If a quota project is provided explicitly while initiating ADC, override the quota project in the credential with the explicit value. _[END]_
1. Else if the `GOOGLE_CLOUD_QUOTA_PROJECT` environment variable is set, override the quota project in the credential with this value. _[END]_
## Changelog
- **2019-08-13**: Add a link to virtual machine flow (AIP 4115).
- **2019-08-18**: Remove STS support from ADC.
- **2021-01-20**: Add identity token flow (AIP 4116).
- **2021-06-29**: Guidance for GOOGLE_API_KEY temporarily removed until consensus can be established.
- **2021-12-10**: Add external account credentials (AIP 4117).
- **2023-01-23**: Add Quota Project Environment variable.
<!-- prettier-ignore-start -->
[0]: https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login
[1]: https://developers.google.com/identity/protocols/oauth2/native-app
[3]: ./4115
[4]: ./4111
[5]: ./4113
[6]: ./4112
[7]: ./4116
[8]: ./4117
[9]: ./4119
<!-- prettier-ignore-end -->
>>>> 4111.md
---
id: 4111
scope: auth
state: approved
created: 2020-07-23
---
# Self-signed JWT
Self-signed JWT (JSON Web Token) for Google APIs was introduced in 2014 to
provide a more efficient auth stack for Cloud APIs being accessed via service
accounts by bypassing the intermediate step of exchanging client assertions for
OAuth tokens. A self-signed JWT is a spec-conforming JWT that is locally signed
by the service account private key. Since many Cloud APIs accept self-signed JWT
in place of traditional access tokens, it is now considered a standard
authentication method.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
This section describes the general guidance of supporting self-signed
JWT as a means of authentication.
### Application Default Credentials
Supporting self-signed JWT is considered a part of [application default
credentials][0]. To understand the overall flow, please read [AIP-4110][0].
### Scope vs. Audience
In the past, the self-signed JWT was only applicable with `audience`. Recently,
the usage of the "scope" claim in self-signed JWTs is now allowed in the Google
Cloud auth ecosystem, which makes the self-signed JWT usable with both audience
and scope. However, caller **must** not specify scope and audience at the same
time to prevent potential conflicts.
#### Opt-in Strategy
Currently, this scope support for JWT is gated behind a configurable boolean
option `UseJWTAccessWithScope`(the actual naming may vary depending on the
programming language), as the feature is not supported for all Cloud API service
backends. The default value of this boolean is false, and to use scope with
self-signed JWT, caller **must** specify this option as `true`. Otherwise the
OAuth exchange flow **should** be used instead when scope is provided. The API
clients **should** try to automatically determine this option with the
appropriate value based on the APIs, so this complexity is not exposed to the
upper-layer developer.
### Default Authentication Method For Service Account Keys
Considering its better efficiency and reliability comparing with OAuth flow
(bypassing the exchanging step), [ADC][0] **should** use self-signed JWT as the
default authentication flow when service account key is provided as the source
credential.
In other words, [ADC][0] **should** choose the OAuth flow over the self-signed
JWT flow only if scope is provided and JWT is not enabled via
`UseJWTAccessWithScope`.
### Expected Behavior
To support self-signed JWT, the auth libraries **must** follow the steps below:
1\. Load the [service account ID JSON file][2]. Please note that the self-signed
JWT only supports service account ID credential type.
2\. Using any standard JWT library, such as one found at [jwt.io][1], create a JWT
with a header and payload like the following examples:
(1) Example of using audience:
```json
{
"alg": "RS256",
"typ": "JWT",
"kid": "abcdef1234567890"
}
{
"iss": "[email protected]",
"sub": "[email protected]",
"aud": "https://pubsub.googleapis.com/",
"iat": 1511900000,
"exp": 1511903600
}
```
(2) Example of using scope:
```json
{
"alg": "RS256",
"typ": "JWT",
"kid": "abcdef1234567890"
}
{
"iss": "[email protected]",
"sub": "[email protected]",
"scope": "https://www.googleapis.com/auth/cloud-platform",
"iat": 1511900000,
"exp": 1511903600
}
```
* For the `kid` field in the header, specify the service account's private key
ID. You can find this value in the `private_key_id` field of the service account
JSON file.
* For the `iss` and `sub` fields, specify the service account's email address.
You can find this value in the `client_email` field of the service account JSON
file.
* For the `aud` field, specify the audience. The default audience value
**should** be `https://[SERVICE]/`. (e.g. `https://pubsub.googleapis.com/`)
* For the `scope` field, specify the scope. Please note that `aud` and `scope`
cannot be specified at the same time.
* For the `iat` field, specify the current Unix time, and for the `exp` field,
the value **must** be exactly 3600 seconds later, when the JWT will expire.
3\. Sign the JWT with RSA-256 using the private key found in the service
account JSON file.
## Changelog
- **2019-08-13**: Add a section for application default credential.
- **2021-06-02**: Update scope support.
<!-- prettier-ignore-start -->
[0]: https://google.aip.dev/auth/4110
[1]: https://jwt.io/#libraries-io
[2]: https://google.aip.dev/auth/4112
<!-- prettier-ignore-end -->
>>>> 4112.md
---
id: 4112
scope: auth
state: approved
created: 2020-08-13
---
# Service Account Keys
A service account is a special kind of account used by an application or a
virtual machine (VM) instance, not a person. Applications use service accounts
to make authorized API calls, authorized as either the service account itself,
or as Google Workspace or Cloud Identity users through domain-wide delegation.
Service accounts are associated with private/public RSA key-pairs that are used
for authentication, which is the focus of this AIP.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
This section describes the general guidance of supporting Service Account Key
authentication using traditional OAuth flow. For Service Account Key
authentication using self-signed JWT flow, please refer to the [self-signed
JWT][0] AIP.
### Service Account Key Authentication Using OAuth
Under the latest [ADC][1] guidance, when Service Account Key is provided as the
source credential, we prefer to use the [self-signed JWT][0] flow by default,
which does not use the OAuth "scope" argument. Therefore, the client **should**
choose the OAuth flow over self-signed JWT flow only if the OAuth "scope" is
explicitly provided to the client.
### Expected Behavior
To support Service Account Key authentication using OAuth flow, the auth
libraries **must** follow the steps below:
1\. Load the **Service Account ID JSON file** from the file path specified by
**GOOGLE_APPLICATION_CREDENTIALS** environment variable, or an explicit
**credentials** file path specified through client options. The JSON file will
look like below:
```json
{
"type": "service_account",
"project_id": "testproject",
"private_key_id": "redacted",
"private_key": "redacted",
"client_email": "[email protected]",
"client_id": "113258942105700140798",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/testserviceaccount%40test.iam.gserviceaccount.com"
}
```
**Note:** A Service Account ID JSON file can be downloaded from [Google Cloud Console][2].
2\. Using any standard JWT library, such as one found at [jwt.io][3], create a
JWT with a header and claims set like the following example:
```json
{
"alg": "RS256",
"typ": "JWT",
"kid": "abcdef1234567890"
}
{
"iss": "[email protected]",
"sub": "[email protected]",
"scope": "https://www.googleapis.com/auth/pubsub",
}
```
* For the `kid` field in the header, specify the service account's private
key ID. You can find this value in the **private_key_id** field of the
service account JSON file.
* For the `iss` and `sub` fields, specify the service account's email
address. You can find this value in the **client_email** field of the
service account JSON file.
* For the `scope` field, specify the **scope** parameter provided by the client.
Multiple scopes can be specified using single space (" ") as delimiter.
3\. Sign the JWT with RSA-256 using the **private_key** found in the service
account JSON file. The output will be used as the request payload (a.k.a.
assertion) for the 2-legged OAuth flow.
4\. Define a parameter "grant_type" as
"urn:ietf:params:oauth:grant-type:jwt-bearer". Define a parameter "assertion" as
the payload from step 3.
5\. Make a POST request to the **token_uri** endpoint with the parameters from
step 4 to obtain an OAuth2 Access Token.
<!-- prettier-ignore-start -->
[0]: https://google.aip.dev/auth/4111
[1]: https://google.aip.dev/auth/4110
[2]: https://console.cloud.google.com/apis/credentials
[3]: https://jwt.io/#libraries-io
<!-- prettier-ignore-end -->
>>>> 4113.md
---
id: 4113
scope: auth
state: approved
created: 2020-07-31
---
# gcloud CLI Integration
[gcloud][0] is the CLI that manages authentication, local configuration,
developer workflow, and interactions with the Google Cloud Platform (GCP) APIs.
gcloud can generate default credentials by obtaining user access credentials
via 3-legged OAuth (3LO) web flow and saving them to well-known locations in
JSON format. These credentials are intended to be consumed by applications
other than gcloud. [Application Default Credentials (ADC) and Google Unified
Auth Clients (GUAC)][1] **must** support the gcloud default credentials.
**Note:** Although the sample code is written in Python, this AIP describes
guidance and requirements in a language-neutral way. It uses generic
terminology which may be imprecise or inappropriate in certain languages or
environments.
## Guidance
This section describes the general guidance of supporting the gcloud default
credentials.
### Credentials Generation
gcloud default credentials can be generated via command ‘[gcloud auth
application-default login][2]’.
```
$ gcloud auth application-default login
```
The generated credentials are saved to well-known locations which vary as
platforms:
- **Linux, Mac**: \$HOME/.config/gcloud/application_default_credentials.json
- **Windows**: %APPDATA%/gcloud/application_default_credentials.json
Below is an example of the gcloud default credentials,
```json
{
"client_id": "fake_id.apps.googleusercontent.com",
"client_secret": "fake_secret",
"quota_project_id": "fake_project",
"refresh_token": "fake_token",
"type": "authorized_user"
}
```
All the fields are populated by the login response from the Google
authorization backend except for ‘quota_project_id’ which is retrieved from
gcloud’s context. Additionally, the users can override ‘quota_project_id’ with
the ‘--client-id-file’ flag,
```
$ gcloud auth application-default login --client-id-file=clientid.json
```
### Expected Behaviors
The auth libraries will use the information in the gcloud default credentials
to exchange access tokens with Google authorization backend. The resulting
access tokens will be further used by applications to call GCP APIs.
The auth libraries and applications **must** follow the steps below:
- The auth library loads the gcloud default credentials from the well-known
location (see previous [section](#credentials-generation)) of the platform on
which it runs.
- The auth library calls Google authorization backend with the refresh token,
client ID and client secret in the gcloud default credentials and procures an
access token. The URL for acquiring access tokens from Google authorization
backend is https://oauth2.googleapis.com/token. Below is an example code with
[google-auth][3] as the auth library and [urllib3][4] as the http transport.
```python
from google.oauth2 import _client as google_auth_client
import google.auth.transport.urllib3 as google_auth_urllib3
import urllib3
# Create an http transport for communicating with Google authorization backend.
http = urllib3.PoolManager()
request = google_auth_urllib3.Request(http)
# Build parameters for the access token request. Assume the gcloud default
# credentials are deserialized are into a dictionary ‘gcloud_default’
# in the previous step.
token_uri = 'https://oauth2.googleapis.com/token'
refresh_token = gcloud_default['refresh_token']
client_id = gcloud_default['client_id']
client_secret = gcloud_default['client_secret']
scopes = ['https://www.googleapis.com/auth/cloud-platform']
# Obtain an access token.
access_token, _, _, _ = google_auth_client.refresh_grant(
request, token_uri, refresh_token, client_id, client_secret, scopes)
```
- The application calls GCP API with the access token obtained from the
previous step. ‘quota_project_id’ in the gcloud default credentials
**should** be added to the ‘X-Goog-User-Project’ http header so that the
associated account will be charged for billing and quota.
<!-- prettier-ignore-start -->
[0]: https://cloud.google.com/sdk/gcloud/reference
[1]: https://google.aip.dev/auth/4110
[2]: https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login
[3]: https://google-auth.readthedocs.io/en/latest/index.html
[4]: https://urllib3.readthedocs.io/en/latest/
<!-- prettier-ignore-end -->
>>>> 4114.md
---
id: 4114
scope: auth
state: approved
created: 2020-07-28
---
# Device Certificate Authentication via Mutual TLS
Mutual TLS (a.k.a mTLS) authentication enables authentication of both client and
server identities in a TLS handshake and provides the foundation for Device
Certificate Authentication for Context Aware Access for GCP enterprise
customers. With Device Certificate Authentication, the server takes into account
the client certificate in the transport layer when making authorization
decisions. The use of a client certificate provides a stronger indication of the
access originating from a trusted device. This allows enterprise customers to
restrict access from untrusted devices.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
This section describes the general guidance of supporting Device Certificate
Authentication (a.k.a. DCA) via mTLS.
### Application Default Credentials for DCA
Users **should** enable DCA through [ADC][0] instead of manual configuration via
client options.
There are two aspects of ADC for DCA. The first is the automatic procurement of
a device certificate. The second is the automatic switching of the service
endpoint to the mTLS version of the service endpoint. Clients **must** support
both of these aspects when supporting ADC for DCA.
### Expected Behavior
Client support for DCA **must** give priority to user overrides specified via
client options. The following decision tree **should** be used:
1. If user specifies both device certificate and endpoint override via client
options, use them as is.
1. If user does not specify device certificate, attempt to procure and use
a default device certificate.
1. If user does not specify endpoint override, use the default mTLS endpoint if
a device certificate is available and the default regular endpoint otherwise.
Implications of the above logic:
1. If user specifies a non-mTLS endpoint override but a device certificate is
available, pass along the certificate anyway and let the server decide what
to do.
1. If user specifies an mTLS endpoint override but device certificate is not
available, do not fail-fast, but let server return error when connecting.
The above behavior avoids introducing client-side logic that parses whether the
endpoint override is an mTLS url, since the url pattern may change at anytime.
### Obtaining the Default mTLS Endpoint
The default mTLS endpoint for a service **should** be read from the Discovery
Document field **"mtlsRootUrl"** instead of generated via regex patterns.
### Obtaining the Default Device Certificate via SecureConnect
The default device certificate **should** be procured using the
[EndpointVerification][1] workflow, which fetches the certificate from a
platform-specific credential store (ex. KeyChain in macOS) via a native helper.
Exporting the certificate via the native helper involves executing a **"cert
provider command"** specified in a well-known gcloud metadata file of the
following format:
```json
{
"version": 1
"min_cloud_sdk_version": "240.0.0"
"has_client_cert": true
"endpoint_verification_error": ""
"cert_provider_command": "[absolute_path_to_provider_command] --fetch_client_cert"
}
```
For Linux and macOS platforms, the above metadata file is located at
**"~/.secureConnect/context_aware_metadata.json"**.
The cert provider command will print the certificate to stdout, which will be in
the form of an X.509 cert followed immediately by the private key:
-----BEGIN CERTIFICATE-----
Common Name: Google Endpoint Verification
Valid From: November 10, 2019
Valid To: November 10, 2020
Serial Number: 4921083229008411918 (0x444b331faf2dbd0e)
...
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
### Obtaining the Default Device Certificate via ECP
The Enterprise Certificate Proxy (ECP) project is the newly recommended way to
procure device certificates. It has two major advantages compared to the legacy
SecureConnect mechanism:
1. Allows usage of enterprise certs and private keys stored in native keystores and TPMs
instead of relying on self-signed certs.
1. Delegates signing operations to keystores, so private keys never leave the security realm.
Please see [ECP Public Documentation][2] for details on ECP configuration.
### Environment Variables
There are situations where the ADC for DCA behavior needs to be modified, such
as for integration testing, or for failsafe. To accomodate those scenarios, the
following environment variables **should** be supported.
**GOOGLE_API_USE_MTLS_ENDPOINT**: If **"always"**, always use mTLS endpoint. If
**"never"**, always use regular endpoint. If **"auto"**, use the default
behavior, which is to use the mTLS endpoint if a device certificate is
available. The default value of this environment variable will be "auto".
**GOOGLE_API_USE_CLIENT_CERTIFICATE**: If **"true"**, device certificate
authentication will be supported as described in the general guidance. If
**"false"**, the device certificate **must** not be used, even if specified by
the user. The default value **should** be "true" as of May 3, 2024. Users who
wish to disable DCA feature **must** explicitly set this environment variable
to "false".
### Firewall and IP Address Guidance
For any given GCP service, its mTLS endpoint has a different IP address compared
to the non-mTLS endpoint but is expected to fall within the same IP range. The
GCP guidance for IP range management is on a service by service basis. See
[Compute Engine IP Range Documentation][3] for example. In the unlikely event
that an end-user has configured firewall rules based on exact IP addresses
instead of an IP range, they may be impacted by the "auto" mTLS endpoint
upgrade behavior. The best-practice recommendation in this case would be to
avoid adding rules that expect exact IP address matches, and instead use
range-based IP rules following public GCP documentation.
## Changelog
- **2024-11-25*: GOOGLE_API_USE_CLIENT_CERTIFICATE should default to "true" as of May 3, 2024.
- **2025-02-20*: Add Firewall and IP Address Guidance
<!-- prettier-ignore-start -->
[0]: https://google.aip.dev/auth/4110
[1]: https://cloud.google.com/endpoint-verification/docs/overview
[2]: https://cloud.google.com/beyondcorp-enterprise/docs/enable-cba-enterprise-certificates
[3]: https://cloud.google.com/compute/docs/faq#find_ip_range
<!-- prettier-ignore-end -->
>>>> 4115.md
---
id: 4115
scope: auth
state: approved
created: 2020-08-13
---
# Default Credentials For Google Cloud Virtual Environments
If the client runs on Google cloud virtual environments such as [Google Compute Engine (GCE)][0],
[Serverless][1], or [Google Kubernetes Engine (GKE)][2], the auth library **may** leverage
Google’s default mutual TLS (mTLS) credentials and obtain bound tokens for the instance.
The auth library **may** use the default mTLS credentials and bound tokens to access Google APIs.
mTLS authentication enables authentication of both client and server identities in a TLS handshake.
Applications running in Google virtual environments can authenticate to Google APIs using X.509
SPIFFE Verifiable Identity Documents (SVIDs). These SVIDs are X.509 certificates that contain SPIFFE
IDs specifying the identity of the certificate owner.
Bound tokens are access tokens that are bound to some property of the credentials used to establish
the mTLS connection. The advantage of bound tokens is that they can be used over secure channels
established via mTLS credentials with the correct binding information, when appropriate access
policies have been put in place. Therefore, using bound tokens is more secure than bearer tokens,
which can be stolen and adversarially replayed.
This AIP describes the flow of:
1. Retrieving a configuration through a metadata server (MDS) endpoint. The configuration specifies
how to access Google’s default mTLS credentials.
2. Requesting bound tokens.
**Note:** Because this AIP describes guidance and requirements in a language-neutral way, it uses
generic terminology which may be imprecise or inappropriate in certain languages or environments.
## Guidance
### Access Default mTLS Credentials
**Note:** Before trying to use Google’s default mTLS credentials, the client **must** first check if the remote
Google API endpoint supports mTLS. If the remote endpoint does NOT support mTLS, the client **should**
connect to the endpoint using TLS. How to check if an endpoint supports mTLS is out of the scope of this
AIP. If the remote endpoint does support mTLS, the client **should** try to connect using mTLS first
before falling back to TLS. How to find the remote API’s mTLS endpoint is out of the scope of this AIP.
If users enabled [Device Certificate Authentication (DCA)](4), the client **should** give priority to DCA
as mTLS credentials.
To leverage Google’s default mTLS credentials, the client **should** retrieve configurations from
MDS. The MDS in all virtual environments (GCE, Serverless, and GKE) exposes an HTTP endpoint that
serves a configuration that specifies how to access Google's default mTLS credentials. This endpoint
is called the mTLS configuration endpoint.
The URL of the MDS's mTLS configuration endpoint is:
```
http://metadata.google.internal/computeMetadata/v1/instance/platform-security/auto-mtls-configuration
```
The request to the MDS's mTLS configuration endpoint **should** be an HTTP GET request without any
parameter or payload.
The response from the MDS's mTLS configuration endpoint **should** contain the following
information:
* The **Secure Session Agent** address: the client doesn’t have direct access to mTLS credentials.
The Secure Session Agent manages default mTLS credentials. The client can only use mTLS
credentials through the Secure Session Agent. The address can be an IP:port address or a file path
representing a Unix Domain Socket (UDS).
The client **must** follow the steps below to access Google’s default mTLS credentials.
1. Check if the remote endpoint supports mTLS.
* If yes, go to step (2).
* If not, go to step (3).
2. Send a request to the MDS's mTLS configuration endpoint. If the request is successful and the
response contains a Secure Session Agent address, use the address to access Google's default mTLS
credentials, and go to step (4). If the request fails or the response contains an empty address,
go to step (3).
3. Fall back to TLS [END].
4. Configure the TLS library to use the Secure Session Agent ([example][3]) for client authentication
during the mTLS handshake.
### Request Bound Tokens
To access Google APIs with bound tokens, the client **should** request tokens from MDS. The MDS in
all virtual environments (GCE, Serverless, and GKE) exposes an HTTP endpoint that serves access tokens.
This endpoint is called the access token endpoint.
The URL of the MDS's access token endpoint is:
```
http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token
```
The request to the MDS's access token endpoint **should** be an HTTP GET request. The request **may**
have a “scopes” URL parameter with a list of comma-separated scopes. The auth library **should** allow
the caller to optionally specify a list of custom scopes, and add the “scopes” parameter to the request
when needed. Depending on the runtime environment, the request for custom scopes **may** be transparently
ignored or fulfilled by the server.
The response from the MDS's access token endpoint **should** contain an access token in the following
JSON format:
```json
{
"access_token": "YOUR_ACCESS_TOKEN",
"expires_in": 3599,
"token_type": "Bearer"
}
```
The client **must** follow the steps below to request new access tokens for Google APIs if existing
tokens expire.
1. Send an HTTP request to the MDS access token endpoint, retrieve the access token from the response
and go to step (2).
2. Attach the token from step (1) to the request to Google APIs.
## Changelog
- **2020-12-14**: Replace note on scopes with more detailed discussion.
- **2021-07-13**: Clarify GCE equivalent runtimes
- **2023-02-16**: Add mTLS configuration endpoint and unify the token binding flow.
<!-- prettier-ignore-start -->
[0]: https://cloud.google.com/compute
[1]: https://cloud.google.com/serverless
[2]: https://cloud.google.com/kubernetes-engine
[3]: https://github.com/google/s2a-go/tree/main/example
[4]: https://google.aip.dev/auth/4114
<!-- prettier-ignore-end -->
>>>> 4116.md
---
id: 4116
scope: auth
state: draft
created: 2021-01-06
---
# Identity Tokens
Identity Tokens (or ID Tokens) are JSON Web Tokens (JWT) with additional claims pertaining to the identity of the client or end-user (such as "email"). Which additional claims fields are included in the ID token are dependent on the ID
token provider.
The **fetching** of ID tokens is used to:
* request a [protected Cloud Run service][0], which requires a valid ID token in the `Authentication` header.
* request an [IAP-protected resource][2], which requires a valid ID token in the `Authentication` header.
The **validation** of ID tokens should be done by:
* services receiving an [authenticated PubSub push message][1] sent outside of Google Cloud.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
This section describes the general guidance of ID tokens as a means of
authentication.
### ID tokens obtained by Application Default Credentials
When a target audience is provided by the developer, [ADC][3] **should** fetch
an ID token with the credentials provided. For example, if service account keys
are provided, ADC will request an ID token from the OAuth token endpoint. If
[virtual machine credentials][4] are provided, ADC will request an ID token from
the virtual machine server.
### Expected Behavior for Fetching ID Tokens
To support fetching ID Tokens, the auth libraries **must** follow the steps
below:
1\. Accept a target audience parameter when Application Default Credentials
are requested. If a target audience and scope are both provided to the auth library, an exception **must** be thrown.
2\. Virtual machine credentials return the ID token retrieved from the metadata
server's `instance/service-accounts/default/identity` endpoint, setting the
`audience` querystring parameter to the provided target audience.
3\. Service account Keys use the [JWT Bearer][5] grant type, and set the
`target_audience` claim to the provided target audience. This will return a JSON
object containing an `id_token` key.
User Identity flows may be used to issue ID tokens, but are not required to.
### Expected Behavior for Verifying ID Tokens
To support verifying ID Tokens, the auth libraries **must** provide a method to
validate a JWT token which can do the following:
1\. Validate a JWT with an ES256 signature.
2\. Validate JWT `aud` claim against a user-provided audience.
3\. Validate JWT `exp` claim against the current timestamp.
<!-- prettier-ignore-start -->
[0]: https://cloud.google.com/run/docs/authenticating/service-to-service
[1]: https://cloud.google.com/pubsub/docs/push#setting_up_for_push_authentication
[2]: https://cloud.google.com/iap/docs/signed-headers-howto#securing_iap_headers
[3]: https://google.aip.dev/auth/4110
[4]: https://google.aip.dev/auth/4115
[5]: https://tools.ietf.org/html/rfc7523
<!-- prettier-ignore-end -->
>>>> 4117.md
---
id: 4117
scope: auth
state: approved
created: 2020-12-10
---
# External Account Credentials (Workload Identity Federation)
Using workload identity federation, your application can access Google Cloud
resources from Amazon Web Services (AWS), Microsoft Azure or any identity
provider that supports OpenID Connect (OIDC) or SAML 2.0.
Traditionally, applications running outside Google Cloud have used service
account keys to access Google Cloud resources. Using identity federation,
you can allow your workload to impersonate a service account. This lets you
access Google Cloud resources directly, eliminating the maintenance and
security burden associated with service account keys.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
This section describes the general guidance of supporting non-Google external
credentials (AWS, Azure, OIDC and SAML IdPs, etc) as a means of authentication.
### Prerequisite
In order to use workload identity federation to access Google cloud resources
from non-Google cloud platforms, the following steps are needed to configure
workload identity pools, providers, service account impersonation and generate
the JSON configuration file to be used by the auth libraries.
- [Configure Workload Identity Federation from AWS][0]
- [Configure Workload Identity Federation from Microsoft Azure][1]
- [Configure Workload Identity Federation from an OIDC identity provider][2]
- [Configure Workload Identity Federation from a SAML identity provider][3]
### Configuration File Generation and Usage
After workload identity federation is configured, the JSON configuration file
should be generated. This sample shows how an AWS configuration file is
generated:
```bash
$ gcloud iam workload-identity-pools create-cred-config \
projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
--service-account=$SERVICE_ACCOUNT_EMAIL \
--aws \
--output-file=$FILEPATH.json
```
The following values would need to be replaced:
- **PROJECT_NUMBER**: Project number of the project that contains the workload
identity pool.
- **POOL_ID**: ID of the workload identity pool.
- **PROVIDER_ID**: ID of the workload identity pool provider.
- **SERVICE_ACCOUNT_EMAIL**: Email address of the service account to
impersonate.
- **FILEPATH**: File to save configuration to.
If you are using [AWS IMDSv2](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html),
an additional flag `--enable-imdsv2` should be added to the [`gcloud iam workload-identity-pools create-cred-config`](/sdk/gcloud/reference/iam/workload-identity-pools/create-cred-config) command:
```bash
$ gcloud iam workload-identity-pools create-cred-config \
projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
--service-account=$SERVICE_ACCOUNT_EMAIL \
--aws \
--enable-imdsv2 \
--output-file=$FILEPATH.json
```
If you wish to configure the service account access token lifetime,
an additional flag `--service-account-token-lifetime-seconds` should be added to the [`gcloud iam workload-identity-pools create-cred-config`](/sdk/gcloud/reference/iam/workload-identity-pools/create-cred-config) command (this example uses an AWS configuration, but the token lifetime can be configured for all workload identity federation providers):
```bash
$ gcloud iam workload-identity-pools create-cred-config \
projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID \
--service-account=$SERVICE_ACCOUNT_EMAIL \
--aws \
--service-account-token-lifetime-seconds=$TOKEN_LIFETIME \
--output-file=$FILEPATH.json
```
The service-account-token-lifetime-seconds flag is optional. If not provided, this defaults to one hour. The minimum allowed value is 600 (10 minutes) and the maximum allowed value is 43200 (12 hours). If a lifetime greater than one hour is required, the service account must be added as an allowed value in an Organization Policy that enforces the `constraints/iam.allowServiceAccountCredentialLifetimeExtension` constraint.
The external identities configuration file can be used with
[Application Default Credentials][6]. In order to use external identities with
Application Default Credentials, the full path to this file should be stored
in the `GOOGLE_APPLICATION_CREDENTIALS` environment variable.
```bash
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/config.json
```
The library can now automatically choose the right type of client and initialize
credentials from the context provided in the configuration file:
```python
import google.auth
credentials, project = google.auth.default()
```
External account credentials can also be initialized explicitly using the
generated configuration file.
```python
# Sample for Azure or OIDC/SAML providers.
import json
from google.auth import identity_pool
json_config_info = json.loads(function_to_get_json_config())
credentials = identity_pool.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
['https://www.googleapis.com/auth/cloud-platform'])
```
```python
# Sample for AWS.
import json
from google.auth import aws
json_config_info = json.loads(function_to_get_json_config())
credentials = aws.Credentials.from_info(json_config_info)
scoped_credentials = credentials.with_scopes(
['https://www.googleapis.com/auth/cloud-platform'])
```
## Expected Behavior
The auth libraries should use the information in the JSON configuration file to
retrieve the external credentials and exchange them for Google access tokens
using the GCP Security Token Service (via the token exchange endpoint
`https://sts.googleapis.com/v1/token`) and then impersonating a service account
by calling the **IamCredentials** [generateAccessToken][5] API to access GCP
resources.
All external account JSON files must share the following fields:
| Field Name | Required | Description |
|-----------------------------------|----------|:------------|
| type | Yes | This identifies the new type of credential object. This must be "external_account" |
| audience | Yes | This is the STS audience which contains the resource name for the workload identity pool and the provider identifier in that pool. |
| subject_token_type | Yes | This is the STS subject token type based on the [OAuth 2.0 token exchange spec][7]. |
| service_account_impersonation_url | No | This is the URL for the service account impersonation request. If this is not available, the STS returned access token should be directly used without impersonation. |
| service_account_impersonation.* | No | This object defines additional service account impersonation options. Only one field is currently supported: “token_lifetime_seconds": This is the requested access token lifetime, e.g. `2800`. |
| token_url | Yes | This is the STS token exchange endpoint. |
| credential_source.* | Yes | This object defines the mechanism used to retrieve the external credential from the local environment so that it can be exchanged for a GCP access token via the STS endpoint. |
The auth libraries and applications **must** follow the steps below for all
types of external account credentials:
- Check **credential_source** to determine the necessary logic to retrieve the
external credential which should be used to construct the subject token to
pass to the STS endpoint. This is covered in detail for every credential
configuration below.
- Construct the STS request, based on [rfc8693][4]:
- STS audience should be constructed using the **audience** field.
- **grant_type** must be `urn:ietf:params:oauth:grant-type:token-exchange`
- **requested_token_type** must be
`urn:ietf:params:oauth:token-type:access_token`
- **subject_token_type** is the **subject_token_type** field as described in
the RFC.
- **subject_token** is the retrieved external credentials. Check the
subsequent sections on how this is retrieved in various environments.
- **scope**: the list of space-delimited, case-insensitive OAuth scopes that
specify the desired scopes of the requested security token in the context
of the service or resource where the token should be used. If service
account impersonation is used, the cloud platform or IAM scope should be
passed to STS and then the customer provided scopes should be passed in the
**IamCredentials** call to [generateAccessToken][5].
- The STS token exchange URL should be the **token_url** (e.g.
`https://sts.googleapis.com/v1/token`).
- Send the STS token exchange request to get the Google access token and its
expiration.
- If the **service_account_impersonation_url** is available, trigger service
account impersonation flow by POSTing to that endpoint with the previously
returned Google access token.
- If this is not available, end the flow and just use the STS access token
for authorization.
- The list of scopes also need to be provided for this endpoint. The customer
provided scopes should be used for this endpoint.
- In order to access this API, the (Cloud platform
`https://www.googleapis.com/auth/cloud-platform` or IAM scope
`https://www.googleapis.com/auth/iam`) are required in the underlying
access token.
- The service account access token lifetime also needs to be provided for this endpoint. The value in
**service_account_impersonation.token_lifetime_seconds** will be used if it
was provided, otherwise it will default to 1 hour.
### Determining the subject token in AWS
External account configuration JSON files should contain the following
information in the `credential_source` object to facilitate retrieval of AWS
credentials to be passed as subject tokens to the GCP STS token exchange
endpoint.
| Field Name | Required | Description |
|--------------------------------------------------|----------|:------------|
| environment_id | Yes | This is the environment identifier, of format `aws${version}`. A version should be specified to indicate to the auth library whether breaking changes were introduced to the underlying AWS implementation. So if aws1 is supported in the current version of the library but a credential file with aws2 is provided, an error should be thrown instructing the developer to upgrade to a newer version of the library. |
| region_url | No | This URL should be used to determine the current AWS region needed for the signed request construction when the region environment variables are not present. |
| url | No | This AWS metadata server URL should be used to retrieve the access key, secret key and security token needed to sign the `GetCallerIdentity` request. The $ROLE_NAME should be retrieved from calling this endpoint without any parameter and then calling again with the returned role name appended to this URL: http://169.254.169.254/latest/meta-data/iam/security-credentials/$ROLE_NAME |
| regional_cred_verification_url | Yes | This defines the regional AWS `GetCallerIdentity` action URL. This URL should be used to determine the AWS account ID and its roles. This should not actually be called by the Auth libraries. It should be called on the STS token server. The region should be substituted by SDK, e.g. `sts.eu-west-1.amazonaws`.com. |
| imdsv2_session_token_url | No | Presence of this URL enforces the auth libraries to fetch a Session Token from AWS. This field is required for EC2 instances using IMDSv2. This Session Token would later be used while making calls to the metadata endpoint. |
The JSON file for AWS configuration files should have the following form:
```json
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
"token_url": "https://sts.googleapis.com/v1/token",
"credential_source": {
"environment_id": "aws1",
"region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
"url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
"regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
"imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
}
}
```
The auth libraries and applications **must** follow the steps below:
- Check **credential_source** for environment ID. If the environment ID is
`aws${version}`, this should be an AWS native credential.
- Inspect the version in the environment ID. If this is a newer unexpected
error, trigger an error that the auth library needs to be updated to handle
this type of credentials.
- Validate the host for the **url**, **regional_url** and
**imdsv2_session_token_url** fields if they are provided. The host should
either be **169.254.169.254** or **fd00:ec2::254**.
- If **imdsv2_session_token_url** is available, then fetch session token
from **imdsv2_session_token_url**.
- Check the environment variables in the following order (`AWS_REGION` and
then the `AWS_DEFAULT_REGION`) to determine the AWS region. If found, skip
using the AWS metadata server to determine this value.
- If the region environment variables are not provided, use the **region_url**
to determine the current AWS region. The API returns the zone name, e.g.
`us-east-1d`. The region should be determined by stripping the last
character, e.g. `us-east-1`.
- Check the environment variables `AWS_ACCESS_KEY_ID`,
`AWS_SECRET_ACCESS_KEY` and the optional `AWS_SESSION_TOKEN` for the AWS
security credentials. If found, skip using the AWS metadata server to
determine these values.
- If **url** is available and the security credentials environment variables
are not provided:
- Call **url** to retrieve the attached AWS IAM role name to the current
instance.
- Call **url/$ROLE_NAME** to get the access key, secret key and security
token needed to sign the `GetCallerIdentity` request.
- Construct the AWS signed request ([AWS Signature Version 4][8]) using the
[GetCallerIdentity][9] **regional_cred_verification_url** (with the region
substituted). This should be serialized by formatting it as a url-encoded
JSON and passed as the **subject_token** to STS endpoint.
Here is a sample of the JSON format used:
```json
{
"url": "https://sts.us-east-1.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
"headers": [
{
"value": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"key": "x-goog-cloud-target-resource"
},
{
"value": "20200228T225005Z",
"key": "x-amz-date"
},
{
"value": "AWS4-HMAC-SHA256 Credential=AKIASOZTBDV4D7ABCDEDF/20200228/us-east-1/sts/aws4_request, SignedHeaders=host;x-amz-date,Signature=abcedefdfedfd",
"key": "Authorization"
},
{
"value": "sts.us-east-1.amazonaws.com",
"key": "host"
},
{
"value": "IQoJb3JpZ2luX2VjEIz//////////wEaCXVzLWVh...",
"key": "x-amz-security-token"
}
],
"method": "POST",
"body": ""
}
```
For the AWS token, STS requires a special header `x-goog-cloud-endpoint` to recognize that the token is for a specific workload identity provider.
### Determining the subject token in Microsoft Azure and URL-sourced credentials
External account configuration JSON files should contain the following
information in the `credential_source` object to facilitate retrieval of Azure
and other URL-sourced credentials to be passed as subject tokens to the GCP STS
token exchange endpoint.
| Field Name | Required | Description |
|---------------------------------------------------|----------|:-------------|
| url | Yes | This defines the local metadata server to retrieve the external credentials from. For Azure, this should be the Azure Instance Metadata Service (IMDS) URL used to retrieve the Azure AD access token. |
| headers | No | This defines the headers to append to the GET request to credential_source.url. |
| format.type | No | This indicates the format of the URL response. This can be either "text" or "json". The default should be "text". |
| format.subject_token_field_name | No | Required for JSON URL responses. This indicates the JSON field name where the subject_token should be stored. |
The JSON file for URL-sourced configuration files (OIDC / SAML) should have the
following form:
```json
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
"token_url": "https://sts.googleapis.com/v1/token",
"credential_source": {
"url": "http://localhost:5000/token",
"format": {
"type": "json",
"subject_token_field_name": "id_token"
}
}
}
```
Azure configuration files are a type of OIDC URL-sourced credentials.
```json
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
"token_url": "https://sts.googleapis.com/v1/token",
"credential_source": {
"headers": {
"Metadata": "True"
},
"url": "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"format": {
"type": "json",
"subject_token_field_name": "access_token"
}
}
}
```
The auth libraries and applications **must** follow the steps below:
- Check **credential_source** has a **url** field and no **environment_id**,
otherwise skip the rest of the steps.
- An HTTP GET request should be sent to this local **url** while injecting
the **headers** key/values (if provided in the configuration file) in the
request header. The request should respond with the external credentials
subject token to be passed to STS token endpoint.
- Before parsing the token, check the **format** field.
- If the **format** is not available, assume the external credential returned
by the URL response is provided in plain text format.
- If available, check if the type is **json**
- If json, check the **subject_token_field_name**.
For Azure, this is set to **access_token**.
- Parse the file as JSON and then retrieve the external credential from
the field name based on the value of **subject_token_field_name**.
### Determining the subject token in file-sourced credentials
External account configuration JSON files contain the following information
in the `credential_source` object to facilitate retrieval of file-sourced
credentials to be passed as subject tokens to the GCP STS token exchange
endpoint.
| Field Name | Required | Description |
|---------------------------------------------------|----------|:------------|
| file | Yes | This is the source of the credential. This should be used for a credential locally available. This should take precedence over `url` when both are provided. |
| format.type | No | This indicates the format of the file where the token is stored. This can be either "text" or "json". The default should be "text". |
| format.subject_token_field_name | No | Required for JSON file formats. This indicates the JSON field name where the `subject_token` should be stored. |
The JSON file for file-sourced configuration files (OIDC / SAML) should have
the following form:
```json
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/$EMAIL:generateAccessToken",
"token_url": "https://sts.googleapis.com/v1/token",
"credential_source": {
"file": "/var/run/saml/assertion/token"
}
}
```
The auth libraries and applications **must** follow the steps below:
- Check **credential_source** has a **file** field and no **environment_id**. If not,
this is not a file-sourced credential and the proceeding steps do not apply.
- Get the external credential from the file location specified by the
`credential_source.file` field.
- Before parsing the token, check the **format** field.
- If the **format** is not available, assume the external credential is
provided in plain text format.
- If available, check if the type is **json**
- If json, check the **subject_token_field_name**.
- Parse the file as JSON and then retrieve the external credential from
the field name based on the value of **subject_token_field_name**.
### Determining the subject token in executable-sourced credentials
External account configuration JSON files contain the following information
in the `credential_source` object to facilitate retrieval of executable-sourced
credentials to be passed as subject tokens to the GCP STS token exchange
endpoint.
| Field Name | Required | Description |
|---------------------------|----------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| executable | Yes | Holds the information necessary to run the executable. |
| executable.command | Yes | Specifies the full command to run to retrieve the subject token. This can include arguments. Must be an absolute path for the program. |
| executable.timeout_millis | No | Specifies the timeout duration, in milliseconds. Defaults to 30 seconds when not provided. |
| executable.output_file | No | Specifies the absolute path to the output file where the executable will cache the response. By specifying this path, the auth libraries will first check this location before running the executable. The format of the file should match the JSON format expected by the auth libraries defined below. |
The JSON file for executable-sourced configuration files (OIDC / SAML) should have
the following form:
```json
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/$PROJECT_NUMBER/locations/global/workloadIdentityPools/$POOL_ID/providers/$PROVIDER_ID",
"subject_token_type": "urn:ietf:params:oauth:token-type:saml2",
"token_url": "https://sts.googleapis.com/v1/token",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[email protected]:generateAccessToken",
"credential_source": {
"executable": {
"command": "/path/to/executable --arg1=value1 --arg2=value2",
"timeout_millis": 5000,
"output_file": "/path/to/cached/credentials"
}
}
}
```
To use executable-sourced credentials, the `GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES` environment variable must be set to `1`.
Additionally, the executable **must** adhere to the following response format:
Successful responses:
| Field Name | Type | Description |
|---------------------------|---------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| version | number | The version of the JSON output. Currently only version 1 is supported. |
| success | boolean | The status of the response. True in this case. |
| token_type | string | The 3rd party subject token type. Must be *urn:ietf:params:oauth:token-type:jwt*, *urn:ietf:params:oauth:token-type:id_token*, or *urn:ietf:params:oauth:token-type:saml2*. |
| id_token OR saml_response | string | The 3rd party OIDC token or SAML response. |
| expiration_time | number | The optional 3rd party subject token expiration time in seconds (unix epoch time). Only required in the response when an output file is specified in the credential configuration. |
A sample successful executable OIDC response:
```json
{
"version": 1,
"success": true,
"token_type": "urn:ietf:params:oauth:token-type:id_token",
"id_token": "...",
"expiration_time": 1620499962
}
```
A sample successful executable SAML response:
```json
{
"version": 1,
"success": true,
"token_type": "urn:ietf:params:oauth:token-type:saml2",
"saml_response": "...",
"expiration_time": 1620499962
}
```
Error responses:
| Field Name | Type | Description |
|------------|---------|:-----------------------------------------------------------------------|
| version | number | The version of the JSON output. Currently only version 1 is supported. |
| success | boolean | The status of the response. False in this case. | |
| code | string | The error code. |
| message | string | The error message. |
A sample executable error response:
```json
{
"version": 1,
"success": false,
"code": "401",
"message": "Caller not authorized."
}
```
The auth libraries and applications **must** follow the steps below:
- Check **credential_source** has an **executable** field and no **environment_id**. If not,
this is not a executable-sourced credential and the proceeding steps do not apply.
- Retrieve the external credential's executable information from the
**credential_source.executable** field.
- Check that the `GOOGLE_EXTERNAL_ACCOUNT_ALLOW_EXECUTABLES` environment variable is set to **1**. If not, error out.
- Before the next step, check if **credential_source.executable.output_file** was specified in the credential configuration.
- If present, check if there is an executable response at that location.
- If the response is valid and unexpired, or there is no response at that location, continue execution.
- If the response is malformed or invalid, error out.
- Ensure the following environment variables will be available to the executable:
- `GOOGLE_EXTERNAL_ACCOUNT_AUDIENCE`: The audience field from the credential configuration. Must always be present.
- `GOOGLE_EXTERNAL_ACCOUNT_TOKEN_TYPE`: The subject token type. Must always be present.
- `GOOGLE_EXTERNAL_ACCOUNT_IMPERSONATED_EMAIL`: The service account email. Only present when service account impersonation is used.
- `GOOGLE_EXTERNAL_ACCOUNT_OUTPUT_FILE`: The output file location from the credential configuration. Only present when specified in the credential configuration.
- Run the command specified at **credential_source.executable.command**.
- Fail in the following scenarios:
- The executable failed to complete in the timeout duration specified.
- The executable's response is invalid, was unsuccessful or expired.
- The executable finished with a non-zero exit code.
- Parse the executable response as JSON and then retrieve the external credential from
the field name based on the value of **token_type**.
- The token_type value must be **urn:ietf:params:oauth:token-type:jwt**,
**urn:ietf:params:oauth:token-type:id_token**, or **urn:ietf:params:oauth:token-type:saml2**.
- If the **token_type** is **urn:ietf:params:oauth:token-type:saml2**, the subject token will be parsed from the **saml_response** field.
- Otherwise it will be parsed from the **id_token** field.
## Changelog
- **2021-12-10**: Add AIP for External Account Credentials (AIP 4117).
- **2022-05-18**: Document executable-sourced credentials (AIP 4117).
- **2022-08-31**: Document configurable token lifetime (AIP 4117).
- **2023-09-12**: Mark region_url as optional (AIP 4117).
<!-- prettier-ignore-start -->
[0]: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#aws
[1]: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#azure
[2]: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#oidc
[3]: https://cloud.google.com/iam/docs/configuring-workload-identity-federation#saml_1
[4]: https://tools.ietf.org/html/rfc8693
[5]: https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/generateAccessToken
[6]: https://google.aip.dev/auth/4110
[7]: https://tools.ietf.org/html/rfc8693#section-3
[8]: https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
[9]: https://docs.aws.amazon.com/STS/latest/APIReference/API_GetCallerIdentity.html
<!-- prettier-ignore-end -->
>>>> 4118.md
---
id: 4118
scope: auth
state: draft
created: 2022-09-09
---
# Mutual Authentication Using Workload Credentials
Mutual TLS (a.k.a mTLS) authentication enables authentication of both client and
server identities in a TLS handshake. With workload credentials, applications
running in Google Cloud can authenticate to Google APIs using [X.509 SPIFFE
Verifiable Identity Documents][0] (SVIDs). These SVIDs are X.509 certificates
that contain [SPIFFE IDs][1] specifying the identity of the certificate owners.
mTLS authentication using X.509 SVIDs occurs when the client uses an X.509 SVID
when performing the TLS handshake.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
If users enable token binding, they **should** do so via [ADC][2]. This section
describes the general guidance of supporting such authentication.
### Provisioning Workload Credentials in Google Cloud
On Google Cloud, workload credentials should be provisioned using one of the
following methods:
- [Set up service security with Envoy][3].
- [Set up service security with proxyless gRPC][4].
In order for workload credentials to be properly used, the auth libraries
**must** support the automatic switching of the service endpoint to its mTLS
counterpart.
### Using Workload Credentials
Users **should** configure [ADC][2] to use workload credentials via the
certificate configuration gcloud metadata file. Workload credentials can be
added as a **"cert_configs"** type as follows:
```json
{
"version": 1
"cert_configs": {
"workload": {
"cert_path": "path/to/cert/file"
"key_path": "path/to/key/file"
"workload_identity_provider": "..."
"authenticate_as_identity_type": "gsa/native"
"service_account_email": "..."
},
"keychain": {
...
},
"pkcs11": {
...
},
"windows": {
...
},
},
"libs": {
...
}
}
```
For Linux and macOS platforms, the above metadata file is located in the
well-known gcloud config directory at
**"~/.config/gcloud/certificate_config.json"**. Note that the default location
of this file can be changed using the `GOOGLE_API_CERTIFICATE_CONFIG`
environment variable.
The following lists the fields of the **"workload"** certificate info type that
are relevant to workload credentials:
- **"cert_path"**: The specified value will be used as the full path to locate
the workload certificate file. This file **must** contain a PEM-encoded
X.509 certificate chain (ordered from leaf to root) where the leaf
certificate is a valid X.509 SVID. The chain __may__ consist of only the
leaf certificate.
- **"key_path"**: The specified value will be used as the full path to locate
the workload private key file. This file must contain a PEM-formatted
private key associated with the X.509 certificate specified by
**“cert_path”**.
The description of the **"workload_identity_provider"**,
**"authenticate_as_identity_type"** and **"service_account_email"** fields can
be found in [mTLS Token Binding][5].
To enable mutual authentication to Google APIs using workload credentials, the
**"workload"** section and its **"cert_path"** and **"key_path"** values must be
present in the **"~/.config/gcloud/certificate_config.json"** configuration
file.
### Expected Behavior
Support for mTLS authentication to Google APIs using workload credentials
**must** give priority to user mTLS endpoint override via client options. The
auth libraries **must** follow the steps below:
- Locate the workload certificate and private key files using the above
config file. If one of these files is not present, mTLS using workload
credentials may be disabled. The auth libraries **must** check that the
public and private keys in the certificate and key files match before
passing them to the TLS library.
- Occasional mismatches may happen, since during certificate rotation the
client library may read the two files while another process is replacing
them. In that case, the library **must** retry reading the certificate
and private key files and checking their match status, up to a maximum
of four attempts. The library **should** wait for 5 seconds between
attempts.
- If the certificate and private key files are loaded in memory (as
opposed to being read from disk for every mTLS connection), the auth
libraries **must** periodically reload them (at least every 10 minutes
or when the certificate expires) to refresh their copies in memory after
the infrastructure rotates them. Refreshing the credentials **must** be
done in a background thread and not upon usage.
- Configure the TLS library to use the found, and matched, certificate and
key for client authentication during the TLS handshake.
- If the user specifies the endpoint override via client options, use it as is
and connect to the specified endpoint using mTLS.
- If the user does not specify the endpoint override, use the default mTLS
endpoint if the certificate and key files exist and the default regular
endpoint otherwise.
Note that mTLS 1.3 **must** be the only supported version to preserve client
identity and certificate confidentiality.
One implication of the above logic is that if the user enables mTLS
authentication using workload credentials, provides valid certificate and key
files, and specifies a non-mTLS endpoint override, the client libraries
**should** use the certificate and key anyway and let the server decide what to
do. This avoids introducing client-side logic that parses whether the endpoint
override is an mTLS URL, since the URL pattern may change at any time.
### Obtaining the Default mTLS Endpoint
The default mTLS endpoint for a service **should** be read from the Discovery
Document field **"mtlsRootUrl"** instead of generated via regex patterns.
<!-- prettier-ignore-start -->
[0]: https://github.com/spiffe/spiffe/blob/main/standards/X509-SVID.md
[1]: https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE-ID.md#2-spiffe-identity
[2]: https://google.aip.dev/auth/4110
[3]: https://cloud.google.com/traffic-director/docs/security-envoy-setup
[4]: https://cloud.google.com/traffic-director/docs/security-proxyless-setup
[5]: https://google.aip.dev/auth/4119
<!-- prettier-ignore-end -->
>>>> 4119.md
---
id: 4119
scope: auth
state: draft
created: 2022-09-09
---
# mTLS Token Binding
Token binding allows issuing Google access tokens that are bound to mTLS
credentials. The advantage of such mTLS bound tokens is that they are meant to
only be used over secure channels established via mTLS credentials they are
bound to. Therefore, using bound tokens is more secure than bearer tokens which
can be stolen and adversarially replayed.
This AIP describes the flow of (1) obtaining access tokens bound to X.509
certificate identities, called identity-bound tokens and (2) how to use them to
access Google APIs using the Google auth libraries.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments.
## Guidance
If users enable token binding, they **should** do so via [ADC][0]. This section
describes the general guidance of supporting such tokens.
### Prerequisites
Identity-bound access tokens require that the clients have
[X.509 SPIFFE Verifiable Identity Documents][1] (SVIDs). [Mutual Authentication
Using Workload Credentials][2] describes how such SVIDs are provisioned in
Google Cloud.
Additionally, identity-bound access tokens tokens require configuring a workload
identity pool and identity provider with Google Cloud's IAM. The instructions on
how to do this are out of scope of this AIP.
### Using mTLS Token Binding
The auth libraries **must** support the following values in the
**"~/.config/gcloud/certificate_config.json"** configuration file. Note that the
default location of this file can be changed using the
`GOOGLE_API_CERTIFICATE_CONFIG` environment variable.
```json
{
"version": 1
"cert_configs": {
"workload": {
"cert_path": "path/to/cert/file"
"key_path": "path/to/key/file"
"workload_identity_provider": "..."
"authenticate_as_identity_type": "gsa/native"
"service_account_email": "..."
},
"keychain": {
...
},
"pkcs11": {
...
},
"windows": {
...
},
},
"libs": {
...
}
}
```
The following lists the fields relevant to mTLS token binding configuration:
- **"workload_identity_provider"**: The specified value will be used to
populate the request to Security Token Service (STS) to request
identity-bound access tokens. This value refers to the fully qualified name
of the workload identity pool and identity provider configured in IAM. The
specified value **must** be of the following format.
```
"workload_identity_provider":"//iam.googleapis.com/projects/<project_number>/locations/global/workloadIdentityPools/<pool_identifier>/providers/<provider_identifier>"
```
- **"authenticate_as_identity_type"**: This field specifies what identity is
used to authenticate to Google APIs. The value can be set to `gsa` or
`native`, where `gsa` is the GCP service account of the workload, e.g., the
GCP service account of a GCE VM, and `native` is the native workload
identity, e.g., the GKE pod kubernetes service account. If not specified,
the default value is `gsa`.
- **"service_account_email"**: If set, the specified value will be used to
populate the request to the IAM Credentials service to request
identity-bound access tokens. This value refers to the service account email
to be used for resource access. If not set, the service account email will
be determined automatically by querying the following Metadata Service
endpoint:
`http://metadata/computeMetadata/v1/instance/service-accounts/default/email`.
The value of this field is only relevant if
**"authenticate_as_identity_type"** is set to `gsa`.
The description of the **"cert_path"** and **"key_path"** fields can be found in
[Mutual Authentication Using Workload Credentials][2].
To enable using token binding when communicating with Google APIs the following
conditions are required:
- [Mutual Authentication Using Workload Credentials][2] **must** be enabled.
- The **"workload_identity_provider"** **must** be present,
**"authenticate_as_identity_type"** __may__ be set and
**"service_account_email"** __may__ be set in the **"workload"**
section of the **"~/.config/gcloud/certificate_config.json"** configuration
file.
### Expected Behavior
To support the usage of identity-bound access tokens, the auth libraries
**must** follow the steps below when sending requests to Google APIs:
1. Connect to the mTLS endpoint of the [STS API][3] using the workload
credentials provisioned as described in [Mutual Authentication Using
Workload Credentials][2]. This endpoint **must** be
`sts.mtls.googleapis.com`.
1. Send an HTTP request to STS’s [ExchangeToken][5] method requesting an
identity-bound token using the information in the
**"workload_identity_provider"** field in the
**"~/.config/gcloud/certificate_config.json"** configuration file. The
scope of the requested token **must** be
`https://www.googleapis.com/auth/iam`.
1. Connect to the mTLS endpoint of the [IAM Credentials Service API][4] using
the workload credentials provisioned as described in [Mutual Authentication
Using Workload Credentials][2]. This endpoint **must** be
`iamcredentials.mtls.googleapis.com`.
1. If **"authenticate_as_identity_type"** is set to `gsa`, send an HTTP
request to the IAM Credentials Service’s [GenerateAccessToken][6] method
requesting an identity bound token asserting the service account email in
the **"service_account_email"** field in the
**"~/.config/gcloud/certificate_config.json"** configuration file. The
scope of this token **must** be the same scope defined by the user for
accessing the requested Google API.
1. Attach the returned token in Step 4 to the request. Note that this request
**must** be sent over an mTLS channel using the same workload credentials
in Step 1.
<!-- prettier-ignore-start -->
[0]: https://google.aip.dev/auth/4110
[1]: https://github.com/spiffe/spiffe/blob/main/standards/X509-SVID.md
[2]: https://google.aip.dev/auth/4118
[3]: https://cloud.google.com/iam/docs/reference/sts/rest
[4]: https://cloud.google.com/iam/docs/reference/credentials/rest
[5]: https://cloud.google.com/iam/docs/reference/sts/rest/v1/TopLevel/token
[6]: https://cloud.google.com/iam/docs/reference/credentials/rest/v1/projects.serviceAccounts/generateAccessToken
<!-- prettier-ignore-end -->
>>>> 4210.md
---
id: 4210
state: approved
created: 2018-06-22
js_scripts:
- /assets/js/graphviz/viz.js
- /assets/js/graphviz/lite.render.js
- /assets/js/aip/aip-graphviz.js
---
# Client library generators
API guidelines exist in order to promote simple, intuitive, and consistent
APIs. Users familiar with APIs that generally adhere to AIP guidance are able
to take what they learn in prior APIs and apply it to new ones.
Client libraries provide a mechanism for users to get started with APIs more
quickly, by simplifying common concerns (such as auth) and by a language-native
way to call API endpoints and receive language-native responses. However, for
these libraries to provide the most value, they also must be simple, intuitive,
and consistent. Code generators provide a means for producing consistent client
libraries at scale.
Code generators following the standards in these AIPs are known as "generated
API client generators", or _GAPIC generators_ for short. The resulting
libraries are colloquially called _GAPICs_.
**Note:** Because this AIP describes guidance and requirements in a
language-neutral way, it uses generic terminology which may be imprecise or
inappropriate in certain languages or environments (for example, the use of the
term `class` even though languages such as Go do not have classes). This AIP's
particular use of vocabulary is best understood as an explanation of
principles, and precise adherence to exact vocabulary in this AIP is not an
expectation.
## Guidance
The general flow of code generation for client libraries and supporting code in
most supported languages is outlined below.
**Note:** Exceptions to this pattern are typically due to use of a unique stack
e.g. Node.js use of `protobuf.js` and `grpc-node` which don't have code
generation, or Python using a protobuf wrapper in `proto-plus-python`, but the
general GAPIC flow remains the same.
```graphviz
digraph {
node [ style="filled,solid" shape=box fontname="Roboto" ];
splines=ortho;
nodesep=0.3;
center=true;
proto [ label="API Protobuf\nDescriptors" shape=rectangle fillcolor=aliceblue ];
subgraph cluster_code_generators {
rank = same;
style = filled;
fillcolor = lightgrey;
node [ shape=oval ];
protobuf [ label="protobuf\ngenerator" fillcolor=deepskyblue3 ];
grpc [ label="gRPC\ngenerator" fillcolor=gold3 ];
gapic [ label="GAPIC\ngenerator" fillcolor=darkseagreen ];
}
proto -> protobuf;
proto -> grpc;
proto -> gapic;
subgraph cluster_generated_code {
rank = same;
style = filled;
fillcolor = lightgrey;
node [ shape=rectangle ];
protobuf_output [ label="Message & Enum\nCode" fillcolor=deepskyblue3 ];
grpc_output [ label="Server & Client\nStubs" fillcolor=gold3 ];
gapic_output [ label="Google API\nClient" fillcolor=darkseagreen ];
}
protobuf -> protobuf_output;
grpc -> grpc_output;
gapic -> gapic_output;
assembly [ label="Package\nassembly" shape=oval fillcolor=aliceblue ];
protobuf_output -> assembly
grpc_output -> assembly
gapic_output -> assembly
assembled_package [ label="Package of\ngenerated code" fillcolor=aliceblue ];
assembly -> assembled_package
}
```
The following sections focus on the "GAPIC generator" in the above diagram.
### Protobuf plugins
The protobuf compiler, `protoc`, supports a [plugin system][0] for code
generation. The plugin system allows plugins to be written _in_ and _for_ any
language.
Code generators **must** be implemented as `protoc` plugins. The following
rules apply to the implementation of a client library generator as a `protoc`
plugin:
- The plugin **should** be written in the language being targeted for
generation.
- `protoc` expects plugins to be an executable in `$PATH`, and named
`protoc-gen-{plugin_name}`, corresponding to the `--{plugin_name}_out` option
sent to the `protoc` executable. As such:
- the plugin executable **should** be named `protoc-gen-{lang}_gapic`
- the plugin option **should** follow the convention `--{lang}_gapic_out`
- The plugin **must not** leverage `protoc` "insertion points". Despite the
`protoc` plugin documentation indicating the existence of insertion points,
their use is unsupported and discouraged by the Protobuf team.
### CLI options
Code generators **should** be able to run without any options or flags if at
all possible, and be able to generate a valid library from only the protos. If
options are required, `protoc` allows them to be passed as
`--{plugin_name}_opt`, and the string provided here becomes set as the
`parameter` string on the `CodeGeneratorRequest`.
**Important:** The `CodeGeneratorRequest.parameter` value is a comma-delimited
string of _all_ associated plugin option values that appear at execution time.
This means that commas cannot be used to delimit list-like plugin option values.
Code generators **must not** rely on environment variables for configuration.
## Expected behavior
This section outlines the expected behavioral attributes of the _output of_ the
client library generator (in other words: the libraries that the generators
write). Client libraries **must** implement these concepts in order to be
considered complete.
### Messages and Enums
Client library generators **should not** generate code for `message` or `enum`
descriptors which are already generated by the Protobuf-provided code
generators.
### Services and methods
Each of the `service` and `rpc` directives in the requested protos **must** be
represented in the client library output, unless the language or transport is
unable to support it.
**Note:** While how to accomplish this may vary from language to language, in
most classical languages it is probably a class for each service, containing
methods for each RPC.
- The classes generated for each `service` directive **must** honor the
`google.api.default_host` annotation if it is provided, and use that host as
the default hostname. These classes **should** provide a mechanism for the
end user to override the hostname.
- If the `google.api.default_host` annotation is not present on the `service`
directive, then the generated class **should** require a hostname when it
is instantiated.
- Additionally, if the classes generated for each service support using OAuth
and service credentials, they **must** honor the `google.api.oauth_scopes`
annotation (if it is provided), and use these scopes by default.
- Services that have set the `deprecated` protobuf option to `true` **should** have an
equivalent deprecation tag generated in the generated class. If applicable, this
tag may include a comment that specifies when the service will be removed, which
is typically the next major version update. Similarly, RPCs with this option set
to `true` **should** have their generated language method(s) marked as deprecated.
- Finally, service classes **must** also accept credentials, which are used
appropriately when requests are made. (Accepting a custom gRPC channel
satisfies this requirement.)
- Code generators **must not** generate client _stub_ classes, that would
normally be generated by gRPC, in addition to client library classes.
### Long-running operations
<!-- TODO(1145): Move to its own client library AIP. -->
An RPC is considered to be a "long-running" RPC if (and only if) the RPC's
return type is [`google.longrunning.Operation`][3]. Any API which has one or
more RPCs returning an `Operation` is expected to implement the `Operations`
service.
Because the `response` and `metadata` fields in [`Operation`][3] are of the
type [`google.protobuf.Any`][4], it is necessary to know what message to use to
deserialize them. This is annotated on the RPC using the
[`google.longrunning.operation_info`][5] annotation.
**Note:** The values in this struct are _strings_, not message objects; the
code generator uses the string to determine the appropriate message to use.
Strings with no period (`.`) character refer to a message in the same proto
package.
Code generators **should** fail with an error if a type is provided in the
[`operation_info`][5] annotation which was not imported, or if no response type
or metadata type is provided. Code generators **should** fail with an error if
_either_ the `response_type` or `metadata_type` keys are omitted.
Client libraries **must** honor the LRO interface; if an RPC has an
[`Operation`][3] as its return type, the generated method **must** intercept it
and return an appropriate idiomatic object for resolving the LRO (such as a
`Future` or `Promise` bound to the underlying [`Operation`][3] object).
### Streaming
Client libraries **must** implement streaming to the extent that their
supporting transports allow. An RPC is considered to be streaming if the
`stream` keyword is present on the argument or response type. This is present
in the [`MethodDescriptorProto`][6] message using the `client_streaming` and
`server_streaming` keys.
<!-- prettier-ignore-start -->
[0]: https://protobuf.dev/reference/other
[1]: https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/plugin.proto
[2]: https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto
[3]: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto#L122
[4]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/any.proto
[5]: https://github.com/googleapis/googleapis/blob/master/google/longrunning/operations.proto#L222
[6]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/descriptor.proto#L269
<!-- prettier-ignore-end -->
## Changelog
- **2023-06-22**: Added code gen diagram, message/enum guidance, and cleaned up
plugin & option guidance.
>>>> 4221.md
---
id: 4221
state: approved
created: 2019-06-17
---
# Client-side retry
Many APIs have error modes that a client should retry. These error modes vary
across APIs, leaving users to hand-write retry logic around client libraries
based on available documentation. Client libraries have the opportunity to help
by implementing automatic client-side retries of well-known error modes.
## Guidance
Client libraries **should** provide automatic, client-side retry logic. Client
libraries with automatic client-side retry logic **should** provide a mechanism
for users to specify error codes to be retried and delays for those retries.
Client libraries for API systems that support remotely-resolved client retry
configuration **should** respect the remotely-resolved configuration. However,
user configuration **must** be honored.
Client library generators implementing this feature **must** accept a retry
configuration. This retry configuration **may** be supplied via a `protoc`
plugin option. In the absence of a given retry configuration, client library
generators **should not** generate a default retry configuration.
### Retry implementation
Client libraries **should** make client-side retry transparent to the user. The
user **should not** have to opt-in to client-side retry explicitly, but the
user **must** have a way to disable client-side retry altogether.
### Retry configuration mechanisms
Client libraries **should** surface a mechanism through which users may control
the client-side retry configuration, including disabling client-side retry
altogether.
For example, Go client libraries for gRPC services can supply an option,
[WithDisableRetry][0], at client initialization to disable the use of the
automatic client-side retry logic.
```go
opts := []grpc.DialOption{
grpc.WithDisableRetry(),
grpc.WithTransportCredentials(creds),
}
cc, err := grpc.Dial("my.api.net:443", opts...)
if err != nil {
// ...
}
```
### Remotely-resolved client configuration
Some API systems have built-in mechanisms for clients to retrieve a
remotely-defined configuration that includes client-side retry configuration.
For example, gRPC supports the [resolution of client configuration][1] that
includes configuration for [automatic client-side retry][2].
Client libraries **should** respect the remotely-resolved configuration, except
when a user overrides it via the aforementioned client library
[retry configuration mechanisms](#retry-configuration-mechanisms).
### Client library generator retry configuration
Client library generators that implement client-side retry **must** accept a
retry configuration. This is to enable API producers to supply a retry
configuration that best-suits their service.
For example, gRPC client-side retry is configured with a [RetryPolicy][3]
within a [gRPC Service Config][4]. Here is a configuration that applies the
`RetryPolicy` to all methods in the `google.example.library.v1.LibraryService`
service, except for those that are explicitly named which get no `RetryPolicy`.
<!-- prettier-ignore-start -->
```json
{
"methodConfig": [{
"name": [{ "service": "google.example.library.v1.LibraryService" }],
"waitForReady": true,
"timeout": "60s",
"retryPolicy": {
"maxAttempts": 3,
"initialBackoff": "0.01s",
"maxBackoff": "60s",
"backoffMultiplier": 1.3,
"retryableStatusCodes": ["UNAVAILABLE"]
}
},
{
"name": [
{ "service": "google.example.library.v1.LibraryService", "method": "CreatePublisher" },
{ "service": "google.example.library.v1.LibraryService", "method": "DeletePublisher" },
{ "service": "google.example.library.v1.LibraryService", "method": "CreateBook" },
{ "service": "google.example.library.v1.LibraryService", "method": "DeleteBook" },
{ "service": "google.example.library.v1.LibraryService", "method": "UpdateBook" },
{ "service": "google.example.library.v1.LibraryService", "method": "MoveBook" }
],
"waitForReady": true,
"timeout": "60s"
}]
}
```
<!-- prettier-ignore-end -->
The retry configuration **may** be a `protoc` plugin option. For example, a
generator could accept the file path of the configuration with an option like:
--{plugin_name}_opt="retry-config=/path/to/config.file"
In the absence of a retry configuration, a generator **should not** generate a
"default" retry configuration. This results in a generated client library that
does not retry anything unless configured to do so by the user.
<!-- prettier-ignore-start -->
[0]: https://godoc.org/google.golang.org/grpc#WithDisableRetry
[1]: https://github.com/grpc/grpc/blob/837a99e1d49a892e6f2c46ee09a1b6b8405571c6/doc/naming.md#resolver-plugins
[2]: https://github.com/grpc/proposal/blob/d4fc009e55f95297374e821d67d679b931753a59/A6-client-retries.md
[3]: https://github.com/grpc/proposal/blob/d4fc009e55f95297374e821d67d679b931753a59/A6-client-retries.md#retry-policy
[4]: https://github.com/grpc/proposal/blob/d4fc009e55f95297374e821d67d679b931753a59/A6-client-retries.md#integration-with-service-config
<!-- prettier-ignore-end -->
## Further reading
- For which error codes to retry, see [AIP-194](https://aip.dev/194).
## Changelog
- **2020-09-23**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
>>>> 4222.md
---
id: 4222
state: approved
created: 2018-06-22
---
# Routing headers
In some situations, a gRPC API backend is able to route traffic more
efficiently if it knows about some values in the request; however, the request
payload can not be reasonably deconstructed on the wire to perform that
routing.
## Guidance
Code generators **must** use the annotations to generate client libraries that,
on a per-RPC basis, extract routing information from the request payload and
add that information to the routing header.
There are two annotations that specify how to extract routing information from
the request payload:
* the [`google.api.routing`][routing] annotation that specifies how to construct
routing headers explicitly
* the [`google.api.http`][http] annotation that may specify how to construct
routing headers implicitly.
For any given RPC, if the explicit routing headers annotation is present, the code
generators **must** use it and ignore any routing headers that might be implicitly
specified in the [`google.api.http`][http] annotation. If the explicit routing
headers annotation is absent, the code generators **must** parse the
[`google.api.http`][http] annotation to see if it specifies routing headers
implicitly, and use that specification.
## Explicit Routing Headers (`google.api.routing`)
For an unary or server-streaming RPC the code generator **must** look at the routing
parameters specified in the [`google.api.routing`][routing] annotation, if present.
Any given routing parameter specifies a field name and a pattern with exactly one
named resource ID path segment. For example:
```proto
rpc CreateTopic(CreateTopicRequest) {
option (google.api.routing) = {
routing_parameters {
field: "parent"
path_template: "{project=projects/*}/**"
}
}
}
```
The value of the field `field` **must** be one of the following:
1. a name of a field in the top-level of the request message
1. a dot-separated path of field names leading to a field in a sub-message of
the request message e.g. `"book.author.name"` where `book` is a message field in
the request message, `author` is a message field in the `book` message, and
`name` is a `string` field in the `author` message
The _actual field_ specified in the field `field` **must** have the following
characteristics:
- it is type `string`
- it either has a path-like value format resembling a resource name or contains
an unstructured value that would be appropriate as an individual path segment
e.g. a `project_id`.
**Note:** An empty `google.api.routing` annotation is acceptable. It means that no
routing headers should be generated for the RPC, when they otherwise would be
e.g. implicitly from the `google.api.http` annotation.
**Note:** It is acceptable to omit the pattern in the resource ID segment, `{parent}`
for example, is equivalent to `{parent=*}` and **must** be parsed, e.g.:
```proto
routing_parameters {
field: "parent"
path_template: "projects/{parent}"
}
```
is the same as
```proto
routing_parameters {
field: "parent"
path_template: "projects/{parent=*}"
}
```
**Note:** It is acceptable to omit the `path_template` field altogether. An omitted
`path_template` is equivalent to a `path_template` with the same resource ID name as
the field and the pattern `**`, and **must** be parsed, e.g.:
```proto
routing_parameters {
field: "parent"
}
```
is the same as
```proto
routing_parameters {
field: "parent"
path_template: "{parent=**}"
}
```
**Note:** An omitted `path_template` field does not indicate that key-value
pairs with empty values can be sent. It's merely a shorthand.
When the user supplies an instance of `CreateTopicRequest` to the method, the
client library **must** match all the routing parameters in the order specified
to the fields of that instance. For each routing parameter, the pattern in the
`path_template` **must** be matched to the input message field specified by the
routing parameter's `field` field. In case of a match, the name of the resource ID
path segment must be used as a key, and the value of the resource ID path segment match
must be used as a value of a key-value pair to be appended to the `x-goog-request-params`
header.
Both the key and the value **must** be URL-encoded per [RFC 6570 §3.2.2][].
This can be done with standard library URL encoding. For example, adding this header
to a gRPC request in Ruby:
```ruby
header_params = {}
if (pattern_matches("{project=projects/*}/**", request.parent))
header_params["project"] = extract_match_value("{project=projects/*}/**", request.parent)
end
request_params_header = URI.encode_www_form header_params
metadata[:"x-goog-request-params"] = request_params_header
```
In cases when multiple routing parameters have the same resource ID path segment name,
thus referencing the same header key, the "last one wins" rule is used to determine
which value to send. The "last" here is meant in terms of the order in which they're specified in the annotation. If some of the routing parameters with the same resource ID segment
name have failed to match the field, or if the field was unset, or if the extracted matched value
is an empty string, these parameters are not considered when determining which value
to send.
Example:
```proto
option (google.api.routing) = {
routing_parameters {
field: "parent"
path_template: "{project=projects/*}/**"
}
routing_parameters {
field: "parent"
path_template: "{project=projects/*/subprojects/*}/**"
}
routing_parameters {
field: "billing_project"
path_template: "{project=**}"
}
}
```
In this case if in a given request the `billing_project` field is set to an non-empty value,
its value will be sent with the `project` key because the routing parameter looking at `billing_project` field is specified last. If the `billing_project` field is not set, the `parent` field will be considered, first trying to send a
project with a subproject specified, and then without. Note that if a given request has a
`parent` field with a value e.g. `projects/100/subprojects/200/foo`, patterns in both first and second `routing_parameters` will match it, but the second one will "win" since it is specified "last".
If all the routing parameters with the same resource ID segment name have failed
to match the field, the key-value pair corresponding to those routing parameters'
resource ID path segment name **must not** be sent.
If none of the routing parameters matched their respective fields, the routing header
**must not** be sent.
Much like URL parameters, if there is more than one key-value pair to be sent, the `&`
character is used as the separator.
### `path_template` syntax
As seen in the above examples, the `path_template` can use a variety of symbols
that are interpreted by code generators during conversion to regular expressions
or non-regular expression matcher implementations. The `path_template` consists
of segments delimited by the segment delimiter. The syntax for `path_template`
is as follows:
- The only acceptable segment delimiter is `/`.
- The last symbol in a `path_template` **may** be a delimiter - it will be
ignored.
- A segment **must** be of one of the following types:
- `*`: A single-segment wildcard. Corresponds to 1 or more non-`/` symbols.
The regex describing it is `[^/]+`.
- A Single-segment wildcard typically represents a resource ID.
- `**`: A multi-segment wildcard. Corresponds to 0 or more segments.
- A multi-segment wildcard **must** only appear as the final segment or
make up the entire `path_template`.
- In a multi-segment `path_template`, a multi-segment wildcard **must**
appear immediately following a segment delimiter. This delimiter is
consumed while matching so a `path_template` like `foo/**` matches all of
the following: `foo`, `foo/`, `foo/bar/baz`.
- In a multi-segment `path_template`, when used as the last segment the
regex describing it is `([:/].*)?`.
- When used as the entire `path_template`, the regex describing it is `.*`.
- Segment delimiters are consumed while matching, including any preceding
delimiter.
- `LITERAL`: A literal segment. A literal segment can contain any
alphanumeric symbol.
- A literal segment **must not** contain a symbol reserved in this syntax.
- Literal segments typically represent a resource collection ID or base
path.
- `{}`: A variable segment. This matches part of the path as specified by its
template.
- A variable segment can be either of the following:
- `{key}`, where `key` is the name to be used in the key-value pair of the
header
- `{key=template}`, where the `template` is the segment(s) (expressed in
this `path_template` syntax) to extract as the value paired with `key`
- A variable segment of just `{key}` defaults to a template of `*` which
matches 1 or more non-`/` symbols.
- While `{key=*}` is technically valid syntax, the simpler syntax of
`{key}` **should** be used.
- A variable segment **must not** contain other variable segments. This
syntax is not recursive.
- A segment **must not** represent a complex resource ID as described in
[AIP-4231][]. A Generator **should** emit an error in this case.
## Implicit Routing Headers (`google.api.http`)
**Note:** For an RPC annotated with the [`google.api.routing`][routing] annotation,
the [`google.api.http`][http] annotation must be ignored for the purpose of adding
routing headers.
If an unary or server-streaming RPC is not annotated with the [`google.api.routing`][routing]
annotation, code generators **must** look at URI-based variables declared in the
[`google.api.http`][http] annotation and transcribe these into the
`x-goog-request-params` header in unary calls. A URI-based variable is a
variable declared as a key in curly braces in the URI string. For example:
```proto
rpc CreateTopic(CreateTopicRequest) {
option (google.api.http).post = "{parent=projects/*}/topics";
}
```
**Note:** It is acceptable to omit the pattern in the resource ID segment, `{parent}`
for example, is equivalent to `{parent=*}` and **must** be parsed.
In this case, the applicable variable is `parent`, and it refers to the
`parent` field in `CreateTopicRequest`. When the user provides an instance of
`CreateTopicRequest` to the method (or once the client library has built it, in
the case of method overloads), the client library must extract the key and
value, and append them to the `x-goog-request-params` header. Both the key and
the value **must** be URL-encoded per [RFC 6570 §3.2.2][rfc 6570 §3.2.2]. This can be done
with standard library URL encoding. For example, adding this header to a gRPC
request in Go:
```go
md := metadata.Pairs("x-goog-request-params",
url.QueryEscape("parent") + "=" + url.QueryEscape(req.GetParent()))
```
At runtime, if a field with the same name as the named parameter is unset on the
request message, the key-value pair corresponding to that parameter **must not**
be included in the routing header. If none of the parameters must be included in
the routing header, the routing header **must not** be sent.
If the [`google.api.http`][http] annotation contains `additional_bindings`,
these patterns **must** be parsed for additional request parameters. Fields
not duplicated in the top-level (or `additional_bindings`) pattern **must**
be included in request parameters, encoded in the same way.
Much like URL parameters, if there is more than one key-value pair, the `&`
character is used as the separator.
<!-- prettier-ignore -->
[http]: https://github.com/googleapis/googleapis/blob/master/google/api/http.proto
[routing]: https://github.com/googleapis/googleapis/blob/master/google/api/routing.proto
[rfc 6570 §3.2.2]: https://tools.ietf.org/html/rfc6570#section-3.2.2
[AIP-4231]: ./4231.md#complex-resource-id-path-segments
## Changelog
- **2023-07-07**: Include `path_template` syntax.
- **2022-07-13**: Updated to include the new `google.api.routing` annotation.
- **2020-04-21**: Explicitly parse path variables missing a trailing segment.
- **2019-11-27**: Include `additional_bindings` as a request parameter source.
- **2019-06-26**: Fix wording and example of key-value pair encoding.
- **2019-06-20**: Specify encoding of header parameters.
>>>> 4223.md
---
id: 4223
state: approved
created: 2022-05-13
---
# Client-side payload validation
Client-side payload validation is the practice of inspecting the request payload
for any misconfigurations (e.g. missing required fields, attempting to update an
immutable field) prior to executing a network call with the payload. Typically,
documentation communicates the expectations of the service with regards to each
request field, including any requirements on presence or format. The
`google.api.field_behavior` annotation defined in AIP-203 is the
machine-readable format that services use to document behavior in relation to
specific request fields.
## Guidance
Client libraries **must not** implement client-side payload validation based
on the `google.api.field_behavior` annotation, except to prevent
[local failures](4223.md#local-failures).
The reason for this is that the `google.api.field_behavior` annotation is
primarily a machine-readable form of _documentation_, and **not** a
configuration for payload validation. The annotation conveys how the service
_already_ handles the field, and alludes to a service's own payload validation
or expectations.
Admittedly, depending on services to implement payload validation means clients
will sometimes make network calls that are bound to result in errors. Furthermore, users
then depend on services to provide actionable errors, when the client could have
provided a language-idiomatic, localized error. However, robust server-side
payload validation means that _all types_ of client (first party, third party,
`curl`, etc.) will benefit from the **same** validation, and that there is no
lag between service and clients when the field behavior changes.
### Local failures
Client libraries **may** implement client-side payload validation based on the
`google.api.field_behavior` annotation only to the extent that it prevents
local failures, such as crashes within the client library code itself.
>>>> 4231.md
---
id: 4231
state: approved
created: 2018-06-22
---
# Parsing resource names
In resource-oriented design ([AIP-121][]), _resources_ represent the primary
nouns within APIs, and often have _resource names_ ([AIP-122][]). These
resource names convey information about the structure and hierarchy of the
resource structure in that API. APIs accept these resource names as parameters
when retrieving and modifying resources, and when referencing them in other
objects. However, users may struggle to piece together resource names to send,
and client libraries have the opportunity to make this easier through the use
of helper components.
## Guidance
Client libraries **may** provide helper classes or functions to make
constructing resource names more straightforward. However, client libraries
that choose to implement this feature **must** always accept the plain strings
also, for two reasons:
1. An existing API that adds resource descriptor annotations should be able to
do so without incurring a breaking change.
2. Resource name patterns occasionally evolve, and users need to be able to
send and receive resource names that a statically-compiled client library
may not yet acknowledge.
### Resource messages
A resource in an API always has a message that describes the representation of
that resource in that API. Client library generators are able to recognize
these messages when they are annotated with the `google.api.resource`
annotation:
```proto
// A representation of a Topic in Pub/Sub.
message Topic {
option (google.api.resource) = {
type: "pubsub.googleapis.com/Topic"
pattern: "projects/{project}/topics/{topic}"
};
// name and so on...
}
```
- The `type` field provides the [unified resource type][aip-123] name.
- Client libraries **should** name their helper component based on this
value.
- The universal resource type usually (but not always) ends with a name that
matches the name of the message.
- The `pattern` field provides the structure of this resource type's names.
- The components in braces represent variable substitutions. Client libraries
implementing this feature **must** accept variables based on these names
when building resource name strings.
- Variable substitution names are usually specified in `snake_case`, but this
is not guaranteed. Client libraries **should** be able to accept any
annotation that uses any coherent case system.
- Patterns are usually slash-separated, but this is not guaranteed. Client
libraries **should** use string interpolation to piece together the
resource name.
- The defining message is expected to contain a field called `name`, which is
the field holding the resource name.
- APIs are able to override the name field's name by setting the `name_field`
property on the `google.api.resource` annotation.
- Code generators **should** fail with an error if a message is annotated as
a resource and has no name field (either the default of `name` or the field
provided in the `name_field` property of the annotation). Code generators
**should** also fail with an error if the field is not a string).
### Multi-pattern resources
Occasionally, a resource may have more than one pattern. The common case for
this is when a resource can live under more than one parent type. In this
situation, the `pattern` field on the annotation can be specified more than
once:
```proto
message LogEntry {
option (google.api.resource) = {
type: "logging.googleapis.com/Log"
pattern: "projects/{project}/logs/{log}"
pattern: "organizations/{organization}/logs/{log}"
pattern: "folders/{folder}/logs/{log}"
pattern: "billingAccounts/{billing_account}/logs/{log}"
};
// name and so on...
}
```
If necessary, client libraries **may** create a separate helper component for
each pattern, and **may** provide a rollup component.
### Resources without messages
Occasionally, a resource may be implicitly defined by an API service, but not
have an explicit message representing that resource. (For example, the
[Firestore API][] defines databases as a common ancestor to its resources, but
does not define a database message.)
In this situation, APIs annotate the resource on the _file_ instead of on a
_message_, using the `google.api.resource_definition` annotation:
```proto
option (google.api.resource_definition) = {
type: "firestore.googleapis.com/Database"
pattern: "projects/{project}/databases/{database}"
};
```
Client library generators implementing this feature **must** generate the same
utility components that would be generated when encountering a
[resource message](#resource-messages).
### Referencing other resources
APIs often use resource names for referencing fields defined elsewhere. This is
particularly common with the request messages for the standard methods, such as
[Get][aip-131] and [Update][aip-134]; however, resources and other structures
use resource name strings as references also.
Client libraries implementing this feature **should** also provide their helper
components when resources are being referenced. Client libraries are able to
recognize these fields when they are annotated with the
`google.api.resource_reference` annotation:
```proto
message GetTopicRequest {
// The name of the topic to retrieve.
string name = 1 [(google.api.resource_reference) = {
type: "pubsub.googleapis.com/Topic"
}];
}
```
The resource reference references the [unified resource type][aip-123] name.
Some methods also refer to the _parent_ of a type, and in situations where
there are multiple parents, it is repetitive and error-prone to refer to each
individual parent type. In these situations, API producers specify `child_type`
rather than `type`:
```proto
message ListLogEntriesRequest {
// The collection of log entries to list.
string parent = 1 [(google.api.resource_reference) = {
child_type: "logging.googleapis.com/Log"
}];
}
```
In this situation, client library generators implementing this feature **must**
derive the set of parent resources from the child type. Client library
generators **must** fail with an error if both `type` and `child_type` are
provided.
### Referencing an arbitrary resource
Occasionally, a field may reference an arbitrary resource. In this case, APIs
use the special value `*` in their resource reference.
```proto
message GetIamPolicyRequest {
string resource = 1 [(google.api.resource_reference) = {
type: "*"
}];
}
```
In this situation, client library generators implementing this feature **may**
provide a generic utility class or function to address that resource name.
### Complex resource ID path segments
**Warning:** Complex resource ID path segments **should not** generally be used
in new APIs. [AIP-124][] contains advice on handling many-to-many associations.
Resource patterns **may** contain resource ID path segments which contain
multiple pattern variables separated by a variable separator:
```proto
message FeedItemTarget {
option (google.api.resource) = {
type: "googleads.googleapis.com/FeedItemTarget"
pattern: "customers/{customer}/feedItemTargets/{feed}~{feed_item}"
};
// name and other fields...
}
```
This is only used when the resource ID is naturally in multiple parts and it is
useful for the user to be able to manipulate the separate ID parts.
- A variable separator is one character long, and **must** only one of: `_`,
`-`, `.`, `~` (underscore, hyphen, period, tilde).
- A variable separator **must not** appear before the first pattern variable or
after the last pattern variable.
### Backwards compatibility
Client libraries implementing helper components for resources **must** conform
to the following backwards-compatibility expectations:
- The addition of a `google.api.resource` annotation on an existing message
**must** be a backwards-compatible change.
- An existing resource **must** be able to add new patterns (including the "\*"
wildcard pattern) without breaking changes as long as the following
conditions are met:
- New patterns must always be appended to the list.
- New patterns must use a distinct sequence of collection identifiers (see
[AIP-122][]) compared with all existing patterns within this resource.
- The identifiers within the pattern variables are final - they **must not** be
changed. These identifiers are used in the surface of generated client
libraries.
- The addition of a `google.api.resource_reference` annotation on an existing
field **must** be a backwards-compatible change.
- Changing a `google.api.resource_reference` from `child_type` to `type`
**must** be a backwards-compatible change when the `child_type` referenced
has a single `pattern` and the newly referenced `type` is the matching
parent resource `pattern` (see the
[Referencing other resources](#referencing-other-resources) section for
more info).
- For references that appear in a request, changing from `type` to
`child_type` **must** be a backwards-compatible change when the resource
`pattern` originally referenced via the `type` is included in the set of
possible resource `pattern`s derived through `child_type` resolution (see
the above [Multi-pattern resources](#multi-pattern-resources) and
[Referencing other resources](#referencing-other-resources) sections for
more examples).
Note: The `ORIGINALLY_SINGLE_PATTERN` and `FUTURE_MULTI_PATTERN` flags are
deprecated, and must not be used.
## Further reading
- For more on resource names and patterns, see [AIP-122][].
- For more on unified resource types, see [AIP-123][].
## Changelog
- **2023-03-25**: Include compatibility guidance for `type` and `child_type`.
- **2022-10-28**: Pattern variables are considered final
- **2020-09-14**: Disallow simultaneous use of both `type` and `child_type`.
- **2020-05-14**: Added complex resource ID path segments.
- **2020-05-07**: Updated backwards compatibility guidance.
- **2019-09-16**: Added guidance for resources without messages.
<!-- prettier-ignore-start -->
[aip-121]: ../0121.md
[aip-122]: ../0122.md
[aip-123]: ../0123.md
[aip-124]: ../0124.md
[aip-131]: ../0131.md
[aip-134]: ../0134.md
[firestore api]: https://github.com/googleapis/googleapis/tree/master/google/firestore/v1
<!-- prettier-ignore-end -->
>>>> 4232.md
---
id: 4232
state: approved
created: 2018-06-22
---
# Method signatures
In protocol buffer RPCs, each RPC takes exactly one argument: a message.
However, sending a full message structure can be cumbersome in the case of
extremely simple requests.
Many RPCs provide information about which pieces of the request are important
and commonly used. In many languages, functions and methods take multiple
positional or keyword arguments.
## Guidance
Some APIs provide annotations to hint how to effectively translate from a
single request object to individual arguments, and client libraries **may**
provide overloads based on these hints.
However, client libraries implementing this feature **must** retain the default
behavior of accepting the full request object. Put another way, if an API
_adds_ this annotation to an already-published API, the resulting library
change **must** be backwards-compatible.
Client library generators **may** also choose to provide this functionality in
some cases but not others, as appropriate in the environment. For example, any
of the following strategies would be permissible:
- Providing overloads iff all arguments in the signature are primitives.
- Providing overloads only for the first of multiple signatures when providing
more than one would produce a conflict.
- Any combination of the above.
In all of these situations, the requirement that the request object is always
accepted still applies.
Furthermore, client library generators **may** choose to support this
functionality for a subset of RPC types, those being:
- Unary
- Server Streaming
- Client Streaming
- Bi-directional Streaming
### Method Signatures
An RPC with the [`google.api.method_signature`][method_signature] annotation
indicates that an overload with a flattened method signature is desired where
supported. The string contains comma-separated arguments, in order. If a
field's name contains a period (`.`) character, this indicates a nested field.
An RPC can provide this annotation more than once to specify multiple
signatures. Order matters here: In some situations, it may not be possible to
generate an overload for every signature provided. In this situation, client
library generators **must** follow a "first match wins" strategy (generate an
overload for the first signature in the conflict set, and drop the rest).
**Note:** A corollary to this is that it is only guaranteed to be a
backwards-compatible change to _append_ method signature annotations.
### Required Arguments
Often, certain fields on the request message are consistently required, as
described in [AIP-203][].
While client libraries generally should not perform validation on this (that is
the server's role), client libraries **may** distinguish required arguments in
method signatures from optional ones if appropriate for the language. A field
is considered required for this purpose if annotated with the
`google.api.field_behavior` annotation value of `REQUIRED`:
```proto
message TranslateTextRequest {
// The text to translate.
string q = 1 [(google.api.field_behavior) = REQUIRED];
}
```
**Note:** The annotation for field behavior is attached to _the field_, not the
method.
### Restrictions
If an RPC lists a nested field in
[`google.api.method_signature`][method_signature] (for example,
`"foo.bar.baz"`), none of the individual component fields may be repeated
except for the last one (continuing the example, `baz` could be repeated but
`foo` or `bar` could not be). Code generators implementing this feature
**must** error with a descriptive error message if encountering a non-terminal
repeated field as a field name.
If any fields are required arguments, all required arguments are expected to
appear before any optional ones. Code generators implementing this feature
**should** error with a descriptive error message if encountering a required
field after an optional one, and **must** do so if the resulting client library
would not be valid.
### Compatibility
Removing, reordering or altering an existing `google.api.method_signature`
entry is a breaking change for client libraries. The associated generated
method is either removed entirely or its signature is altered, both of which
break user code.
## Changelog
- **2020-07-14**: Added caveat for supporting some RPC types
- **2019-09-27**: Added a Compatibility section.
<!-- prettier-ignore-start -->
[aip-203]: ../0203.md
[method_signature]: https://github.com/googleapis/googleapis/blob/master/google/api/client.proto#L100
<!-- prettier-ignore-end -->
>>>> 4233.md
---
id: 4233
state: approved
created: 2018-06-22
---
# Automatic pagination
Many API responses, particularly to list and search methods, are paginated
([AIP-158][]). Users calling these methods are then required to implement their
own pagination logic, which is common boilerplate.
## Guidance
Client libraries **may** provide automatic resolution of pagination (meaning
that it performs requests in the background on an as-needed basis).
Pagination can be inferred for an RPC when _all_ of the following conditions
are met:
- The request message contains an `int32 page_size` field.
- The request message contains a `string page_token` field.
- The response message contains a `string next_page_token` field.
- The response message contains one non-primitive `repeated` field.
Client library generators implementing this feature **must** ensure that it is
a backwards-compatible change to add client-side pagination functionality where
it was previously absent.
### Implementing pagination
For the client library to implement pagination (as defined by [AIP-158][]), it
**should** use the `next_page_token` value from the response message to
populate the `page_token` value of an otherwise-identical request message,
continuing indefinitely until the `next_page_token` value is empty.
**Important:** Client libraries that are implementing automatic resolution of
pagination should only perform requests for future pages on an as-needed basis,
and avoid greedily resolving potentially long and unnecessary result sets.
Client libraries that are implementing automatic pagination **must** still
provide access to the individual fields on the response message, in the usual
fashion.
**Note:** If the response message has more than one non-primitive `repeated`
field, the first one (in order of appearance in the file _and_ field number) is
used. If the first field by order of appearance in the message and the first
field by field number do not match, code generators that implement automatic
pagination **should** fail with an error.
[aip-158]: ../0158.md
>>>> 4234.md
---
id: 4234
state: approved
created: 2021-06-07
---
# Common service client mixins
Often, APIs can have common features for administering resources that are
separated into utility services. These common, utility services are shared among
the discrete product services. These common services each have a
centrally-defined surface, but individual instances are hosted alongside each
product service. Thus, the surfaces of each common service hosted by the product
service are "mixed in" at runtime.
## Guidance
Client libraries **may** provide client methods for the common services that its
API declares to be mixed-in. Such methods improve the user experience by
presenting the mixin methods from a client configured to communicate with the
product service that hosts the mixin service.
If client libraries support mixin services, they **must** support the following
common services (and **may** support others):
- `google.cloud.location.Locations`
- `google.iam.v1.IAMPolicy`
- `google.longrunning.Operations`
**Note:** The list of supported mixin services and the mixin services
themselves both change infrequently. Any new common service to be supported
**must** be added to the list here and generators **must** be updated.
To be generated into a client library, a mixin service **must** be declared
under [apis] in a [google.api.Service]. Furthermore, only the RPCs with
[google.api.http] bindings declared in the [http] configuration of the same
[google.api.Service] can be generated (more details on these bindings in
AIP-127). If a mixin service RPC does not have a [google.api.http] rule declared
in the [google.api.Service], it **must not** be generated.
## Implementing mixin support
### Generator configuration
Client library generators **must** accept the file path of a
[google.api.Service] in YAML form. This file is specified via a flag that
**must** be optional. In other words, the flag **must not** be required for
basic client generation that omits the mixin methods.
### Mixin API client configuration
If a mixin API utilizes client library configuration (i.e. annotations, default
retry settings, etc.) such as `google.api.method_signature` or request header
injection, generators **may** support it in the generated mixin methods if the
configuration is accessible.
Where client library configuration calls for a fully-qualified name, the
_mixin's_ fully-qualified name **must** be used. That is to say, the host
service **must not** refer to a mixin element as if it was in its own package.
For example, configuring `google.iam.v1.IAMPolicy.GetIAMPolicy` with default
retry and timeout settings would be as follows:
```json
{
"name": [{ "service": "google.iam.v1.IAMPolicy", "method": "GetIAMPolicy" }],
"timeout": "60s",
"retryPolicy": {
"initialBackoff": "0.1s",
"maxBackoff": "5s",
"backoffMultiplier": 1.3,
"retryableStatusCodes": ["UNKNOWN"]
}
}
```
### Generating mixin methods
The mixin API RPCs **should** be generated as methods on the surface of the
host API's service client library, alongside the host service's RPCs. This
presents them as top-level methods on the client, but under-the-hood, the
appropriate mixin API stub or request should be invoked. However, generators
**may** choose to present the mixin API RPCs in a different manner if it is more
language idiomatic.
**Note:** For gRPC clients, the mixin API gRPC stub **must** be used in order
to properly construct the gRPC request. For example, the generated, Go gRPC stub
[LocationsClient] must be used to invoke RPCs for the
`google.cloud.location.Locations` mixin, as opposed to that of the host
service's gRPC stub.
### Multiple host service clients
If there are multiple services defined by the host API that would result in
multiple clients being generated, generators **must** include the mixin
methods on all eligible, generated service clients.
### Overriding a duplicate RPC
Client library generators **must not** generate a mixin method on any host
service client if a host service already defines an RPC with the same name. For
example, take the following service definition:
```proto
service LibraryService {
rpc ListBooks(ListBooksRequest) returns (ListBooksResponse);
// Other host API RPCs...
// Redefinitions of the google.iam.v1.IAMPolicy mixin service.
rpc GetIamPolicy(google.iam.v1.GetIamPolicyRequest)
returns (google.iam.v1.Policy);
rpc SetIamPolicy(google.iam.v1.SetIamPolicyRequest)
returns (google.iam.v1.Policy);
rpc TestIamPermissions(google.iam.v1.TestIamPermissionsRequest)
returns (google.iam.v1.TestIamPermissionsResponse);
}
```
Should the host API declare the `google.iam.v1.IAMPolicy` as a mixin service,
client library generators **must not** generate the resulting mixin methods that
match the names explicitly declared above for _any_ service client in the host
proto package, but they **must** generate the other methods for this mixin. In
this case, the generator **may** log a warning indicating that a collision was
avoided, or silently skip the mixin methods in question.
[apis]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto#L96
[google.api.Service]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto
[google.api.Http]: https://github.com/googleapis/googleapis/blob/master/google/api/http.proto
[http]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto#L124
[LocationsClient]: https://pkg.go.dev/google.golang.org/[email protected]/googleapis/cloud/location#LocationsClient
>>>> 4235.md
---
id: 4235
state: approved
created: 2024-03-07
---
# Automatically populate fields in the request message
For APIs that leverage request idempotency as described via [AIP-155], APIs may
choose to have the client libraries automatically populate fields such as
`request_id` if they are not already set by the customer.
**Note:** This feature is primarily written for `request_id` fields within the
request message. Nonetheless, this feature **must** work for a field of any
name, as long as the conditions below hold true.
## Guidance
APIs **may** configure fields in the request message for automatic population.
For a field to be automatically populated, **all** the conditions below
**must** be true:
- The field **must** be of type `string`
- The field **must** be at the top-level of the request message
- The RPC **must** be a unary RPC (i.e. streaming RPCs are not supported)
- The field **must not** be annotated
with [`google.api.field_behavior = REQUIRED`][required].
- The field **must** be annotated
with [`google.api.field_info.format = UUID4`][uuid4].
- The field name **must** be listed
in the [`google.api.MethodSettings.auto_populated_fields`][apf] entry
in [`google.api.Publishing.method_settings`][apf]
for the target method.
### Expected Generator and Client Library Behavior
If the aforementioned requirements are met for a given field, client library
generators **must** enable automatic population of said field in the
generated client.
The field **must** be automatically populated if and only if one of the
following conditions holds:
- The field supports explicit presence, and has **not** been set by the user
- The field doesn't support explicit presence, and its value is the empty
string (i.e. the default value)
If a field is specified in the `auto_populated_fields`, but does not meet the
structural requirements, the client library generators **must not** enable
automatic population for that field. Client library generators **may** emit an
error during generation.
Client libraries **must** reuse automatically populated values for retries of
the same request. In other words, the automatically populated fields
**must not** be regenerated for each RPC attempt with the same request message.
[AIP-155]: https://google.aip.dev/155
[apf]: https://github.com/googleapis/googleapis/blob/master/google/api/client.proto
[uuid4]: https://google.aip.dev/202#uuid4
[required]: https://google.aip.dev/203#required
>>>> 4236.md
---
id: 4236
state: reviewing
created: 2024-05-16
---
# Version-aware clients
APIs can annotate services with [`google.api.api_version`][]. If
`google.api.api_version` is specified, version-aware clients **must**
include the value of `google.api.api_version` in the request to the API.
### Expected Generator and Client Library Behavior
If a service is annotated with `google.api.api_version`, client library
generators **must** include either an HTTP query parameter `$apiVersion`
or HTTP header `X-Goog-Api-Version`, but a request **must not** contain both.
Generated documentation for a given service **may** include the value of
`google.api.api_version`, if it exists in the source protos.
Clients **must** treat the value of `google.api.api_version` as opaque to ensure
robust compatibility. This means that the specific format or structure of the
version string **must not** be parsed or interpreted for any purpose beyond identifying
the intended API version.
## Rationale
### Necessity for Versioning
Explicit API versioning using the `google.api.api_version` annotation is essential
for maintaining compatibility between clients and services over time. As services
evolve, their schemas and behaviors may change. By specifying the API version, a
client communicates its expectations to the service. This allows the service to
respond in a manner consistent with the client's understanding, preventing errors
or unexpected results due to incompatible changes.
### Importance of Opaque Treatment
Treating the `google.api.api_version` value as opaque is important for ensuring robust
compatibility guarantees. By relying on this opaque identifier, clients avoid making
assumptions about the underlying versioning scheme, which may change independently of
the API itself. This flexibility allows service providers to evolve their versioning
strategies without impacting client compatibility.
### Mutual Exclusivity of Query and Header
Both the query parameter and header mechanisms exist to provide flexibility for different
client environments. However, allowing both simultaneously could lead to ambiguity if the
values differ. To ensure consistent version identification and prevent potential conflicts,
only one mechanism should be used at a time.
[`google.api.api_version`]: https://github.com/googleapis/googleapis/blob/master/google/api/client.proto
>>>> 4290.md
---
id: 4290
state: approved
created: 2018-11-08
---
# Docker interface
A consequence of using individual generators for API client library generation
is that each generator has its own set of dependencies and requirements in
order to run.
This is reasonable for a user who wishes to generate many libraries for a
single environment, but presents challenges for a user wishing to generate a
single API for many languages or environments. Users need a way to generate
libraries easily and quickly, with minimal ramp-up per language.
## Guidance
Client library generators **should** ship [Docker][] images providing the
generator and exposing a common interface, so that generating the same API in
multiple languages usually only requires substituting in the appropriate Docker
image. Docker images for Google-authored generators will follow a consistent
scheme.
### CLI usage
The expected user command to invoke the code generator in a Docker image (from
the proto import root, on a POSIX machine):
```bash
$ docker run --rm --user $UID \
--mount type=bind,source=`pwd`/a/b/c/v1/,destination=/in/a/b/c/v1/,readonly \
--mount type=bind,source=/path/to/dest/,destination=/out/ \
gcr.io/gapic-images/{GENERATOR} \
[-- additional options...]
```
**Note:** Even though each component of this is standard in the Docker
ecosystem (other than the destination path issue, which is a result of how
protoc handles imports), this is still a rather long command. We can provide a
[shortcut script][] to further simplify this, but such a script would be for
convenience and not a replacement for this interface.
### Container composition
Containers **must** include:
- A current version of protoc, the protocol buffer compiler.
- Common protos permitted to be used by all APIs ([googleapis][]).
- The applicable code generator plugin, as well as any dependencies it
requires.
- The code generator plugin itself **should** be added using an `ADD` or
`COPY` statement from the host machine at build time and installed locally;
it **should not** pull from a package manager. (This leads to catch-22
situations when cutting releases.)
- Images **may** include either a pre-compiled binary of the plugin, or the
installed source code, depending on the needs of the applicable ecosystem.
- Installation of dependencies **should** use appropriate package managers.
The common protos and the protoc compiler are supplied by an independent image
(`gcr.io/gapic-images/api-common-protos`). Both `protoc` and the common protos
can be retrieved from this image into a generator's image using the
`COPY --from` syntax (see [multi-stage builds][]). This is the preferred
approach as it follows Docker conventions, and allows the protos to be
versioned independently.
### Base images
**TL;DR:** Each language probably wants `language:x.y-alpine` or
`language:x.y-slim`. For example, `ruby:2.5-alpine` or `python:3.7-slim`.
(Alpine images are smaller but idiosyncratic.)
The following guidelines apply to selecting base images (sorted roughly from
most important to least important):
- Images **should** generally be based off an official image for the latest
stable version of the language in which the generator is implemented.
- Images **should** be able to install required system dependencies from a
well-understood package manager.
- Images **should** be ultimately based off of Alpine, Debian, or Ubuntu. This
is to ensure we benefit from GCR's [vulnerability scanning][].
- Images should endeavor to be as small as possible, in line with the general
expectations of the Docker community:
- Use the smallest base image you can. Alpine-based images are great if
possible, but may not always be reasonable. "Slim" Debian images are
usually the next best (and probably significantly more feasible in many
situations).
### Mount points
`protoc` must read protos (representing the API to be generated) from disk, and
must write the final output (the client library) to disk. Because the user has
the API protos on the host machine, and will ultimately need the output to go
to said host machine, Docker images **should** use two mount points. This
creates a hole in the abstraction layer: the user must mount the appropriate
locations on the host machine to the appropriate locations in the container.
The expected locations in the container **must** be constant, and consistent
between all generator container images:
- `/in/`: The location of the protos to be generated. This **must** be the
import root.
- Example: If generating protos for the Language API, the protos in the
Docker image must live in `/in/google/cloud/language/v1/`.
- `/out/`: The location to which the client library shall be written.
### Plugin options
Some micro-generators support configuration provided via protoc plugin options.
In such cases, the options must be routed from the CLI input to the protoc
command.
The ultimate protoc invocation could look like the following:
```bash
protoc --proto_path {path/to/common/protos} --proto_path /in/ \
--{LANG}_gapic_out /out/ \
--{LANG}_gapic_opt "go-gapic-package=GO_PACKAGE_VALUE" \
`find /in/ -name *.proto`
```
A resulting invocation of the Docker image would be as follows:
```bash
$ docker run --rm --user $UID \
--mount type=bind,source=`pwd`/a/b/c/v1/,destination=/in/a/b/c/v1/,readonly \
--mount type=bind,source=/path/to/dest/,destination=/out/ \
gcr.io/gapic-images/{GENERATOR} \
--go-gapic-package GO_PACKAGE_VALUE
```
Thus shortcut scripts written to wrap the Docker image invocation **must** pass
all options occurring after `--` to the underlying docker run command. The
internal Docker image **must** provide the conversion from usual shell syntax
to the protoc option syntax.
Client library generators that make use of plugin options **must** accept those
options as either flags or `key=value` pairs. (If a generator receives a string
without an `=` character, that is a flag, and the implied value is `true`.) If
multiple options are provided, they are comma-separated, to conform with the
protoc behavior if multiple `--opt` flags are specified.
Additionally, generators **should** prefix all understood option keys with the
target language for that generator (e.g. `go-gapic-package`,
`java-gapic-package`), and **should** use `kebab-case` for keys (in order to
match Docker, since protoc is inconsistent).
Microgenerators **must not** error on option keys that they do not recognize,
although they **may** issue a warning.
### Publishing images
Images for Google-created generators **should** be published in
`gcr.io/gapic-images`, a dedicated project in [Google Container Registry][].
Images **should** follow the naming scheme:
gcr.io/gapic-images/gapic-generator-{lang}
CI **should** be configured to push a new Docker image to the registry when
releases are made. When a release is tagged in GitHub (with a version number,
such as 1.0.3), the CI service should build an image based on the code at that
tag.
The resulting image **should** be tagged with each component of the version
number, as well as `latest`, and the resulting tags pushed the registry. (This
is in addition to pushing to a package manager if appropriate, which is outside
the scope of this AIP.)
This means that a release tag of `1.0.3` in GitHub would result in pushing the
following four tags to GCR:
- `gcr.io/gapic-images/gapic-generator-{lang}:1`
- `gcr.io/gapic-images/gapic-generator-{lang}:1.0`
- `gcr.io/gapic-images/gapic-generator-{lang}:1.0.3`
- `gcr.io/gapic-images/gapic-generator-{lang}:latest`
**Note:** These rules assumes that releases have ever-increasing version
numbers; this process will need to be amended slightly if a generator needs to
maintain multiple version streams simultaneously.
<!-- prettier-ignore-start -->
[googleapis]: https://github.com/googleapis/googleapis
[docker]: https://docker.com/
[google container registry]: https://cloud.google.com/container-registry/
[multi-stage builds]: https://docs.docker.com/develop/develop-images/multistage-build/
[shortcut script]: https://github.com/googleapis/gapic-generator-python/blob/master/gapic.sh
[vulnerability scanning]: https://cloud.google.com/container-registry/docs/container-analysis#vulnerability_source
<!-- prettier-ignore-end -->
>>>> CONTRIBUTING.md
# Contributing
We'd love to accept your patches and contributions to this project.
## Development Environment
If you are contributing AIP content (rather than code) and want to be able to
view it in your browser, the easiest way to do so is to run the provided
development server.
We use [GitHub Pages][1] to make this documentation available, and a specific
[site generator][2] to build the site.
If you have [Docker][3] installed, clone this repository and run the `serve.sh`
file at the root of the repository. This script does two things:
- It builds the provided Docker image (unless you already have it) and tags it
as `aip-site`.
- It runs the `aip-site` image.
The development server uses port 4000; point your web browser to
`http://localhost:4000`, and you should see the site.
**Note:** After building the Docker image for the first time, you may
experience issues if Python dependencies change underneath you. If this
happens, remove your Docker image (`docker rmi aip-site`) and run `serve.sh`
again.
### Arguments
Any arguments provided to `serve.sh` (or `docker run`) are forwarded (however,
the current site generator does not honor any; this may change in the future).
### Hot reloading
The development server recognizes when files change (including static files)
and local changes will be automatically reflected in your browser upon reload.
### Local Installation
It is possible to run the development server locally also. The general gist of
how to do so correctly is:
- Install Python 3.8 if you do not already have it (direct install is fine, but
[pyenv][5] is probably the best way if you have other Python projects).
- Create a Python 3.8 [venv][6]. Once it is created, activate it in your shell
(`source path/to/venv/bin/activate`).
- `pip install git+https://github.com/aip-dev/site-generator.git`
- `aip-site-serve .`
## Contributor License Agreement
Contributions to this project must be accompanied by a Contributor License
Agreement. You (or your employer) retain the copyright to your contribution,
this simply gives us permission to use and redistribute your contributions as
part of the project. Head over to <https://cla.developers.google.com/> to see
your current agreements on file or to sign a new one.
You generally only need to submit a CLA once, so if you have already submitted
one (even if it was for a different project), you probably do not need to do it
again.
## Code reviews
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.
### Formatting
We use [prettier][4] to format Markdown, JavaScript, and (most) HTML, in order
to ensure a consistent style throughout our source. You can add prettier as a
plugin in most development environments.
[1]: https://pages.github.com/
[2]: https://github.com/aip-dev/site-generator
[3]: https://docker.com/
[4]: https://prettier.io/
[5]: https://github.com/pyenv/pyenv
[6]: https://docs.python.org/3/library/venv.html
>>>> LICENSE.md
Except as otherwise noted, the content of this repository is licensed under the
[Creative Commons Attribution 4.0 License][1], and code samples are licensed
under the [Apache 2.0 License][2]. For details, see the developer.google.com
[Site Policies][3].
[1]: https://creativecommons.org/licenses/by/4.0/
[2]: https://www.apache.org/licenses/LICENSE-2.0
[3]: https://developers.google.com/terms/site-policies
---
The Creative Commons Attribution 4.0 License reads:
```
Creative Commons Attribution 4.0 International Public License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution 4.0 International Public License ("Public License"). To the
extent this Public License may be interpreted as a contract, You are
granted the Licensed Rights in consideration of Your acceptance of
these terms and conditions, and the Licensor grants You such rights in
consideration of benefits the Licensor receives from making the
Licensed Material available under these terms and conditions.
Section 1 -- Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
performance, broadcast, sound recording, and Sui Generis Database
Rights, without regard to how the rights are labeled or
categorized. For purposes of this Public License, the rights
specified in Section 2(b)(1)-(2) are not Copyright and Similar
Rights.
d. Effective Technological Measures means those measures that, in the
absence of proper authority, may not be circumvented under laws
fulfilling obligations under Article 11 of the WIPO Copyright
Treaty adopted on December 20, 1996, and/or similar international
agreements.
e. Exceptions and Limitations means fair use, fair dealing, and/or
any other exception or limitation to Copyright and Similar Rights
that applies to Your use of the Licensed Material.
f. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
g. Licensed Rights means the rights granted to You subject to the
terms and conditions of this Public License, which are limited to
all Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
h. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
i. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
j. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and of
the Council of 11 March 1996 on the legal protection of databases,
as amended and/or succeeded, as well as other essentially
equivalent rights anywhere in the world.
k. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 -- Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt, where
Exceptions and Limitations apply to Your use, this Public
License does not apply, and You do not need to comply with
its terms and conditions.
3. Term. The term of this Public License is specified in Section
6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor -- Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. No downstream restrictions. You may not offer or impose
any additional or different terms or conditions on, or
apply any Effective Technological Measures to, the
Licensed Material if doing so restricts exercise of the
Licensed Rights by any recipient of the Licensed
Material.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights.
1. Moral rights, such as the right of integrity, are not
licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to
the extent possible, the Licensor waives and/or agrees not to
assert any such rights held by the Licensor to the limited
extent necessary to allow You to exercise the Licensed
Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this
Public License.
3. To the extent possible, the Licensor waives any right to
collect royalties from You for the exercise of the Licensed
Rights, whether directly or through a collecting society
under any voluntary or waivable statutory or compulsory
licensing scheme. In all other cases the Licensor expressly
reserves any right to collect such royalties.
Section 3 -- License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the
following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the Licensor
with the Licensed Material:
i. identification of the creator(s) of the Licensed
Material and any others designated to receive
attribution, in any reasonable manner requested by
the Licensor (including by pseudonym if
designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to the
extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI or
hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may be
reasonable to satisfy the conditions by providing a URI or
hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
4. If You Share Adapted Material You produce, the Adapter's
License You apply must not prevent recipients of the Adapted
Material from complying with this Public License.
Section 4 -- Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material; and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 -- Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply with
this Public License, then Your rights under this Public License
terminate automatically.
b. Where Your right to use the Licensed Material has terminated under
Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided
it is cured within 30 days of Your discovery of the
violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 -- Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 -- Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is
deemed unenforceable, it shall be automatically reformed to the
minimum extent necessary to make it enforceable. If the provision
cannot be reformed, it shall be severed from this Public License
without affecting the enforceability of the remaining terms and
conditions.
c. No term or condition of this Public License will be waived and no
failure to comply consented to unless expressly agreed to by the
Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
```
---
The Apache 2.0 License reads:
```
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```
>>>> README.md
# API Improvement Proposals
**TL;DR:** AIPs are lots of documents on how Google does APIs.
## Overview
AIP stands for **API Improvement Proposal**, which is a design document
providing high-level, concise documentation for API development. The goal is
for these documents to serve as the source of truth for API-related
documentation at Google and the way API teams discuss and come to consensus on
API guidance. The program is named and styled after Python's enhancement
proposals (PEPs) which have seemed to work pretty well over the years.
### Specific areas inside Google
While much of the API-related guidance is general and spans across all the
different products at Google, we've found that some teams working in different
areas may have different customs, styles, or guidance. To accommodate these
historical differences, we've provided separate blocks of numbers for those
areas where they might override or extend the more general guidance.
## Getting started
### New to AIPs?
If you're **new to AIPs**, check out the [Frequently Asked Questions][] which
answer some common questions about how AIPs work and what you need to know.
### Want to use this in your company?
If you like what you see and **want to adopt the general AIPs for your
organization**, check out our guide on [Adopting AIPs in your company][]. This
guide walks you through how to start using AIPs and write your own guidance
specific to your organization.
### Have an idea for an AIP?
If you **have an idea for an AIP that isn't written yet** (yes, there are
plenty!) check out [Contributing to the project][] to see how you can write
AIPs for others to follow.
[frequently asked questions]: https://google.aip.dev/faq
[adopting aips in your company]: https://google.aip.dev/adopting
[contributing to the project]: ./CONTRIBUTING.md
## License
Except as otherwise noted, the content of this repository is licensed under the
[Creative Commons Attribution 4.0 License][1], and code samples are licensed
under the [Apache 2.0 License][2].
For the full text of each license, see [`LICENSE.md`](./LICENSE.md). For
additional details, see the developer.google.com [Site Policies][3].
[1]: https://creativecommons.org/licenses/by/4.0/
[2]: https://www.apache.org/licenses/LICENSE-2.0
[3]: https://developers.google.com/terms/site-policies
>>>> adopting.md
# Adopting AIPs in your company
**Note:** We're working on some tooling to make this better. Keep an eye on
[this GitHub issue][] for progress.
While AIPs originated at Google and were aimed for Googlers writing Google
APIs, much of the guidance documented is useful outside of Google. This
document describes how you might adopt AIPs as the way you document your own
APIs even if you don't work at Google.
## The problem
Sometimes private organizations might have API guidance that they don't want or
care to share with the rest of the world. For example, maybe in your company
(let's say, Acme, Inc), you identify your resources with a special field called
`string acme_id`.
This rule could be written as an AIP, but there's no reason to share this rule
with everyone -- it's only for you and your development team. However, you
don't want this to conflict with a future AIP (e.g., if you make this AIP-1234,
maybe that AIP will get written in the future and then you'll have a conflict).
So what do you do?
## The 9000 block
Much like the Unicode specification, we've reserved the 9000 block of AIP
numbers (that is, AIP-9000 through AIP-9999) to be for "internal use". This
means that these AIPs will never have public documentation and are free for
private companies to use for their own API guidance. As a result, all you have
to do is write your AIPs as usual and give them a number in the 9000 block.
## Custom AIP domains
We're working on a separate fork-able repository that you can copy and use as
your 9000 block AIP repository. The page rendered by GitHub Pages will serve
only the AIPs in that block and redirect all other pages to [aip.dev][].
In short, this means that you can create your own AIP domain (e.g.,
`aip.example.com`) and point that to the forked GitHub repository which will
redirect for all well-known AIPs and serve all internal AIPs from your own
repository. Once you've done that, you can cite all AIPs specifically using
that domain name (e.g., `aip.example.com/1234`) and you'll always get sent to
the right place.
## Forking AIP
You can fork the AIP project and run it in your own domain if necessary. This
will allow you to customize the AIPs to your organization's needs. Or you could
use the infrastructure to create your own set of AIPs.
### Updating the URL
To run the AIP infrastructure as a GitHub Page in a another repository, the
`_config.yaml` file must be updated to work correctly in the new repo.
If a new custom domain and CNAME have been created for your AIP, only the `url`
property will need to be updated to the new domain.
```
url: https://aip.dev
```
If you are not creating a new domain, it will be necessary to add the `baseurl`
property to the `_config.yaml`. This property should contain any additional
path information that may be appended to the domain in the `url`.
For example, assume GitHub user jdoe123 forked the aip project into a
repository named my-aips. If this user served the content from their master
branch, the url to the GitHub pages would be
`https://jdoe123.github.io/my-aips/`. The accompanying `_config.yaml` would be
configured as follows:
```
url: https://jdoe123.github.io
baseurl: /my-aips
```
For more information about about how these values are used by GitHub Pages, see
the [release notes][] that standardized these configurations in Jekyll.
### Configuring Navigation
The navigation headers and bar are generated dynamically based on the
`_data/header.yaml` and `_data/nav.yaml` files respectively.
The schema for the navigation bar can be viewed under
`assets/schemas/nav-schema.yaml`. It supports two types of navigation
components; `static_group` and `matter_group`. A `static_group` menu component
will always show the same navigation elements, regardless of the content of the
page and the repository. A `matter_group` component is generated dynamically
based on the AIPs in the domain, or the current page in the site. The
configurations for a `matter_group` can be viewed in the
`assets/schemas/nav-components.yaml#defintions/matter_group` schema.
The header is just a specially rendered `static_group` component. The schema
can be viewed at `assets/schemas/static_group.yaml`.
### Testing Configuration
The `tests` folder contains an npm test module that will validate your data
files. Running these tests requires npm and mocha. Once these are installed
that tests can be ran with the `npm test` command.
<!-- prettier-ignore-start -->
[this github issue]: https://github.com/googleapis/aip/issues/98
[npm]: https://www.npmjs.com/get-npm
[mocha]: https://www.npmjs.com/package/mocha
[release notes]: https://jekyllrb.com/news/2016/10/06/jekyll-3-3-is-here/#2-relative_url-and-absolute_url-filters
<!-- prettier-ignore-end -->
>>>> faq.md
# Frequently asked questions
**Note:** Have a question not answered here? [Ask us on GitHub!][bug]
### Why Protocol Buffers?
At Google, we define all of our APIs using Protocol Buffers (`.proto` files).
While we realize there are lots of other, newer ways of defining these
interfaces (e.g., OpenAPI, RAML, etc), we had to standardize on something and
since we're Google, we use protos.
This also leads to some very Google-specific ways of mapping proto RPCs to
RESTful URIs (`google.api.http` annotations). If you're not using protos, feel
free to follow the _spirit_ of the API guidance since you won't be able to
follow the example snippets exactly as provided.
### What's all this Googley stuff?
Originally, our API guidance was written by and for Googlers only. We're in the
process of making this guidance more general to the world, but that's far from
complete. If you see things like "Submit a CL" or "Talk to your PA lead", feel
free to send pull requests to fix these oversights.
(CL stands for "Change List" which is sort of like a GitHub Pull Request, and
PA stands for "Product Area" meaning things like "Google Maps" or
"Google Workspace".)
### I like most of these AIPs, but some make no sense for me. What do I do?
Sometimes guidance from AIPs makes lots of sense for Google (perhaps due to its
size or scale) but no sense at all for others. This is much more common than we
originally thought, so we added a way for sub-teams to override guidance that
doesn't make sense for those teams.
If you want to adopt most of the AIPs but have a few that make no sense for
your team, you can override those few and stick to just the ones that make
sense for you.
### Why aren't all of Google's products listed here?
When this project was first published, the AIP program was still relatively new.
Even with time, not all of Google has adopted AIPs as the way of documenting
their API guidance and policies. We're working on broadening the scope to more
and more areas, both inside and outside Google, so this will continue to evolve.
If you work at Google and want your team to adopt AIPs for documenting how your
team does APIs, feel free to reach out to get in touch with us!
### I have a question that isn't answered here. What do I do?
We're still pretty new and haven't built up a long list of frequently asked
questions, so if you have a question not answered here, you can always [ask us
on GitHub][bug] by filing an issue.
And if you already know the answer to a question that would be helpful for
others, [send us a pull request adding it to this page][pull request]!
[bug]: https://github.com/aip-dev/google.aip.dev/issues/new?labels=question
[pull request]: https://github.com/aip-dev/google.aip.dev/edit/master/pages/general/faq.md
>>>> licensing.md
# Content licensing
We are pleased to license much of the AIP content under terms that explicitly
encourage people to take, modify, reuse, re-purpose, and remix our work as they
see fit.
You will find the following notice at the bottom of many pages:
> Except as otherwise noted, the content of this page is licensed under the
> [Creative Commons Attribution 4.0 License][ccal-4.0], and code samples are
> licensed under the [Apache 2.0 License][apache-2.0]. For details, see
> [content licensing][licensing].
When you see a page with this notice you are free to use
[nearly everything](#restrictions) on the page in your own creations. For
example, you could quote the text in a book, cut-and-paste sections to your
blog, record it as an audiobook for the visually impaired, or even translate it
into Swahili. Really. That's what open content licenses are all about. We just
ask that you give us [attribution](#attribution) when you reuse our work.
In addition, you are free to use the computer source code that appears in the
content (such as in examples) in the code of your own projects.
## Restrictions
We say "nearly everything" as there are a few simple conditions that apply.
Google's trademarks and other brand features are not included in this license.
Please see our [standard guidelines for third party use of Google brand
features][branding] for information about this usage.
In some cases, a page may include content consisting of images, audio or video
material, or a link to content on a different webpage (such as videos or slide
decks). This content is not covered by the license, unless specifically noted.
## Attribution
Proper attribution is required when you reuse or create modified versions of
content that appears on a page made available under the terms of the Creative
Commons Attribution license. The complete requirements for attribution can be
found in section 3 of the [Creative Commons legal code][legal-code].
In practice we ask that you provide attribution to Google to the best of the
ability of the medium in which you are producing the work.
There are several typical ways in which this might apply:
### Exact reproductions
If your online work _exactly reproduces_ text or images from this site, in
whole or in part, please include a paragraph at the bottom of your page that
reads:
> Portions of this page are reproduced from work created and [shared by
> Google][licensing] and used according to terms described in the [Creative
> Commons 4.0 Attribution License][ccal-4.0].
Also, please link back to the original source page so that readers can refer to
it for more information.
### Modified versions
If your online work shows modified text or images based on the content from
this site, please include a paragraph at the bottom of your page that reads:
> Portions of this page are modifications based on work created and [shared by
> Google][licensing] and used according to terms described in the [Creative
> Commons 4.0 Attribution License][ccal-4.0].
Again, please link back to the original source page so that readers can refer
to it for more information. This is even more important when the content has
been modified.
### Other media
If you produce non-hypertext works, such as books, audio, or video, we ask that
you make a best effort to include a spoken or written attribution in the spirit
of the messages above.
[apache-2.0]: https://www.apache.org/licenses/LICENSE-2.0
[ccal-4.0]: https://creativecommons.org/licenses/by/4.0/
[branding]: http://www.google.com/permissions/guidelines.html
[legal-code]: https://creativecommons.org/licenses/by/4.0/legalcode
[licensing]: ./licensing.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment