From OpenAPI to Documentation

openapi-to-docs.md

From OpenAPI to Documentation

We are at a ripe time to formalize our API development process by using OpenAPI as our means to describe the APIs we develop and all changes we want to make to them. This repository holds a description of how that process may look like and will act as a prototype of its initial implementation.

Rationale

We already have a Developer Portal in which we have invested a lot of energy to get a neat hierarchy of folders containing Markdown files with sequence diagrams provided (in time) by PlantUML and Mermaid, illustrations and video (coming) provided by a designer, custom navigation written in a Ruby gem, a custom Jekyll Theme, etc.

Jekyll Please

All of this investment gives us great flexibility in how we compose our documentation, making it easy for our technical writers to just write and not having to think too much about anything else.

Legacy

We also have huge swaths of APIs that are not and will never be described by OpenAPI, so some solution that does not use OpenAPI as the single source of truth for all content published to the developer portal is an absolute requirement.

Therefore, we need a hybrid solution that allows us to publish our developer portal as a regular Jekyll site like today, but draw information in from OpenAPI where possible. The most obvious information to draw from OpenAPI are field tables and code examples. In time, I believe OpenAPI can give us even more, but this is good enough as a start.

Please No JavaScript

While it's not an absolute requirement, it would be great if the solution we find does not involve client-site JavaScript in order to render the information in a web browser. Being able to generate the information at build-time to static HTML gives us the freedom to index it in a custom search engine using Searchyll, gives us accessibility, WCAG and related law compliancy out of the box and doesn't burden the developer portal with client-side frameworks that have so far not been needed for anything else.

The description of how a hybrid solution may work is provided below.

Suggested Process

1. API developer opens or creates an openapi.yml file in the API repository using Stoplight Studio (or another favorite text editor).
   1. If the repository was cloned with an SSH URL, reject Stoplight Studio's offer to change it to an HTTPS URL. Just use your regular Git client for Git and Stoplight Studio as an editor.
2. The review is performed, feedback is resolved, the proposed changes are implemented in the API and the pull request is merged in the API repository.
3. The merge triggers a GitHub Action that notices a change in openapi.yml, which creates a draft pull request in the documentation repository.
   1. The pull request will contain a Markdown file (or changes to an existing one) that does an {% include %} on one or more HTML files containing the auto-generated documentation from the openapi.yml. The rest of the Markdown file will have whatever front matter, etc., Jekyll needs to render the file, make it available in navigation, etc.
   2. Having the OpenAPI converted to HTML at this stage gives us most freedom of tooling, since as long as it executes on the command line, it should be good to go.
   3. The HTML should contain the URL to the openapi.yml file it was generated from, so the HTML file can be regenerated within the documentation repository when there's a change in the HTML template or something similar.
4. The documentation pull request will be marked as ready for review once the same GitHub Action detects via communication from Octopus Deploy that the openapi.yml file has been deployed to the Stage environment.
5. The pull request is assigned to @techwriters who will start writing missing prose documentation, task up the design of illustrations, write sequence diagrams, etc., as soon as possible.
6. The documentation is finished, merged to develop and published to the Stage environment.
7. Once the changed openapi.yml is deployed to the Sandbox environment, a comment stating this information will be posted to the documentation pull request to let @techwriters know the API is live and documentation can be published to production.
8. A release is created in the documentation repository, eventually leading to the documentation being published to production.

Prerequisites

1. All repositories needs to be set up with CODEOWNERS for openapi.yml @techwriters so when pull requests for changes in openapi.yml are created, @techwriters will be assigned for review.
2. All repositories must have the GitHub Super Linter set up, to validate that the openapi.yml file is valid.
3. Contract testing must be set up to ensure that the written openapi.yml file corresponds with reality.

Missing Pieces

GitHub Action

We need to figure out how to compose the documentation pull request once the openapi.yml file is merged in the API repository.

Autogeneration

What tool should we use to convert OpenAPI to field tables, examples and whatever else we can extract from the OpenAPI metadata?

For code examples, httpsnippets looks like a good contender. For SDK models, etc., OpenAPI Generator looks promising. People have alsow written blog posts about this that are worth looking into.

Hierarchy

How can the GitHub Action know which Markdown file to create in the documentation repository, where it fits in the navigation, etc?

Octopus Deploy

We need to develop a build step in Octopus Deploy that either:

1. Directly finds the documentation pull request on GitHub and manipulates it according to the environment the openapi.yml file is currently deployed to.
2. Or communicates its deployment status somewhere that a daemon can pick it up and manipulate the pull request.

Future Riches

With all this in place, we can in due time harvest the riches of our OpenAPI investment by getting the following pretty much for free:

1. Client-side validation based on JSON Schema.
2. Auto-generation of SDK request and response models.

Credits

Illustrations are sourced from the Noun Project:

• programmer by Eucalyp.
• writer by Eucalyp.
• artist by Eucalyp.
• pull-request by Nick Bluth.
• publish by Adrien Coquet.
• comment by iconsphere.