Skip to content

Instantly share code, notes, and snippets.

@beccasaurus
Last active June 27, 2019 02:28
Show Gist options
  • Save beccasaurus/e548b579656c36787b986f26df7d2b1b to your computer and use it in GitHub Desktop.
Save beccasaurus/e548b579656c36787b986f26df7d2b1b to your computer and use it in GitHub Desktop.
$ gapicify-samples (polyfill to support samples in separate files)

$ gapicify-samples

Polyfill to support authoring samples in separate files (not in the _gapic config)

This feature is currently in development – this script lets you try it out today!

See also: include_samples polyfill for running sample tests for library repos

Sample YAML files

GAPIC Code Sample Configurations are currently stored in _gapic.yaml files.

  • Sample configurations are changing.
  • Sample configurations are moving into their own, separate files.
  • Sample configurations will no longer be the _gapic.yaml files.

The new YAML format to support samples in individual files is being finalized.

Meanwhile, you can enjoy the benefits of this planned, upcoming functionality today by using this script.

This script reads files written in the latest/new format and adds them to a provided _gapic.yaml file (in the "old" format which is still used by the latest stable release of the sample generator).

Usage: gapicify-samples product_gapic.yaml samples/*

Examples:

gapicify-samples speech_gapic.yaml sample1.yaml sample2.yaml
gapicify-samples v1/speech_gapic.yaml v1/samples/ more/samples/
gapicify-samples v1/speech_gapic.yaml v1/samples/**/*.yaml more/samples/**/*.yaml

Download:

curl -LO https://gist.github.com/beccasaurus/e548b579656c36787b986f26df7d2b1b/raw/gapicify-samples

Dependencies:

  • Ruby (version that comes with Mac and many linux distros should work fine)

YAML format:

Only valid YAML files with these top-level keys and values are processed:

  • type: com.google.api.codegen.SampleConfigProto
  • schema_version: 1.2.0 alpha1
  • samples: key with array value (list of samples)

Notes:

Running gapicify-samples removes all existing code samples from the gapic config before adding samples.
To change this behavior, set KEEP_EXISTING=true to append/overwrite samples but not delete existing ones.

Running gapicify-samples does not preserve comments/whitespace, all comments in _gapic.yaml will be removed.

#! /usr/bin/env ruby
require "yaml"
class SampleGAPICifier
def self.usage
puts %(
Usage: gapicify-samples product_gapic.yaml samples/*
Examples:
gapicify-samples speech_gapic.yaml sample1.yaml sample2.yaml
gapicify-samples v1/speech_gapic.yaml v1/samples/ more/samples/
gapicify-samples v1/speech_gapic.yaml v1/samples/**/*.yaml more/samples/**/*.yaml
YAML format:
Only valid YAML files with these top-level keys and values are processed:
type: com.google.api.codegen.SampleConfigProto
schema_version: 1.2.0 alpha1
samples:
- [...]
Notes:
gapicify-samples removes all standalone code samples from the gapic config
before adding the provided samples. to change this behavior, set KEEP=true
to append/overwrite samples and not delete existing ones.
)
end
def self.usage!
SampleGAPICifier.usage
exit 1
end
def self.run
usage! unless ARGV.size >= 2
SampleGAPICifier.new.main *ARGV
end
attr_reader :gapic_config, :samples
def initialize
@samples = []
end
def main *args
if args.any? &method(:is_gapic_config)
gapic_config_file = args.find &method(:is_gapic_config)
args.reject! &method(:is_gapic_config)
else
gapic_config_file = args.shift
end
sample_file_glob_paths = args
load_gapic_config! gapic_config_file
remove_existing_samples! unless ENV["KEEP_EXISTING"] == "true"
load_sample_configs! sample_file_glob_paths
merged_gapic_structure = merge_samples_into_gapic_config
save_gapic_config_file! gapic_config_file
end
def is_gapic_config path
File.file?(path.to_s) && path.to_s.end_with?("_gapic.yaml")
end
def load_gapic_config! file_path
@gapic_config = load_yaml_from_file file_path
raise "Invalid GAPIC configuration file: #{file_path}" unless gapic_config_valid?
end
def remove_existing_samples!
gapic_config["interfaces"].each do |interface|
interface["methods"].each do |method|
if method.has_key?("samples") && method["samples"].has_key?("standalone")
method["samples"]["standalone"].each do |standalone_sample|
if method.has_key? "sample_value_sets"
method["sample_value_sets"].delete_if { |s| s["id"] == standalone_sample["region_tag"] }
end
end
end
end
end
end
def load_sample_configs! sample_file_glob_paths
yaml_files = []
sample_file_glob_paths.each do |path|
if File.directory? path
yaml_files.concat Dir.glob(File.join path, "**", "*.yaml")
else
yaml_files.concat Dir.glob(path).find_all {|file| file.end_with? ".yaml" }
end
end
yaml_files.each do |yaml_file_path|
deserialized_structure = load_yaml_from_file yaml_file_path, ignore_errors: true
# TODO(beccasaurus) check for type: com.google.api.codegen.SampleConfigProto / schema_version: 1.2.0 alpha1
if deserialized_structure && deserialized_structure.is_a?(Hash) && deserialized_structure.has_key?("samples")
samples.concat deserialized_structure["samples"]
end
end
end
def merge_samples_into_gapic_config
samples.each do |sample|
service_name = sample.delete "service"
rpc_name = sample.delete "rpc"
calling_forms = sample.delete "calling_forms"
if region_tag = sample.delete("region_tag")
sample["id"] = region_tag
end
if service_config = gapic_config["interfaces"].find { |s| s["name"].end_with? service_name }
if rpc_config = service_config["methods"].find { |m| m["name"] == rpc_name }
rpc_config["samples"] ||= {}
rpc_config["samples"]["standalone"] ||= []
rpc_config["samples"]["standalone"].delete_if {|s| s["region_tag"] == sample["id"] }
standalone_sample = {
"region_tag" => sample["id"],
"value_sets" => [ sample["id"] ]
}
standalone_sample["calling_forms"] = calling_forms if calling_forms
rpc_config["samples"]["standalone"].push standalone_sample
rpc_config["sample_value_sets"] ||= []
rpc_config["sample_value_sets"].delete_if { |s| s["id"] == sample["id"] }
rpc_config["sample_value_sets"].push format_for_gapic_config(sample)
puts sample["id"]
else
puts "Could not find rpc config #{rpc_name} for #{service_name}"
end
else
puts "Could not find service config #{service_name}"
end
end
end
def format_for_gapic_config sample
# [Support for v1.2 sample format]
if response_body = sample.delete("response")
sample["on_success"] = response_body # same format, different key
end
if request_fields = sample.delete("request")
defaults = []
attributes = []
request_fields.each do |field|
defaults.push "#{field["field"]} = #{field["value"].inspect}"
attribute_config = { "parameter" => field["field"] }
attribute_config["description"] = field["comment"] if field["comment"]
attribute_config["sample_argument_name"] = field["input_parameter"] if field["input_parameter"]
attribute_config["read_file"] = true if field["value_is_file"]
attributes.push attribute_config
end
sample["parameters"] = { "defaults" => defaults, "attributes" => attributes }
end
sample
end
def save_gapic_config_file! gapic_config_file
# * removes the '---' which Ruby to_yaml adds by default as the first line
gapic_config_plaintext = gapic_config.to_yaml.lines[1..-1].join
File.write gapic_config_file, gapic_config_plaintext
puts "Wrote #{samples.size} samples to #{gapic_config_file}"
end
private
def load_yaml_from_file file_path, ignore_errors: false
YAML.load_file file_path
rescue
raise unless ignore_errors
end
def gapic_config_valid?
return gapic_config.is_a?(Hash) && gapic_config["interfaces"].is_a?(Array)
end
end
SampleGAPICifier.run if __FILE__ == $0
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment