# clone Java google cloud client libraries repo and cd into the google cloud language library folder
git clone https://github.com/googleapis/google-cloud-java.git
cd google-cloud-java/
cd google-cloud-clients/google-cloud/language/
# update synth.py so it generates samples/v1/ directory
https://gist.githubusercontent.com/beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e/raw/2e3da0eaff96921adf6b1f6d38af9a0bd10e11f1/synth.py
# install and run synthtool
python3 -m pip install --user --upgrade git+https://github.com/googleapis/synthtool.git
python3 -m synthtool
cd samples/v1/
# generate pom.xml which has execution profiles for running each sample via its region tag
# if you choose to use this, it only works with Ruby 2.5+, sorry, Mac has a lower version
# brew install rbenv && rbenv install 2.5.5 && gem install httparty <== dependencies (not intending this to be used for long!)
curl -O https://gist.githubusercontent.com/beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e/raw/2e3da0eaff96921adf6b1f6d38af9a0bd10e11f1/generate-pom
curl -O https://gist.githubusercontent.com/beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e/raw/2e3da0eaff96921adf6b1f6d38af9a0bd10e11f1/pom.xml.erb
chmod +x generate-pom
./generate-pom language v1 src/ > pom.xml
mvn clean package # compile
# generate samples.manifest.yaml which maps region tags to how to invoke samples using mvn exec:java to run the profiles
python3 -m pip install --user --upgrade git+https://github.com/beccasaurus/sample-tester.git@add-invocation-to-manifest
sample-tester gen-manifest --env java --invocation "mvn exec:java -q -D{sample} -Dexec.args='@args'" --output samples.manifest.yaml "src/**/*.java"
# download tests from googleapis
curl -O https://raw.githubusercontent.com/googleapis/googleapis/master/google/cloud/language/v1/language.tests.yaml
# run tests
sample-tester *.yaml
RUNNING: Test environment: "java"
RUNNING: Test suite: "Natural Language V1"
PASSED: Test case: "Analyze Syntax"
PASSED: Test case: "Analyze Syntax – GCS"
PASSED: Test case: "Analyze Sentiment"
PASSED: Test case: "Analyze Sentiment – Negative"
PASSED: Test case: "Analyze Sentiment – GCS"
PASSED: Test case: "Analyze Sentiment – GCS – Negative"
PASSED: Test case: "Analyze Entities"
PASSED: Test case: "Analyze Entities – GCS"
PASSED: Test case: "Analyze Entity Sentiment"
PASSED: Test case: "Analyze Entity Sentiment – GCS"
PASSED: Test case: "Classify Text"
PASSED: Test case: "Classify Text – GCS"-
-
Save beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e to your computer and use it in GitHub Desktop.
Let's use Natural Language which already has (a) samples (b) tests
You will need https://github.com/googleapis/google-cloud-java
You will also need the latest version of https://github.com/googleapis/synthtool
python3 -m pip install --user --upgrade git+https://github.com/googleapis/synthtool.git
And the latest version of https://github.com/googleapis/sample-tester
python3 -m pip install --user --upgrade git+https://github.com/hzyi-google/sample-tester.git@python-manifest
Actually, for right now...
python3 -m pip install --user --upgrade git+https://github.com/beccasaurus/sample-tester.git@add-invocation-to-manifest
cd google-cloud-java
cd google-cloud-clients/google-cloud-language/Update the synth.py
# Add this to java_library call
generator_args=["--dev_samples"]
# Add this below, in the versions loop
s.copy(library / f'gapic-google-cloud-{service}-{version}/samples', f'samples/{version}')Run synth
python3 -m synthtool
This generates a samples/v1/ directory
samples/
└── v1
├── build.gradle
└── src
└── main
└── java
└── com
└── google
└── cloud
└── examples
└── language
└── v1
├── LanguageClassifyGcs.java
├── LanguageClassifyText.java
├── LanguageEntitiesGcs.java
├── LanguageEntitiesText.java
├── LanguageEntitySentimentGcs.java
├── LanguageEntitySentimentText.java
├── LanguageSentimentGcs.java
├── LanguageSentimentText.java
├── LanguageSyntaxGcs.java
└── LanguageSyntaxText.java
Let's create a pom.xml that knows how to run these.
And then setup sample-tester so it knows how to execute them via the pom.xml.
And then run samples-tester.
This could also work using the build.gradle.
cd samples/v1/
We're going to create a pom.xml here in samples/v1/
I have a Ruby script which:
- Reads all Java files in a src directory and maps their region tags to class names and uses those for Maven execution profiles
- Finds the most recent Maven package to include for the given version of the API
Download generate-pom and pom.xml.erb into the samples/v1/ directory
curl -O https://gist.githubusercontent.com/beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e/raw/d8df94ca17e5779ec028215884747333889602cb/generate-pomcurl -O https://gist.githubusercontent.com/beccasaurus/487a6b089d0bfc97c8b48f77c45f5c2e/raw/d8df94ca17e5779ec028215884747333889602cb/pom.xml.erb
Generate the pom.xml by running this from samples/v1/
ruby generate-pom language v1 src/ > pom.xml
Now compile
mvn clean package
Now try running a sample (with -q so only the STDOUT printed from the sample prints out)
mvn exec:java -q -Dlanguage_syntax_text-
Part of speech: DET Text: This Part of speech: VERB Text: is Part of speech: DET Text: a Part of speech: ADJ Text: short Part of speech: NOUN Text: sentence Part of speech: PUNCT Text: .
Now try passing an argument to customize the text passed to the sample
mvn exec:java -q -Dlanguage_syntax_text -Dexec.arguments='--text_content="Hello, world."'-
Part of speech: PUNCT Text: " Part of speech: X Text: Hello
Sweet. Now we need to generate a sample.manifest.yaml for sample-tester so that,
when a test tries to invoke region tag anguage_syntax_text, it runs the above command.
We'll save it as samples/v1/samples.manifest.yaml
-
sample-tester gen-manifest --env java \ --invocation "mvn exec:java -q -D{sample} -Dexec.arguments=@args" \ --output samples.manifest.yaml \ "src/**/*.java"
Using
-Dexec.arguments=@argsis probably problematic with multiple params, will test to see... using-Dexec.arguments='@args'messes up string processing by the Java CLI
If you look at samples.manifest.yaml, it should include a list of region tags and how to invoke them (in addition to the paths to the individual .java files, which we don't actually need for running samples)
cat samples.manifest.yaml-
version: 2 sets:
- environment: java
invocation: mvn exec:java -q -D{sample} -Dexec.arguments=@args
path: /Users/rebeccataylor/code/beccasaurus/google-cloud-java/google-cloud-clients/google-cloud-language/samples/v1/
items:
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageEntitySentimentText.java sample: language_entity_sentiment_text
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageSentimentGcs.java sample: language_sentiment_gcs
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageEntitySentimentGcs.java sample: language_entity_sentiment_gcs
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageEntitiesText.java sample: language_entities_text
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageSentimentText.java sample: language_sentiment_text
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageSyntaxGcs.java sample: language_syntax_gcs
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageSyntaxText.java sample: language_syntax_text
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageClassifyGcs.java sample: language_classify_gcs
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageEntitiesGcs.java sample: language_entities_gcs
- path: src/main/java/com/google/cloud/examples/language/v1/AnalyzeTextSentiment.java sample: analyze_text_sentiment
- path: src/main/java/com/google/cloud/examples/language/v1/LanguageClassifyText.java sample: language_classify_text
Cool, now copy the tests YAML into samples/v1/ from googleapis: https://github.com/googleapis/googleapis/blob/master/google/cloud/language/v1/language.tests.yaml
- `curl -O https://raw.githubusercontent.com/googleapis/googleapis/master/google/cloud/language/v1/language.tests.yaml`
Now let's try to run the tests!
- `sample-tester *.yaml`
- ```
RUNNING: Test environment: "java"
RUNNING: Test suite: "Natural Language V1"
PASSED: Test case: "Analyze Syntax"
PASSED: Test case: "Analyze Syntax – GCS"
PASSED: Test case: "Analyze Sentiment"
PASSED: Test case: "Analyze Sentiment – Negative"
PASSED: Test case: "Analyze Sentiment – GCS"
PASSED: Test case: "Analyze Sentiment – GCS – Negative"
PASSED: Test case: "Analyze Entities"
PASSED: Test case: "Analyze Entities – GCS"
PASSED: Test case: "Analyze Entity Sentiment"
PASSED: Test case: "Analyze Entity Sentiment – GCS"
PASSED: Test case: "Classify Text"
PASSED: Test case: "Classify Text – GCS"
Tests passe
Annnndddddd.... Done :)
You should have this, in your final state (not showing the compiled .class files)
.
├── build.gradle
├── generate-pom
├── language.tests.yaml
├── pom.xml
├── pom.xml.erb
├── samples.manifest.yaml
├── src
│ └── main
│ └── java
│ └── com
│ └── google
│ └── cloud
│ └── examples
│ └── language
│ └── v1
│ ├── LanguageClassifyGcs.java
│ ├── LanguageClassifyText.java
│ ├── LanguageEntitiesGcs.java
│ ├── LanguageEntitiesText.java
│ ├── LanguageEntitySentimentGcs.java
│ ├── LanguageEntitySentimentText.java
│ ├── LanguageSentimentGcs.java
│ ├── LanguageSentimentText.java
│ ├── LanguageSyntaxGcs.java
│ └── LanguageSyntaxText.java
| #! /usr/bin/env ruby | |
| def usage! | |
| puts "Usage: $0 api_name api_version path/to/java/dir/containing/a/src/dir" | |
| exit 1 | |
| end | |
| usage! unless ARGV.length == 3 | |
| POM_TEMPLATE = "pom.xml.erb" | |
| API_NAME = ARGV.shift | |
| API_VERSION = ARGV.shift | |
| SRC_DIR = ARGV.shift | |
| usage! unless File.directory? SRC_DIR | |
| library_name = "google-cloud-#{API_NAME}" | |
| ## | |
| # Create a mapping of region_tags to their class names (based on file names) | |
| # | |
| # "language_analyze_text" => "LanguageAnalyzeText" | |
| # | |
| # This could lazily simply presume every JavaClass.java has a java_class region tag | |
| ## | |
| region_tags_to_class_names = {} | |
| java_files = Dir.glob File.join(SRC_DIR, "**", "*.java") | |
| java_files.each do |path| | |
| class_name = File.basename path, ".java" | |
| java_source = File.read path | |
| region_tags = java_source.scan(/\[START ([^\]]+)\]/).flatten. | |
| reject { |region_tag| region_tag.end_with? "_core" } # ignore _core region tags | |
| region_tags.each do |region_tag| | |
| region_tags_to_class_names[region_tag] = class_name | |
| end | |
| end | |
| ## | |
| # Lookup the latest version of the Maven packagbe | |
| ## | |
| require "httparty" # gem install httparty (could use backed in Net::HTTP but I'm lazy) | |
| url = %{http://search.maven.org/solrsearch/select?q=g:"com.google.cloud"+AND+a:"#{library_name}"&rows=20&core=gav} | |
| response = HTTParty.get url | |
| packages = response["response"]["docs"] | |
| first_package = packages.first | |
| if first_package && (first_package["a"] == library_name) | |
| library_version = first_package["v"] | |
| else | |
| STDERR.puts "Couldn't find package: #{library_name}" | |
| exit | |
| end | |
| ## | |
| # Render pom.xml template (prints it to STDOUT) | |
| ## | |
| template_variables = { | |
| api_name: API_NAME, | |
| api_version: API_VERSION, | |
| library_name: library_name, | |
| library_version: library_version, | |
| region_tags_to_class_names: region_tags_to_class_names | |
| } | |
| require "erb" | |
| puts ERB.new(File.read POM_TEMPLATE).result_with_hash(template_variables) |
| <?xml version="1.0"?> | |
| <project xmlns="http://maven.apache.org/POM/4.0.0" | |
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | |
| xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> | |
| <modelVersion>4.0.0</modelVersion> | |
| <artifactId>google-cloud-language</artifactId> | |
| <version>1.65.1-SNAPSHOT</version><!-- {x-version-update:google-cloud-language:current} --> | |
| <packaging>jar</packaging> | |
| <name>Google Cloud Natural Language</name> | |
| <url>https://github.com/googleapis/google-cloud-java/tree/master/google-cloud-clients/google-cloud-language | |
| </url> | |
| <description> | |
| Java idiomatic client for Google Cloud Natural Language. | |
| </description> | |
| <parent> | |
| <groupId>com.google.cloud</groupId> | |
| <artifactId>google-cloud-clients</artifactId> | |
| <version>0.83.1-alpha-SNAPSHOT</version><!-- {x-version-update:google-cloud-clients:current} --> | |
| </parent> | |
| <properties> | |
| <site.installationModule>google-cloud-language</site.installationModule> | |
| </properties> | |
| <dependencies> | |
| <dependency> | |
| <groupId>${project.groupId}</groupId> | |
| <artifactId>google-cloud-core</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>${project.groupId}</groupId> | |
| <artifactId>google-cloud-core-grpc</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>com.google.api.grpc</groupId> | |
| <artifactId>proto-google-cloud-language-v1</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>com.google.api.grpc</groupId> | |
| <artifactId>proto-google-cloud-language-v1beta2</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>io.grpc</groupId> | |
| <artifactId>grpc-netty-shaded</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>io.grpc</groupId> | |
| <artifactId>grpc-stub</artifactId> | |
| </dependency> | |
| <dependency> | |
| <groupId>io.grpc</groupId> | |
| <artifactId>grpc-auth</artifactId> | |
| </dependency> | |
| <!-- Test dependencies --> | |
| <dependency> | |
| <groupId>junit</groupId> | |
| <artifactId>junit</artifactId> | |
| <scope>test</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.easymock</groupId> | |
| <artifactId>easymock</artifactId> | |
| <scope>test</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>org.objenesis</groupId> | |
| <artifactId>objenesis</artifactId> | |
| <scope>test</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>com.google.api.grpc</groupId> | |
| <artifactId>grpc-google-cloud-language-v1</artifactId> | |
| <scope>test</scope> | |
| </dependency> | |
| <dependency> | |
| <groupId>com.google.api.grpc</groupId> | |
| <artifactId>grpc-google-cloud-language-v1beta2</artifactId> | |
| <scope>test</scope> | |
| </dependency> | |
| <!-- Need testing utility classes for generated gRPC clients tests --> | |
| <dependency> | |
| <groupId>com.google.api</groupId> | |
| <artifactId>gax-grpc</artifactId> | |
| <classifier>testlib</classifier> | |
| <scope>test</scope> | |
| </dependency> | |
| </dependencies> | |
| </project> |
| <!-- | |
| Copyright 2019, Google Inc. | |
| Licensed under the Apache License, Version 2.0 (the "License"); | |
| you may not use this file except in compliance with the License. | |
| You may obtain a copy of the License at | |
| http://www.apache.org/licenses/LICENSE-2.0 | |
| Unless required by applicable law or agreed to in writing, software | |
| distributed under the License is distributed on an "AS IS" BASIS, | |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| See the License for the specific language governing permissions and | |
| limitations under the License. | |
| --> | |
| <project> | |
| <modelVersion>4.0.0</modelVersion> | |
| <groupId>ccom.google.cloud.examples.language.v1h</groupId> | |
| <artifactId>language-v1-google-cloud-samples</artifactId> | |
| <packaging>jar</packaging> | |
| <parent> | |
| <groupId>com.google.cloud.samples</groupId> | |
| <artifactId>shared-configuration</artifactId> | |
| <version>1.0.10</version> | |
| </parent> | |
| <properties> | |
| <maven.compiler.target>1.8</maven.compiler.target> | |
| <maven.compiler.source>1.8</maven.compiler.source> | |
| <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> | |
| </properties> | |
| <dependencies> | |
| <!-- Published GAPIC Client Library --> | |
| <!-- Latest version via Maven Central --> | |
| <dependency> | |
| <groupId>com.google.cloud</groupId> | |
| <artifactId><%= library_name %></artifactId> | |
| <version><%= library_version %></version> | |
| </dependency> | |
| <!-- Command Line Arguments for Samples --> | |
| <dependency> | |
| <groupId>commons-cli</groupId> | |
| <artifactId>commons-cli</artifactId> | |
| <version>1.4</version> | |
| </dependency> | |
| </dependencies> | |
| <profiles> | |
| <!-- Profiles for `mvn exec:java -D[profile name]` for each sample --> | |
| <% region_tags_to_class_names.each do |region_tag, main_class| %> | |
| <profile> | |
| <id><%= region_tag %></id> | |
| <activation> | |
| <property> | |
| <name><%= region_tag %></name> | |
| </property> | |
| </activation> | |
| <build> | |
| <plugins> | |
| <plugin> | |
| <groupId>org.codehaus.mojo</groupId> | |
| <artifactId>exec-maven-plugin</artifactId> | |
| <version>1.6.0</version> | |
| <executions> | |
| <execution> | |
| <goals> | |
| <goal>java</goal> | |
| </goals> | |
| </execution> | |
| </executions> | |
| <configuration> | |
| <mainClass>com.google.cloud.examples.<%= api_name %>.<%= api_version %>.<%= main_class %></mainClass> | |
| <cleanupDaemonThreads>false</cleanupDaemonThreads> | |
| </configuration> | |
| </plugin> | |
| </plugins> | |
| </build> | |
| </profile> | |
| <% end %> | |
| </profiles> | |
| </project> |
| # Copyright 2018 Google LLC | |
| # | |
| # Licensed under the Apache License, Version 2.0 (the "License"); | |
| # you may not use this file except in compliance with the License. | |
| # You may obtain a copy of the License at | |
| # | |
| # http://www.apache.org/licenses/LICENSE-2.0 | |
| # | |
| # Unless required by applicable law or agreed to in writing, software | |
| # distributed under the License is distributed on an "AS IS" BASIS, | |
| # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| # See the License for the specific language governing permissions and | |
| # limitations under the License. | |
| """This script is used to synthesize generated parts of this library.""" | |
| import synthtool as s | |
| import synthtool.gcp as gcp | |
| import synthtool.languages.java as java | |
| gapic = gcp.GAPICGenerator() | |
| service = 'language' | |
| versions = ['v1', 'v1beta2'] | |
| config_pattern = '/google/cloud/language/artman_language_{version}.yaml' | |
| for version in versions: | |
| library = gapic.java_library( | |
| service=service, | |
| version=version, | |
| config_path=config_pattern.format(version=version), | |
| artman_output_name='', generator_args=["--dev_samples"]) | |
| s.copy(library / f'gapic-google-cloud-{service}-{version}/src', 'src') | |
| s.copy(library / f'gapic-google-cloud-{service}-{version}/samples', f'samples/{version}') | |
| s.copy(library / f'grpc-google-cloud-{service}-{version}/src', f'../../google-api-grpc/grpc-google-cloud-{service}-{version}/src') | |
| s.copy(library / f'proto-google-cloud-{service}-{version}/src', f'../../google-api-grpc/proto-google-cloud-{service}-{version}/src') | |
| java.format_code('./src') | |
| java.format_code(f'../../google-api-grpc/grpc-google-cloud-{service}-{version}/src') | |
| java.format_code(f'../../google-api-grpc/proto-google-cloud-{service}-{version}/src') |