Yu Watanabe yuwtennis
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ywatanabe@laptop-archlinux beam]$ ./gradlew :sdks:java:io:expansion-service:runExpansionService -PconstructionService.port=18089 | |
| Configuration on demand is an incubating feature. | |
| > Task :sdks:java:io:expansion-service:runExpansionService | |
| Starting expansion service at localhost:18089 | |
| Aug 17, 2022 6:49:19 PM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms | |
| beam:transform:org.apache.beam:kafka_read_with_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@663c9e7a | |
| INFO: Registering external transforms: [beam:transform:org.apache.beam:kafka_read_with_metadata:v1, beam:transform:org.apache.beam:kafka_read_without_metadata:v1, beam:transform:org.apache.beam:kafka_write:v1, beam:external:java:generate_sequence:v1] | |
| beam:transform:org.apache.beam:kafka_read_without_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@19e4653c | |
| beam:transform:org.apache.beam:kafka_w |
yuwtennis
/ elasticsearchio
Last active
August 28, 2022 11:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package elasticsearchio | |
| import ( | |
| "fmt" | |
| "github.com/apache/beam/sdks/v2/go/pkg/beam" | |
| "github.com/apache/beam/sdks/v2/go/pkg/beam/core/typex" | |
| "github.com/apache/beam/sdks/v2/go/pkg/beam/core/util/reflectx" | |
| ) | |
| const ( |
yuwtennis
/ az-consumption
Last active
December 29, 2022 03:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| SUBSCRIPTION_ID: str = os.environ.get("SUBSCRIPTION_ID", None) | |
| SCOPE: str =f'/subscriptions/{SUBSCRIPTION_ID}' | |
| logging.basicConfig(filename='./az-sdk-consumption-api-8-0-0-b1.log', level=logging.DEBUG, force=True) | |
| current_timestamp = datetime.datetime.now(datetime.timezone.utc).strftime("%c %") | |
| print(f"Querying Microsoft REST API. Started on {current_timestamp}") | |
| consumption_client = ConsumptionManagementClient( | |
| credential=DefaultAzureCredential(), | |
| subscription_id=SUBSCRIPTION_ID |
yuwtennis
/ DagOptions.java
Created
July 14, 2024 09:54
Practical example for setting custom options as beamTestPipelineOptions for TestPipeline in apache beam
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| public interface DagOptions extends PipelineOptions { | |
| @Description("Dag options") | |
| @Default.String("HELLOWORLD") | |
| String getDagType(); | |
| void setDagType(String dagType); | |
| } |
yuwtennis
/ gist:1abf229918161ec2a54f54092af78c58
Last active
March 9, 2025 07:41
Using A record as DNS for Cloud SQL Auth Proxy connection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| resource "google_compute_network" "vpc" { | |
| name = "dsongcp" | |
| auto_create_subnetworks = false | |
| } | |
| resource "google_compute_subnetwork" "private_subnet" { | |
| name = "prv-subnet" | |
| ip_cidr_range = "192.168.0.0/28" | |
| network = google_compute_network.vpc.id | |
| region = local.region |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ywatanabe@lemp13 beam]$ ./start-build-env.sh ./gradlew checkSetup | |
| ++ dirname ./start-build-env.sh | |
| + cd . | |
| + DOCKER_DIR=dev-support/docker | |
| + DOCKER_FILE=dev-support/docker/Dockerfile | |
| + CONTAINER_NAME=beam-dev-ywatanabe-202434 | |
| ++ command -v docker | |
| + '[' '!' -x /usr/bin/docker ']' | |
| + docker build -t beam-build -f dev-support/docker/Dockerfile dev-support/docker | |
| [+] Building 211.8s (24/24) FINISHED docker:default |
OlderNewer