This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Set your GCP project ID | |
PROJECT_ID=$(gcloud config get-value project) | |
# List all functions in the project and iterate over them | |
for FUNCTION_NAME in $(gcloud functions list --project=$PROJECT_ID --format="value(name)") | |
do | |
echo "Function: $FUNCTION_NAME" | |
# Describe each function to get the dockerRegistry value |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
This (Google Cloud) Function will scan all a list of provided projects and get all Bigquery datasets for each of those projects. | |
It will then update the storage billing model for each dataset to be 'PHYSICAL' instead of 'LOGICAL'. This is useful for large datasets that are frequently accessed, as it can reduce costs. | |
Required Permissions: | |
- The service account running this function must have the following roles: | |
to be updated--- | |
Returns: | |
string: A message indicating the function has completed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import functions_framework | |
from google.cloud import bigquery | |
def extract_dataset_name(full_path): | |
""" | |
Extracts the dataset name from a full BigQuery dataset resource path. | |
Args: | |
- full_path (str): The full resource path of the dataset, | |
formatted as 'projects/{project_id}/datasets/{dataset_id}'. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
steps: | |
- name: golang | |
entrypoint: go | |
args: [ 'run', 'github.com/GoogleCloudPlatform/artifact-registry-go-tools/cmd/[email protected]', 'add-locations', '--locations=${_LOCATION}' ] | |
env: | |
# Set GOPROXY to the public proxy to pull the credential helper tool | |
- 'GOPROXY=proxy.golang.org' | |
- name: golang | |
entrypoint: go |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# prompt user for monorepo name | |
read -p "Enter monorepo name: " monorepo_name | |
# check if monorepo directory already exists | |
if [ -d "$monorepo_name" ]; then | |
echo "Monorepo directory already exists. Skipping creation process." | |
else | |
# create the main directory |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
class GCPCostCalculator: | |
def __init__(self): | |
self.sql_cores = 0 | |
self.sql_ram = 0 | |
self.sql_disk_size = 0 | |
self.sql_disk_type = "" | |
self.sql_num_machines = 0 | |
self.spanner_nodes = 0 | |
self.spanner_region = "" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gcp_machine_family_pricing = { | |
'N1': { | |
'us-central1': { | |
'core_cost': 0.031611, # Cost per core-hour | |
'ram_cost': 0.004237, # Cost per GB-hour of RAM | |
'cud_discount': { | |
'1-year': 0.3, # 30% discount | |
'3-year': 0.57 # 57% discount | |
} | |
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
for project in $(gcloud projects list --format="value(projectId)") | |
do | |
for recommendation in $(gcloud recommender recommendations list --project=$project --location=global --recommender=google.compute.instance.MachineTypeRecommender) | |
do | |
# Here is where the logic for your 'what to do with each recommendation' goes | |
echo "$recommendation" | |
done | |
done |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WITH | |
[CHURN STAT] AS | |
(SELECT * | |
FROM [TABLE] | |
WHERE [STAT START] < '[DATE]' | |
AND ( | |
([STAT END] >= '2018-01-01') | |
OR ([STAT END]] IS NULL)) | |
), | |
status AS |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" Warning - this script automatically submits a request for GCP DLP job creation | |
to scan for every table, in every dataset available to a user. This will fail if the following APIs / Permissions | |
are not enabled. | |
This script can, and may, cost you actual $$. Outcomes can be seen in the DLP console. | |
DLP - jobs.create, jobs.get, jobs.list | |
BQ - bigquery.user (role) |