This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
""" | |
This python script improves the usability of terraform 0.12 by | |
eliminating the display of redundant changes, typically found in | |
resource attributes like policy/policy_data for AWS/GCP providers. | |
This is a known limitation with the legacy terraform SDK, as described here: | |
https://github.com/hashicorp/terraform/issues/21901 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python3 | |
""" | |
This python script improves the usability of terraform 0.12 by | |
eliminating the display of redundant changes, typically found in | |
resource attributes like policy/policy_data for AWS/GCP providers. | |
This is a known limitation with the legacy terraform SDK, as described here: | |
https://github.com/hashicorp/terraform/issues/21901 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# for all tables modified in the last 24 hours | |
# relax all columns to be NULLABLE instead of REQUIRED | |
# Python3 | |
from google.cloud import bigquery | |
from datetime import datetime, timedelta | |
CLIENT = bigquery.Client() # auth using default credentials/project | |
DATASET = 'your_dataset' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# I wrote this script to alter 700 redshift tables to diststyle all (and remove sort keys) | |
# but it is partially setup to specify any dist style and sort key on a table by table basis | |
# all that's needed is to modify the main() function to accept a dict with the table configs | |
# Redshift clusters with large node types will waste disk space and network bandwidth | |
# when small tables use EVEN or DISTKEY dist styles | |
# sort keys will double the minimum size of a table, also wasting space | |
# see here for more on minimum table size calculation: | |
# https://aws.amazon.com/premiumsupport/knowledge-center/redshift-cluster-storage-space/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import random | |
import argparse | |
import mock | |
from where_ever.stream_handler import * | |
from where_ever.tests.test_stream_handler import * | |
""" | |
This will check an avro schema against an example data record |