This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| alter table {0} rename to {1}; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from redshift_utils import Messages | |
| from redshift_utils import ScriptReader | |
| from redshift_utils import RedshiftDataManager | |
| from settings import SCRIPT_PATH | |
| from settings import DB_CONNECTION | |
| def lambda_handler(event, context): | |
| table_name = event.get('table_name') | |
| new_table_name = event.get('new_table_name') | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import psycopg2 | |
| # Functions for reading scripts | |
| class ScriptReader(object): | |
| @staticmethod | |
| def get_script(path): | |
| return open(path, 'r').read() | |
| # Utils for messages |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ############################################### | |
| # Script settings and constants. | |
| ############################################### | |
| SCRIPT_PATH = 'script.sql' | |
| DB_CONNECTION = { | |
| 'db_host': 'myhost.redshift.amazonaws.com', | |
| 'db_name': 'somedb', | |
| 'db_username': 'user', | |
| 'db_password': 'pA$$word' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var encrypt = function(plaintext, shiftAmount) { | |
| var ciphertext = ""; | |
| for(var i = 0; i < plaintext.length; i++) { | |
| var plainCharacter = plaintext.charCodeAt(i); | |
| if(plainCharacter >= 97 && plainCharacter <= 122) { | |
| ciphertext += String.fromCharCode((plainCharacter - 97 + shiftAmount) % 26 + 97); | |
| } else if(plainCharacter >= 65 && plainCharacter <= 90) { | |
| ciphertext += String.fromCharCode((plainCharacter - 65 + shiftAmount) % 26 + 65); | |
| } else { | |
| ciphertext += String.fromCharCode(plainCharacter); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from pyspark.sql import SparkSession | |
| spark = SparkSession.builder.master("local").appName("rel subject").getOrCreate() | |
| columns = ['id', 'referringUrl'] | |
| vals = [ | |
| (1, 'http://spark.apache.org/docs/2.1.0/api/python/pyspark.sql.html'), | |
| (2, 'https://stackoverflow.com/questions/33224740/best-way-to-get-the-max-value-in-a-spark-dataframe-column'), | |
| (3, 'https://datascience.stackexchange.com/questions/11284/key-parameter-in-max-function-in-pyspark'), | |
| ] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from pyspark.context import SparkContext | |
| spark = SparkSession.builder.master("local").appName("rel subject").getOrCreate() | |
| file_path = 'some_file.csv' | |
| dataframe = spark.read.csv(path=file_path, header=True) | |
| dataframe = dataframe.filter(dataframe['value'] >= 0.5) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ################################## | |
| # The Philosopher: A program that | |
| # generates philosophic quotes... | |
| ################################# | |
| import random | |
| R = ["without", "under", | |
| "between", "over", "on", | |
| "after", "by", "because of", "next to", "on top", | |
| "like", "through", "until", "from", "into", "to", "in", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Import necessary modules | |
| from sklearn.metrics import roc_auc_score | |
| from sklearn.model_selection import cross_val_score | |
| # Compute predicted probabilities: y_pred_prob | |
| y_pred_prob = logreg.predict_proba(X_test)[:,1] | |
| # Compute and print AUC score | |
| print("AUC: {}".format(roc_auc_score(y_test, y_pred_prob))) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Import necessary modules | |
| from sklearn.metrics import roc_curve | |
| # Compute predicted probabilities: y_pred_prob | |
| y_pred_prob = logreg.predict_proba(X_test)[:,1] | |
| # Generate ROC curve values: fpr, tpr, thresholds | |
| fpr, tpr, thresholds = roc_curve(y_test, y_pred_prob) | |
| # Plot ROC curve |