Skip to content

Instantly share code, notes, and snippets.

@exy02
exy02 / simple_pgres_query_5.py
Last active October 10, 2020 20:26
Simplified PostgreSQL querying (with or without SSH) - General Queries
sql_statement = """
SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'ey_test_table'
;
"""
query_df = pgres.query(db='database_name', query=sql_statement)
query_df
#returns the results of an sql statement as a pandas dataframe.
#This example returns the column names and data types of table 'ey_test_table'.
@exy02
exy02 / xml_interpreter_converter.py
Last active October 5, 2021 19:48
xml_interpreter_converter
import pandas as pd
import requests
import pickle
import base64
import getpass
import boto3
import json
def get_access_token():
user_successful_login = False
@exy02
exy02 / airmap-demo-map_data.py
Created October 9, 2021 19:57
map data using AirMap
Internal_Import_Format1 = pd.read_csv("Internal_Import_Format1.csv",dtype=str)
Internal_Import_Format2 = pd.read_csv("Internal_Import_Format2.csv",dtype=str)
Internal_Table_Format1 = pd.read_csv("Internal_Table_Format1.csv",dtype=str)
data = {'Internal_Import_Format1' : Internal_Import_Format1,
'Internal_Import_Format2' : Internal_Import_Format2,
'Internal_Table_Format1' : Internal_Table_Format1 }
airMap = airtableDataMapper(base_key,airAPI_key)
airMap.findMappingView("Project A - Client Summary").mapData(data)
@exy02
exy02 / airmap-demo-view_map.py
Created October 9, 2021 20:15
view the details of a chosen data map
airMap.findMappingView("Project A - Client Summary").viewMap()
@exy02
exy02 / airmap-demo-viewMap.csv
Created October 9, 2021 21:51
sample map table output from AirMap
OutputColumn InputColumn InputSource MaxLength ColumnPosition DataType Format Active Description Required
Date of Service ServiceDate Internal_Import_Format1 10 1 Date mm-dd-yyyy Yes Date of services rendered Y
PurchaserDOB ClientDOB Internal_Import_Format1 10 2 Date mm-dd-yyyy Yes Client's date of birth Y
ServiceProviderID ProviderID Internal_Import_Format1 10 3 String ########## Yes Service providers ID Y
PurchaseAmount PurchaseAmount Internal_Import_Format1 20 4 String $dd.cc Yes Client's purchase Y
Line Service ID ServiceLineID Internal_Table_Format1 8 5 String ######## Yes Service providers ID Y
Document DocumentID Internal_Import_Format2 10 6 String NaN Yes DocumentID N
@exy02
exy02 / airmap-demo-mapData.csv
Created October 9, 2021 21:55
Resulting mapped data output from AirMap
Line Service ID ServiceProviderID Date of Service PurchaserDOB PurchaseAmount Document
20394822 1938472929 10/09/2021 7/4/1776 $103.20 AJ3fLN32jS1SK2
20428284 1293018083 10/07/2021 12/30/1899 $120.12 J2sjldw3nSk4S2l
19428292 9420410832 10/08/2021 1/26/1980 $84.23 2Pn28dh1lsdD0Q
import subprocess
import shutil
import boto3
import json
import os
def lambda_handler(event, context):
#Parameters =============================
pipPackage = "requests"
packageVersion = "2.26.0"