Skip to content

Instantly share code, notes, and snippets.

@crspiccin
crspiccin / load_dataframe_from_http_zip_csv_file.py
Last active June 5, 2023 22:06
Pandas: read a csv file inside a zip located on a remote path
import zipfile
import io
import requests
def load_dataframe_from_http_zip_csv_file(endpoint, csv_filename, engine='python', encoding='ISO-8859-1', delimiter=';', skiprows=0, index_col=None):
response = requests.get(endpoint)
zip_data = zipfile.ZipFile(io.BytesIO(response.content))
zip_data.extract(csv_filename, path='extracted_directory')
df = pd.read_csv(f'extracted_directory/{csv_filename}', encoding=encoding, delimiter=delimiter, engine=engine, skiprows=skiprows, index_col=index_col)
return df
@crspiccin
crspiccin / cognito_user_extraction.sh
Last active October 6, 2021 16:07
Parse users form cognito user poll extraction with awk
# list users from a cognito specific poll
aws cognito-idp list-users --user-pool-id us-east-1_aaaaa > users.txt
# awk script save to script_parse_users.awk
# run the script printing "username,id" csv like
awk -f script_parse_users.awk users.txt
@crspiccin
crspiccin / gist:790796a68e7178404de4
Last active March 16, 2024 21:00
Node.js convert an image to Base 64
//http://www.hacksparrow.com/base64-encoding-decoding-in-node-js.html
var fs = require('fs');
// function to encode file data to base64 encoded string
function base64_encode(file) {
// read binary data
var bitmap = fs.readFileSync(file);
// convert binary data to base64 encoded string
return new Buffer(bitmap).toString('base64');
}
@crspiccin
crspiccin / gist:ed8d1b5f8f32b7c4bed1
Created August 25, 2014 13:44
Get classloader libs
// Get classloader libs
ClassLoader tcl = Thread.currentThread().getContextClassLoader();
Enumeration<URL> roots = tcl.getResources("");
for(;roots.hasMoreElements();){
URL url = roots.nextElement();
System.out.println("URL: " + url.getPath());
}