Skip to content

Instantly share code, notes, and snippets.

@tomdean
tomdean / bulk.py
Last active September 9, 2023 23:56
from collections import OrderedDict
import datetime
from typing import Iterator, List, Sized, Union
import numpy as np
import pandas as pd
from psycopg2.extensions import QuotedString
from sqlalchemy import and_, exists, MetaData, Table, Column as SAColumn
import logging
@satwikkansal
satwikkansal / cheatsheet.cpp
Last active August 12, 2024 04:14
C++ STL cheatsheet for competitive progrmming
/*
This a header file that includes every standard library.
You can use it to save time.
NOTE: This header file may not be recognized by compilers
other than gcc.
*/
#include <bits/stdc++.h>
/*
//Use this if the above header file doesn't work.
@uhho
uhho / pandas_s3_streaming.py
Last active December 2, 2022 18:57
Streaming pandas DataFrame to/from S3 with on-the-fly processing and GZIP compression
def s3_to_pandas(client, bucket, key, header=None):
# get key using boto3 client
obj = client.get_object(Bucket=bucket, Key=key)
gz = gzip.GzipFile(fileobj=obj['Body'])
# load stream directly to DF
return pd.read_csv(gz, header=header, dtype=str)
def s3_to_pandas_with_processing(client, bucket, key, header=None):
@kdubovikov
kdubovikov / tensorflow_mnist.py
Created June 18, 2017 07:29
TensorFlow MNIST example
import numpy as np
import tensorflow as tf
from tensorflow.contrib import learn
from tensorflow.contrib.learn.python.learn.estimators import model_fn as model_fn_lib
tf.logging.set_verbosity(tf.logging.INFO)
def cnn_model_fn(features, labels, mode):
@thomas-a-neil
thomas-a-neil / test_s3_download.py
Created August 22, 2016 20:59
Sample DAG to download from S3, sleep, and reupload
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators.python_operator import PythonOperator
from datetime import datetime, timedelta
# default arguments for each task
default_args = {
'owner': 'nthomas',
@malexer
malexer / sqlalchemy_upsert.py
Last active January 26, 2024 14:09
Modelling UPSERT in SQLAlchemy (well actually it is not upsert but speed improvement is significant in comparison with simple session.merge)
# Note: it is a copy of great answer by "mgoldwasser" from Stackoverflow
# Check the original answer here: http://stackoverflow.com/a/26018934/1032439
# Imagine that post1, post5, and post1000 are posts objects with ids 1, 5 and 1000 respectively
# The goal is to "upsert" these posts.
# we initialize a dict which maps id to the post object
my_new_posts = {1: post1, 5: post5, 1000: post1000}
for each in posts.query.filter(posts.id.in_(my_new_posts.keys())).all():
@DaniSancas
DaniSancas / neo4j_cypher_cheatsheet.md
Created June 14, 2016 23:52
Neo4j's Cypher queries cheatsheet

Neo4j Tutorial

Fundamentals

Store any kind of data using the following graph concepts:

  • Node: Graph data records
  • Relationship: Connect nodes (has direction and a type)
  • Property: Stores data in key-value pair in nodes and relationships
  • Label: Groups nodes and relationships (optional)
@aldraco
aldraco / gist:15236a4dfaf76e866fc0
Created September 28, 2015 16:28
Learning Python - class for Rational numbers, inspired by interactivepython.org's exercises.
def gcd(m,n):
while m % n != 0:
oldm = m
oldn = n
m = oldn
n = oldm % oldn
return n
class Rational:
def __init__(self, n, d):
@tsiege
tsiege / The Technical Interview Cheat Sheet.md
Last active November 14, 2024 15:40
This is my technical interview cheat sheet. Feel free to fork it or do whatever you want with it. PLEASE let me know if there are any errors or if anything crucial is missing. I will add more links soon.

ANNOUNCEMENT

I have moved this over to the Tech Interview Cheat Sheet Repo and has been expanded and even has code challenges you can run and practice against!






\

@cmslewis
cmslewis / sort.c
Created February 3, 2013 09:30
A few simple sorting algorithms
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
/* ==========================================================================
* COMPARATORS
* ========================================================================== */