Skip to content

Instantly share code, notes, and snippets.

View duncangh's full-sized avatar
💭
👨‍💻

Graham Duncan duncangh

💭
👨‍💻
View GitHub Profile

https://github.com/reireias/dotseeker

grep -r -E -v -h '^\s*#' files | grep --color=none -E "^\s*alias " | sed -e 's/#.*//g' -e 's/^\s*//g' -e 's/\s*$//g' | sort | uniq -c | sort -nr
 90 alias ls='ls --color=auto'
 87 alias grep='grep --color=auto'
 59 alias fgrep='fgrep --color=auto'

59 alias egrep='egrep --color=auto'

@alphapapa
alphapapa / wget-page.sh
Created November 23, 2018 09:07
Archive web pages with wget, optionally compressing with tar
#!/bin/bash
# * Defaults
compression=xz
subdir="web"
# * Functions
function debug {

Preface

This article walks you through an example of deploying a Python 3.6 application that uses Pandas and AWS S3 on AWS Lambda using Boto3 in Python in 2018. No shell, no bash, no web console, everything is automated in Python. The previous article of a Hello World example can be found here.

Again, the reason to use Python Boto3 to interact with AWS is that,

  1. I'm more familiar with Python than Bash, which means a Python script can be more flexible and powerful than Bash for me.
  2. I'm not a fun of the AWS web console. It might be easier to do certain things, but it is definitely not automated.

Introduction

"""
Django ORM Optimization Tips
Caveats:
* Only use optimizations that obfuscate the code if you need to.
* Not all of these tips are hard and fast rules.
* Use your judgement to determine what improvements are appropriate for your code.
"""
# ---------------------------------------------------------------------------
@jaklinger
jaklinger / s3_to_pandas.py
Last active April 7, 2019 03:01
Read CSV (or JSON etc) from AWS S3 to a Pandas dataframe
import boto3
import pandas as pd
from io import BytesIO
bucket, filename = "bucket_name", "filename.csv"
s3 = boto3.resource('s3')
obj = s3.Object(bucket, filename)
with BytesIO(obj.get()['Body'].read()) as bio:
df = pd.read_csv(bio)
"""
@Will Reynolds 2018
this is a financial data gathering and manipulation program.
"""
# import libraries
# you must have the following packages installed to run the program
import requests
@parente
parente / README.md
Last active September 14, 2023 13:25
Jupyter Tidbit: Run and say "done!"

Summary

Many modern web browsers provide a speech synthesis API for JavaScript. You can write and invoke a function to have your notebook speak when it finishes executing certain cells, whether you're running it in JupyterLab (>=0.34) or classic Jupyter Notebook.

def speak(text):
    from IPython.display import Javascript as js, clear_output
 # Escape single quotes
@Nikasa1889
Nikasa1889 / bisect_sort.py
Last active July 26, 2020 03:35
Test performance of python's bisect and best data struture to work with it
import timeit
setup='''
import random, bisect
from collections import deque
random.seed('slartibartfast')
s = [random.random() for i in range(1000)]
timsort = sorted
@simonw
simonw / vega-view-update.md
Created June 24, 2018 00:21
How to update an existing vega view with new fetched data

How to update an existing vega view with new fetched data

This took quite a bit of figuring out because the web is littered with Vega 2 examples but Vega 3 is a bit different.

https://vega.github.io/vega/docs/api/view/#view_change

var spec = {
  data: {"url": "https://fivethirtyeight.datasettes.com/fivethirtyeight-45d758d/most-common-name%2Fsurnames.json?_shape=array"},
  mark: "bar",
@conquistadorjd
conquistadorjd / convert_daily_to_monthly.py
Last active August 1, 2022 14:24
Pandas Equity Market
################################################################################################
# name: convert_daily_to_monthly.py
# desc: takes inout as daily prices and convert into monthly data
# date: 2018-06-15
# Author: conquistadorjd
################################################################################################
import pandas as pd
import numpy as np
print('*** Program Started ***')