Skip to content

Instantly share code, notes, and snippets.

View binfeng's full-sized avatar
🎯
Focusing

Alex binfeng

🎯
Focusing
  • UIUC
  • Urbana-Champaign
View GitHub Profile
@binfeng
binfeng / cloudSettings
Created July 21, 2020 05:46
Visual Studio Code Settings Sync Gist
{"lastUpload":"2020-07-21T05:46:34.873Z","extensionVersion":"v3.4.3"}
@woswos
woswos / tor-browser-selenium-wire.py
Last active November 23, 2024 02:09
This script creates a proxy server between the Tor Browser and Tor to capture requests/responses, using the seleniumwire library. You can access and modify the HTTP headers that are being sent/received, including the onion services. Note: You need to have Tor installed and running on the localhost while running this script.
import os
from seleniumwire import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
# Uncomment these if you need additional information for debugging
#import logging
#logging.basicConfig(level=logging.DEBUG)
# The location of the Tor Browser bundle
@duhaime
duhaime / headless.py
Last active February 16, 2023 23:19
Python Selenium Headless Chrome OSX
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.binary_location = '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'
driver = webdriver.Chrome(executable_path='/usr/local/bin/chromedriver', chrome_options=chrome_options)
driver.get("http://www.duo.com")
import pandas as pd
import requests
import json
def getPushshiftData(after, sub):
url = 'https://api.pushshift.io/reddit/search/submission?&size=1000&after='+str(after)+'&subreddit='+str(sub)
r = requests.get(url)
data = json.loads(r.text)
return data['data']
@linwoodc3
linwoodc3 / utilities.py
Last active February 14, 2025 01:49
A python script to scrape text from websites. This works surprisingly well on most news websites when you have the URL to the story. Use GDELT urls for the best results.
# Author: Linwood Creekmore
# Email: [email protected]
# Description: Python script to pull content from a website (works on news stories).
#Licensed under GNU GPLv3; see https://choosealicense.com/licenses/lgpl-3.0/ for details
# Notes
"""
23 Oct 2017: updated to include readability based on PyCon talk: https://github.com/DistrictDataLabs/PyCon2016/blob/master/notebooks/tutorial/Working%20with%20Text%20Corpora.ipynb
18 Jul 2018: added keywords and summary
@AtulKsol
AtulKsol / psql-error-fix.md
Last active November 13, 2024 12:43
Solution of psql: FATAL: Peer authentication failed for user “postgres” (or any user)

psql: FATAL: Peer authentication failed for user “postgres” (or any user)

The connection failed because by default psql connects over UNIX sockets using peer authentication, that requires the current UNIX user to have the same user name as psql. So you will have to create the UNIX user postgres and then login as postgres or use sudo -u postgres psql database-name for accessing the database (and psql should not ask for a password).

If you cannot or do not want to create the UNIX user, like if you just want to connect to your database for ad hoc queries, forcing a socket connection using psql --host=localhost --dbname=database-name --username=postgres (as pointed out by @meyerson answer) will solve your immediate problem.

But if you intend to force password authentication over Unix sockets instead of the peer method, try changing the following pg_hba.conf* line:

from

@elbaulp
elbaulp / addImageSizeToIMGTag.py
Last active June 29, 2020 21:33
Cómo Añadir Automáticamente El Tamaño De Una Imagen en HTML Con Python
#!/bin/python
from BeautifulSoup import BeautifulSoup
from os.path import basename, splitext
from PIL import Image
import glob
import io
path = "/ruta/ficheros/*.markdown"
@yrevar
yrevar / imagenet1000_clsidx_to_labels.txt
Last active April 19, 2025 13:21
text: imagenet 1000 class idx to human readable labels (Fox, E., & Guestrin, C. (n.d.). Coursera Machine Learning Specialization.)
{0: 'tench, Tinca tinca',
1: 'goldfish, Carassius auratus',
2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias',
3: 'tiger shark, Galeocerdo cuvieri',
4: 'hammerhead, hammerhead shark',
5: 'electric ray, crampfish, numbfish, torpedo',
6: 'stingray',
7: 'cock',
8: 'hen',
9: 'ostrich, Struthio camelus',
@binfeng
binfeng / autopush.sh
Created June 11, 2015 07:45
Auto commit and push to Github (experimental)
#!/bin/bash
#eval "$(ssh-agent -s)"
#SERVICE='ssh-agent'
#if ps ax | grep -v grep | grep $SERVICE > /dev/null
#then
# :
# #echo "$SERVICE service running, everything is fine"
#else
@ergoithz
ergoithz / xpath_soup.py
Last active January 11, 2025 16:34
Generate unique XPATH for BeautifulSoup element
#!/usr/bin/python
# -*- coding: utf-8 -*-
import bs4
def xpath_soup(element):
# type: (typing.Union[bs4.element.Tag, bs4.element.NavigableString]) -> str
"""
Generate xpath from BeautifulSoup4 element.