Skip to content

Instantly share code, notes, and snippets.

View junjuew's full-sized avatar

Junjue Wang junjuew

View GitHub Profile
@junjuew
junjuew / phash_dedup.py
Last active November 3, 2020 18:45
Deduplicate Images Based on Their Perceptual Hash
import pathlib
import cv2 as cv
import fire
import imagehash
from PIL import Image
def dedup(video_uri: str, output_dir: str, min_hash_difference_between_key_frames: float=10):
cap = cv.VideoCapture(video_uri)
@junjuew
junjuew / cpu-freq-performance.sh
Created May 23, 2020 23:06
set CPU frequency to performance on linux
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# https://wiki.archlinux.org/index.php/CPU_frequency_scaling
@junjuew
junjuew / test.py
Last active February 2, 2023 20:29
Set abseil / Tensorflow flags from python script
"""Example for passing Tensorflow configuration variables from python script.
Many tensorflow scripts variable are defined with absl, which parses inputs from
command line by default. Here is a way to set the flags in a python script.
"""
from absl import flags
FLAGS = flags.FLAGS
flags.DEFINE_integer('my_version', 0, 'Version number.')
@junjuew
junjuew / gist:b0578e9b486df7461f40435ddbc0c787
Created November 18, 2019 21:20
Python One-Liner to Show a Single Example from TFRecord
python -c 'import sys; import tensorflow as tf; ex=next(tf.python_io.tf_record_iterator(sys.argv[1])); print(tf.train.Example.FromString(ex))' <path-to-tfrecord>
@junjuew
junjuew / docker-firewall.sh
Last active August 19, 2019 05:12
Set up firewalls for docker container to allow internal/CMU IPs only
# in Docker >v17, DOCKER-USER chain is introduced. It is a
# iptable chain in FILTER table, called before DOCKER chain.
# By default, it accepts all connections.
# delete default return rule
sudo iptables -D DOCKER-USER -j RETURN
# allow default container and private subnet
sudo iptables -C DOCKER-USER -s 172.16.0.0/12 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 172.16.0.0/12 -j ACCEPT
sudo iptables -C DOCKER-USER -s 192.168.0.0/16 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 192.168.0.0/16 -j ACCEPT
sudo iptables -C DOCKER-USER -s 10.0.0.0/8 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 10.0.0.0/8 -j ACCEPT
# allow CMU ips
@junjuew
junjuew / tensorflow-object-detection-benchmark.py
Last active February 16, 2020 22:40
tensorflow object detection inference speed benchmark
import os
import tarfile
import time
import numpy as np
import six.moves.urllib as urllib
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.INFO)
@junjuew
junjuew / uninstall_cloud_init_add_passwd.sh
Created July 10, 2018 19:05
Add password login to cloud images
virt-customize -a /var/lib/libvirt/images/debian9-vm1.qcow2 --root-password password:NewRootUserPasswordHere --uninstall cloud-init
@junjuew
junjuew / vm.xml
Last active March 18, 2023 13:37
libvirt xml with port forwarding
# Use qemu directly with kvm
# sudo qemu-system-i386 -machine accel=kvm -m 2048 mn-trusty64server-170321-14-17-08/mininet-vm-x86_64.vmdk -net nic,model=virtio -net user,net=192.168.101.0/24,hostfwd=tcp::8022-:22
# sample libvirt xml with port forwarding enabled
# copy the following
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>mininet</name>
<uuid>cd4dbb45-1628-4310-9a1a-f8bc62ca91c0</uuid>
<memory unit='KiB'>4194304</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>2</vcpu>
@junjuew
junjuew / keybindings.json
Created June 21, 2018 17:01
Custom Visual Studio Code Keybinding
// Place your key bindings in this file to overwrite the defaults
// File path on Ubuntu is $HOME/.config/Code/User/keybindings.json
[
// ctrl+shift to navigate among UI elements
{
"key": "ctrl+shift+up",
"command": "-workbench.action.terminal.scrollUp",
"when": "terminalFocus"
},
@junjuew
junjuew / license.py
Created May 6, 2018 22:21
A bare-minimal script to add and remove licenses on top of python files.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import fire
LICENSE_CONTENT = {}
LICENSE_CONTENT['apache'] = """# Copyright (C) 2018 Carnegie Mellon University. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");