Skip to content

Instantly share code, notes, and snippets.

@vsooda
vsooda / doc_line_split.py
Created May 9, 2018 07:44
doc line split
#! /usr/bin/env python
# -*- coding: utf-8 -*-
import os
import cv2
import numpy as np
import matplotlib.pyplot as plt
# http://chongdata.com/articles/?p=32
@vsooda
vsooda / 2.csv
Created April 17, 2018 09:59
csv2json
default 1 2 5 11 9 2 4 1
4990 3 2 0 1 3 1
5238 3 2 0 4 6 1
4999 3 0 1 1
3387 2 2 5 11 9 2 5 1
5232 1
5018 3 0 3 1
@vsooda
vsooda / gist:cf8d07ad20056f3ac2510e74259d18ef
Created January 2, 2018 07:11
caffe mac Makefile.config
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
@vsooda
vsooda / gist:3f230c233714ecb4955338370f126aac
Created November 30, 2017 14:05
andriod mxnet predict build
FROM pythonlib_cpu:latest
LABEL maintainer="Erwan BERNARD https://github.com/edmBernard/DockerFiles"
ENV OPENCV_DIR "$LIB_DIR/opencv"
RUN mkdir -p "$OPENCV_DIR"
ENV NDK_VERSION "android-ndk-r14b"
ENV NDK_TOOLCHAIN_DIR "$LIB_DIR/arm7-toolchain"
ENV OPENBLAS_DIR "$LIB_DIR/OpenBLAS_build"
ENV MXNET_DIR "$LIB_DIR/mxnet"
@vsooda
vsooda / icdar2013_convert.py
Last active November 20, 2017 11:08
icdar2013 converter
#! /usr/bin/env python2.7
#coding=utf-8
import os
import cv2
import codecs
import ast
from lxml import etree
# do_crop the image for recognization

A Tour of PyTorch Internals (Part I)

The fundamental unit in PyTorch is the Tensor. This post will serve as an overview for how we implement Tensors in PyTorch, such that the user can interact with it from the Python shell. In particular, we want to answer four main questions:

  1. How does PyTorch extend the Python interpreter to define a Tensor type that can be manipulated from Python code?
  2. How does PyTorch wrap the C libraries that actually define the Tensor's properties and methods?
  3. How does PyTorch cwrap work to generate code for Tensor methods?
  4. How does PyTorch's build system take all of these components to compile and generate a workable application?

Extending the Python Interpreter

PyTorch defines a new package torch. In this post we will consider the ._C module. This module is known as an "extension module" - a Python module written in C. Such modules allow us to define new built-in object types (e.g. the Tensor) and to call C/C++ functions.

std::string Encrypt(const std::string& src)
{
aes_ks_t key;
const unsigned char test_key3[16] = "abcdef";
unsigned char test_v[128] = { 0 };
int len = src.length();
int srcLen = len + sizeof(len);
srcLen = srcLen + (AES_BLOCK_SIZE - (srcLen%AES_BLOCK_SIZE));
char* srcBuffer = new char[srcLen];
memset(srcBuffer, 0, srcLen);
@vsooda
vsooda / pg-pong.py
Last active June 5, 2016 14:07 — forked from karpathy/pg-pong.py
Training a Neural Network ATARI Pong agent with Policy Gradients from raw pixels
#! /usr/bin/env python2.7
# coding=utf-8
""" Trains an agent with (stochastic) Policy Gradients on Pong. Uses OpenAI Gym. """
import numpy as np
import cPickle as pickle
import gym
# hyperparameters
H = 200 # number of hidden layer neurons
batch_size = 10 # every how many episodes to do a param update?
@vsooda
vsooda / keras_prediction.py
Created April 10, 2016 07:14 — forked from Nemitek/keras_prediction.py
Predicting sequences of vectors (regression) in Keras using RNN - LSTM (original by danielhnyk.cz) - fixed for Keras 0.2.0
import pandas as pd
from random import random
flow = (list(range(1,10,1)) + list(range(10,1,-1)))*1000
pdata = pd.DataFrame({"a":flow, "b":flow})
pdata.b = pdata.b.shift(9)
data = pdata.iloc[10:] * random() # some noise
import numpy as np
@vsooda
vsooda / lstm.png
Last active September 23, 2018 11:12 — forked from karpathy/gist:587454dc0146a6ae21fc
An efficient, batched LSTM.
"""
This is a batched LSTM forward and backward pass
the comment is writen by karpathy, except the comment start with #sooda:
#sooda: will add some comment corresponding the equtions (ref: lstm.png)
"""
import numpy as np
import code
class LSTM: