Skip to content

Instantly share code, notes, and snippets.

View arundasan91's full-sized avatar

Arun Das arundasan91

View GitHub Profile
@arundasan91
arundasan91 / CaffeInstallation.md
Created April 2, 2016 22:16
Caffe Installation Tutorial for beginners

Caffe

Freshly brewed !

With the availability of huge amount of data for research and powerfull machines to run your code on, Machine Learning and Neural Networks is gaining their foot again and impacting us more than ever in our everyday lives. With huge players like Google opensourcing part of their Machine Learning systems like the TensorFlow software library for numerical computation, there are many options for someone interested in starting off with Machine Learning/Neural Nets to choose from. Caffe, a deep learning framework developed by the Berkeley Vision and Learning Center (BVLC) and its contributors, comes to the play with a fresh cup of coffee.

Installation Instructions (Ubuntu 14 Trusty)

The following section is divided in to two parts. Caffe's documentation suggest

@arundasan91
arundasan91 / bazel_install.md
Created December 10, 2016 05:26
Install Bazel and Java OpenJDK 1.8 in Ubuntu 14.04

Install OpenJDK 1.8

sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update && sudo apt-get -y install openjdk-8-jdk

Install Bazel

echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
@arundasan91
arundasan91 / core-site.xml
Created December 10, 2016 05:33
core-site.xml file for integrating openstack swift with hadoop
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:9000</value>
</property>
<property>
<name>fs.swift.impl</name>
<value>org.apache.hadoop.fs.swift.snative.SwiftNativeFileSystem</value>
</property>
<property>
@arundasan91
arundasan91 / torch_build_1
Created February 23, 2017 06:46
Build is successful, dual xeon, NVIDIA M40, plenty RAM
CMake Warning at /usr/share/cmake-3.5/Modules/FindCUDA/make2cmake.cmake:65 (message):
Removing non-existent dependency file: generic/THCTensorMathScan.h
CMake Warning at /usr/share/cmake-3.5/Modules/FindCUDA/make2cmake.cmake:65 (message):
Removing non-existent dependency file: generic/THCTensorMasked.h
@arundasan91
arundasan91 / show_video_ipython.py
Created April 14, 2017 19:54
Function to display any video in Ipython or Jupyter Notebook given a directory in which the video exist and the video file name.
def showanyVideo(baseDir=None,fname=None):
"""Function to display any video in Ipython or Jupyter Notebook given a directory in which the video exist and the video file name.
Args:
baseDir: Directory containing the video
fname: Filename of video.
"""
from IPython.display import HTML
import os
location = baseDir + fname
if os.path.isfile(location):
@arundasan91
arundasan91 / opencv_video_to_frames.py
Created April 14, 2017 19:56
Function to extract frames from input video file using OpenCV and save them as separate frames in an output directory.
def video_to_frames(input_loc, output_loc):
"""Function to extract frames from input video file and save them as separate frames in an output directory.
Args:
input_loc: Input video file.
output_loc: Output directory to save the frames.
Returns:
None
"""
@arundasan91
arundasan91 / download_youtube_video.py
Created April 14, 2017 19:57
Function to download video's from YouTube to a specific location. Requires 'youtube-dl'.
def download_video(url, output_dir, filename, ext='mp4'):
"""Function to download video's from YouTube to a specific location.
Args:
url: YouTube URL.
output_dir: Output directory to save the video.
filename: Filename of the video.
ext: File type to save the video to. Default to mp4.
"""
import os
heat_template_version: 2015-10-15
description: Implementation of Apache MXNET Distributed Deep Learning Appliance for OpenStack.
parameters:
key_name:
type: string
label: Key name
description: Name of a key pair to enable SSH access to the instance
default: default
#!/bin/bash
ls -1 *.tiff | parallel convert '{}' '{.}.jpg'
find . -name "*.tiff" -print0 | xargs -0 -I {} rm {}
@arundasan91
arundasan91 / install-packages-CC-Ubuntu1604-GPU.sh
Last active February 21, 2018 01:30
This bash script will create OpenStack `openrc` file for the current tenant, source it, download required files from OpenStack Swift Object Storage and then runs the scripts to install TensorFlow 1.3, OpenCV from source and some other user defined packages.
#!/usr/bin/env bash
# OpenRC extraction
function extract_json_key {
RESULT=$(echo "$2" | sed "s/.*$1\": \"//g" | sed 's/".*//g')
echo "$RESULT"
}
JSON_VENDOR_DATA=$(curl -s http://169.254.169.254/openstack/latest/vendor_data.json)
SITE=$(extract_json_key "site" "$JSON_VENDOR_DATA")