jerinphilip’s gists

jerinphilip / lmdb_stuff.py

Created July 10, 2019 16:35

LMBD stuff

	import os
	import lmdb
	import pickle
	import numpy as np

	class LMDBCorpusWriter:
	def __init__(self, corpus):
	map_size = LMDBCorpusWriter.corpus_map_size(corpus)
	self.corpus = corpus
	path = '{}.processed.lmdb'.format(corpus.path)

jerinphilip / README.md

Last active February 22, 2019 08:05

SLURM Script, resources across nodes.

Put run.sh and auxilliary.sh in the same directory.

$ env | grep SLURM
SLURM_CHECKPOINT_IMAGE_DIR=/var/slurm/checkpoint
SLURM_NODELIST=gnode[10,20,25,37]
SLURM_JOB_NAME=test-multi-nodes
SLURMD_NODENAME=gnode10
SLURM_TOPOLOGY_ADDR=gnode10
SLURM_NTASKS_PER_NODE=4

jerinphilip / sample-sbatch.sh

Created February 16, 2019 06:02

Handling TimeOut in SLURM Clusters

	#!/bin/bash
	#SBATCH -n 1
	#SBATCH -c 36
	#SBATCH --gres=gpu:4
	#SBATCH --mem=120G
	#SBATCH --mail-type=END

	## The following sends a SIGHUP signal 900 seconds before the job
	## gets sent a kill signal on TimeOut. We can thus premptively clean
	## up and export any data saved automatically.

jerinphilip / slurm-f.sh

Created September 1, 2018 05:35

SLURM related Bash functions and aliases

	function summarize-user {
	check_time=$(date +%Y-%m-%d -d "7 days ago")
	sacct -S $check_time -u $1 --format=User,Account,Jobname,elapsed,ncpus,AllocTRES
	}


	function modquota {
	echo sacctmgr -i modify account $1 set GrpTRES=gres/gpu=$2 GrpTRESMin=gres/gpu=$3;
	sacctmgr -i modify account $1 set GrpTRES=gres/gpu=$2 GrpTRESMin=gres/gpu=$3;
	}

jerinphilip / helper-function.sh

Created July 26, 2018 17:06

Add to bash profile?

	jupyter-launch ()
	{
	# jupyter-notebook

	LAB_MACHINE='10.2.16.59'
	USERNAME='jerin'
	ADA_PORT='1618'
	LAB_PORT='1617'

	ssh -N -f -R 0.0.0.0:${LAB_PORT}:localhost:${ADA_PORT} ${USERNAME}@${LAB_MACHINE};

jerinphilip / temperature.ipynb

Last active July 4, 2018 12:06

Temperature and it's effects.

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

jerinphilip / decoder.py

Last active April 12, 2018 08:32

Bug Report

	mport torch
	import torch.nn as nn
	from torch.autograd import Variable
	import torch.nn.functional as F
	from ilmt.utils.gpu_profile import counter

jerinphilip / snippet.py

Created April 12, 2018 06:35

PyTorch: Where'd my memory go?

	import torch
	import gc
	for obj in gc.get_objects():
	if torch.is_tensor(obj) or (hasattr(obj, 'data') and torch.is_tensor(obj.data)):
	print(type(obj), obj.size())

jerinphilip / .block

Last active April 12, 2018 04:41

Corpus Growth

license: mit

jerinphilip / render.py

Last active June 30, 2023 09:35 — forked from santhoshtr/render.py

PangoCairo Text rendering

	#!/usr/bin/python3
	#-- coding:utf8 --
	# Modified for python3
	# python2 version https://gist.github.com/santhoshtr/bea3da00651bcd18e282
	import cairo
	from gi.repository import Gtk, Gdk, Pango, PangoCairo
	import cairo

	surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, 600, 100)
	context = cairo.Context(surface)

Jerin Philip jerinphilip