Skip to content

Instantly share code, notes, and snippets.

View tomas-wood's full-sized avatar

Hoemas Ruud tomas-wood

View GitHub Profile
  1. Make sure you have nvidia-docker installed (I've got this covered on 35 for all users, I believe)
  2. Run nvidia-docker run -it odellus/(sg_srvc:version1|detectron_srvc:version5) and it will take you inside the docker container.
  3. When you get finished with the work you're doing, don't just exit, open a new ssh connection and commit the changes you've made using docker commit (here is how to do it) by entering
nvidia-docker ps (get the container id this way or just read it off the beginning of bash prompt in other terminal)
nvidia-docker commit <container id> odellus/(sg_srvc:version1|detectron_srvc:verstion5)
nvidia-docker push

It will probably ask you for my password, which I can send you through eSpace messenger.

@tomas-wood
tomas-wood / nvidia-smi
Created September 18, 2018 22:38
every GPU is being used.
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 5760 C /usr/bin/python2.7 11739MiB |
| 1 5760 C /usr/bin/python2.7 11589MiB |
| 2 5760 C /usr/bin/python2.7 11589MiB |
| 3 5760 C /usr/bin/python2.7 11589MiB |
| 4 5760 C /usr/bin/python2.7 11589MiB |
| 5 5760 C /usr/bin/python2.7 11589MiB |
@tomas-wood
tomas-wood / gen_rpn_app.py
Created September 18, 2018 23:15
generate roidb with rpn
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
################################################################
#
# This script takes an image as input and outputs the
# region proposals
#
# Author: Aniket Adnaik ([email protected])
################################################################
@tomas-wood
tomas-wood / server.py
Last active September 21, 2018 18:43
flask restful api for calling detectron on an image via url.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from collections import defaultdict
import argparse
import cv2 # NOQA (Must import before importing caffe2 due to bug in cv2)
import glob
import logging
@tomas-wood
tomas-wood / detectron_api.py
Created September 19, 2018 22:58
simple detectron api example
import urllib3
import pickle
import json
http = urllib3.PoolManager()
url = "http://localhost:5000/detectron"
img_url="https://cdn7.dissolve.com/p/D1028_98_301/D1028_98_301_1200.jpg"
@tomas-wood
tomas-wood / use_curl.sh
Created September 20, 2018 18:05
API call to GES
curl -H "Content-Type:application/json" -X POST --data '{"command":"graph=EywaGraph.open(\"ges_abfe\");g=graph.traversal();g.V().limit(10)"}' http://49.4.88.7/graph/ges/v1.0/1111/1/graphs/222/execute-gremlin-query
{"errorMessage":"Resource not allowed!","errorCode":"GES.8006"}
Wed Sep 19 11:48:26 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.37 Driver Version: 396.37 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 00000000:04:00.0 Off | N/A |
| 23% 40C P2 50W / 250W | 893MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
@tomas-wood
tomas-wood / fetch_data.py
Created September 25, 2018 22:48
get visual genome data
def fetch_data():
urls = [
"https://visualgenome.org/static/data/dataset/objects.json.zip",
"https://visualgenome.org/static/data/dataset/relationships.json.zip",
"https://visualgenome.org/static/data/dataset/object_alias.txt",
"https://visualgenome.org/static/data/dataset/relationship_alias.txt"
]
if not os.path.exists('data'):
os.mkdir('data')
else:
@tomas-wood
tomas-wood / whiteboard_algorithm.md
Last active November 13, 2020 17:11
partition graph for mini-batch gcn

Partitioning Graph for Mini-Batch Approximate Graph Convolutional Networks (GCNs)

So we want to be able to create partitions of the graph which we can perform graph convolutions on without having to load the entire dataset into memory.

Terms: A, the adjacency matrix of the graph (normalized and with identity matrix added to it, but still A). With N nodes in the graph, A is N X N.
X, a dense matrix of features such that each feature is of dimension F, then for all N nodes we get a dense matrix of size N X F

In the GCN paper and code, the entire dataset and the entire adjacency matrix are loaded into memory. The weight updates are done by