Skip to content

Instantly share code, notes, and snippets.

View tomas-wood's full-sized avatar

Hoemas Ruud tomas-wood

View GitHub Profile
@tomas-wood
tomas-wood / test_api.py
Created October 16, 2018 17:46
GES API`
import urllib3
http = urllib3.PoolManager()
with open('token', 'r') as f:
token = f.read()
# Headers are the way we pass through the token.
headers = {}
headers['Content-Type'] = 'application/json'
@tomas-wood
tomas-wood / whiteboard_algorithm.md
Last active November 13, 2020 17:11
partition graph for mini-batch gcn

Partitioning Graph for Mini-Batch Approximate Graph Convolutional Networks (GCNs)

So we want to be able to create partitions of the graph which we can perform graph convolutions on without having to load the entire dataset into memory.

Terms: A, the adjacency matrix of the graph (normalized and with identity matrix added to it, but still A). With N nodes in the graph, A is N X N.
X, a dense matrix of features such that each feature is of dimension F, then for all N nodes we get a dense matrix of size N X F

In the GCN paper and code, the entire dataset and the entire adjacency matrix are loaded into memory. The weight updates are done by

@tomas-wood
tomas-wood / fetch_data.py
Created September 25, 2018 22:48
get visual genome data
def fetch_data():
urls = [
"https://visualgenome.org/static/data/dataset/objects.json.zip",
"https://visualgenome.org/static/data/dataset/relationships.json.zip",
"https://visualgenome.org/static/data/dataset/object_alias.txt",
"https://visualgenome.org/static/data/dataset/relationship_alias.txt"
]
if not os.path.exists('data'):
os.mkdir('data')
else:
Wed Sep 19 11:48:26 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 396.37 Driver Version: 396.37 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 TITAN X (Pascal) Off | 00000000:04:00.0 Off | N/A |
| 23% 40C P2 50W / 250W | 893MiB / 12196MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
@tomas-wood
tomas-wood / use_curl.sh
Created September 20, 2018 18:05
API call to GES
curl -H "Content-Type:application/json" -X POST --data '{"command":"graph=EywaGraph.open(\"ges_abfe\");g=graph.traversal();g.V().limit(10)"}' http://49.4.88.7/graph/ges/v1.0/1111/1/graphs/222/execute-gremlin-query
{"errorMessage":"Resource not allowed!","errorCode":"GES.8006"}
@tomas-wood
tomas-wood / detectron_api.py
Created September 19, 2018 22:58
simple detectron api example
import urllib3
import pickle
import json
http = urllib3.PoolManager()
url = "http://localhost:5000/detectron"
img_url="https://cdn7.dissolve.com/p/D1028_98_301/D1028_98_301_1200.jpg"
@tomas-wood
tomas-wood / server.py
Last active September 21, 2018 18:43
flask restful api for calling detectron on an image via url.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
from collections import defaultdict
import argparse
import cv2 # NOQA (Must import before importing caffe2 due to bug in cv2)
import glob
import logging
@tomas-wood
tomas-wood / gen_rpn_app.py
Created September 18, 2018 23:15
generate roidb with rpn
#!/usr/bin/env python2
# -*- coding: utf-8 -*-
################################################################
#
# This script takes an image as input and outputs the
# region proposals
#
# Author: Aniket Adnaik ([email protected])
################################################################
@tomas-wood
tomas-wood / nvidia-smi
Created September 18, 2018 22:38
every GPU is being used.
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 5760 C /usr/bin/python2.7 11739MiB |
| 1 5760 C /usr/bin/python2.7 11589MiB |
| 2 5760 C /usr/bin/python2.7 11589MiB |
| 3 5760 C /usr/bin/python2.7 11589MiB |
| 4 5760 C /usr/bin/python2.7 11589MiB |
| 5 5760 C /usr/bin/python2.7 11589MiB |