Skip to content

Instantly share code, notes, and snippets.

View fabianvf's full-sized avatar

Fabian von Feilitzsch fabianvf

View GitHub Profile
@fabianvf
fabianvf / Marvel REST commands
Last active August 29, 2015 14:01
Marvel commands for dummy hierarchy in ElasticSearch
GET _search
{
"query": {
"match_all": {}
}
}
DELETE website
PUT website
@fabianvf
fabianvf / pip requirements
Last active February 1, 2017 11:37
ansible pip install lxml
- name: Install lxml requirements
apt: "pkg={{ item }} state=present update_cache=yes"
sudo: yes
with_items:
- python-dev
- libxml2
- libxml2-dev
- libxslt1-dev
- lib32z1-dev
- libssl-dev
### Keybase proof
I hereby claim:
* I am fabianvf on github.
* I am fabianvf (https://keybase.io/fabianvf) on keybase.
* I have a public key whose fingerprint is 0D2E 3DAC 3091 715B 7687 2B45 A87E 7040 BF11 F6F0
To claim this, I am signing this object:
@fabianvf
fabianvf / DEBIAN | UBUNTU
Last active August 29, 2015 14:13
Getting Elasticsearch1.4.2 with wikipedia data loaded
# Install elasticsearch
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.4.2.deb
sudo dpkg -i elasticsearch-1.4.2.deb
# Install the wikipedia river
/usr/share/elasticsearch/bin/plugin -i elasticsearch/elasticsearch-river-wikipedia/2.4.1
# Start elasticsearch
sudo service elasticsearch start
{
"title": "Scrapi",
"services":
{
"query":
{
@fabianvf
fabianvf / Get
Last active August 29, 2015 14:17
How to get data from the OSF elasticsearch endpoint
import requests
response = requests.get('https://osf.io/api/v1/search/', params={
'q': 'test AND category:project' # Full lucene syntax accepted: http://lucene.apache.org/core/2_9_4/queryparsersyntax.html
})
results = response.json()
@fabianvf
fabianvf / example.py
Last active August 29, 2015 14:17
Graph data osf
# coding: utf-8
import json
import requests
HEADERS = {
'Content-type': 'application/json'
}
URL = 'https://osf.io/api/v1/search/'
@fabianvf
fabianvf / Example document.json
Last active August 29, 2015 14:18
json schema
{
"collectionTime": "2014-03-14T00:00:00Z",
"creationDate": "2014-09-12",
"creator": [
{
"name": "Roger Danger Ebert",
"sameAs": [
"/wrgr2/",
"DAE"
],
import os
import json
from scrapi.processing.base import BaseProcessor
class StorageProcessor(BaseProcessor):
NAME = 'storage'
def process_raw(self, raw):
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{