lawlesst

#!/bin/sh -e

set -x # Print commands and their arguments as they are executed.

JETTY_HOME=/dock/jetty7.6.14
VITRO_HOME=/tmp/vitrohome
STAGE_DIR=/tmp/vbuild

ant clean distribute -Dskiptests=true
#ant distribute -Dskiptests=true

lawlesst

"""

Example code for connecting to Stardog (http://stardog.com/) with
Python's RDFLib (http://github.com/rdflib).

See longer description: http://lawlesst.github.io/notebook/rdflib-stardog.html

"""

lawlesst

##University Linking

Using OrgRef to link universities represented in VIVOs to DBPedia and to each other.

• Exploring how VIVO instances could share data like this and reuse URIs.
• Crawled eight VIVO instances and pulled back data about Universities.
• Matched each site to OrgRef using simple string normalization and equality.
• Matched OrgRef to DBPedia using the Wikipedia Page IDs included by OrgRef.
• Matched VIVOs to DBPedia
• Loaded data into a Linked Data Fragments server: http://ldf-vivo.herokuapp.com/orgref

lawlesst

@prefix :        <http://localhost/jena_example/#> .
@prefix rdf:     <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs:    <http://www.w3.org/2000/01/rdf-schema#> .
@prefix tdb:     <http://jena.hpl.hp.com/2008/tdb#> .
@prefix ja:      <http://jena.hpl.hp.com/2005/11/Assembler#> .
@prefix text:    <http://jena.apache.org/text#> .
@prefix fuseki:  <http://jena.apache.org/fuseki#> .

## Example of a TDB dataset and text index
## Initialize TDB

lawlesst

"""

Used chardet library to find encoding - it reported:
{'confidence': 0.7158930206047512, 'encoding': 'ISO-8859-2'}

Use line writing to print ntriples as they are read to avoid
memory issues with storing all statements in an in memory 
RDFLib Graph.

"""

lawlesst

Sample files for pulling data into a VIVO instance from Linked Data Fragments, Wikidata, and Wikipedia.

lawlesst

petl

Python ETL

• Documentation: https://petl.readthedocs.org/en/latest/index.html
• Source: https://github.com/alimanfoo/petl
• Mailing list: http://groups.google.com/group/python-etl

Many in the VIVO community use Python for data manipulation and transformation tasks (ETL). petl is a framework for reading data from various sources, transforming it, and passing it along to another source.

lawlesst

September 25, 2015

Steps to run ldf-server on Heroku with an HDT backend.

• Read the Node troubleshooting guide from Heroku for background.
• Run the commands in the Start with a Blank Slate section of the guide.
• Add package.json to your git repository if it isn't already.
• For my project, I had to explicitly add hdt - 1.2.2 as a dependendency.

package.json

lawlesst

birthCity,birthState,birthYear,lat,long
Aberdeen,MD,1935,39.515856,-76.170180
Aberdeen,MD,1869,39.515856,-76.170180
Aberdeen,MS,1901,33.828299,-88.554741
Aberdeen,SD,1977,45.464678,-98.468104
Aberdeen,SD,1893,45.464678,-98.468104
Aberdeen,SD,1959,45.464678,-98.468104
Aberdeen,SD,1943,45.464678,-98.468104
Aberdeen,WA,1959,46.976677,-123.809718
Aberdeen,WA,1914,46.976677,-123.809718

lawlesst

<?xml version="1.0" encoding="UTF-8" ?>
<request xmlns="http://www.isinet.com/xrpc41" src="app.id=InternalVIVODemo">
  <fn name="LinksAMR.retrieve">
    <list>
      <!-- authentication -->
      <map>
        <val name="username">xxx</val>
        <val name="password">xxx</val>
      </map>
      <!-- what to to return -->