This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from SPARQLWrapper import SPARQLWrapper, JSON | |
| sparql = SPARQLWrapper("http://linkeddata.uriburner.com/sparql") | |
| sparql.setQuery(""" | |
| PREFIX xsd: <http://www.w3.org/2001/XMLSchema#> | |
| PREFIX foaf: <http://xmlns.com/foaf/0.1/> | |
| PREFIX owl: <http://www.w3.org/2002/07/owl#> | |
| PREFIX wd: <http://www.wikidata.org/entity/> | |
| PREFIX wdt: <http://www.wikidata.org/prop/direct/> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| """ | |
| Test du Stanford NER tagger avec les modèles CRF d'Europeana | |
| entrainés sur des journaux : | |
| http://lab.kbresearch.nl/static/html/eunews.html | |
| La fonction est lente --> songer au multiprocessing | |
| """ | |
| import warnings | |
| with warnings.catch_warnings(): | |
| warnings.filterwarnings("ignore",category=DeprecationWarning) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| library(rvest) | |
| library(dplyr) | |
| #création des urls de départ | |
| main_url <- "https://fr.wikipedia.org/wiki/Décès_en_" | |
| annee <- 2010:2018 | |
| urls <- paste0(main_url, annee) | |
| #vecteur vide qui contiendra les urls des pages | |
| liens <- vector() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| import functools | |
| from ngram import NGram | |
| def get_similar(data, target): | |
| G = NGram(target) | |
| return G.find(data) | |
| class cache(object): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import subprocess | |
| console = subprocess.run('openrefine_cli.exe --help', shell=True, stdout=subprocess.PIPE).stdout.decode('utf8') | |
| print(console) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import requests | |
| import requests_cache | |
| from time import sleep | |
| # Nominatim bloque les requêtes répétées | |
| requests_cache.install_cache('nominatim_cache') | |
| def get_nominatim(value, countrycodes=['BE',''], limit=5, lang="fr"): | |
| # doc : https://wiki.openstreetmap.org/wiki/Nominatim |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # -*- coding: utf-8 -*- | |
| """ | |
| Created on Fri Jan 5 12:35:54 2018 | |
| @author: ettor | |
| """ | |
| import pandas as pd | |
| import requests | |
| import requests_cache |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import itertools | |
| def glance(d): | |
| """aperçu d'un échantillon non ordonné d'un dictionnaire""" | |
| return dict(itertools.islice(d.items(), 3)) | |
| glance(json_file) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <qualifieddc xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:dc="http://purl.org/dc/elements/1.1" xmlns:dcterms="http://purl.org/dc/terms" xmlns:marcrel="http://www.loc.gov/marc.relators" xsi:schemaLocation="http://www.loc.gov/marc.relators http://imlsdcc2.grainger.illinois.edu/registry/marcrel.xsd" xsi:noNamespaceSchemaLocation="http://dublincore.org/schemas/xmls/qdc/2008/02/11/qualifieddc.xsd"> | |
| {{forNonBlank(cells["id"], v, "<dc:identifier>"+v.value+"</dc:identifier>", "")}} | |
| {{forNonBlank(cells["Title"], v, "<dc:title>"+v.value+"</dc:title>", "")}} | |
| {{forNonBlank(cells["Creator"], v, "<dc:creator>"+v.value+"</dc:creator>", "")}} | |
| {{forNonBlank(cells["Date"], v, "<dc:date>"+v.value+"</dc:date>", "")}} | |
| {{forNonBlank(cells["Description"], v, "<dc:description>"+v.value+"</dc:description>", "")}} | |
| {{forNonBlank(cells["Description2"], v, "<dc:description>"+v.value+"</dc:description>", "")}} | |
| {{forNonBlank(cells["Rights"], v, "<dc:rights>"+v.value+"</dc:rights>", "")}} | |
| {{forNonBlank(cells["Type"], v, "<dc: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import json | |
| from pprint import pprint | |
| with open(r'C:\Users\ettor\Desktop\lod-data.json') as f: | |
| data = json.load(f) | |
| myfile = open('test.txt', 'a') | |
| for key in data.keys(): |