Skip to content

Instantly share code, notes, and snippets.

@michimau

michimau/air

Created March 11, 2021 10:26
Show Gist options
  • Save michimau/dcac6a3f078d23bd5f7b5a591c9bcc3b to your computer and use it in GitHub Desktop.
Save michimau/dcac6a3f078d23bd5f7b5a591c9bcc3b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
air
11/03/2021 11:24:34[2021-03-11 10:24:34.377] - DEBUG --- eionet.cr.harvest.scheduled.UrgentHarvestQueue : Pull harvest added to the urgent queue, url = http://www.eea.europa.eu/portal_vocabularies/themes/air - (UrgentHarvestQueue.java:108)
11/03/2021 11:24:49[2021-03-11 10:24:49.845] - DEBUG --- eionet.cr.harvest.BaseHarvest : - (BaseHarvest.java:289)
11/03/2021 11:24:49[2021-03-11 10:24:49.846] - INFO --- eionet.cr.harvest.BaseHarvest : Starting harvest [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (BaseHarvest.java:290)
11/03/2021 11:24:49[2021-03-11 10:24:49.869] - DEBUG --- eionet.cr.harvest.PullHarvest : Connecting http://www.eea.europa.eu/portal_vocabularies/themes/air [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:321)
11/03/2021 11:24:49[2021-03-11 10:24:49.873] - DEBUG --- eionet.cr.harvest.BaseHarvest : Timeout set to the maximum 129600000 ms, last harvest duration could not be detected [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (BaseHarvest.java:1146)
11/03/2021 11:24:49[2021-03-11 10:24:49.939] - DEBUG --- eionet.cr.harvest.PullHarvest : Received code 301 from http://www.eea.europa.eu/portal_vocabularies/themes/air [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:327)
11/03/2021 11:24:49[2021-03-11 10:24:49.939] - DEBUG --- eionet.cr.harvest.PullHarvest : http://www.eea.europa.eu/portal_vocabularies/themes/air redirects to https://www.eea.europa.eu/portal_vocabularies/themes/air [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:353)
11/03/2021 11:24:49[2021-03-11 10:24:49.939] - DEBUG --- eionet.cr.harvest.PullHarvest : Connecting https://www.eea.europa.eu/portal_vocabularies/themes/air [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:321)
11/03/2021 11:24:50[2021-03-11 10:24:50.083] - DEBUG --- eionet.cr.harvest.PullHarvest : Received code 303 from https://www.eea.europa.eu/portal_vocabularies/themes/air [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:327)
11/03/2021 11:24:50[2021-03-11 10:24:50.083] - DEBUG --- eionet.cr.harvest.PullHarvest : https://www.eea.europa.eu/portal_vocabularies/themes/air redirects to http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:353)
11/03/2021 11:24:50[2021-03-11 10:24:50.084] - DEBUG --- eionet.cr.harvest.PullHarvest : Connecting http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:321)
11/03/2021 11:24:50[2021-03-11 10:24:50.284] - DEBUG --- eionet.cr.harvest.PullHarvest : Received code 200 from http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:327)
11/03/2021 11:24:50[2021-03-11 10:24:50.285] - DEBUG --- eionet.cr.harvest.BaseHarvest : Switching context to http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf [http://www.eea.europa.eu/portal_vocabularies/themes/air] - (BaseHarvest.java:815)
11/03/2021 11:24:50[2021-03-11 10:24:50.287] - DEBUG --- eionet.cr.harvest.BaseHarvest : - (BaseHarvest.java:289)
11/03/2021 11:24:50[2021-03-11 10:24:50.287] - INFO --- eionet.cr.harvest.BaseHarvest : Starting harvest [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:290)
11/03/2021 11:24:50[2021-03-11 10:24:50.309] - DEBUG --- eionet.cr.harvest.PullHarvest : Downloading file [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (PullHarvest.java:723)
11/03/2021 11:24:50[2021-03-11 10:24:50.314] - DEBUG --- eionet.cr.harvest.BaseHarvest : File is in RDF or web feed format [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:1217)
11/03/2021 11:24:50[2021-03-11 10:24:50.314] - DEBUG --- eionet.cr.harvest.BaseHarvest : Loading file into triple store, loader class is RDFFormatLoader [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:1219)
11/03/2021 11:24:50[2021-03-11 10:24:50.314] - DEBUG --- eionet.cr.harvest.BaseHarvest : Running "Pre-purge" scripts... [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:416)
11/03/2021 11:24:50[2021-03-11 10:24:50.525] - DEBUG --- eionet.cr.harvest.BaseHarvest : No active "Pre-purge" scripts were found for this source [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:447)
11/03/2021 11:24:50[2021-03-11 10:24:50.526] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Starting content loading ... [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (VirtuosoHarvestSourceDAO.java:1612)
11/03/2021 11:24:50[2021-03-11 10:24:50.531] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Current triples count = -1, file size = 13633, direct loading flag = FALSE [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (VirtuosoHarvestSourceDAO.java:1642)
11/03/2021 11:24:50[2021-03-11 10:24:50.536] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Going to load triples into ORIGINAL graph [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (VirtuosoHarvestSourceDAO.java:1678)
11/03/2021 11:24:50[2021-03-11 10:24:50.537] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Executing file loading command: DB.DBA.RDF_LOAD_RDFXML_MT(file_open('/var/tmp/eionet.cr.tempfile-1615458290309-547ec754-0c11-487e-bc1b-7dc210847ac9'), 'http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf', 'http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf', 1, 8, 1) - (VirtuosoHarvestSourceDAO.java:2020)
11/03/2021 11:24:50[2021-03-11 10:24:50.889] - DEBUG --- eionet.cr.util.FileDeletionJob : File successfully deleted: /var/tmp/eionet.cr.tempfile-1615458290309-547ec754-0c11-487e-bc1b-7dc210847ac9 - (FileDeletionJob.java:140)
11/03/2021 11:24:50[2021-03-11 10:24:50.889] - DEBUG --- eionet.cr.harvest.PullHarvest : 169 triples loaded [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (PullHarvest.java:380)
11/03/2021 11:24:50[2021-03-11 10:24:50.889] - DEBUG --- eionet.cr.harvest.BaseHarvest : Running "Post-import" scripts... [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:416)
11/03/2021 11:24:50[2021-03-11 10:24:50.893] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing all-source "Post-import" script titled "Delete Yago classes" - (BaseHarvest.java:494)
11/03/2021 11:24:50[2021-03-11 10:24:50.894] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing parsedQuery: DELETE FROM <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50{ ?s a ?o }
11/03/2021 11:24:50FROM <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50WHERE {
11/03/2021 11:24:50 ?s a ?o
11/03/2021 11:24:50 FILTER(REGEX(?o,'^http://dbpedia.org/class/yago/'))
11/03/2021 11:24:50} - (BaseHarvest.java:497)
11/03/2021 11:24:50[2021-03-11 10:24:50.991] - DEBUG --- eionet.cr.harvest.BaseHarvest : Script's update count was 0 - (BaseHarvest.java:572)
11/03/2021 11:24:50[2021-03-11 10:24:50.992] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing all-source "Post-import" script titled "Create labels from foaf:name if there is none" - (BaseHarvest.java:494)
11/03/2021 11:24:50[2021-03-11 10:24:50.992] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing parsedQuery: PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
11/03/2021 11:24:50PREFIX foaf: <http://xmlns.com/foaf/0.1/>
11/03/2021 11:24:50
11/03/2021 11:24:50INSERT INTO <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50{ ?s rdfs:label `STR(?o)` }
11/03/2021 11:24:50FROM <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50WHERE {
11/03/2021 11:24:50 ?s foaf:name ?o FILTER(LANG(?o) IN ('en','') )
11/03/2021 11:24:50 OPTIONAL {?s rdfs:label ?currlabel } FILTER(!BOUND(?currlabel))
11/03/2021 11:24:50} - (BaseHarvest.java:497)
11/03/2021 11:24:50[2021-03-11 10:24:50.997] - DEBUG --- eionet.cr.harvest.BaseHarvest : Script's update count was 0 - (BaseHarvest.java:572)
11/03/2021 11:24:50[2021-03-11 10:24:50.997] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing all-source "Post-import" script titled "Inherit topics from parent content if none" - (BaseHarvest.java:494)
11/03/2021 11:24:50[2021-03-11 10:24:50.997] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing parsedQuery: # Owner/Maintainer: Antonio De Marinis
11/03/2021 11:24:50# to suggest changes/report issues, please contact owner.
11/03/2021 11:24:50
11/03/2021 11:24:50PREFIX pt: <http://www.eea.europa.eu/portal_types#>
11/03/2021 11:24:50PREFIX dct: <http://purl.org/dc/terms/>
11/03/2021 11:24:50
11/03/2021 11:24:50INSERT INTO <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50{
11/03/2021 11:24:50 ?s pt:topic ?topic.
11/03/2021 11:24:50}
11/03/2021 11:24:50FROM
11/03/2021 11:24:50 <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:50WHERE {
11/03/2021 11:24:50 ?s dct:isPartOf ?parent.
11/03/2021 11:24:50 OPTIONAL {?s pt:topic ?mytopic } FILTER(!BOUND(?mytopic)).
11/03/2021 11:24:50 GRAPH ?g
11/03/2021 11:24:50 {
11/03/2021 11:24:50 ?parent pt:topic ?topic
11/03/2021 11:24:50 }
11/03/2021 11:24:50 filter regex(str(<http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>), "http://www.eea.europa.eu")
11/03/2021 11:24:50} - (BaseHarvest.java:497)
11/03/2021 11:24:51[2021-03-11 10:24:51.017] - DEBUG --- eionet.cr.harvest.BaseHarvest : Script's update count was 0 - (BaseHarvest.java:572)
11/03/2021 11:24:51[2021-03-11 10:24:51.091] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing type-specific "Post-import" script titled "Match to country code or title and match to countries skos" - (BaseHarvest.java:494)
11/03/2021 11:24:51[2021-03-11 10:24:51.091] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing parsedQuery: PREFIX eea: <http://rdfdata.eionet.europa.eu/eea/ontology/>
11/03/2021 11:24:51PREFIX dcterms: <http://purl.org/dc/terms/>
11/03/2021 11:24:51PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
11/03/2021 11:24:51PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
11/03/2021 11:24:51
11/03/2021 11:24:51INSERT INTO <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:51{ ?rowuri skos:exactMatch ?countryuri .
11/03/2021 11:24:51 ?rowuri rdfs:label ?label
11/03/2021 11:24:51}
11/03/2021 11:24:51
11/03/2021 11:24:51WHERE {
11/03/2021 11:24:51 GRAPH <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf> {
11/03/2021 11:24:51 ?rowuri dcterms:title ?value
11/03/2021 11:24:51 }
11/03/2021 11:24:51 GRAPH <http://rdfdata.eionet.europa.eu/eea/countries.rdf> {
11/03/2021 11:24:51 ?countryuri skos:notation ?code;
11/03/2021 11:24:51 rdfs:label ?label
11/03/2021 11:24:51 }
11/03/2021 11:24:51 FILTER (bif:lower(STR(?label)) = bif:lower(STR(?value))
11/03/2021 11:24:51 || bif:lower(STR(?code)) = bif:lower(STR(?value))
11/03/2021 11:24:51 )
11/03/2021 11:24:51} - (BaseHarvest.java:497)
11/03/2021 11:24:51[2021-03-11 10:24:51.255] - DEBUG --- eionet.cr.harvest.BaseHarvest : Script's update count was 0 - (BaseHarvest.java:572)
11/03/2021 11:24:51[2021-03-11 10:24:51.255] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing type-specific "Post-import" script titled "Make all simple vocabulary terms be skos concepts with one broader term and fixed scheme" - (BaseHarvest.java:494)
11/03/2021 11:24:51[2021-03-11 10:24:51.255] - DEBUG --- eionet.cr.harvest.BaseHarvest : Executing parsedQuery: PREFIX cr: <http://cr.eionet.europa.eu/ontologies/contreg.rdf#>
11/03/2021 11:24:51PREFIX e: <http://www.eea.europa.eu/portal_types/SimpleVocabularyTerm#>
11/03/2021 11:24:51PREFIX dcterms: <http://purl.org/dc/terms/>
11/03/2021 11:24:51PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
11/03/2021 11:24:51
11/03/2021 11:24:51INSERT INTO <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:51 { ?s skos:notation ?id .
11/03/2021 11:24:51 ?s a skos:Concept .
11/03/2021 11:24:51 ?s rdfs:label ?title .
11/03/2021 11:24:51 ?s skos:prefLabel ?title .
11/03/2021 11:24:51 ?s skos:broader ?broader .
11/03/2021 11:24:51 ?s skos:inScheme <http://www.eea.europa.eu/portal_vocabularies>
11/03/2021 11:24:51}
11/03/2021 11:24:51FROM <http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf>
11/03/2021 11:24:51WHERE {
11/03/2021 11:24:51 ?s dcterms:isPartOf ?broader .
11/03/2021 11:24:51 ?s e:id ?id .
11/03/2021 11:24:51 OPTIONAL {?s dcterms:title ?title}
11/03/2021 11:24:51} - (BaseHarvest.java:497)
11/03/2021 11:24:51[2021-03-11 10:24:51.305] - DEBUG --- eionet.cr.harvest.BaseHarvest : Script's update count was 6 - (BaseHarvest.java:572)
11/03/2021 11:24:51[2021-03-11 10:24:51.305] - DEBUG --- eionet.cr.harvest.BaseHarvest : Running "Push" scripts... [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:416)
11/03/2021 11:24:51[2021-03-11 10:24:51.374] - DEBUG --- eionet.cr.harvest.BaseHarvest : No active "Push" scripts were found for this source [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:447)
11/03/2021 11:24:51[2021-03-11 10:24:51.375] - DEBUG --- eionet.cr.harvest.BaseHarvest : Updating harvest source record [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:699)
11/03/2021 11:24:51[2021-03-11 10:24:51.376] - DEBUG --- eionet.cr.harvest.BaseHarvest : Updating harvest record, saving harvest messages [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:686)
11/03/2021 11:24:51[2021-03-11 10:24:51.396] - DEBUG --- eionet.cr.harvest.BaseHarvest : Saving 6 triples of harvest source metadata, cleaning all previous metadata first [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:666)
11/03/2021 11:24:51[2021-03-11 10:24:51.417] - DEBUG --- eionet.cr.harvest.BaseHarvest : Inferring new harvest sources [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:896)
11/03/2021 11:24:51[2021-03-11 10:24:51.432] - DEBUG --- eionet.cr.harvest.BaseHarvest : 0 new harvest sources found and inserted [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:900)
11/03/2021 11:24:51[2021-03-11 10:24:51.432] - DEBUG --- eionet.cr.harvest.BaseHarvest : Deleting old harvests history of the harvested source ... [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:581)
11/03/2021 11:24:51[2021-03-11 10:24:51.434] - DEBUG --- eionet.cr.harvest.BaseHarvest : Checking sources that need removal [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:608)
11/03/2021 11:24:51[2021-03-11 10:24:51.434] - INFO --- eionet.cr.harvest.BaseHarvest : Harvest finished [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (BaseHarvest.java:384)
11/03/2021 11:24:51[2021-03-11 10:24:51.434] - DEBUG --- eionet.cr.harvest.BaseHarvest : - (BaseHarvest.java:385)
11/03/2021 11:24:51[2021-03-11 10:24:51.435] - DEBUG --- eionet.cr.harvest.PullHarvest : Clearing these sources: [http://www.eea.europa.eu/portal_vocabularies/themes/air, https://www.eea.europa.eu/portal_vocabularies/themes/air] - (PullHarvest.java:1180)
11/03/2021 11:24:51[2021-03-11 10:24:51.436] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Deleting this resource from harvester graph - (VirtuosoHarvestSourceDAO.java:680)
11/03/2021 11:24:51[2021-03-11 10:24:51.436] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Deleting this resource from harvester graph - (VirtuosoHarvestSourceDAO.java:680)
11/03/2021 11:24:51[2021-03-11 10:24:51.441] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Deleting requested sources from harvest scripts table. - (VirtuosoHarvestSourceDAO.java:603)
11/03/2021 11:24:51[2021-03-11 10:24:51.442] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Deleting requested sources from harvest source table. - (VirtuosoHarvestSourceDAO.java:605)
11/03/2021 11:24:51[2021-03-11 10:24:51.483] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Deleting requested sources from urgent harvest queue. - (VirtuosoHarvestSourceDAO.java:607)
11/03/2021 11:24:51[2021-03-11 10:24:51.484] - DEBUG --- eionet.cr.dao.virtuoso.VirtuosoHarvestSourceDAO : Clearing graphs: [http://www.eea.europa.eu/portal_vocabularies/themes/air, https://www.eea.europa.eu/portal_vocabularies/themes/air] - (VirtuosoHarvestSourceDAO.java:717)
11/03/2021 11:24:51[2021-03-11 10:24:51.488] - DEBUG --- eionet.cr.harvest.PullHarvest : Saving redirection metadata: [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (PullHarvest.java:1189)
11/03/2021 11:24:51[2021-03-11 10:24:51.501] - DEBUG --- eionet.cr.harvest.PullHarvest : Saving redirection metadata: [http://www.eea.europa.eu/portal_vocabularies/themes/air/@@rdf] - (PullHarvest.java:1189)
11/03/2021 11:24:51[2021-03-11 10:24:51.516] - DEBUG --- eionet.cr.harvest.PullHarvest : Clearing the graph of http://www.eea.europa.eu/portal_vocabularies/themes/air - (PullHarvest.java:1203)
11/03/2021 11:24:51[2021-03-11 10:24:51.516] - DEBUG --- eionet.cr.harvest.PullHarvest : Clearing the graph of https://www.eea.europa.eu/portal_vocabularies/themes/air - (PullHarvest.java:1203)
Connected
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment