SKOS concept mapping

Current:

Data retrieved in Twig during rendering-time.

[
  'skos:closeMatch': { # ConceptProperty
    'prop': 'skos:closeMatch',
    'super': null,
    'label': null,
    'values': [
      { # ConceptMappingPropertyValue
        'type': 'skos:closeMatch',
        'clan': 'en',
        'labelcache': [ "http://id.loc.gov/.../sh12345" ],
        'vocab': { ... # Vocabulary },
        'order': [],
        'model': { ... # Model },
        'resource': { ... # EasyRdf/Resource }
      }
    ],
    'is_sorted': false,
    'sort_by_notation': false
  }
]

What's used in Twig:

ConceptProperty.type
ConceptProperty.description
ConceptProperty.label
ConceptProperty.values (iterate through ConceptMappingPropertyValue)
ConceptMappingPropertyValue.uri
ConceptMappingPropertyValue.label
ConceptMappingPropertyValue.exvocab
ConceptMappingPropertyValue.notation
ConceptMappingPropertyValue.vocabname

During this call (Concept.php, Concept::getMappingProperties) to retrieve Concept Properties, it also populates the model... But appears somewhere else the JSON-LD requirements, demand that this is kept. Which is tricky....

The concept http://localhost:8000/ysopaikat/en/page/?uri=http://www.yso.fi/onto/yso/p188863, loads in 1.67 seconds, without the mapping properties.

With the mapping properties, it loads in 20.41 seconds.

Yup, we definitely have a performance penalty for doing that.

With the pull request NatLibFi/Skosmos#803, http://localhost:8000/rest/v1/ysopaikat/mappings?uri=http%3A%2F%2Fwww.yso.fi%2Fonto%2Fyso%2Fp125927 returns:

[
  {
    "type": [
      "skos:closeMatch"
    ],
    "from": {
      "memberSet": [
        {
          "uri": "http://www.yso.fi/onto/yso/p125927"
        }
      ]
    },
    "to": {
      "memberSet": [
        {
          "uri": "http://www.wikidata.org/entity/Q5401",
          "prefLabel": {
            "de": "Eurasien"
          }
        }
      ]
    },
    "fromScheme": {
      "uri": "http://www.yso.fi/onto/yso/places/"
    }
  },
  {
    "type": [
      "skos:exactMatch"
    ],
    "from": {
      "memberSet": [
        {
          "uri": "http://www.yso.fi/onto/yso/p125927"
        }
      ]
    },
    "to": {
      "memberSet": [
        {
          "uri": "http://www.yso.fi/onto/allars/Y31177",
          "prefLabel": {
            "sv": "Eurasien"
          }
        }
      ]
    },
    "fromScheme": {
      "uri": "http://www.yso.fi/onto/yso/places/"
    }
  },
  {
    "type": [
      "skos:exactMatch"
    ],
    "from": {
      "memberSet": [
        {
          "uri": "http://www.yso.fi/onto/yso/p125927"
        }
      ]
    },
    "to": {
      "memberSet": [
        {
          "uri": "http://www.yso.fi/onto/ysa/Y125927",
          "prefLabel": {
            "-": "ysa:Y125927"
          }
        }
      ]
    },
    "fromScheme": {
      "uri": "http://www.yso.fi/onto/yso/places/"
    },
    "toScheme": {
      "uri": ""
    }
  }
]

Concept query

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX skosxl: <http://www.w3.org/2008/05/skos-xl#>

CONSTRUCT {
 ?s ?p ?uri .
 ?sp ?uri ?op .
 ?uri ?p ?o .
 ?p rdfs:label ?proplabel .
 ?p rdfs:subPropertyOf ?pp .
 ?pp rdfs:label ?plabel .
 ?o a ?ot .
 ?o skos:prefLabel ?opl .
 ?o rdfs:label ?ol .
 ?o rdf:value ?ov .
 ?o skos:notation ?on .
 ?o ?oprop ?oval .
 ?o ?xlprop ?xlval .
 ?directgroup skos:member ?uri .
 ?parent skos:member ?group .
 ?group skos:prefLabel ?grouplabel .
 ?b1 rdf:first ?item .
 ?b1 rdf:rest ?b2 .
 ?item a ?it .
 ?item skos:prefLabel ?il .
 ?group a ?grouptype . 
 ?x skos:member ?o . ?x skos:prefLabel ?xl . ?x a <http://purl.org/iso25964/skos-thes#ThesaurusArray> .
} FROM NAMED <http://www.yso.fi/onto/yso/places/>  WHERE {
 VALUES (?uri) { (<http://www.yso.fi/onto/yso/p107650>) }
 GRAPH <http://www.yso.fi/onto/yso/places/> {
  {
    ?s ?p ?uri .
    FILTER(!isBlank(?s))
    FILTER(?p != skos:inScheme)
  }
  UNION
  { ?sp ?uri ?op . }
  UNION
  {
    ?directgroup skos:member ?uri .
    ?group skos:member+ ?uri .
    ?group skos:prefLabel ?grouplabel .
    ?group a ?grouptype .
    OPTIONAL { ?parent skos:member ?group }
  }
  UNION
  {
   ?uri ?p ?o .
   OPTIONAL {
     ?o rdf:rest* ?b1 .
     ?b1 rdf:first ?item .
     ?b1 rdf:rest ?b2 .
     OPTIONAL { ?item a ?it . }
     OPTIONAL { ?item skos:prefLabel ?il . }
   }
   OPTIONAL {
     { ?p rdfs:label ?proplabel . }
     UNION
     { ?p rdfs:subPropertyOf ?pp . }
   }
   OPTIONAL {
     { ?o a ?ot . }
     UNION
     { ?o skos:prefLabel ?opl . }
     UNION
     { ?o rdfs:label ?ol . }
     UNION
     { ?o rdf:value ?ov . 
       OPTIONAL { ?o ?oprop ?oval . }
     }
     UNION
     { ?o skos:notation ?on . }
     UNION
     { ?o a skosxl:Label .
       ?o ?xlprop ?xlval }
   } 
 OPTIONAL {
                      ?x skos:member ?o .
                      ?x a <http://purl.org/iso25964/skos-thes#ThesaurusArray> .
                      ?x skos:prefLabel ?xl .
                      FILTER NOT EXISTS {
                        ?x skos:member ?other .
                        MINUS { ?other skos:broader ?uri }
                      }
                    }
  }
 }
}

Wikidata HTTP requests

Example call: curl -H "accept: application/sparql-results+json,application/sparql-results+xml;q=0.8" -XGET https://www.wikidata.org/wiki/Special:EntityData/Q27275.nt

Example return:

<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Dataset> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://schema.org/about> <http://www.wikidata.org/entity/Q27275> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://creativecommons.org/ns#license> <http://creativecommons.org/publicdomain/zero/1.0/> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://schema.org/softwareVersion> "1.0.0" .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://schema.org/version> "770772895"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://schema.org/dateModified> "2018-10-23T01:05:56Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://wikiba.se/ontology#statements> "51"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://wikiba.se/ontology#identifiers> "20"^^<http://www.w3.org/2001/XMLSchema#integer> .
<https://www.wikidata.org/wiki/Special:EntityData/Q27275> <http://wikiba.se/ontology#sitelinks> "152"^^<http://www.w3.org/2001/XMLSchema#integer> .
<http://www.wikidata.org/entity/Q27275> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://wikiba.se/ontology#Item> .
<https://ko.wikipedia.org/wiki/%EC%A4%91%EC%95%99%EC%95%84%EC%8B%9C%EC%95%84> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://ko.wikipedia.org/wiki/%EC%A4%91%EC%95%99%EC%95%84%EC%8B%9C%EC%95%84> <http://schema.org/about> <http://www.wikidata.org/entity/Q27275> .
<https://ko.wikipedia.org/wiki/%EC%A4%91%EC%95%99%EC%95%84%EC%8B%9C%EC%95%84> <http://schema.org/inLanguage> "ko" .
<https://ko.wikipedia.org/wiki/%EC%A4%91%EC%95%99%EC%95%84%EC%8B%9C%EC%95%84> <http://schema.org/isPartOf> <https://ko.wikipedia.org/> .
<https://ko.wikipedia.org/wiki/%EC%A4%91%EC%95%99%EC%95%84%EC%8B%9C%EC%95%84> <http://schema.org/name> "\uC911\uC559\uC544\uC2DC\uC544"@ko .
<https://ko.wikipedia.org/> <http://wikiba.se/ontology#wikiGroup> "wikipedia" .
<https://ace.wikipedia.org/wiki/Asia_Teung%C3%B6h> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://ace.wikipedia.org/wiki/Asia_Teung%C3%B6h> <http://schema.org/about> <http://www.wikidata.org/entity/Q27275> .
<https://ace.wikipedia.org/wiki/Asia_Teung%C3%B6h> <http://schema.org/inLanguage> "ace" .
<https://ace.wikipedia.org/wiki/Asia_Teung%C3%B6h> <http://schema.org/isPartOf> <https://ace.wikipedia.org/> .
<https://ace.wikipedia.org/wiki/Asia_Teung%C3%B6h> <http://schema.org/name> "Asia Teung\u00F6h"@ace .
<https://ace.wikipedia.org/> <http://wikiba.se/ontology#wikiGroup> "wikipedia" .
<https://an.wikipedia.org/wiki/Asia_Central> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://schema.org/Article> .
<https://an.wikipedia.org/wiki/Asia_Central> <http://schema.org/about> <http://www.wikidata.org/entity/Q27275> .
<https://an.wikipedia.org/wiki/Asia_Central> <http://schema.org/inLanguage> "an" .
<https://an.wikipedia.org/wiki/Asia_Central> <http://schema.org/isPartOf> <https://an.wikipedia.org/> .
<https://an.wikipedia.org/wiki/Asia_Central> <http://schema.org/name> "Asia Central"@an .
...
...

kinow/skosmos-performance-issue-1.md

Current:

Concept query

Wikidata HTTP requests

kinow commented Dec 19, 2018 •

edited

Loading

kinow commented Dec 19, 2018

kinow commented Dec 19, 2018

kinow commented Dec 27, 2018

kinow/skosmos-performance-issue-1.md

Current:

Concept query

Wikidata HTTP requests

kinow commented Dec 19, 2018 • edited Loading

kinow commented Dec 19, 2018

kinow commented Dec 19, 2018

kinow commented Dec 27, 2018

kinow commented Dec 19, 2018 •

edited

Loading