Skip to content

Instantly share code, notes, and snippets.

@HanOostdijk
Last active November 27, 2018 14:03
Show Gist options
  • Save HanOostdijk/49b3c59110dd3718ac5e3cc5856e5117 to your computer and use it in GitHub Desktop.
Save HanOostdijk/49b3c59110dd3718ac5e3cc5856e5117 to your computer and use it in GitHub Desktop.
filter on 'Value' column of OData v4 CBS tables
library(glue)
get_table_odata4 <- function (root=NULL,table="",subtable=NULL,query=NULL,verbose=F,encode=T) {
if (is.null(subtable)) {
subtable =""
} else subtable =glue("/{subtable}")
if (is.null(root))
root = "https://acc-ccb.cbs.nl"
if (is.null(query)) {
query =""
} else {
if (encode ==TRUE) {
query1=URLencode(query)
} else {
query1= query
}
query1 =glue("&{query1}")
}
url1 = glue("{root}/{table}{subtable}?$format=json{query1}")
if (verbose==T ) {
cat(glue("generated url: {url1}"))
cat(glue("unencoded query: {query}"))
}
get_table_odata4_int(url1)
}
get_table_odata4_int <- function (url1) {
res1 = httr::GET(url1)
if (httr::http_error(res1))
return(httr::http_status(res1)$message)
else
return(jsonlite::fromJSON(httr::content(res1,as="text"))$value)
}
q1 = get_table_odata4(table="83765NED",subtable="Observations",
query = paste0("$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) ",
"&$top=5 "),verbose=T)
class(q1)
q2 = get_table_odata4(table="83765NED",subtable="Observations",
query = paste0("$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) and ",
"contains(Value,'Amstel') "),verbose=T,encode = T)
print(q2)
start UPDATE 27NOV2018:
updated functions are now available in package HOQCutil ( https://github.com/HanOostdijk/HOQCutil )
as get_table_cbs_odata4 and get_table_cbs_odata4_GET
end UPDATE 27NOV2018
start UPDATE 30SEP2018:
see https://www.linkedin.com/feed/update/urn:li:activity:6439587327596732416
In his comment on that post Dennis Ramondt answered that 'Value' columns as from now will only be exclusive numeric or exclusive character.
end UPDATE 30SEP2018
There is new implementation of (some of) the CBS (Statistics Netherlands) tables.
This implementation uses Odata v4.
The actual data is in a subtable 'Observations' that contains a column (field) 'Value'.
I would appreciate some help in building a filter on this column.
Up till now I get 'Server error' messages.
Using table "83765NED" the following filtering works:
$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) &$top=5
The url https://acc-ccb.cbs.nl/83765NED/Observations?$format=json&$filter=(Measure%20eq%20'Gemeentenaam_1')%20and%20
(IsNumeric%20ne%20true)%20&$top=5%20
returns 5 observations with 6 fields
Expanding the filter to select only records from 'Amstelveen' I tried
$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) and contains(Value,'Amstel')
The url https://acc-ccb.cbs.nl/83765NED/Observations?$format=json&$filter=(Measure%20eq%20'Gemeentenaam_1')%20and%20
(IsNumeric%20ne%20true)%20and%20contains(Value,'Amstel')%20
returns "Server error: (500) Internal Server Error"
The R code used can be found in 'code.R' and the version of the packages used in 'sessionInfo.txt'.
sessionInfo()
R version 3.5.0 (2018-04-23)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] glue_1.3.0
loaded via a namespace (and not attached):
[1] httr_1.3.1 compiler_3.5.0 R6_2.2.2 tools_3.5.0 curl_3.2 yaml_2.2.0 jsonlite_1.5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment