Last active
November 27, 2018 14:03
-
-
Save HanOostdijk/49b3c59110dd3718ac5e3cc5856e5117 to your computer and use it in GitHub Desktop.
filter on 'Value' column of OData v4 CBS tables
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
library(glue) | |
get_table_odata4 <- function (root=NULL,table="",subtable=NULL,query=NULL,verbose=F,encode=T) { | |
if (is.null(subtable)) { | |
subtable ="" | |
} else subtable =glue("/{subtable}") | |
if (is.null(root)) | |
root = "https://acc-ccb.cbs.nl" | |
if (is.null(query)) { | |
query ="" | |
} else { | |
if (encode ==TRUE) { | |
query1=URLencode(query) | |
} else { | |
query1= query | |
} | |
query1 =glue("&{query1}") | |
} | |
url1 = glue("{root}/{table}{subtable}?$format=json{query1}") | |
if (verbose==T ) { | |
cat(glue("generated url: {url1}")) | |
cat(glue("unencoded query: {query}")) | |
} | |
get_table_odata4_int(url1) | |
} | |
get_table_odata4_int <- function (url1) { | |
res1 = httr::GET(url1) | |
if (httr::http_error(res1)) | |
return(httr::http_status(res1)$message) | |
else | |
return(jsonlite::fromJSON(httr::content(res1,as="text"))$value) | |
} | |
q1 = get_table_odata4(table="83765NED",subtable="Observations", | |
query = paste0("$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) ", | |
"&$top=5 "),verbose=T) | |
class(q1) | |
q2 = get_table_odata4(table="83765NED",subtable="Observations", | |
query = paste0("$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) and ", | |
"contains(Value,'Amstel') "),verbose=T,encode = T) | |
print(q2) | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
start UPDATE 27NOV2018: | |
updated functions are now available in package HOQCutil ( https://github.com/HanOostdijk/HOQCutil ) | |
as get_table_cbs_odata4 and get_table_cbs_odata4_GET | |
end UPDATE 27NOV2018 | |
start UPDATE 30SEP2018: | |
see https://www.linkedin.com/feed/update/urn:li:activity:6439587327596732416 | |
In his comment on that post Dennis Ramondt answered that 'Value' columns as from now will only be exclusive numeric or exclusive character. | |
end UPDATE 30SEP2018 | |
There is new implementation of (some of) the CBS (Statistics Netherlands) tables. | |
This implementation uses Odata v4. | |
The actual data is in a subtable 'Observations' that contains a column (field) 'Value'. | |
I would appreciate some help in building a filter on this column. | |
Up till now I get 'Server error' messages. | |
Using table "83765NED" the following filtering works: | |
$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) &$top=5 | |
The url https://acc-ccb.cbs.nl/83765NED/Observations?$format=json&$filter=(Measure%20eq%20'Gemeentenaam_1')%20and%20 | |
(IsNumeric%20ne%20true)%20&$top=5%20 | |
returns 5 observations with 6 fields | |
Expanding the filter to select only records from 'Amstelveen' I tried | |
$filter=(Measure eq 'Gemeentenaam_1') and (IsNumeric ne true) and contains(Value,'Amstel') | |
The url https://acc-ccb.cbs.nl/83765NED/Observations?$format=json&$filter=(Measure%20eq%20'Gemeentenaam_1')%20and%20 | |
(IsNumeric%20ne%20true)%20and%20contains(Value,'Amstel')%20 | |
returns "Server error: (500) Internal Server Error" | |
The R code used can be found in 'code.R' and the version of the packages used in 'sessionInfo.txt'. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sessionInfo() | |
R version 3.5.0 (2018-04-23) | |
Platform: x86_64-w64-mingw32/x64 (64-bit) | |
Running under: Windows >= 8 x64 (build 9200) | |
Matrix products: default | |
locale: | |
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252 | |
[4] LC_NUMERIC=C LC_TIME=English_United States.1252 | |
attached base packages: | |
[1] stats graphics grDevices utils datasets methods base | |
other attached packages: | |
[1] glue_1.3.0 | |
loaded via a namespace (and not attached): | |
[1] httr_1.3.1 compiler_3.5.0 R6_2.2.2 tools_3.5.0 curl_3.2 yaml_2.2.0 jsonlite_1.5 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment