Skip to content

Instantly share code, notes, and snippets.

@max-mapper
Last active February 28, 2017 21:28
Show Gist options
  • Select an option

  • Save max-mapper/b67d4b1853a056e9f72175570224647b to your computer and use it in GitHub Desktop.

Select an option

Save max-mapper/b67d4b1853a056e9f72175570224647b to your computer and use it in GitHub Desktop.
opendataphilly metadata stats
~/Desktop 🐈  cat results-opendataphilly.json | jsonfilter url | awk -F/ '{print $1}' | sort | uniq -c | sort -rn
1183 "http:
466 "https:
36 "ftp:

~/Desktop 🐈  cat results-opendataphilly.json | jsonfilter status | sort | uniq -c | sort -rn
1575 200
  35 404
  10 500
  10 303
   4 202
   3 502
   3 401
   2 403
   1 410
   1 400
   
~/Desktop 🐈  cat results-opendataphilly.json | jsonfilter headers.content-type | sort | uniq -c | sort -rn
 324 "text/html; charset=utf-8"
 219 "text/csv; charset=utf-8"
 202 "application/octet-stream"
 164 "application/json"
 123 "application/vnd.google-earth.kml+xml"
 111 "text/html"
  91 "text/html; charset=UTF-8"
  52 "text/html;charset=utf-8"
  51 "text/plain; charset=utf-8"
  50 "application/json; charset=utf-8"
  29 "application/zip"
  22 "application/x-zip-compressed"
  11 "application/vnd.geo+json; charset=UTF-8"
  11 "application/atom+xml"
   6 "application/json; charset=UTF-8"
   4 "text/xml;charset=UTF-8"
   4 "text/html;charset=UTF-8"
   3 "text/xml"
   3 "application/rss+xml; charset=utf-8"
   2 "text/html;charset=ISO-8859-1"
   2 "text/html; charset=iso-8859-1"
   2 "application/rss+xml; charset=ISO-8859-1"
   1 "text/xml; charset=utf-8"
   1 "text/xml; charset=UTF-8"
   1 "text/plain; charset=UTF-8"
   1 "text/plain"
   1 "application/xml;charset=utf-8"
   1 "application/xml"
   1 "application/x-gzip"
   1 "application/vnd.google-earth.kml+xml; charset=utf-8"
   1 "application/rss+xml;charset=UTF-8"
   1 "application/rss+xml"
   1 "application/pdf"
   
~/Desktop 🐈  cat results-opendataphilly.json | jsonfilter url | awk -F/ '{print $3}' | sort | uniq -c | sort -rn
530 data.phl.opendata.arcgis.com
190 metadata.phila.gov
165 data.phila.gov
142 services.arcgis.com
109 www.pasda.psu.edu
57 dev.socrata.com
50 gis.phila.gov
45 www.phila.gov
44 raw.githubusercontent.com
43 www.arcgis.com
38 dvrpc.dvrpcgis.opendata.arcgis.com
27 github.com
26 webgui.phila.k12.pa.us
15 www3.septa.org
14 arcgis.dvrpc.org
11 www.phillyspacefinder.com
11 tiles.arcgis.com
10 s3.amazonaws.com
 8 cityofphiladelphia.github.io
 7 www.preservationalliance.com
 5 libwww.freelibrary.org
 4 www.yelp.com
 4 www.phillyhistory.org
 4 www.philadelphiadance.org
 4 www.google.com
 4 opendata.arcgis.com
 3 www.trulia.com
 3 www.septa.org
 3 www.philageohistory.org
 3 www.pde.state.pa.us
 3 www.greatschools.org
 3 phlapi.com
 3 philart.net
 3 philadelphia.craigslist.org
 3 nis.cml.upenn.edu
 3 api.phila.gov
 3 api.everyblock.com
 3 alpha.phila.gov
 2 www.zillow.com
 2 www.walkscore.com
 2 www.philamuseum.org
 2 www.philadelphiavotes.com
 2 www.opendataphilly.org
 2 www.flickr.com
 2 www.everyblock.com
 2 www.dvrpc.org
 2 www.analyzethevote.com
 2 webapps.philasd.org
 2 skookul.com
 2 philly.councilmatic.org
 2 aws.redistricting.state.pa.us
 2 api.flickr.com
 1 www2.septa.org
 1 www.walkshed.org
 1 www.theatrealliance.org
 1 www.seeclickfix.com
 1 www.rideindego.com
 1 www.randalolson.com
 1 www.preservephiladelphia.org
 1 www.phillytreemap.org
 1 www.phillykeyspots.org
 1 www.phillyfunguide.com
 1 www.phillydancespaces.com
 1 www.philly.com
 1 www.philart.net
 1 www.philaplace.org
 1 www.philadems.org
 1 www.philadelphiabuildings.org
 1 www.philadelinquency.com
 1 www.pha.phila.gov
 1 www.nws.noaa.gov
 1 www.drinkphilly.com
 1 www.compass.state.pa.us
 1 www.childcaremap.org
 1 visualization.phillybuildingbenchmarking.com
 1 services.phila.gov
 1 septa.org
 1 plis.cloudapp.net
 1 phor.net
 1 phlprop.us"
 1 phl.maps.arcgis.com
 1 phillyhoods.net
 1 philapark.org
 1 philadox.phila.gov
 1 philadelphiadance.org
 1 philadelphia.maps.arcgis.com
 1 phila.mwdsbe.com
 1 phila-records.com
 1 opendata.pgworks.com
 1 maps.psiee.psu.edu
 1 lti.planphilly.com
 1 jasonsladinski.maps.arcgis.com
 1 help.arcgis.com
 1 haverfordds.cartodb.com
 1 guide.seventy.org
 1 ftp.phila-records.com
 1 flightinfo.phlapi.com
 1 feeds2.feedburner.com
 1 dvrpc.org
 1 developer.trulia.com
 1 dev.seeclickfix.com
 1 connectthecircuit.org
 1 cdb.io
 1 beta.phila.gov
 1 addtransit.com
 1 %20https

downloaded using https://gist.github.com/maxogden/7f09ff2dbb061a2517cd678341b50eaf on feb 1, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment