Convert from json to csv datasets from donnees.ville.montreal.qc.ca

README.md

How to convert a json file to csv for a given ckan api call

Requirements

• httpie

Get the json file

This command will retrieve the last 100 datasets created with the metadata_created date from 2014-01-01 from 2014-12-15 from donnees.ville.montreal.qc.ca

http donnees.ville.montreal.qc.ca/api/3/action/package_search q=metadata_created:'[2014-01-01T00:00:00Z TO 2014-12-15T23:59:59Z]' rows=100

Then you want to save to a json the results

http donnees.ville.montreal.qc.ca/api/3/action/package_search q=metadata_created:'[2014-01-01T00:00:00Z TO 2014-12-15T23:59:59Z]' rows=2 > datasets.json

To convert the json file to csv file

Simply run this command:

python json-to-csv-vdmdo.py datasets.json results.csv

json-to-csv-vdmdo.py

# -*- coding: utf8 -*-
import json
import csv
import argparse

# TODO(fprieur): add the http call directly in this script

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('json_filename',
                        help='Path and name to jsonfile to convert')
    parser.add_argument('csv_outputfile',
                        help='Name of the csv output file')
    args = parser.parse_args()

    data = []

    with open(args.json_filename) as json_file:
        json_data = json.load(json_file)
        data.append(json_data)

    with open(args.csv_outputfile, 'wb') as csvfile:
        csvwriter = csv.writer(csvfile)
        csvwriter.writerow(["Nom ensemble de données", 'Nom ressource',
                            'Date de creation de donnees', 'Url resource',
                            'Description ressource'])

        for results in data[0]["result"]["results"]:
            for resources in results["resources"]:
                csvwriter.writerow([unicode(results["title"])
                                    .encode('utf-8'),
                                    unicode(resources["name"])
                                    .encode('utf-8'),
                                    unicode(resources["created"])
                                    .encode('utf-8'),
                                    unicode(resources["url"])
                                    .encode('utf-8'),
                                    unicode(resources["description"])
                                    .encode('utf-8')])