Skip to content

Instantly share code, notes, and snippets.

@HouzuoGuo
Last active March 22, 2020 11:43
Show Gist options
  • Save HouzuoGuo/83eaab9a46a8ab3c7473323b34f9ea62 to your computer and use it in GitHub Desktop.
Save HouzuoGuo/83eaab9a46a8ab3c7473323b34f9ea62 to your computer and use it in GitHub Desktop.
Use PostgreSQL to convert CSV into JSON

Start the server

docker run -it --rm -e POSTGRES_PASSWORD=demo -e POSTGRES_HOST_AUTH_METHOD=trust postgres:10

Prepare DB client

cat <<EOF >test.csv
title,url
"NHS facing Italian-style coronavirus crisis if we don't stay at home, says Boris Johnson",https://www.telegraph.co.uk/politics/2020/03/21/nhs-facing-italian-style-crisis-dont-stay-home-says-boris-johnson/
Italy: PM warns of worst crisis since WW2 as coronavirus deaths leap by almost 800,https://www.theguardian.com/world/2020/mar/22/italian-pm-warns-of-worst-crisis-since-ww2-as-coronavirus-deaths-leap-by-almost-800
Scottish government 'furious' at travellers to Highlands and Islands,https://www.theguardian.com/world/2020/mar/21/scottish-government-furious-at-travellers-to-highlands-and-islands
EOF

apt install postgresql-client

psql -h 172.17.0.2 -U postgres

SQL client magic

create database news;
\c news
create table article (title varchar(1000), url varchar(1000));
\copy article from 'test.csv' with (format csv)
\copy (select to_json(article) from article) to 'articles.json'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment