Convert large json files into flat, line-by-line format. With and without surrounding brackets & commas

head -n 10 data.json

[
  {
    "name": "Angela Merkel",
    "phone": "+49123456789",
    "dob": {
        "$date": "1954-07-17T00:00:00.000+00:00"
    },
    "address": {
      "street": "Reichstagstraße 1",
      "city": "Berlin",
...

jq -c '.[]' data.json > data-flat.json

cat data-flat.json

{"name":"Angela Merkel","phone":"+49123456789","dob":{"$date":"1954-07-17T00:00:00.000+00:00"},"address":{"street":"Reichstagstraße 1","city":"Berlin","state":"","postal_code":"11011","country":"Germany"},"email":"[email protected]","ssn":"123-45-6789","credit_card":{"number":"5555-5555-5555-4444","type":"Visa","expiration":"06/2022"},"avatar":"🇩🇪"}
{"name":"Narendra Modi","phone":"+91234-567-890","dob":{"$date":"1950-09-17T00:00:00.000+00:00"},"address":{"street":"Race Course Road","city":"New Delhi","state":"Delhi","postal_code":"110001","country":"India"},"email":"[email protected]","ssn":"123-45-6790","credit_card":{"number":"5555-5555-5555-5555","type":"JCB","expiration":"09/2021"},"avatar":"🇮🇳"}
{"name":"Cristiano Ronaldo","phone":"+351912345678","dob":{"$date":"1985-05-05T00:00:00.000+00:00"},"address":{"street":"Rua Dr. José Vieira de Carvalho","city":"Funchal","state":"Madeira","postal_code":"9000-310","country":"Portugal"},"email":"[email protected]","ssn":"123-45-6791","credit_card":{"number":"5555-5555-5555-6666","type":"American Express","expiration":"05/2022"},"avatar":"🇵🇹"}
{"name":"Taylor Swift","avatar":"💙","ssn":"901-10-4516","dob":{"$date":"1989-12-13T00:00:00.000Z"},"contacts":{"address":"1700 Broadway","city":"New York","state":"NY","postalCode":"10019","phone":"646-454-5900"},"email":"[email protected]","card":{"number":"371449635398431","type":"Amex","expiration":"05/2026"}}

Add commas after all but the last line and surround documents with top-level braces:

sed '1s/^/[/;$!s/$/,/;$s/$/]/' data-flat.json

[{"name":"Angela Merkel","phone":"+49123456789","dob":{"$date":"1954-07-17T00:00:00.000+00:00"},"address":{"street":"Reichstagstraße 1","city":"Berlin","state":"","postal_code":"11011","country":"Germany"},"email":"[email protected]","ssn":"123-45-6789","credit_card":{"number":"5555-5555-5555-4444","type":"Visa","expiration":"06/2022"},"avatar":"🇩🇪"},
{"name":"Narendra Modi","phone":"+91234-567-890","dob":{"$date":"1950-09-17T00:00:00.000+00:00"},"address":{"street":"Race Course Road","city":"New Delhi","state":"Delhi","postal_code":"110001","country":"India"},"email":"[email protected]","ssn":"123-45-6790","credit_card":{"number":"5555-5555-5555-5555","type":"JCB","expiration":"09/2021"},"avatar":"🇮🇳"},
{"name":"Cristiano Ronaldo","phone":"+351912345678","dob":{"$date":"1985-05-05T00:00:00.000+00:00"},"address":{"street":"Rua Dr. José Vieira de Carvalho","city":"Funchal","state":"Madeira","postal_code":"9000-310","country":"Portugal"},"email":"[email protected]","ssn":"123-45-6791","credit_card":{"number":"5555-5555-5555-6666","type":"American Express","expiration":"05/2022"},"avatar":"🇵🇹"},
{"name":"Taylor Swift","avatar":"💙","ssn":"901-10-4516","dob":{"$date":"1989-12-13T00:00:00.000Z"},"contacts":{"address":"1700 Broadway","city":"New York","state":"NY","postalCode":"10019","phone":"646-454-5900"},"email":"[email protected]","card":{"number":"371449635398431","type":"Amex","expiration":"05/2026"}}]

To do the same thing, but put opening & closing braces on their own lines:

sed '1s/^/[\n/;$!s/$/,/;$s/$/\n]/' data-flat.json

[
{"name":"Angela Merkel","phone":"+49123456789","dob":{"$date":"1954-07-17T00:00:00.000+00:00"},"address":{"street":"Reichstagstraße 1","city":"Berlin","state":"","postal_code":"11011","country":"Germany"},"email":"[email protected]","ssn":"123-45-6789","credit_card":{"number":"5555-5555-5555-4444","type":"Visa","expiration":"06/2022"},"avatar":"🇩🇪"},
{"name":"Narendra Modi","phone":"+91234-567-890","dob":{"$date":"1950-09-17T00:00:00.000+00:00"},"address":{"street":"Race Course Road","city":"New Delhi","state":"Delhi","postal_code":"110001","country":"India"},"email":"[email protected]","ssn":"123-45-6790","credit_card":{"number":"5555-5555-5555-5555","type":"JCB","expiration":"09/2021"},"avatar":"🇮🇳"},
{"name":"Cristiano Ronaldo","phone":"+351912345678","dob":{"$date":"1985-05-05T00:00:00.000+00:00"},"address":{"street":"Rua Dr. José Vieira de Carvalho","city":"Funchal","state":"Madeira","postal_code":"9000-310","country":"Portugal"},"email":"[email protected]","ssn":"123-45-6791","credit_card":{"number":"5555-5555-5555-6666","type":"American Express","expiration":"05/2022"},"avatar":"🇵🇹"},
{"name":"Taylor Swift","avatar":"💙","ssn":"901-10-4516","dob":{"$date":"1989-12-13T00:00:00.000Z"},"contacts":{"address":"1700 Broadway","city":"New York","state":"NY","postalCode":"10019","phone":"646-454-5900"},"email":"[email protected]","card":{"number":"371449635398431","type":"Amex","expiration":"05/2026"}}
]

kennwhite/flatten_json_examples.md

Convert large json files into flat, line-by-line format. With and without surrounding brackets & commas