handling-json-files

SKILL.md

name	handling-json-files
description	Use when about to read, open, inspect, or query any JSON file, or before writing a script to extract/filter data from JSON — covers when to inspect schema first instead of reading raw, and requires jq instead of ad-hoc scripts for querying.

Handling JSON Files

Overview

JSON files are either small configs (safe to read directly) or larger index/data files (expensive to read raw, and read is often unnecessary — you usually just need the shape). This skill sets the rule for which path to take, plus quick reference for the two tools involved: json-to-schema and jq.

Violating the letter of these rules is violating the spirit — "it's only slightly over 3KB" or "I'll just peek at the top of the file" are not exceptions.

When to Use

• About to Read a .json file
• About to write a Python/JS/shell script to filter, extract, or transform JSON data
• Deciding whether you need the full file contents or just its structure

Decision Rule

digraph json_read_decision {
    "About to read a .json file" [shape=box];
    "Is it index/data (not config)?" [shape=diamond];
    "File > 3KB?" [shape=diamond];
    "Run json-to-schema first" [shape=box];
    "Read file directly" [shape=box];

    "About to read a .json file" -> "Is it index/data (not config)?";
    "Is it index/data (not config)?" -> "Run json-to-schema first" [label="yes"];
    "Is it index/data (not config)?" -> "File > 3KB?" [label="no (it's config)"];
    "File > 3KB?" -> "Run json-to-schema first" [label="yes"];
    "File > 3KB?" -> "Read file directly" [label="no"];
}

Rules, in order:

1. Data/index files (datasets, search indices, generated data, API dumps, caches — anything that isn't hand-edited config) → always run json-to-schema first, regardless of size. Never read the raw file first.
2. Config files (package.json, tsconfig.json, app settings, etc.) → check size. If > 3KB, run json-to-schema first. If ≤ 3KB, reading directly is fine.
3. Whenever you need to query/filter/extract values from JSON data (not just see its shape), use jq at the command line. Do not write a throwaway Python/Node script for this — jq is faster to write and verify, and needs no cleanup.

No exceptions:

• Don't skip json-to-schema because "the schema is probably obvious"
• Don't read "just the first N lines" of a large data file instead of getting the schema
• Don't write a Python script to do what a one-line jq filter does

json-to-schema Reference

Infers a JSON Schema from a file — use this to understand structure without reading raw content.

json-to-schema -i data.json                    # print inferred schema to stdout
json-to-schema -i data.json -o schema.json      # write schema to file
json-to-schema -i data.json --minify            # compact output

Useful flags:

Flag	Purpose
--validate SCHEMA_FILE	Validate -i input (or stdin) against an existing schema
--infer-all-bounds	Infer min/max constraints for all numeric/length fields
--infer-all-enum	Infer enum constraints for fields with a small set of values
--infer-bounds FIELD... / --infer-enum FIELD...	Same, limited to named fields
--additional-properties false|true	Control whether unlisted object properties are allowed (default: false)
--schema-title TITLE / --schema-description TEXT	Set root schema metadata
--field-title FIELD=TITLE / --field-description FIELD=TEXT	Set per-field metadata (dot paths, [] for array items)

Run json-to-schema -h for the full list.

jq Quick Reference

Task	Command
Pretty-print	jq '.' file.json
Get a field	jq '.user.name' file.json
Get array element	jq '.items[0]' file.json
Map over array	jq '.items[].id' file.json
Filter array by condition	jq '.items[] | select(.active == true)' file.json
Count matches	jq '[.items[] | select(.active)] | length' file.json
Extract into array	jq '[.items[].id]' file.json
Multiple fields as row	jq '.items[] | {id, name}' file.json
Raw string output (no quotes)	jq -r '.name' file.json
From stdin	cat file.json | jq '.foo'
Compact/minified	jq -c '.' file.json

Chain filters with |; use select() to filter, map() to transform, [...] to collect into an array. Run jq --help or man jq for the full language.

Common Mistakes

• Reading a 50KB data.json directly "just to check one field" — run json-to-schema first, then use jq to pull the one field.
• Writing a Python script with json.load + a loop to filter records — that's a one-line jq select().
• Treating a large config file the same as a data file — configs ≤3KB are fine to read directly even though they're config, not data.