Last active
May 8, 2017 21:04
-
-
Save jlongman/e1557e478245212694a98bfbbfe2ddcf to your computer and use it in GitHub Desktop.
jq for selecting key and nested value
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# sample data: | |
echo '{"count": {}, "block": | |
{ | |
"67.215.237.26": { | |
"paths": { | |
"/js/compiled.js": 1, | |
"/css/global-compiled.css": 1, | |
"/js/pace.min.js": 1 | |
}, | |
"max_req_per_min": 1, | |
"updated_at": "2017-05-08 13:59:40" | |
}, | |
"70.186.138.162": { | |
"paths": { | |
"/includes/gallery.php": 2 | |
}, | |
"max_req_per_min": 2, | |
"updated_at": "2017-05-08 13:59:40" | |
} | |
} | |
}' | \ | |
jq '.block| to_entries| .[] | select(.value.paths | keys | length ==1 ) | [.]| from_entries' | |
# or jq '[.block| to_entries| .[] | select(.value.paths | keys | length ==1 ) ] | from_entries' | |
# if you need them in an array |
This parses the output of https://aws.amazon.com/blogs/security/how-to-configure-rate-based-blacklisting-with-aws-waf-and-aws-lambda/ when parsed from a local file (my changes), which lists the pages accessed (my changes), then this code is used so that we can determine if an IP is only ever hitting one page (like we saw recently in a DDoS). The later addition searches for the path, "/"
in this case.
Using this answer maybe we can merge the json outputs to get whether different pages were hit, just across log files.
Note these mostly serve as an independent verification from the main parser which is solving a slightly different problem (hitting any page too often rather than only hitting one page, or one specific page, too often)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
You're a crazy man, longman:
jq '[.block| to_entries| .[] | select(.value.paths | keys | length ==1 ) ] | from_entries' /tmp/foo.json | jq '.| to_entries | .[] | select(.value.paths |keys|.[0] == "/")| (.key, .value.max_req_per_min)