Skip to content

Instantly share code, notes, and snippets.

@blohr-hs
Created November 2, 2017 22:19
Show Gist options
  • Save blohr-hs/b980b39ec4f13d61bdd703263388b6d7 to your computer and use it in GitHub Desktop.
Save blohr-hs/b980b39ec4f13d61bdd703263388b6d7 to your computer and use it in GitHub Desktop.
Get actual Elasticseach input when Firehose fails to insert it
#!/bin/bash
set -u
set -e
# Download a failed payload file from the "elasticsearch-failed/" folder in the S3 bucket, and pass its path to this script.
error_batch=$1
for raw in $(jq --raw-output '.rawData' $error_batch); do
echo $raw | base64 -D
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment