Skip to content

Instantly share code, notes, and snippets.

@devlatte
Last active December 29, 2020 14:10
Show Gist options
  • Save devlatte/755690c89cb4f1467c8885ed3c76f84b to your computer and use it in GitHub Desktop.
Save devlatte/755690c89cb4f1467c8885ed3c76f84b to your computer and use it in GitHub Desktop.
kinesis firehose transformation, origin: json log, add kinesis approximateArrivalTimestamp to log, append newline string for s3 json line
exports.handler = async (event, context) => {
const output = event.records.map((record) => {
const arrived_ts = record.approximateArrivalTimestamp;
const buff = Buffer.from(record.data, 'base64');
const obj = JSON.parse(buff.toString());
obj.arrived_ts = arrived_ts;
record.data = Buffer.from(JSON.stringify(obj) + "\n").toString('base64');
record.result = 'Ok';
return record;
});
return { records: output };
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment