Skip to content

Instantly share code, notes, and snippets.

View alopresto's full-sized avatar
🔏
Focusing on NiFi security...

Andy LoPresto alopresto

🔏
Focusing on NiFi security...
View GitHub Profile
@alopresto
alopresto / encrypt_openssl.xml
Created December 18, 2018 19:50
This flow demonstrates generating content, encrypting it with OpenSSL-compatible ciphers, and then writing the encrypted output to a file and decrypting it within NiFi as well.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This flow demonstrates generating content, encrypting it with OpenSSL-compatible ciphers, and then writing the encrypted output to a file and decrypting it within NiFi as well. </description>
<groupId>c2c4192b-0167-1000-0a5f-09f8e34812a7</groupId>
<name>Encrypting and Decrypting with OpenSSL compatible ciphers</name>
<snippet>
<connections>
<id>38949c23-f08c-3ee2-0000-000000000000</id>
<parentGroupId>13ac3bab-d237-331f-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / chc.xml
Created December 18, 2018 17:27
Generates a flowfile containing a static string, calculates the SHA-256 hash value of that content, and logs the result.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>Generates a flowfile containing a static string, calculates the SHA-256 hash value of that content, and logs the result. </description>
<groupId>af5b27c5-0167-1000-2569-5ab3a92d0ccf</groupId>
<name>CryptographicHashContent Example</name>
<snippet>
<processors>
<id>29f3bfab-eb9a-3c48-0000-000000000000</id>
<parentGroupId>03225898-7487-38e6-0000-000000000000</parentGroupId>
<position>
@alopresto
alopresto / invoke_http_authorization.xml
Created October 18, 2018 11:15
InvokeHTTP processor which sends POST request and passes an Authorization header set via a dynamic property on the InvokeHTTP processor. This attribute overrides an Authorization header set on the incoming flowfile via GenerateFlowFile. Also runs a simple HTTP server via ListenHTTP and LogAttribute.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>InvokeHTTP processor which sends POST request and passes an Authorization header set via a dynamic property on the InvokeHTTP processor. This attribute overrides an Authorization header set on the incoming flowfile via GenerateFlowFile.
Also runs a simple HTTP server via ListenHTTP and LogAttribute. </description>
<groupId>47042e50-0166-1000-3961-4d76268fc5f3</groupId>
<name>Demo of InvokeHTTP with Authorization Header</name>
<snippet>
<connections>
<id>3bfce6e7-2e8c-340b-0000-000000000000</id>
@alopresto
alopresto / hash_components_legacy.xml
Created September 4, 2018 01:57
This template was created on a NiFi 1.7.1 instance and will be exported to a 1.8.0-SNAPSHOT instance to see if the legacy component versions are affected.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This template was created on a NiFi 1.7.1 instance and will be exported to a 1.8.0-SNAPSHOT instance to see if the legacy component versions are affected. </description>
<groupId>a1a3dd7b-0165-1000-0fc8-878cfc1226e5</groupId>
<name>Hash Components Legacy</name>
<snippet>
<connections>
<id>0e7ec0a5-dc47-3b80-0000-000000000000</id>
<parentGroupId>78fbab03-6bc3-3ad5-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / hash_components_new.xml
Created September 4, 2018 01:56
Apache NiFi template that exercises the HashContent, CryptographicHashContent, HashAttribute, and CryptographicHashAttribute processors against static and dynamic input.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This flow exercises the HashContent, CryptographicHashContent, HashAttribute, and CryptographicHashAttribute processors against static and dynamic input. </description>
<groupId>a185a5af-0165-1000-95c8-c262426efb7c</groupId>
<name>Hash Component Test Flow</name>
<snippet>
<connections>
<id>0541e6d6-1707-3f80-0000-000000000000</id>
<parentGroupId>d96ffce1-d803-3e10-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / replace_json_key_delimiters
Created September 3, 2018 20:49
Apache NiFi template that replaces JSON keys that contain '.' as a delimiter with '_' while leaving JSON values unmodified.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>Replaces JSON keys that contain '.' as a delimiter with '_' while leaving JSON values unmodified. </description>
<groupId>a0f3dfb7-0165-1000-8310-4fb72628adaa</groupId>
<name>Replace dots in JSON keys with underscores</name>
<snippet>
<connections>
<id>359b6ca9-02ac-37c3-0000-000000000000</id>
<parentGroupId>e223d59f-ac39-3ea4-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / pr2973.xml
Created August 30, 2018 04:19
As described in NIFI-5561, this PR allows a SiteToSiteProvenanceReportingTask to filter the components included by accept list & deny list regex. This flow exercises the new functionality.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>As described in NIFI-5561, this PR allows a SiteToSiteProvenanceReportingTask to filter the components included by accept list &amp; deny list regex. This flow exercises the new functionality. </description>
<groupId>890136a9-0165-1000-7e60-2ed87888f3ab</groupId>
<name>Flow to verify PR 2973</name>
<snippet>
<connections>
<id>2cde3fb8-1773-3ebf-0000-000000000000</id>
<parentGroupId>3bc70cea-1997-36c0-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / PR 2919 groovy-eclipse-compiler issue.sh
Created August 7, 2018 03:07
Bash commands demonstrating the Groovy tests being skipped in Apache NiFi PR 2919.
~W/nifi (master) 😉
🔒 0s @ 19:50:01 $ git --no-pager log -n 1 --show-signature
commit a19134f32560ae044cb898a7654ada31338e20c2 (HEAD -> master, upstream/master, origin/master, origin/HEAD, apache/master, 2919)
Author: Lars Francke <[email protected]>
Date: Thu Jun 28 23:22:49 2018 +0200
NIFI-5350 Add a way to provide arbitrary Java options in shell scripts
Signed-off-by: Mike Moser <[email protected]>
@alopresto
alopresto / hash_attribute.xml
Created August 1, 2018 18:44
This template demonstrates an ExecuteScript processor containing a Groovy script to calculate the SHA-256 hash of various attributes and add that value as a new attribute on the flowfile. It also compares the ssdeep locality-sensitive hash (LSH) for reference. To deploy locally, replace the *Module Lib* property of *ExecuteScript* with the path …
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This template demonstrates an ExecuteScript processor containing a Groovy script to calculate the SHA-256 hash of various attributes and add that value as a new attribute on the flowfile. It also compares the ssdeep locality-sensitive hash (LSH) for reference. </description>
<groupId>f6aec6a2-0164-1000-e01d-dcaf5d2d2f5d</groupId>
<name>HashAttribute Template</name>
<snippet>
<connections>
<id>13e1392c-72b1-38ed-0000-000000000000</id>
<parentGroupId>b01ab23b-41d5-389b-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@alopresto
alopresto / bc_upgrade.xml
Created March 8, 2018 00:40
This flow generates a plaintext flowfile, encrypts it using a PGP public key (key not provided), logs the cipher text, decrypts it using the corresponding PGP private key, and logs the recovered plaintext.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description>This flow generates a plaintext flowfile, encrypts it using a PGP public key (key not provided), logs the cipher text, decrypts it using the corresponding PGP private key, and logs the recovered plaintext. </description>
<groupId>f8e24946-0161-1000-38fc-63a7eb3ca9d1</groupId>
<name>BC Upgrade Test</name>
<snippet>
<processors>
<id>a7e90833-9bc7-3dc0-0000-000000000000</id>
<parentGroupId>479f822d-6b06-3370-0000-000000000000</parentGroupId>
<position>