Skip to content

Instantly share code, notes, and snippets.

@ijokarumawak

ijokarumawak/0.README.md

Last active February 15, 2022 11:05
Show Gist options
  • Save ijokarumawak/375915c45071c7cbfddd34d5032c8e90 to your computer and use it in GitHub Desktop.
Save ijokarumawak/375915c45071c7cbfddd34d5032c8e90 to your computer and use it in GitHub Desktop.
Download ZIP
NiFi example template, using Wait and Notify with different counter names.
Raw
0.README.md

NiFi example template, using Wait and Notify with different counter names

Configurations to note:

  • Notify
    • Release Signal Identifier: ${batch.id}
    • Signal Counter Name: ${filetype}
  • Wait
    • Release Signal Identifier: ${batch.id}
    • Signal Counter Name: blank (to count total)
    • Target Signal Count: 3 (to wait a, b and c)
    • Releasable FlowFile Count: 3 (to release a, b and c)
  • RouteOnAttribute:
    • Used to check if all a, b and c are arrived
    • valid: ${wait.counter.a:equals(1):and(${wait.counter.b:equals(1):and(${wait.counter.c:equals(1)})})}
Raw
Wait_Notify_Batch_Example.xml
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description></description>
<groupId>ed7f0fda-015d-1000-8559-1fc6c72cf36e</groupId>
<name>Wait Notify Batch Example</name>
<snippet>
<processGroups>
<id>62865661-b1c9-3465-0000-000000000000</id>
<parentGroupId>35ad49a9-2392-3ad2-0000-000000000000</parentGroupId>
<position>
<x>0.0</x>
<y>0.0</y>
</position>
<comments></comments>
<contents>
<connections>
<id>a4a985db-8b61-3526-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>20c1bcf7-2c89-3398-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>failure</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>61469364-8abf-3e85-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>bfe4654a-55b2-36e7-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>61469364-8abf-3e85-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>79715730-e5b4-3695-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>cdeccbaa-e4bf-3a18-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<bends>
<x>948.5900244140626</x>
<y>721.730028076172</y>
</bends>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>20c1bcf7-2c89-3398-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>expired</selectedRelationships>
<selectedRelationships>failure</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>ce792c54-e108-3c17-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>2fe620b6-3d8f-316f-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>cf1a4794-c08e-3cd5-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>61469364-8abf-3e85-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>c9ec0aa4-cfa5-36b6-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>f52c20e7-8e8c-39b9-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>61469364-8abf-3e85-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>fcb41f94-db0c-3108-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<bends>
<x>1402.6998291015625</x>
<y>977.8198852539062</y>
</bends>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>20c1bcf7-2c89-3398-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>unmatched</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>2fe620b6-3d8f-316f-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>315b38b1-74d5-32da-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<bends>
<x>460.93994140625</x>
<y>817.9798583984375</y>
</bends>
<bends>
<x>478.2199401855469</x>
<y>920.2198486328125</y>
</bends>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>wait</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>6a1d6594-ab42-3cf7-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>20c1bcf7-2c89-3398-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>valid</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>2fe620b6-3d8f-316f-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<connections>
<id>70feef29-2c55-3a1b-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
<backPressureObjectThreshold>10000</backPressureObjectThreshold>
<destination>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>61469364-8abf-3e85-0000-000000000000</id>
<type>PROCESSOR</type>
</destination>
<flowFileExpiration>0 sec</flowFileExpiration>
<labelIndex>1</labelIndex>
<name></name>
<selectedRelationships>success</selectedRelationships>
<source>
<groupId>62865661-b1c9-3465-0000-000000000000</groupId>
<id>26a85cf8-e96d-3d57-0000-000000000000</id>
<type>PROCESSOR</type>
</source>
<zIndex>0</zIndex>
</connections>
<controllerServices>
<id>bb9e9b6f-4c90-3e93-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<bundle>
<artifact>nifi-distributed-cache-services-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<comments></comments>
<descriptors>
<entry>
<key>Server Hostname</key>
<value>
<name>Server Hostname</name>
</value>
</entry>
<entry>
<key>Server Port</key>
<value>
<name>Server Port</name>
</value>
</entry>
<entry>
<key>SSL Context Service</key>
<value>
<identifiesControllerService>org.apache.nifi.ssl.SSLContextService</identifiesControllerService>
<name>SSL Context Service</name>
</value>
</entry>
<entry>
<key>Communications Timeout</key>
<value>
<name>Communications Timeout</name>
</value>
</entry>
</descriptors>
<name>DistributedMapCacheClientService</name>
<persistsState>false</persistsState>
<properties>
<entry>
<key>Server Hostname</key>
<value>localhost</value>
</entry>
<entry>
<key>Server Port</key>
</entry>
<entry>
<key>SSL Context Service</key>
</entry>
<entry>
<key>Communications Timeout</key>
</entry>
</properties>
<state>ENABLED</state>
<type>org.apache.nifi.distributed.cache.client.DistributedMapCacheClientService</type>
</controllerServices>
<processors>
<id>9bf50b86-723c-3d95-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>516.0</x>
<y>729.0</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>release-signal-id</key>
<value>
<name>release-signal-id</name>
</value>
</entry>
<entry>
<key>target-signal-count</key>
<value>
<name>target-signal-count</name>
</value>
</entry>
<entry>
<key>signal-counter-name</key>
<value>
<name>signal-counter-name</name>
</value>
</entry>
<entry>
<key>wait-buffer-count</key>
<value>
<name>wait-buffer-count</name>
</value>
</entry>
<entry>
<key>releasable-flowfile-count</key>
<value>
<name>releasable-flowfile-count</name>
</value>
</entry>
<entry>
<key>expiration-duration</key>
<value>
<name>expiration-duration</name>
</value>
</entry>
<entry>
<key>distributed-cache-service</key>
<value>
<identifiesControllerService>org.apache.nifi.distributed.cache.client.AtomicDistributedMapCacheClient</identifiesControllerService>
<name>distributed-cache-service</name>
</value>
</entry>
<entry>
<key>attribute-copy-mode</key>
<value>
<name>attribute-copy-mode</name>
</value>
</entry>
<entry>
<key>wait-mode</key>
<value>
<name>wait-mode</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>release-signal-id</key>
<value>${batch.id}</value>
</entry>
<entry>
<key>target-signal-count</key>
<value>3</value>
</entry>
<entry>
<key>signal-counter-name</key>
</entry>
<entry>
<key>wait-buffer-count</key>
<value>100</value>
</entry>
<entry>
<key>releasable-flowfile-count</key>
<value>3</value>
</entry>
<entry>
<key>expiration-duration</key>
<value>10 min</value>
</entry>
<entry>
<key>distributed-cache-service</key>
<value>bb9e9b6f-4c90-3e93-0000-000000000000</value>
</entry>
<entry>
<key>attribute-copy-mode</key>
<value>keeporiginal</value>
</entry>
<entry>
<key>wait-mode</key>
<value>wait</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>Wait</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>expired</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>wait</name>
</relationships>
<state>RUNNING</state>
<style></style>
<type>org.apache.nifi.processors.standard.Wait</type>
</processors>
<processors>
<id>c9ec0aa4-cfa5-36b6-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>567.6800537109375</x>
<y>243.93999481201172</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>File Size</key>
<value>
<name>File Size</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>Data Format</key>
<value>
<name>Data Format</name>
</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>
<name>Unique FlowFiles</name>
</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>
<name>generate-ff-custom-text</name>
</value>
</entry>
<entry>
<key>batch.id</key>
<value>
<name>batch.id</name>
</value>
</entry>
<entry>
<key>filename</key>
<value>
<name>filename</name>
</value>
</entry>
<entry>
<key>filetype</key>
<value>
<name>filetype</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>File Size</key>
<value>0B</value>
</entry>
<entry>
<key>Batch Size</key>
<value>1</value>
</entry>
<entry>
<key>Data Format</key>
<value>Text</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>false</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>Content A.</value>
</entry>
<entry>
<key>batch.id</key>
<value>1</value>
</entry>
<entry>
<key>filename</key>
<value>batch-1-b.txt</value>
</entry>
<entry>
<key>filetype</key>
<value>b</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>1d</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>Generate File B</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>RUNNING</state>
<style></style>
<type>org.apache.nifi.processors.standard.GenerateFlowFile</type>
</processors>
<processors>
<id>20c1bcf7-2c89-3398-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>1031.180048828125</x>
<y>734.4600561523438</y>
</position>
<bundle>
<artifact>nifi-update-attribute-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Delete Attributes Expression</key>
<value>
<name>Delete Attributes Expression</name>
</value>
</entry>
<entry>
<key>Store State</key>
<value>
<name>Store State</name>
</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
<value>
<name>Stateful Variables Initial Value</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Delete Attributes Expression</key>
</entry>
<entry>
<key>Store State</key>
<value>Do not store state</value>
</entry>
<entry>
<key>Stateful Variables Initial Value</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>UpdateAttribute</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style></style>
<type>org.apache.nifi.processors.attributes.UpdateAttribute</type>
</processors>
<processors>
<id>26a85cf8-e96d-3d57-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>195.27996826171875</x>
<y>243.93999481201172</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>File Size</key>
<value>
<name>File Size</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>Data Format</key>
<value>
<name>Data Format</name>
</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>
<name>Unique FlowFiles</name>
</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>
<name>generate-ff-custom-text</name>
</value>
</entry>
<entry>
<key>batch.id</key>
<value>
<name>batch.id</name>
</value>
</entry>
<entry>
<key>filename</key>
<value>
<name>filename</name>
</value>
</entry>
<entry>
<key>filetype</key>
<value>
<name>filetype</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>File Size</key>
<value>0B</value>
</entry>
<entry>
<key>Batch Size</key>
<value>1</value>
</entry>
<entry>
<key>Data Format</key>
<value>Text</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>false</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>Content A.</value>
</entry>
<entry>
<key>batch.id</key>
<value>1</value>
</entry>
<entry>
<key>filename</key>
<value>batch-1-a.txt</value>
</entry>
<entry>
<key>filetype</key>
<value>a</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>1d</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>Generate File A</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>RUNNING</state>
<style></style>
<type>org.apache.nifi.processors.standard.GenerateFlowFile</type>
</processors>
<processors>
<id>2fe620b6-3d8f-316f-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>864.1398535156252</x>
<y>959.1000585937498</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>Routing Strategy</key>
<value>
<name>Routing Strategy</name>
</value>
</entry>
<entry>
<key>valid</key>
<value>
<name>valid</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>Routing Strategy</key>
<value>Route to Property name</value>
</entry>
<entry>
<key>valid</key>
<value>${wait.counter.a:equals(1):and(
${wait.counter.b:equals(1):and(
${wait.counter.c:equals(1)})})}</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>RouteOnAttribute</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>unmatched</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>valid</name>
</relationships>
<state>RUNNING</state>
<style></style>
<type>org.apache.nifi.processors.standard.RouteOnAttribute</type>
</processors>
<processors>
<id>61469364-8abf-3e85-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>511.0</x>
<y>487.0</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>release-signal-id</key>
<value>
<name>release-signal-id</name>
</value>
</entry>
<entry>
<key>signal-counter-name</key>
<value>
<name>signal-counter-name</name>
</value>
</entry>
<entry>
<key>signal-counter-delta</key>
<value>
<name>signal-counter-delta</name>
</value>
</entry>
<entry>
<key>signal-buffer-count</key>
<value>
<name>signal-buffer-count</name>
</value>
</entry>
<entry>
<key>distributed-cache-service</key>
<value>
<identifiesControllerService>org.apache.nifi.distributed.cache.client.AtomicDistributedMapCacheClient</identifiesControllerService>
<name>distributed-cache-service</name>
</value>
</entry>
<entry>
<key>attribute-cache-regex</key>
<value>
<name>attribute-cache-regex</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>release-signal-id</key>
<value>${batch.id}</value>
</entry>
<entry>
<key>signal-counter-name</key>
<value>${filetype}</value>
</entry>
<entry>
<key>signal-counter-delta</key>
<value>1</value>
</entry>
<entry>
<key>signal-buffer-count</key>
<value>100</value>
</entry>
<entry>
<key>distributed-cache-service</key>
<value>bb9e9b6f-4c90-3e93-0000-000000000000</value>
</entry>
<entry>
<key>attribute-cache-regex</key>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>0 sec</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>Notify</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>failure</name>
</relationships>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>RUNNING</state>
<style></style>
<type>org.apache.nifi.processors.standard.Notify</type>
</processors>
<processors>
<id>79715730-e5b4-3695-0000-000000000000</id>
<parentGroupId>62865661-b1c9-3465-0000-000000000000</parentGroupId>
<position>
<x>933.7999877929688</x>
<y>243.93999481201172</y>
</position>
<bundle>
<artifact>nifi-standard-nar</artifact>
<group>org.apache.nifi</group>
<version>1.4.0-SNAPSHOT</version>
</bundle>
<config>
<bulletinLevel>WARN</bulletinLevel>
<comments></comments>
<concurrentlySchedulableTaskCount>1</concurrentlySchedulableTaskCount>
<descriptors>
<entry>
<key>File Size</key>
<value>
<name>File Size</name>
</value>
</entry>
<entry>
<key>Batch Size</key>
<value>
<name>Batch Size</name>
</value>
</entry>
<entry>
<key>Data Format</key>
<value>
<name>Data Format</name>
</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>
<name>Unique FlowFiles</name>
</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>
<name>generate-ff-custom-text</name>
</value>
</entry>
<entry>
<key>batch.id</key>
<value>
<name>batch.id</name>
</value>
</entry>
<entry>
<key>filename</key>
<value>
<name>filename</name>
</value>
</entry>
<entry>
<key>filetype</key>
<value>
<name>filetype</name>
</value>
</entry>
</descriptors>
<executionNode>ALL</executionNode>
<lossTolerant>false</lossTolerant>
<penaltyDuration>30 sec</penaltyDuration>
<properties>
<entry>
<key>File Size</key>
<value>0B</value>
</entry>
<entry>
<key>Batch Size</key>
<value>1</value>
</entry>
<entry>
<key>Data Format</key>
<value>Text</value>
</entry>
<entry>
<key>Unique FlowFiles</key>
<value>false</value>
</entry>
<entry>
<key>generate-ff-custom-text</key>
<value>Content C.</value>
</entry>
<entry>
<key>batch.id</key>
<value>1</value>
</entry>
<entry>
<key>filename</key>
<value>batch-1-c.txt</value>
</entry>
<entry>
<key>filetype</key>
<value>c</value>
</entry>
</properties>
<runDurationMillis>0</runDurationMillis>
<schedulingPeriod>1d</schedulingPeriod>
<schedulingStrategy>TIMER_DRIVEN</schedulingStrategy>
<yieldDuration>1 sec</yieldDuration>
</config>
<name>Generate File C</name>
<relationships>
<autoTerminate>false</autoTerminate>
<name>success</name>
</relationships>
<state>STOPPED</state>
<style></style>
<type>org.apache.nifi.processors.standard.GenerateFlowFile</type>
</processors>
</contents>
<name>Wait Notify Batch Example</name>
</processGroups>
</snippet>
<timestamp>08/17/2017 09:47:16 JST</timestamp>
</template>
@dci-aloughran
Copy link

Thanks! That's really helpful; much appreciated!

@kartheekchoudhary
Copy link

Hi ijokarumawak,
I have downloaded the template to my nifi environment and tried running the flow and the I see flow files are waiting in the wait relationship and not moving forward. Do we need to modify any settings ?

@monsif
Copy link

monsif commented May 2, 2018

@kartheekchoudhary have you tried to Setup DistributedMapCacheClientService and DistributedMapCacheServer ? make sure they are running.

@rakeshsilswal
Copy link

above e.g. doest work when you just want wait to release lock when you have atleast 1 count for A, B, C. how to achieve it? I am trying to test above e.g. as start-stop A for 3 times, then start-stop B for 2 times, then start-stop C for 1 time. one 3 files(a,b,c) should get released from wait. start-stop C. one 3 files(a,b,c) should get released from wait. start-stop B, start-stopC.one 3 files(a,b,c) should get released from wait. wait queue now be empty. And all counters showing correct values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment