Skip to content

Instantly share code, notes, and snippets.

@favila
Created April 26, 2017 18:47
Show Gist options
  • Save favila/bd1a2996dd3daf109f7a11a1ece11fa6 to your computer and use it in GitHub Desktop.
Save favila/bd1a2996dd3daf109f7a11a1ece11fa6 to your computer and use it in GitHub Desktop.
How to use DRPCClient in Java in storm 1.0.3 (at least)
;; Documentation on the storm site http://storm.apache.org/releases/1.0.3/Distributed-RPC.html
;; has an example that constructs DRPCClient in a way that is out-of-date.
;; And all the example code uses the LocalDRPCClient, so there's no docs on how to construct this object.
;; The problem is the new first parameter to the DRPCClient constructors. This should be the storm config map,
;; the same one topologies get and the one configured via storm.yaml. Ideally it is the identical map, but
;; if you are running the DRPCClient on a different process you may not have access to it.
;; Below is the minimum you need to get started on a not-very-customized storm cluster
(import '(org.apache.storm.utils DRPCClient)
'(org.apache.storm Config))
(DRPCClient.
;; NOTE: The transport plugin must match the one configured in cluster's storm.yaml,
;; and must be on the client's classpath.
;; NOTE: SimpleTransportPlugin is deprecated but still seems to be the default in storm v1.0.3
;; If this is changed in storm.yaml to org.apache.storm.security.auth.plain.PlainSaslTransportPlugin
;; (the most likely new candidate), you must change the setting here
;; and you can also remove the *_MAX_BUFFER_SIZE config settings
{Config/STORM_THRIFT_TRANSPORT_PLUGIN "org.apache.storm.security.auth.SimpleTransportPlugin"
Config/NIMBUS_THRIFT_MAX_BUFFER_SIZE 1048576
Config/DRPC_MAX_BUFFER_SIZE 1048576
;; These config names are deceiving. These are actually used to configure
;; the drpc exponential backoff (as well as nimbus retries).
;; You can potentially use different settings here.
Config/STORM_NIMBUS_RETRY_TIMES 5
Config/STORM_NIMBUS_RETRY_INTERVAL 2000
Config/STORM_NIMBUS_RETRY_INTERVAL_CEILING 60000}
;; Remaining args (including optional timeout arg) are as in older DRPCClient classes.
"drpc-server-hostname" 3772)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment