Created
April 26, 2017 18:47
-
-
Save favila/bd1a2996dd3daf109f7a11a1ece11fa6 to your computer and use it in GitHub Desktop.
How to use DRPCClient in Java in storm 1.0.3 (at least)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
;; Documentation on the storm site http://storm.apache.org/releases/1.0.3/Distributed-RPC.html | |
;; has an example that constructs DRPCClient in a way that is out-of-date. | |
;; And all the example code uses the LocalDRPCClient, so there's no docs on how to construct this object. | |
;; The problem is the new first parameter to the DRPCClient constructors. This should be the storm config map, | |
;; the same one topologies get and the one configured via storm.yaml. Ideally it is the identical map, but | |
;; if you are running the DRPCClient on a different process you may not have access to it. | |
;; Below is the minimum you need to get started on a not-very-customized storm cluster | |
(import '(org.apache.storm.utils DRPCClient) | |
'(org.apache.storm Config)) | |
(DRPCClient. | |
;; NOTE: The transport plugin must match the one configured in cluster's storm.yaml, | |
;; and must be on the client's classpath. | |
;; NOTE: SimpleTransportPlugin is deprecated but still seems to be the default in storm v1.0.3 | |
;; If this is changed in storm.yaml to org.apache.storm.security.auth.plain.PlainSaslTransportPlugin | |
;; (the most likely new candidate), you must change the setting here | |
;; and you can also remove the *_MAX_BUFFER_SIZE config settings | |
{Config/STORM_THRIFT_TRANSPORT_PLUGIN "org.apache.storm.security.auth.SimpleTransportPlugin" | |
Config/NIMBUS_THRIFT_MAX_BUFFER_SIZE 1048576 | |
Config/DRPC_MAX_BUFFER_SIZE 1048576 | |
;; These config names are deceiving. These are actually used to configure | |
;; the drpc exponential backoff (as well as nimbus retries). | |
;; You can potentially use different settings here. | |
Config/STORM_NIMBUS_RETRY_TIMES 5 | |
Config/STORM_NIMBUS_RETRY_INTERVAL 2000 | |
Config/STORM_NIMBUS_RETRY_INTERVAL_CEILING 60000} | |
;; Remaining args (including optional timeout arg) are as in older DRPCClient classes. | |
"drpc-server-hostname" 3772) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment