Skip to content

Instantly share code, notes, and snippets.

@aeppert
Last active April 12, 2018 16:49
Show Gist options
  • Select an option

  • Save aeppert/9a623fb6973696e45f8074353796370a to your computer and use it in GitHub Desktop.

Select an option

Save aeppert/9a623fb6973696e45f8074353796370a to your computer and use it in GitHub Desktop.
librdkafka round-robin partitioner
class RRPartitionerCb : public RdKafka::PartitionerCb {
public:
RRPartitionerCb() { partition = 0; }
int32_t partitioner_cb(const RdKafka::Topic *topic,
const std::string *key,
int32_t partition_cnt,
void *msg_opaque) {
if((partition+1) > (partition_cnt-1)) {
partition = 0;
return partition;
}
partition++;
return partition;
}
private:
int32_t partition;
};
@ericorange

Copy link
Copy Markdown

Do you need to do both checks here? Isn't it always true that partition > (partition_cnt-1) when (partition + 1) > (partition_cnt-1)? Wouldn't this mean that partition > (partition_cnt-1) is never true, since the partition will always be set to zero before?

@aeppert

aeppert commented Apr 12, 2018

Copy link
Copy Markdown
Author

Actually, you would with the following scenario:

let partition_cnt = 10
let partition = 9

if(9 > 10) -> fallthrough
partition++ -> 10 ... which is an invalid partition and Kafka will end up placing it in the '-1' partition.

Thus the check for (partition + 1) > (partition_cnt -1)

@ericorange

ericorange commented Apr 12, 2018

Copy link
Copy Markdown

Maybe I wasn't clear before, but I agree that the expression (partition + 1) > (partition_cnt -1) is needed for exactly the case you mentioned. What I was suggesting is that with that expression, you don't need partition > (partition_cnt-1). I would expect that anytime
x > y was true, x + 1 > y would also be true. At least I can't think of a case where you would not have the same result with

if((partition + 1) > (partition_cnt-1)) {
    partition = 0;
    return partition;
} 
        
partition++;
return partition;

@aeppert

aeppert commented Apr 12, 2018

Copy link
Copy Markdown
Author

Ah, indeed you would be correct. That is the simplification. This is what happens when one writes code at 1AM takes a nap and still has the over-complicated logic in one's head... Thank you for that!.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment