Jia Zhai jiazhai

Is this a question, feature request, or bug report?

QUESTION

Have you checked our documentation at http://bookkeeper.apache.org/ , If you could not find an answer there, please consider asking your question in our community forum at [email protected], as it would benefit other members of our community.

FEATURE REQUEST

Please describe the feature you are requesting.

Descriptions of the changes in this PR:

xxxxxx

Be sure to do all of the following to help us incorporate your contribution quickly and easily:

Make sure the PR title is formatted like: : Description of pull request

目标：

在现有的Share的订阅模式之上，添加一种新的订阅模式Key_Failover。这种订阅模式，可以保证同一个订阅中的多个consumer可以共享订阅同一个partiton；同时又可以保证partiton中的每一个Key对应的消息只被发送到一个consumer。

现状和要解决的问题

现有的订阅模式中，share模式可以对同一个partition中的消息进行共享的消费，可以高效的清除partition的backlog。但是在很多场景中，顺序性也是很常见的需求。在share模式中，我们不能保证同一个Key对应的消息会被发送到固定的某一个consumer中。每个consumer对消息的顺序性得不到保证。

我们想要在新的订阅模式Key_Failover中，保证同一个订阅中的多个consumer可以共享订阅同一个partiton；同时又可以保证partiton中的每一个Key对应的消息只被发送到一个consumer。

解决方法

This is based on https://gist.github.com/jiazhai/1cd48ab6c8f6c3012c895df9746799b5

0. prepare a docker image

1， build it from source code:

mvn install -DskipTests -Pdocker

2， or use docker pull

This doc demonstrates how to do geo-replication across multiple pulsar clusters without a global configuration store (zookeeper).

This demo is using docker-compose to start 3 pulsar clusters. Each pulsar cluster has 1 zk, 1 bk, and 1 broker. The docker compose file can be found at https://gist.github.com/jiazhai/72aef9bfbfdd3fe421084c2973247a64

The information of all the three clusters is listed in the following table:

	zk	configuration store	broker


	org.apache.pulsar.tests.integration.offload.TestS3Offload.tiered-storage-test-suite (from TestSuite)

	Failing for the past 1 build (Since 失败#5056 )
	运行时间：1 分 16 秒
	添加说明
	Error Message
	arrays differ firstly at element [15]; expected value is <48> but was <49>.
	Stacktrace
	java.lang.AssertionError: arrays differ firstly at element [15]; expected value is <48> but was <49>.


	tenant: "public"
	namespace: "default"
	name: "debezium-mysql-source"
	topicName: "debezium-mysql-topic"
	archive: "connectors/pulsar-io-debezium-mysql-2.4.0-SNAPSHOT.nar"

	##autoAck: true
	parallelism: 1

	tenant: "public"
	namespace: "default"
	name: "debezium-postgres-source"
	topicName: "debezium-postgres-topic"
	archive: "connectors/pulsar-io-debezium-postgres-2.4.0-SNAPSHOT.nar"

	parallelism: 1

	configs:
	## config for pg, docker image: debezium/example-postgress:0.8

	#
	# Licensed to the Apache Software Foundation (ASF) under one
	# or more contributor license agreements. See the NOTICE file
	# distributed with this work for additional information
	# regarding copyright ownership. The ASF licenses this file
	# to you under the Apache License, Version 2.0 (the
	# "License"); you may not use this file except in compliance
	# with the License. You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0