srnagar

EH

employee.avro -> App user will write the json model specification and generate the class Employee class is generated by Avro --> User will do

Send side -- Org Foo (Avro)

Employee bob EHClient -> configured with a serializer and if this serializer

srnagar

IoT Hub Connection String support in Event Hubs

Background:

Azure IotHub instances have built-in Event Hub instances configured for users to publish/consume events from their IotHub devices. The IotHub instance has a connection string that can be used to authenticate the IotHub instance but cannot be used to authenticate Event Hub instance directly as the endpoints are not the same. However, using the IotHub connection string, an Event Hub connection string can be constructed by getting the endpoint from Event Hub service.

History:

In track 1, Event Hub SDK had support for taking an IotHub connection string and internally converting it into Event Hub connection string. So, users had to simply use the IotHub connection string readily available on the portal to access Event Hub.

Current:

For track 2, the feature team decided that Event Hub SDK should not be coupled with IotHub. So, users cannot use IotHub connection string to connect to Event Hub. To fill this void, all languages instead will

srnagar

Smart send API Options

Event Hub producer client today requires users to create an EventBatch, tryAdd() events to the batch and send the batch when the user is ready or when the next event does not fit into a batch.

While this API is very helpful to ensure we build a batch that would meet the size restrictions of the AMQP link, it is a cumbersome for cases where the customer is confident that their set of events do actually fit within the size restrictions. There has also been feedback that the client should be smart enough to do batching on its own without having the customer do it.

Java Sample

.NET Sample

srnagar

Problem Statement

In track 2 SDK for Event Processor client, the load balancing loop runs at a fixed interval of 10 seconds. When the load is not balanced among active processors, partitions are claimed/stolen one at a time per load balancing loop. This can take a long time for the processors to reach a balanced state. This document contains a few alternate options to speed up the partition-claiming to reach balanced state sooner.

You can jump to the proposed API section if you want to skip all the options considered.

Options

I have listed a few options below and we could potentially combine two or more of them and also make these configurable load balancing strategies.

srnagar

Receive batch of events

Initial release version (5.0.1) of Event Hubs client library supports receiving events primarily through 3 consumer clients

• EventHubConsumerClient
• EventHubAsyncConsumerClient
• EventProcessorClient

EventHubConsumerClient

srnagar

Checkpoint Store

• Start from scratch - no blobs for either ownership or checkpoint
• Ownership and checkpoint blobs for some partitions but not all
• Ownership blobs for all partitions but no checkpoint blobs
• Ownership blobs without ownerid in metadata and checkpoint blobs without sequence number and offset in metadata
• Checkpoint blobs with only sequence number (no offset)
• Checkpoint blobs with only offset (no sequence number)

Producer client

srnagar

Event Hub Producer Client

Feature	Track 1	Track 2
Send single event without creating batch	Yes	No
Tracing	No	Supported

Event Hub Consumer Client

Feature	Track 1	Track 2

srnagar

Error Handling

Error	Where?	Action	Notes
Error from listing ownership	Top level
Error from claiming ownership	Top level
Error from listing checkpoints	Top level
Error getting Event Hub properties	Top level
Error receiving events from a partition	Partition level
Error updating checkpoints	Partition level

srnagar

Querying Event Hub properties

public void queryEventHubProperties() {
    EventHubProducerClient client = new EventHubClientBuilder()
        .connectionString("connection-string")
        .buildProducerClient();

    EventHubProperties eventHubProperties = client.getEventHubProperties();
    System.out.println("Name = " + eventHubProperties.getName());

srnagar

    // ---------------------Send Operations----------------------------------
    /* Send events without specifying partition */
    private static void send(List<EventData> eventDataList) {
        EventHubProducerClient producerClient = new EventHubClientBuilder()
            .connectionString("")
            .buildProducerClient();
        producerClient.send(eventDataList);
    }