bb01100100

Access multiple private-link networked Confluent Cloud environments from a single VPC

If you have a single VPC and use Private Link networking to access your Confluent Cloud clusters, then you'll quickly realise that you can only access a single Confluent Cloud network (Environment) because the PL attachment has a regional DNS domain.

In organisations that segregate dev/uat/test/prod networks from each other, this is almost never an issue. However, if you have a flat network with a mixture of different clients, then this will be a challenge.

In order to connect to multiple (different) Confluent Cloud environments via Private Link, you'll need to either:

• Use split-horizon DNS (e.g. different clients resolve to different PL endpoints)
• Use a proxy

bb01100100

import sys
import json
import argparse


def find_refs(data, key="$ref"):
    """Recursively search for values of a specified key in a nested dictionary or list.
    """
    if isinstance(data, dict):
        for k, v in data.items():

bb01100100

(INDEX($sa[] ; .id) | map_values(.name)) as $dict
| [(.[] | select(.principal[:7] == "User:sa"))]
| group_by(.principal)  
| map(    
    {     
        (first.principal[5:] as $p     
            | ($dict[$p] // ("ACL-for-non-existant-service-account: " + $p))    
        ): map(    
            ((.resource_type | ascii_downcase) + "/"+ .resource_name)    
        )    

bb01100100

# Confirmed on Centos 8.5 (in Sept 2022)
sudo yum group install -y "Development Tools"
sudo yum install -y git openssl-devel cmake libcurl-devel

git clone https://github.com/edenhill/kcat.git
cd kcat
./bootstrap.sh

bb01100100

#!/usr/bin/env bash

# We set up a file descriptor to redirect input/output from bash's tcp pseudodevice
# This allows us to, for example, call Zookeeper 4lws without needing netcat
# Works everywhere that bash works (proven on RHEL 7.x, AIX, MacOS bash 3.2)

ZKHOST="$1"
ZKPORT=2181
ZKCMD="$2"

bb01100100

#!/usr/bin/env bash

# Author: Kel Graham
# Date:   2021-03-30
# Purpose: 
#   This script wraps curl with caroot, cert and key files obtained from the standard
#   /var/ssl/private folder used by cp-ansible for deployed Confluent Platform components.
#   The idea is to allow you to get on with hitting the Schema Registry REST endpoing
#   without faffing around providing repetitive cert args.
#   Does some super basic checks if we are registering Schemas on SR and provides the

bb01100100

#!/usr/bin/env bash

# Author: Kel Graham
# Date:   2021-03-17
# Purpose: 
#   This script uses the linux traffic control `tc` command to add delay, drop packets, etc
#   which is useful when we want to test software behaviour under stress conditions, e.g.
#   such as spiking latency between Zookeeper or Kafka nodes, simulating a data centre outage
#   or restricting bandwidth between components to simulate inter-DC resource contention.

bb01100100

#!/bin/bash


# Author: Kel Graham
# Date:   2022-05-19
# Purpose: Increase the replication factor of one or more topics, using only
#          a connection to the Kafka broker (no Zookeeper, REST APIs etc)
#          and the standard kafka-topics, kafka-reassign-partitions scripts
#          that come with Kafka.
#       

bb01100100

#!/usr/bin/env bash

# Author: Kel Graham
# Date:   2019-12-04
# Purpose: Let's use Kafka CLI tools to get topic counts.
#   This should be usable on any Kafka install since it uses 
#   only the kafka-consumer-groups utility and some standard
#   unix tools to sum up.