onefoursix

#!/bin/bash
#===================================================================================
#
# FILE: get-streamsets-metrics.sh
#
# DESCRIPTION: A bash script that captures Java heap and CPU usage for a StreamSets
#              Data Collector and writes the output in CSV format
#
# CONFIGURATION: Set the following variables at the head of the script:
# 

onefoursix

#!/usr/bin/python3

"""
FILE: add_streamsets_user.py

DESCRIPTION: Adds a user to a StreamSets Platform Organization.
             If the user already exists in the Organization, no action is taken.
             (Users are identified by their email addresses).

ARGS:

onefoursix

In the Groovy Stage's init script, look for this line of code (around line 138):

            // Execute the request      
            HttpResponse response = httpClient.execute(request)

Comment out that line and replace that line with this section of code:

            // mbrooks 06/01/2024 -- comment out this line
            // HttpResponse response = httpClient.execute(request)

onefoursix

import os
from datetime import datetime
from time import time
import sys
from streamsets.sdk import ControlHub
import json

# Set to true to echo the metrics to stdout
print_metrics = True

onefoursix

 #!/usr/bin/env python

'''
This script writes a continuous stream of CPU and Memory metrics for a given SDC
as well as counts of all running pipelines' input records, output records, and error records.

The script uses StreamSets Platform's REST API to pull metrics directly from SDCs; it does not connect to Control Hub

On each refresh interval, the script will record CPU and memory metrics and then the record counts for each running pipeline.

onefoursix

export CRED_ID="..."

export CRED_TOKEN="..."

curl -X GET  "https://rancher.onefoursix.com:31910/rest/v1/system/jmx" \
 -H "X-SS-App-Component-Id: $CRED_ID" \
 -H "X-SS-App-Auth-Token: $CRED_TOKEN"

onefoursix

#!/usr/bin/env python

'''
This script writes a rolling log file that contains running Pipeline names and record counts,
along with SDC CPU usage and JVM heap memory metrics for all Data Collectors registered 
with StreamSets Platform that match the specified set of Labels.

The script writes a sdc-resource-metrics.log as a rolling log file with the pipeline and SDC metrics
as well as a sdc-resource-metrics-messages-and-errors.log file that shows the SDCs that are discovered
and whether connections to them are successful.

onefoursix

#!/usr/bin/env bash

# Script that calls the /provisioning/rest/v1/csp/deployment/${DEP_ID}/unsafeForceStop 
# endpoint to delete StreamSets Kubernetes Deployments stuck in an Activating state.
#
# Use this REST API endpoint only if Kubernetes Deployments are stuck 
# in an Activating state.  
#
# After the script completes, the selected Deployments will be in a 
# "Deactivation Error" state and can be deleted using the Control Hub UI

onefoursix

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: streamsets-deployment-576a6314-076a-4eba-9916-1482c89cae72
  name: streamsets-deployment-576a6314-076a-4eba-9916-1482c89cae72
  namespace: ns100
spec:
  replicas: 1
  selector:

onefoursix

#!/usr/bin/env python

'''
This script writes a rolling log file that contains CPU usage and JVM heap memory metrics
for a given Data Collector registered with Control Hub 3.x, with a user definable refresh
interval, along with the number and names of Jobs running on the Data Collector at the time
of metrics collection.

Prerequisites: