Skip to content

Instantly share code, notes, and snippets.

View tspannhw's full-sized avatar
💭
Unstructured Data, Vector Database, Cloud, AI, Edge, Streaming, SQL

Timothy Spann tspannhw

💭
Unstructured Data, Vector Database, Cloud, AI, Edge, Streaming, SQL
View GitHub Profile
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@tspannhw
tspannhw / JSON_Array_to_String_List.xml
Created January 6, 2024 02:15 — forked from ijokarumawak/JSON_Array_to_String_List.xml
NiFi template that converts JSON Array into String List.
<?xml version="1.0" ?>
<template encoding-version="1.1">
<description></description>
<groupId>1e8778a8-015c-1000-c747-8273276bab45</groupId>
<name>JSON Array to String List</name>
<snippet>
<processGroups>
<id>51eb9288-0a70-311b-0000-000000000000</id>
<parentGroupId>5454b790-94fe-3e28-0000-000000000000</parentGroupId>
<position>
| # |Where |Type|Date |Event |Talk Title |Link |
|---|--------------------------------------------------------------------------------------------------------------|----|----------|------------------------------|--------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------|
|1 |Virtual |C |1/13/2023 |Data Science Online Camp |Pulsar for ML |
@mneedham
mneedham / 0_install.sh
Created October 28, 2023 08:30
Hugging Face's Text Embeddings Inference Library
git clone [email protected]:huggingface/text-embeddings-inference.git
cd text-embeddings-inference
cargo install --path router -F candle -F accelerate
model=BAAI/bge-large-en-v1.5
revision=refs/pr/5
text-embeddings-router --model-id $model --revision $revision --port 8080
@davidad
davidad / lead.py
Created August 4, 2023 20:14
Lead poisoning data analysis (thanks GPT-4)
import pandas as pd
# Load the data
df = pd.read_excel('pnas.2118631119.sd01.xlsx')
import matplotlib.pyplot as plt
# Filter the data for ages 22-35
df_filtered = df[(df['AGE'] >= 22) & (df['AGE'] <= 35) & (df['YEAR'] >= 1955) & (df['YEAR'] <= 2040)]
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.3">
<description></description>
<groupId>bbe01d7d-0188-1000-547f-9f14b009425b</groupId>
<name>ftp</name>
<snippet>
<connections>
<id>07e20afe-0750-325d-0000-000000000000</id>
<parentGroupId>a568e7b3-e3ef-305c-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@stuartlangridge
stuartlangridge / firehose.py
Created July 6, 2023 19:16
Python file to read the Bluesky ("atproto") firehose and show the json of each message. (Doesn't contain the author, because who knows how the hell to do that.)
from atproto.firehose import FirehoseSubscribeReposClient, parse_subscribe_repos_message
import sys
from atproto import CAR, models
from atproto.cbor import decode_dag, decode_dag_multi
from atproto.xrpc_client.models.utils import get_or_create
import json
class JSONExtra(json.JSONEncoder):
"""raw objects sometimes contain CID() objects, which
seem to be references to something elsewhere in bluesky.
@tspannhw
tspannhw / kafka-python-sasl-gssapi.py
Created March 16, 2023 19:36 — forked from asdaraujo/kafka-python-sasl-gssapi.py
kafka-python example with Kerberos auth
# Requirements: kafka-python gssapi krbticket
import os
import time
from kafka import KafkaConsumer, KafkaProducer
from krbticket import KrbConfig, KrbCommand
try:
os.environ['KRB5CCNAME'] = '/tmp/krb5cc_<myusername>'
kconfig = KrbConfig(principal='araujo', keytab='/path/to/<myusername>.keytab')
KrbCommand.kinit(kconfig)
@ferencerdei
ferencerdei / Github_PR_Verification_Bot.json
Created March 7, 2023 12:57
GitHub PR validation CFM Flow
{
"flowContents": {
"identifier": "42063a31-a98b-319f-8617-2086e4b7f30b",
"instanceIdentifier": "a7be953d-0186-1000-5378-dd26f8d2184c",
"name": "Github_PR_Verification_Bot",
"comments": "",
"position": {
"x": 589.4942055903919,
"y": 191.17769189766898
},
@tspannhw
tspannhw / kafka_python_sasl_scram.py
Created February 8, 2023 21:10 — forked from alexlopes/kafka_python_sasl_scram.py
Kafka Python with SASL/SCRAM Authentication Example
import os
from kafka import KafkaProducer, KafkaConsumer
BOOTSTRAP_SERVERS=os.gentenv("KAFKA_BOOTSTRAP_SERVERS").split(",")
TOPIC_NAME="the-topic"
SASL_USERNAME=os.gentenv("KAFKA_SASL_USERNAME")
SASL_PASSWORD=os.gentenv("KAFKA_SASL_PASSWORD")
def consume():
consumer = KafkaConsumer(TOPIC_NAME, security_protocol="SASL_SSL", sasl_mechanism="SCRAM-SHA-512", sasl_plain_username=SASL_USERNAME, sasl_plain_password=SASL_PASSWORD, bootstrap_servers=BOOTSTRAP_SERVERS)