Manuel Garrido manugarri

How to get a stack trace for each thread in a running Python script

Sometimes a Python script will simply hang forever with no indication of what is going wrong. Perhaps it's polling a service that will never return a value that allows the program to move forward.

Here's a way to see where the program is currently stuck, using pyrasite a tool for injecting code into running Python processes.

Install gdb and pyrasite

Install gdb.

Kafka Topics

List existing topics

bin/kafka-topics.sh --zookeeper localhost:2181 --list

Describe a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic mytopic

Purge a topic

bin/kafka-topics.sh --zookeeper localhost:2181 --alter --topic mytopic --config retention.ms=1000

... wait a minute ...

Automation for the People

Long ago, the first time I read "The Pragmatic Programmer", I read some advice that really stuck with me.

"Don't Use Manual Procedures".

This in the chapter on Ubiquitous Automation. To summarize, they want you to automate all the things.

The trouble was that I hadn't much of an idea how to actually go

	from fastai.text import *
	from fastai.structured import proc_df
	import pandas as pd
	import numpy as np

	class MixedInputModelWithText(nn.Module):
	def __init__(self, emb_szs, n_cont, emb_drop, out_sz, szs, drops,
	y_range=None, use_bn=False, is_reg=True, is_multi=False, n_text=0):
	super().__init__()
	for i, (c, s) in enumerate(emb_szs): assert c > 1, f"cardinality must be >=2, got emb_szs[{i}]: ({c},{s})"

	func TestSetCookie(t *testing.T) {
	// Create a new HTTP Recorder (implements http.ResponseWriter)
	recorder := httptest.NewRecorder()

	// Drop a cookie on the recorder.
	http.SetCookie(recorder, &http.Cookie{Name: "test", Value: "expected"})

	// Copy the Cookie over to a new Request
	request := &http.Request{Header: http.Header{"Cookie": recorder.HeaderMap["Set-Cookie"]}}

	Host *.gerardorf.org !bastion.gerardorf.org
	User admin
	IdentityFile ~/.ssh/gerardorf-private-key
	ProxyCommand ssh -W %h:%p [email protected]

	Host bastion.gerardorf.org
	Hostname XXX.XXX.XXX.XXX
	User admin
	IdentityFile ~/.ssh/gerardorf-private-key
	ForwardAgent yes

	{
	"IAB1": "Arts & Entertainment",
	"IAB1-1": "Books & Literature",
	"IAB1-2": "Celebrity Fan/Gossip",
	"IAB1-3": "Fine Art",
	"IAB1-4": "Humor",
	"IAB1-5": "Movies",
	"IAB1-6": "Music",
	"IAB1-7": "Television",
	"IAB2": "Automotive",

	from sklearn.cross_validation import train_test_split
	from keras.preprocessing import sequence, text
	from keras.models import Sequential
	from keras.layers import (Dense, Dropout, Activation, Embedding, LSTM,
	Convolution1D, MaxPooling1D)

	# Embedding
	max_features = 20000
	maxlen = 100
	embedding_size = 32

	#!/usr/bin/python2


	# Copyright (C) 2016 Sixten Bergman
	# License WTFPL
	#
	# This program is free software. It comes without any warranty, to the extent
	# permitted by applicable law.
	# You can redistribute it and/or modify it under the terms of the Do What The
	# Fuck You Want To Public License, Version 2, as published by Sam Hocevar. See