Skip to content

Instantly share code, notes, and snippets.

View thanos's full-sized avatar

thanos vassilakis thanos

View GitHub Profile
@thanos
thanos / batch_process.ex
Created September 4, 2022 14:41
Processing a stream in batches
batch_size = 100
# 8 tasks running at the same time and we don't care about the results order
async_options = [max_concurrency: 8, ordered: false]
csv_rows
|> Stream.chunk(batch_size)
|> Task.async_stream(fn batch ->
batch
|> Enum.map(&CSV.generate_xml/1)
@thanos
thanos / Slug.ex
Last active April 30, 2019 17:26
Using a slug as your primary key with Elixir's Ecto and Phoenix
#
# Define a simple slug type stating the source for the slug and the primary key field name
# see documention https://hexdocs.pm/ecto_autoslug_field/readme.html
#
defmodule CheckpointCharlie.PlayPen.Slug do
use EctoAutoslugField.Slug, from: :name, to: :id
end
import hashlib
from functools import wraps
from django.core.cache import cache
from django.utils.encoding import force_text, force_bytes
def cache_memoize(
timeout,
prefix='',
@thanos
thanos / install-rdkafka.sh
Created March 7, 2018 02:29
A script to install kafka rdkafka drivers for python
sudo apt-get update -y
sudo apt-get upgrade -y
wget -qO - http://packages.confluent.io/deb/3.2/archive.key | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] http://packages.confluent.io/deb/3.2 stable main"
sudo apt-get update && sudo apt-get install confluent-platform-oss-2.11
sudo apt-get install librdkafka-dev
ldconfig -p | grep librdkafka
@thanos
thanos / resize.sh
Created March 5, 2018 14:24
Resize a drive on EC2 Ubuntu 16.04
#resive the volue using EC2 tools.
rm /var/log/somediskhog/*.log.?
growpart /dev/xvda 1
lsblk
resize2fs /dev/xvda1
lsblk
@thanos
thanos / bttc-news.markdown
Created February 9, 2018 15:10
BTTC News
@thanos
thanos / docker_rm_a.sh
Created October 27, 2016 18:52
delete all me docker containers
sudo docker rm `sudo docker ps -a | cut -f 1 -d " "`
@thanos
thanos / couchdb_reduce.js
Created October 17, 2016 01:43
yet Another Coudhdb reduce
function (key, values, rereduce) {
if (rereduce) {
var results = {
'sum': values.reduce(function(a, b) { return a + b.sum }, 0),
'min': values.reduce(function(a, b) { return Math.min(a, b.min) }, Infinity),
'max': values.reduce(function(a, b) { return Math.max(a, b.max) }, -Infinity),
'count': values.reduce(function(a, b) { return a + b.count }, 0),
'sumsqr': values.reduce(function(a, b) { return a + b.sumsqr }, 0)
};
for(var i = 0; i < values.length; i++) {
@thanos
thanos / group.py
Created December 16, 2015 14:47
A custom django has_group conditional tag for testing if a user belongs to a group in a django template.
# A custom django has_group conditional tag
# usage:
#
# {% if request.user|has_group:"Manager" %} {% endif %}
#
# Developer: Thanos Vassilakis, 2002
import re, string
from django import template
@thanos
thanos / ntlog_handler.py
Created September 24, 2015 21:25
I use this when I need logging on a windows 7 or greater machine. Note the line commented out.
# -*- coding: utf-8 -*-
# vim: tabstop=8 expandtab shiftwidth=4 softtabstop=4
import sys, logging, socket, types, os, string, cPickle, struct, time
class NTEventLogHandler(logging.Handler):
"""
A handler class which sends events to the NT Event Log. Adds a
registry entry for the specified application name. If no dllname is