Skip to content

Instantly share code, notes, and snippets.

@NickCrews
NickCrews / coalesce_parquet.py
Last active January 10, 2024 03:48
Coalesce parquet files
"""coalesce_parquets.py
gist of how to coalesce small row groups into larger row groups.
Solves the problem described in https://issues.apache.org/jira/browse/PARQUET-1115
"""
from __future__ import annotations
from pathlib import Path
from typing import Callable, Iterable, TypeVar
@jibing57
jibing57 / aws_elb_access_log_parser.rb
Last active May 26, 2020 18:27
a simple ruby script to parse aws elb access log to csv format file
require 'csv'
require 'optparse'
class AWSELBAccessLogParser
def initialize()
@@elb_access_log_format=%Q(timestamp elb client:port backend:port request_processing_time backend_processing_time response_processing_time elb_status_code backend_status_code received_bytes sent_bytes "request" "user_agent" ssl_cipher ssl_protocol)
# puts "elb_access_log_format is #{elb_access_log_format.split(" ")}"
@@line_regex = /
@peterdm
peterdm / gist:9e31be343a733c44ff0e
Last active February 5, 2023 17:05
Elasticsearch semi-equivalent of Solr "bf=product(query('cat'),query('dog'))"
GET /my_index/my_type/_search
{
"query": {
"function_score": {
"query": { <ORIGINAL QUERY> },
"functions": [
{
"filter" : {
"term" : { "_all" : "cat"}
},
# Multiple inheritance with Modules as an alternative to injected composition
# from Sandi Metz's talk [Nothing is Something](http://confreaks.tv/videos/bathruby2015-nothing-is-something)
# Like Sandi's 'direct' DI method this has behavior outside of the base class
# that gets composed together. However in this gist I compose modules in class
# definitions instead of injecting collaborators.
# Tradeoffs between this and Sandi's version are that in this case the API consumer doesn't
# have to know how to make a RandomEchoHouse (no `house = House.new(formatter: Whatever.new)`),
# but also the API consumer can't make anything not already accounted for either.
@tuxfight3r
tuxfight3r / tcp_flags.txt
Last active April 8, 2025 12:59
tcpdump - reading tcp flags
##TCP FLAGS##
Unskilled Attackers Pester Real Security Folks
==============================================
TCPDUMP FLAGS
Unskilled = URG = (Not Displayed in Flag Field, Displayed elsewhere)
Attackers = ACK = (Not Displayed in Flag Field, Displayed elsewhere)
Pester = PSH = [P] (Push Data)
Real = RST = [R] (Reset Connection)
Security = SYN = [S] (Start Connection)
@takeshixx
takeshixx / hb-test.py
Last active April 14, 2025 17:04
OpenSSL heartbeat PoC with STARTTLS support.
#!/usr/bin/env python2
"""
Author: takeshix <[email protected]>
PoC code for CVE-2014-0160. Original PoC by Jared Stafford ([email protected]).
Supportes all versions of TLS and has STARTTLS support for SMTP,POP3,IMAP,FTP and XMPP.
"""
import sys,struct,socket
from argparse import ArgumentParser
@clintongormley
clintongormley / load_test_data.sh
Last active January 5, 2024 07:32
Run these commands in your shell to setup the test data for Chapter 5
curl -XPUT 'http://localhost:9200/us/user/1?pretty=1' -d '
{
"email" : "[email protected]",
"name" : "John Smith",
"username" : "@john"
}
'
curl -XPUT 'http://localhost:9200/gb/user/2?pretty=1' -d '
{
@thiagooak
thiagooak / kill-rmq-connections.sh
Created December 13, 2013 12:11
kill ALL rabbitmq connections
rabbitmqctl list_connections pid port state user vhost recv_cnt send_cnt send_pend name | awk '{print "rabbitmqctl close_connection \"" $1 "\" \"manually closing idle connection\"" | "/bin/bash" }'
@sloria
sloria / bobp-python.md
Last active April 17, 2025 08:09
A "Best of the Best Practices" (BOBP) guide to developing in Python.

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

  • "Build tools for others that you want to be built for you." - Kenneth Reitz
  • "Simplicity is alway better than functionality." - Pieter Hintjens