Skip to content

Instantly share code, notes, and snippets.

@ian-whitestone
ian-whitestone / notes.md
Last active March 1, 2023 01:45
Best practices for presto sql

Presto Specific

  • Don’t SELECT *, Specify explicit column names (columnar store)
  • Avoid large JOINs (filter each table first)
    • In PRESTO tables are joined in the order they are listed!!
    • Join small tables earlier in the plan and leave larger fact tables to the end
    • Avoid cross joins or 1 to many joins as these can degrade performance
  • Order by and group by take time
    • only use order by in subqueries if it is really necessary
  • When using GROUP BY, order the columns by the highest cardinality (that is, most number of unique values) to the lowest.
@marwei
marwei / how_to_reset_kafka_consumer_group_offset.md
Created November 9, 2017 23:39
How to Reset Kafka Consumer Group Offset

Kafka 0.11.0.0 (Confluent 3.3.0) added support to manipulate offsets for a consumer group via cli kafka-consumer-groups command.

  1. List the topics to which the group is subscribed
kafka-consumer-groups --bootstrap-server <kafkahost:port> --group <group_id> --describe

Note the values under "CURRENT-OFFSET" and "LOG-END-OFFSET". "CURRENT-OFFSET" is the offset where this consumer group is currently at in each of the partitions.

  1. Reset the consumer offset for a topic (preview)
@adamwathan
adamwathan / promise-take-at-least.js
Last active February 26, 2023 14:25
Promise.takeAtLeast
// Creates a new promise that automatically resolves after some timeout:
Promise.delay = function (time) {
return new Promise((resolve, reject) => {
setTimeout(resolve, time)
})
}
// Throttle this promise to resolve no faster than the specified time:
Promise.prototype.takeAtLeast = function (time) {
return new Promise((resolve, reject) => {
@antirez
antirez / lmdb.tcl
Created April 28, 2017 15:40
LMDB -- First version of Redis written in Tcl
# LVDB - LLOOGG Memory DB
# Copyriht (C) 2009 Salvatore Sanfilippo <[email protected]>
# All Rights Reserved
# TODO
# - cron with cleanup of timedout clients, automatic dump
# - the dump should use array startsearch to write it line by line
# and may just use gets to read element by element and load the whole state.
# - 'help','stopserver','saveandstopserver','save','load','reset','keys' commands.
# - ttl with milliseconds resolution 'ttl a 1000'. Check ttl in dump!
@gilyes
gilyes / Backup, restore postgres in docker container
Last active January 18, 2025 00:57
Backup/restore postgres in docker container
Backup:
docker exec -t -u postgres your-db-container pg_dumpall -c > dump_`date +%d-%m-%Y"_"%H_%M_%S`.sql
Restore:
cat your_dump.sql | docker exec -i your-db-container psql -Upostgres
@52cik
52cik / npm.taobao.sh
Last active November 23, 2024 11:42
npm 淘宝镜像配置
npm set registry https://r.npm.taobao.org # 注册模块镜像
npm set disturl https://npm.taobao.org/dist # node-gyp 编译依赖的 node 源码镜像
## 以下选择添加
npm set sass_binary_site https://npm.taobao.org/mirrors/node-sass # node-sass 二进制包镜像
npm set electron_mirror https://npm.taobao.org/mirrors/electron/ # electron 二进制包镜像
npm set ELECTRON_MIRROR https://cdn.npm.taobao.org/dist/electron/ # electron 二进制包镜像
npm set puppeteer_download_host https://npm.taobao.org/mirrors # puppeteer 二进制包镜像
npm set chromedriver_cdnurl https://npm.taobao.org/mirrors/chromedriver # chromedriver 二进制包镜像
npm set operadriver_cdnurl https://npm.taobao.org/mirrors/operadriver # operadriver 二进制包镜像
@rtconner
rtconner / SftpServiceProvider.php
Last active June 27, 2024 09:07
Provider so you can add a 'sftp' connection in Laravel 5 filesystems.php - "Call to undefined method League\Flysystem\Filesystem::createSftpDriver"
<?php
namespace App\Providers;
use League\Flysystem\Sftp\SftpAdapter;
use Storage;
use League\Flysystem\Filesystem;
use Illuminate\Support\ServiceProvider;
/**
@carcinocron
carcinocron / debugger pause beforeunload
Last active March 19, 2025 22:52
Chrome: pause before redirect
// Run this in the F12 javascript console in chrome
// if a redirect happens, the page will pause
// this helps because chrome's network tab's
// "preserve log" seems to technically preserve the log
// but you can't actually LOOK at it...
// also the "replay xhr" feature does not work after reload
// even if you "preserve log".
window.addEventListener("beforeunload", function() { debugger; }, false)
@hgomez
hgomez / download-java8.sh
Created March 19, 2014 20:39
wget to download Java 8
wget --continue --no-check-certificate -O jdk-8-linux-x64.tar.gz --header Cookie: oraclelicense=a http://download.oracle.com/otn-pub/java/jdk/8-b132/jdk-8-linux-x64.tar.gz
@plentz
plentz / nginx.conf
Last active March 28, 2025 17:48
Best nginx configuration for improved security(and performance)
# to generate your dhparam.pem file, run in the terminal
openssl dhparam -out /etc/nginx/ssl/dhparam.pem 2048