Skip to content

Instantly share code, notes, and snippets.

View sivabudh's full-sized avatar

Sivabudh Umpudh sivabudh

29 followers · 3 following
View GitHub Profile
@sivabudh
sivabudh / the-log-readable-jay-kreps.md
Created August 23, 2026 07:09
[Readable Format] The Log: What every software engineer should know about real-time data's unifying abstraction

The Log: What every software engineer should know about real-time data's unifying abstraction

Author: Jay Kreps
Role: Co-founder and former CEO at Confluent
Published: December 16, 2013 Original post: https://www.linkedin.com/blog/engineering/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

I joined LinkedIn about six years ago at a particularly interesting time. We were just beginning to run up against the limits of our monolithic, centralized database and needed to start the transition to a portfolio of specialized distributed systems. This has been an interesting experience: we built, deployed, and run to this day a distributed graph database, a distributed search backend, a Hadoop installation, and a first and second generation key-value store.

One of the most useful things I learned in all this was that many of the things we were building had a very simple concept at their heart: the log. Sometimes called write-ahead logs or commit l

@sivabudh
sivabudh / abbyy.txt
Created January 22, 2020 02:27
admin-classify
django-admin-celery-classify_1 | [2020-01-22 09:22:14,332: WARNING/ForkPoolWorker-2] Declared queue.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,356: WARNING/ForkPoolWorker-2] Initialized callback channel.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,360: WARNING/ForkPoolWorker-2] Sent PDF file to ABBYY.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,370: WARNING/ForkPoolWorker-2] Published.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,371: WARNING/ForkPoolWorker-2] Waiting for response...
django-admin-celery-classify_1 | [2020-01-22 09:22:14,466: WARNING/ForkPoolWorker-3] Declared queue.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,484: WARNING/ForkPoolWorker-3] Initialized callback channel.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,484: WARNING/ForkPoolWorker-3] Sent PDF file to ABBYY.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,486: WARNING/ForkPoolWorker-3] Published.
django-admin-celery-classify_1 | [2020-01-22 09:22:14,487: WARN
@sivabudh
sivabudh / gist:76f1af0e3d6d0af4cf3b7b90543d460a
Created August 7, 2019 14:06
Error from compilemessages
Traceback (most recent call last):
File "/usr/local/lib/python3.5/site-packages/django/db/backends/postgresql/base.py", line 20, in <module>
import psycopg2 as Database
File "/usr/local/lib/python3.5/site-packages/psycopg2/__init__.py", line 50, in <module>
from psycopg2._psycopg import ( # noqa
ImportError: /usr/local/lib/python3.5/site-packages/psycopg2/.libs/libresolv-2-c4c53def.5.so: symbol __res_maybe_init version GLIBC_PRIVATE not defined in file libc.so.6 with link time reference
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 12, in <module>
execute_from_command_line(sys.argv)
@sivabudh
sivabudh / sample.json
Created May 6, 2018 07:30
/api/categories/id/shallow-get/
{
"id": 9713,
"sequence": 1,
"has_children": true,
"name": "LIXIL",
"header_image": null,
"content_image": "http://0.0.0.0:8000/media/foto/content_image/1_LIXIL_content_megqk8u.jpg",
"popup_image": null,
"product_view_type": 1,
"product_selection_type": 1,
@sivabudh
sivabudh / console.sh
Created August 30, 2017 14:40
buddy output
python manage.py test
Hey, I'm Local!
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1122, in _do_get
return self._pool.get(wait, self._timeout)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/queue.py", line 145, in get
raise Empty
sqlalchemy.util.queue.Empty
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
@sivabudh
sivabudh / my_jobs_for_tl.txt
Created August 6, 2017 10:06
Displaying My Jobs for TL role
Logic for displaying "My Jobs" for TL:
From PMJobModel table, any jobs that:
1) PMJobModel.userTL == TL's user ID
AND
2) PMJobModel.JobStatus == "submitted" OR "reviewing" OR "FO reject"
@sivabudh
sivabudh / bad_query.sql
Last active July 15, 2017 04:39
Bad SQL Query taking almost 30 seconds to respond
SELECT "nodes_node"."id", "nodes_node"."alive", "nodes_node"."name", "nodes_node"."node_type", "nodes_node"."province", "nodes_node"."postal_code", "nodes_node"."attributes" FROM "nodes_node" WHERE ("nodes_node"."alive" = true AND "nodes_node"."id" IN (7, 3258, 3260, 8228, 8229, 8230, 8231, 8232, 8233, 8234, 8235, 8236, 8237, 8238, 8239, 8240, 8241, 8242, 8243, 8244, 8245, 8246, 8247, 8248, 8249, 8250, 11409, 8252, 8253, 8254, 8255, 8256, 8257, 8258, 8259, 8260, 11410, 8262, 8263, 8264, 8265, 8266, 8267, 8268, 8269, 11413, 8271, 11412, 8273, 8275, 8276, 8277, 11414, 8280, 8281, 8282, 8283, 8284, 8285, 8286, 8287, 3275, 8289, 8290, 8291, 8292, 8293, 8294, 8295, 8296, 8297, 8298, 8299, 11418, 8301, 8302, 8303, 8304, 8305, 8306, 8307, 8308, 8309, 8310, 8311, 8312, 8313, 8314, 11421, 8316, 8317, 8318, 8319, 8320, 8321, 8322, 8323, 8324, 8325, 8326, 8327, 8328, 8329, 8330, 8331, 8332, 8333, 8334, 8335, 8336, 8337, 8338, 8339, 11425, 8341, 9799, 8342, 8344, 8345, 8346, 8347, 8348, 8349, 8350, 8351, 8352, 8353, 8354
@sivabudh
sivabudh / grit_orders.js
Last active June 23, 2017 12:26
// [POST] to URL: https://eneos-pos-prod.sbrandpos.com/api/orders/
{
"car_data_id" : 10424,
"customer_id" : 7492,
"staff_comment" : "Image Object",
"shop" : 62,
"staff" : 838,
"order_items" : [
{
@sivabudh
sivabudh / error.sh
Created May 26, 2017 05:35
npm error
ERROR in [at-loader] ./yaoms/apps/nav-bar/nav-bar.component.ts:43:22
TS2686: '_' refers to a UMD global, but the current file is a module. Consider adding an import instead.
Child html-webpack-plugin for "index.html":
[0] ./~/lodash/lodash.js 540 kB {0} [built]
[] -> factory:90ms building:4052ms = 4142ms
[1] ./~/html-webpack-plugin/lib/loader.js!./yaoms/index.html 2.72 kB {0} [built]
factory:102ms building:29ms = 131ms
[2] (webpack)/buildin/global.js 509 bytes {0} [built]
[] -> factory:161ms building:1082ms = 1243ms
[3] (webpack)/buildin/module.js 517 bytes {0} [built]
@sivabudh
sivabudh / nginx.conf
Last active April 4, 2017 16:45
Microservices with NGINX
#
# NGINX config for microservices
# Tested with nginx/1.10.3
#
worker_processes 5;
events {
worker_connections 4096;
}