Skip to content

Instantly share code, notes, and snippets.

View cabecada's full-sized avatar

Vijaykumar Jain cabecada

View GitHub Profile
@cabecada
cabecada / gist:d03b7a75f380091d16672b36aedcd013
Last active August 15, 2023 18:43
citus failover testing
postgres@pg:~/citusdb$ cat setup.sh
#!/bin/bash
export PATH=/usr/lib/postgresql/14/bin:$PATH
port=5432
if [[ -d dbcr ]]
then
pg_ctl -D dbcr stop
rm -rf dbcr*
fi
@cabecada
cabecada / gist:95a6d4b4c3adbf4f350f96425c8bbda8
Created June 21, 2023 06:36
play with diff recovery options
https://blog.devgenius.io/setup-continuous-archiving-and-point-in-time-recovery-pitr-with-postgresql-db-7e670523e8e4
PostgreSQL is a powerful open-source relational database management system. It is widely used in various industries and applications due to its reliability, security, and scalability. One of its most useful features is Point-In-Time-Recovery (PITR), which allows you to recover your database to a specific point in time. PITR is essential in disaster recovery scenarios because it can minimize data loss.
Here are the steps to set up PITR with PostgreSQL:
Enable Archiving
In the postgresql.conf file, set the archive_mode configuration parameter to on. This will enable archiving.
Configure Archive Command
In the postgresql.conf file, set the archive_command configuration parameter to a command that will copy the WAL files to a safe location. For example, you can use the following command:
@cabecada
cabecada / gist:3e6352c6e2588aabc99faa7effc7b5ee
Created June 18, 2023 19:57
postmaster.pid file using chatgpt
i need to modify my postmaster.pid file in postgresql to a custom path. how to do it
ChatGPT
To modify the path of the postmaster.pid file in PostgreSQL, you'll need to update the configuration file (postgresql.conf) with the desired location. Here's a step-by-step guide to help you:
Locate the postgresql.conf file: The exact location of the file depends on your operating system and PostgreSQL installation. Common locations include:
Linux: /etc/postgresql/{version}/main/postgresql.conf
macOS: /usr/local/var/postgres/postgresql.conf
Windows: C:\Program Files\PostgreSQL\{version}\data\postgresql.conf
Open the postgresql.conf file using a text editor.
@cabecada
cabecada / Citus 11.1.md
Created April 14, 2023 14:11 — forked from hanefi/Citus 11.1.md
Generated Documentation of Citus using pg_readme

t

pg_extension_name: citus pg_extension_version: 11.1-1 pg_readme_generated_at: 2023-01-30 17:35:52.80893+03 pg_readme_version: 0.5.5

Citus distributed database

#if i bump number of concurrent subscriptions to 400 my vm freezes :), but otherwise it works fine. so may be some resource bottleneck
#is resulting in the errors
postgres@pg:~/temp$ cat run.sh
#!/bin/bash
for i in {1..3}
do
pg_ctl -D db$i -l db${i}.log stop
@cabecada
cabecada / gist:812de8362eaf4219452755cb3d9c50ae
Created March 17, 2023 09:25
pg_filenode.map global deleted and recovery
postgres@pg:~/temp$ initdb -D db1
postgres@pg:~/temp$ pg_ctl -D db1 -l db1.log start
waiting for server to start.... done
server started
postgres@pg:~/temp$
postgres@pg:~/temp$ pg_basebackup -D replica -R -c fast -C -S replica -d "port=5432"
postgres@pg:~/temp$ vim replica/postgresql.auto.conf
postgres@pg:~/temp$ pg_ctl -D db2 -l db2.log start
pg_ctl: directory "db2" does not exist
postgres@pg:~/temp$ pg_ctl -D replica -l replica.log start
@cabecada
cabecada / gist:6ff09966474dfdd2cd93bb479a3d86ec
Last active March 13, 2023 13:03
fun with citus extension
postgres@pg:~/citusdb$ more setup.sh
#!/bin/bash
export PATH=/usr/lib/postgresql/14/bin:$PATH
port=5432
for i in db1 db2 db3 db4
do
pg_ctl -D $i stop
rm -rf $i
initdb -D $i
@cabecada
cabecada / gist:c67442c051404d1ff5548b40a0270398
Last active March 12, 2023 09:42
grouping by bucket size postgres
https://stackoverflow.com/questions/22157718/postgres-group-by-timestamp-into-6-hourly-buckets
select count(1), trunc(extract (minute from x) / 5) from generate_series(now() - '60 minute'::interval, now(), '1 minute'::interval) x group by trunc(extract (minute from x) / 5) order by 2 desc;
count | trunc
-------+-------
5 | 11
5 | 10
5 | 9
5 | 8
5 | 7
@cabecada
cabecada / gist:19cccfc762f6429860f6774f53a16e5c
Created February 24, 2023 13:59
postgresql trace app request to postgresql query in logs and pg_statement
how to generate a unique request id mapping to app request to a db query request.
i have a setup of
1) postgresql
2) flask app running on port 5000
3) haproxy running on port 8000 with backend as flask app. haproxy also generated a uuid header mapping app request to postgresql query.
```
postgres@pg:~$ cat /etc/haproxy/haproxy.cfg
global
@cabecada
cabecada / gist:36a89dc915758973e73f0514af9900eb
Created February 18, 2023 12:50
chatgpt and coredns 101
What is coredns
CoreDNS is an open-source Domain Name System (DNS) server software that can be used to provide DNS services for applications and services running in cloud-native environments. It is written in Go and is designed to be highly modular and extensible, allowing users to customize its functionality to suit their specific needs.
CoreDNS is often used as a replacement for traditional DNS servers like BIND, which can be difficult to configure and manage in cloud-native environments. CoreDNS supports a wide range of DNS features, including DNS-over-TLS, DNS-over-HTTPS, DNSSEC, and plugin-based customization of DNS functionality.
CoreDNS is used in many cloud-native platforms, including Kubernetes, which uses CoreDNS as its default DNS server. It can also be used as a standalone DNS server or as a component in larger cloud-native architectures.