Skip to content

Instantly share code, notes, and snippets.

View emmanuelnk's full-sized avatar

Emmanuel emmanuelnk

View GitHub Profile
@emmanuelnk
emmanuelnk / PG_ROLE_CHANGE_GUIDE.md
Created November 6, 2024 23:56
Postgres Role Change Guide

Migrating from one role to another can be very confusing in Postgres. This is a personal guide with a collection of queries to help aid in this effort.

Definitions

  • old_role -- Refers to a postgres role that currently owns objects.
  • new_role -- Refers to a postgres role that will soon own previous objects.
  • postgres -- Refers to the admin user role.
  • role_database -- Refers to a database that the old_role currently owns but will soon be owned by new_role

Some useful queries

See all the tables a given role in a db owns:

@emmanuelnk
emmanuelnk / useful_lambda_log_insights_queries.md
Created December 7, 2023 19:25
AWS Cloudwatch Log Insights Useful Lambda Queries

Queries

General Performance

This query runs against Lambda function logs to get the following information:

  • InvocationCount -- This is the number of times the Lambda function was invoked.
  • ColdStartCount -- This is the number of times the Lambda function had initialization overhead (cold start).
  • InitDuration (min, avg, max) -- This is the amount of time it took the Lambda function to initialize (cold start time).
  • Duration (min, avg, max) -- This is the amount of time it took the Lambda function to run after initialization.
  • AllocatedMemory -- This is the amount of memory allocated to the Lambda function in MB.
  • MaxMemoryUsed (min, avg, max) -- This is the max amount of memory that the Lambda function used during its invocation in MB.
@emmanuelnk
emmanuelnk / openpr.md
Created November 1, 2023 18:03
Open Github PR from Terminal (SSH Authorization)
  1. Install Dependencies

    brew install jq
    brew install hub
  2. Log into hub (from any repo just do hub pr list)

    • use your github username for username
    • use a personal access token for password (token should have repo access)
@emmanuelnk
emmanuelnk / github-actions-json-outputs.md
Created September 20, 2023 21:29
Extract Job or step output JSON object to GITHUB_ENV in Github Actions

If you have a job or step that has many outputs that you would like to set as envs (in the shell), do this:

name:

workflow_dispatch:

jobs:
  test:
 runs-on: ubuntu-latest
1. Install oh-my-zsh
sh -c "$(wget https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh -O -)"
2. Clone necessary plugins.
git clone https://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions
git clone https://github.com/zsh-users/zsh-history-substring-search ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-history-substring-search
git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
3. Add plugins to ~/.zshrc as
plugins = ( [plugins...] zsh-autosuggestions zsh-history-substring-search zsh-syntax-highlighting)
@emmanuelnk
emmanuelnk / mwaa_cloudwatch_get_worker_count.md
Created April 21, 2023 17:00
Managed Worklows for Apache Airflow get worker Count from Cloudwatch Insights

Cloudwatch does not provide metrics on how many workers you are running in a MWAA Environment. Fortunately we can get this information from Cloudwatch Insights using the MWAA_ENVIRONMENT_NAME-Worker log group. Every log stream has the ip address of the worker in the @logStream field

  1. Open the relevant MWAA_ENVIRONMENT_NAME-Workerlog group
  2. Open in CloudWatch Insights and paste and run the following query for your desired time range.
  3. you can then add the visualization to your desired Cloudwatch dashboard
fields @timestamp
@emmanuelnk
emmanuelnk / Queries.md
Last active April 26, 2023 16:25
Useful Athena Queries

A collection of useful Athena Queries

Web Application Firewall

Create WAF logs with projection (no need to partition)

  • See the docs for reference

  • Replace the values BUCKET_NAME, AWS_ACCOUNT_NUMBER, AWS_REGION, WEB_ACL_NAME accordingly

  • Replace 'projection.date.range' = '2023/01/01/00/00,NOW', with the date your WAF logging begins (find in S3)

    Query
@emmanuelnk
emmanuelnk / albLogFormat.interface.ts
Created February 23, 2023 15:20
AWS Application Load Balancer Access Logs Typescript Interface
export interface AlbAccessLogFormat {
/**
* The type of request or connection. The possible values are as follows (ignore any other values):
*
* * http — HTTP
* * https — HTTP over TLS
* * h2 — HTTP/2 over TLS
* * grpcs— gRPC over TLS
* * ws — WebSockets
* * wss — WebSockets over TLS
@emmanuelnk
emmanuelnk / log_bash_commands.bash
Last active September 16, 2022 22:10
Log bash commands
function log_cmd()
{
"$@"
ret=$?
if [[ $ret -eq 0 ]]
then
echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") user[ $USER ] status[ Success ] code[ $ret ] cmd[ $@ ]" >> bash_command.log
else
echo "$(date -u +"%Y-%m-%dT%H:%M:%SZ") user[ $USER ] status[ Error ] code[ $ret ] cmd[ $@ ]" >> bash_command.log
@emmanuelnk
emmanuelnk / .zshenv
Last active September 13, 2022 16:10
Aliases for jira-cli with Jira Cloud and ZSH
export JIRA_API_TOKEN=XXXXXXXXXXXXXXXXXXX