Skip to content

Instantly share code, notes, and snippets.

@thoroc
thoroc / select_aws_profile.sh
Created April 4, 2025 14:33
Shell function to select an AWS profile when using Leapp
select_aws_profile () {
local profiles=($(leapp session list --output=json | jq -r '[.[] | select(.status=="active")] | .[].profileId'))
if [[ ${#profiles[@]} -eq 0 ]]
then
unset AWS_PROFILE
echo "AWS_PROFILE unset as there are no profiles"
elif [[ ${#profiles[@]} -eq 1 ]]
then
export AWS_PROFILE="${profiles}"
echo "AWS_PROFILE set to ${profiles}"
@thoroc
thoroc / README.md
Last active February 26, 2025 21:22
How I scraped 2.1 million jobs (including 5,335 data science jobs)

How I scraped 2.1 million jobs (including 5,335 data science jobs)

original: https://www.reddit.com/r/datascience/comments/1iynyco/how_i_scraped_21_million_jobs_including_5335_data/

Background

During my PhD in Data Science at Stanford, I got sick and tired of ghost jobs & 3rd party offshore agencies on LinkedIn & Indeed. So I wrote a script that fetches jobs from 30k+ company websites' career pages and uses GPT4o-mini to extract relevant information (ex salary, remote, etc.) from job descriptions. You can use it here: (HiringCafe). Here is a filter for Data science jobs

/*--------------------------------------------------------------
/*! normalize.css v8.0.1
--------------------------------------------------------------*/
html {
line-height: 1.15;
-webkit-text-size-adjust: 100%;
scroll-behavior: smooth;
}
@thoroc
thoroc / GIT_MIRROR.md
Last active February 12, 2025 11:48
Git clone script in Deno/Typescript

git-mirror

This was now moved to: https://github.com/thoroc/git-mirror

Deno script to clone a github/gitlab repo to ~/Projects while keeping a tree structure close to the remote url. If the project is already present, then it'll fetch from the remote.

WARNING this was developed on a MacOS, so no guaranty are offered to run on a different OS.

install

@thoroc
thoroc / README.md
Created October 9, 2024 18:32
Removel sophos AV from MacOS

Remove sophos keychain

sudo rm /Library/"Sophos Anti-Virus"/SophosSecure.keychain

Disable sophos

sudo defaults write /Library/Preferences/com.sophos.sav TamperProtectionEnabled -bool false
@thoroc
thoroc / put-event.json
Created October 7, 2024 16:50
Attempting to setup eventbridge rule to send logs to cloudwatch
[
{
"Source": "my.application",
"DetailType": "TestEvent",
"Detail": "{\"message\": \"Hello World\"}",
"EventBusName": "default"
},
{
"Source": "my.application",
"DetailType": "MyAppEvent",
@thoroc
thoroc / git-checkout-repo.sh
Created October 1, 2024 08:04
Git checkout repository, reproducing the path of the origin locally
#!/bin/bash
# put an alias in your .gitconfig to where the script lives: co-repo = ~/.bin/git-checkout-repo.sh
# don't forget to `chmod +x` it
# Input Git repository URL
REPO_URL="$1"
# Define the target base directory (where all projects should reside)
TARGET_BASE_DIR=~/Projects
@thoroc
thoroc / README.md
Last active April 16, 2024 10:14
Api Gateway with CDK (Python)

Api Gateway with CDK

source: https://www.sccbrasil.com/blog/aws/cdk-api.html

By Wolfgang Unger

Lets have a look how to create a API Gateway with CDK (Python) The first approach is using the RestApi Class and code the resources and methods. The second by using a Swagger/Open API file. Both APIs will use lambda integrations.

@thoroc
thoroc / appl.csv
Created February 25, 2024 20:13
just so I can call it and not host it directly.
date close
2007-04-23 93.24
2007-04-24 95.35
2007-04-25 98.84
2007-04-26 99.92
2007-04-29 99.8
2007-05-01 99.47
2007-05-02 100.39
2007-05-03 100.4
2007-05-04 100.81
@thoroc
thoroc / get_resolvers.sh
Created January 29, 2024 16:18
Example on how to fetch all the functions resolver for an GraphQL API on AWS and output them per template (dealing with VTL)
#!/bin/sh
# set the table list for "DrawTable" "GameTable" "PlayroundTable"
TABLE_NAMES="DrawTable GameTable PlayroundTable"
# API_ID="asdasdadajsdljalskdjalksdj"
# use the following command to get the list of functions
# FUNCTION_FILE=$(aws appsync list-functions --api-id "$API_ID")
FUNCTION_FILE="list_functions_full.json"
rm -rf resolvers