Gordin

cd() {
  # Check if no arguments to make just typing cd<Enter> work
  # Also check if the first argument starts with a - and let cd handle it
  if [ $# -eq 0 ] || [[ $1 == -* ]]
  then
    builtin cd $@
    return
  fi
  # If path exists, just cd into it
  # (also, using $* and not $@ makes it so you don't have to escape spaces any more)

ghing

Identifying which PDFs are images vs. text

Someone who was in my PDF text extraction session at NICAR 2020 asked how to identify image vs. text PDFs when you have thousands of files and they're a mixture of formats with the end goal of only running OCR software on the image PDFs.

This is how I would approach the problem using command-line tools.

Assumptions

• You’re working on a Mac or Linux machine where you have access to some common command-line utilities such as find and sed
• This should work under the Windows Subshell for Linux under Windows also

kinoc

#  So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay 
#  More exploratory coding. It uses the Huggingface model port, deepspeed and reads all text/md files from a target directory
#  It is a fragment of a larger system with remote editing, but that's another story
#  This is the raw, training tester. Items to look out for:
#  - uses DeepSpeed and has a DS config
#  - to save space uses SGD instead of ADAM
#  - uses gradient checkpointing
#  - freezes 25% of the layers to fit

# Assumes you can already run https://gist.github.com/kinoc/2d636a68876cd3de7b6e9c9452b61089

danirukun

#!/usr/bin/env bash

# Small shell script to more easily automatically download and transcribe live stream VODs.
# This uses YT-DLP, ffmpeg and the CPP version of Whisper: https://github.com/ggerganov/whisper.cpp
# Use `./transcribe-vod help` to print help info.

# MIT License

# Copyright (c) 2022 Daniils Petrovs