Skip to content

Instantly share code, notes, and snippets.

@lsaville
Last active January 10, 2023 15:23
Show Gist options
  • Save lsaville/34eca45b3a3be262d4997cba78b8fc2b to your computer and use it in GitHub Desktop.
Save lsaville/34eca45b3a3be262d4997cba78b8fc2b to your computer and use it in GitHub Desktop.

2023-01-10

Changing to iso 1806 style dates cuz who needs anything else amiright?

DISTINCT ON

DISTINCT ON with ORDER BY acts acts like window functions with ranking/row_number. https://www.youtube.com/watch?v=XuGxGP8quMQ

psql get result file workaround

-- Set the variable "query" to the contents of the sql file
\set query `cat my_rad_query.sql`
 
-- Use the variable as the query and copy to csv
copy (:query) TO STDOUT WITH CSV HEADER \g 'my_rad_query_result.csv'

12/4/2022

Since I can never remember the name

cli benchmarking tool https://github.com/sharkdp/hyperfine

1/9/2021

linux mount apple file system (old external drive)

yay -S hfsprogs
mkdir gumption-5
sudo mount -t hfsplus -o force,rw /dev/sda2 ~/gumption-5

sda2 found from lsblk -f

4/7/2020

Xargs -I flag

tldr don't loop

something | xargs -I % sh -c 'command where every occurrence of % is replaced by one of the args from xargs'

echo -e 'meow1\nmeow2\nmeow3\n' | xargs -I % echo "any little % is the spawn of %'s parent"

echo -e 'this\nthat\nthe other\n' | xargs -I % sh -c 'echo %; echo %'

c/o dear Travitz

4/4/2020

WhatsApp video format conversion

To get a video to work with WhatsApp it needs to be MP4 (H264 + aac). From Quora of all places!

ffmpeg -i file-from-guvcviewer.mkv -vcodec libx264 -acodec aac functioning-whatsapp-vid.mp4

3/19/2020

Git list my scratch files in a way that can be | xarg <something>'d

git ls-files --others --exclude-standard

12/13/2019

Run g command on visual selection

Add something to line ending with ...

'<,'>g/\.\.\.$/norm A something

Works with v too!

Sort markdown table on column

Suppose you've got:

| header1 | header 2 |
| ------- | -------- |
| a       | c        |
| b       | b        |

You can sort by header 2 via sort -t'|' -k3... the left bar counts as one, then add for columns. Just do column # plus 1. I used it in vim via visual selection like:

:'<,'>.! sort -t'|' -k3

From Christian Tietze post

12/12/2019

Get an overall diff from start to finish on a branch w/o remote compares "beginning of time" with the tip

git diff --name-only $(git rev-list --max-parents=0 HEAD) HEAD

11/2/2019

Mount OSX formatted drive

mkdir ~/<drive-name-whatevs-shrug>

Find the drive via lsblk -f

sudo mount -t hfsplus -o force,rw /dev/sdXY ^^^^dir-above

udisksctl power-off -b ^^^^dir-above

sudo eject/dev/sdXY

10/17/2019

Show slow running command Postgres

From -> blog post

SELECT
  pid,
  now() - pg_stat_activity.query_start AS duration,
  query,
  state
FROM pg_stat_activity
WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes';

Feh slideshow

This does a 3 second slide delay

feh -rF -D 3 <folder>

10/16/2019

Vim "very magic" with g command

This allows the use of non-escaped parens and | for composing a single g delete command with multiple search terms

g/\v(\d\d:\d\d|^$)/d

This applied to a visual range (could use other ranges also) along with an awk shell out cleans up a copy pasta from codeship.

!awk '{print $1" "$2}'

10/11/2019

json to yaml since I can't figure out yq

python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' < in-file.json > out-file.yaml

also %s/\(\<..$\)/US\1... pretty cool, adds US to someting like area: KY --> area: USKY

10/4/2019

Toss the name of file into the file :)

Used this to grab the filename for a hard to autocomplete migration file.

r! echo %

A little better 🤷?

:exe ":silent ! echo % | xclip -sel clip" | redraw!

8/29/2019

Shell tricks via Travitz

To format json:

nnoremap <leader>j :%!jq ''<CR>

To format html:

nnoremap <leader>h :%!xmllint --format --encode UTF-8 --html -<CR>

8/20/2019

Vim re-number ordered list after rearrangment (Thanks for the prompt Ryan!)

This takes a visual selection from vim and hands it off to awk for 'filtering'.

:'<,'>! awk 'match($0, /^.+\. (.*)$/, a) {print NR". " a[1]}' (This one has unpredicatable behavior... a lacking in my regex prowess certainly)

'<,'>! awk 'match($0, /^[^.]+\. (.*)/, a) {print NR". " a[1]}'

The Awk match function takes a string, a pattern, and a variable name for the resultant array of matches. In this case a[0] would be the whole line and a[1] the one and only capture group from the pattern.

This specifically matches something up to a literal period character followd by a space (the existing numbers), captures everything after that space, and then re-adds the line numbers to the selection based on Awk's built-in NR variable which is the 'number of records so far' starting from zero.

As a personal note, this solution represents a continued resistence to my learning proper vimscript functions. I've looked into some plugins and have a very vague idea of what functions look like, what they can do etc, but there has been a hurdle to making my own that I've been unwilling or unable to cross thus far. In defense I can say that using Awk bolsters familiarity with another Unix tool that can be used outside of Vim, and that this specific problem is an awky/sedy type line problem for which these programs excel.

8/2/2019

Delete untracked files when I want a clean slate after a git reset --hard

git status --porcelain | awk '{print $2}' | xargs rm

7/10/2019

Almost painful enough to do something about.

~/.dropbox-dist/dropboxd &

7/1/2019

Delete all but the most recent in a directory (please be careful!)

for dir in $(ls -t | tail -n +2); do rm -rf $dir; done

The ls -t lists files/directories by most recent. tail -n +2 lists output starting with the second line.

5/20/2019

Recursive macro with search detail

I can't believe I didn't put this in here before (had encountered this problem and solution once already). If you're using / in a recursive macro, if you continue to have matches after the 'first round' you'll keep running through the file doing more action than you had anticipated and it will do wild things until it runs out of search matches. This is because by default vim has a boolean variable called wrapscan set to true. You can turn this feature off with :set nowrapscan and the search will end with the end of the file.

4/16/2019

Dont do it!

Please stop doing Capybara.current_session.driver. You'll have to kill the pry session because the object is so big that it'll scroll seemingly forever. There are more monster objects as well, if I recall a request is similarly large and "scrolly".

3/27/2019

Run macro on lines matching pattern

I knew you could do something like :g/pry/d to delete all lines with pry in them. I use this to clean up from heavy debugging sessions in both js and rb files.

Today I used '<,'>g/do/:norm 0$gEa, data: { action: 'header#linkTest' } do add a bit of text to every line that contained /do/ within a visual selection.

..... time elapses ....

In the aforementioned snippet, I used CTRL-r j to dump the contents of my macro onto the command line. The contents of j were: 0$gEa, data: { action: 'header#linkTest' }. Turns out from reading more of Power of G I could have just done something like:

:g/pattern/normal @q

3/4/2019

Rehash of side by side vim

Occassionally I find myself wanting to put things side by side in the same file in vim. I've used this to easily compare similar parts of Rails schema.rb. While I'd rather not sort the lines in the schema it's easy enough to copy paste the comparison targets and put them side by side in a scratch file. The technique is outlined here and I've forgotten and gone back enough times it merits an entry here!

If you've got the contents butted up against the left side go to the block (the bit you want as a 'right column'). Ctrl-v to start VISUAL BLOCK mode, go to bottom of the selection, $ to the end and d. Move to the 'left column', hit A on the first line and space out until you've cleared the longest line of 'left column'. <esc> to normal mode and p to paste the select. The [link to SO] (https://stackoverflow.com/a/27542895/5889617) has a nice gif show-and-tell if this didn't make sense.

It's a good idea if you're doing line-wise comparisons to sort both the 'left' and 'right' columns by using V to select the lines followed by :sort or :sor. Visually at the vim display line it'll show like: :'<,'>sor. That's just your visual selection on the vim command-line.

Count the occurances of an Ag find with help from Awk

Today I was curious how many places I might need to change a thing in trying to decide my course of action. The command is as follows:

ag -c 'embedded' | awk -F ':' '{n=n+$2}END{print n}'

ag -c PATTERN will give you output like:

path/to/file:3
path/to/file:15
path/to/file:1

The Awk bit breaks the line with a specific Field separator -F ':' and then accumulates the second field (the one with our find count) in the variable n. After Awk is done it will run anything in the END block, in this case just printing out our accumulated variable.

1/11/2019

Open a directory of files in vim

vim $(find app/views/purchase_applications/steps/ | awk '{ if (NR != 1) { print } }')

The awk bit removes the directory itself from the list of files.

Multiline grepping

It occured to me from the last section that what I really want is to multiline grep. Grep can't do that. Neither can ag. Thus the discovery of pcregrep which has a multiline option.

pcregrep -r -M 'hidden {\n display: none;\n}' app/assets/stylesheets/

Just for kicks, a fairly rediculous command to give me the actual number:

pcregrep -c --files-with-matches -r -M 'hidden {\n display: none;\n}' app/assets/stylesheets/ | awk -F ':' '{ n = n + $2; } END { print n }'

-> 13

Lolz.

End command line arguments

You can use -- after some options to a command to explicitly state that you're done with options and that everything after should be argument.

I used this to learn how many times we've duplicated a style that toggles visibility alone (display: none). We use some version of BEM for naming our styles so the command that was failing:

ag --sass '--hidden' <- Causes ag to go 'wait, what option is --hidden?'

The working one:

ag --sass -- --hidden

1/10/2019

Beware the greedy star!

On 12/4/2018 I talked about the following command:

ag --haml -l 'placeholder' | xargs sed -i -e "s/placeholder: '\(.*\)'/placeholder: '\L\1'/g" -e "s/placeholder: '\(.*\)'/placeholder: '\u\1'/g"

This did what I thought I wanted it to, until yesterday when I found out that our javascripty mortgage calculators were broken. It turned out to be a lesson to be wary of using * in Sed replacements. * is greedy! On a handful of lines, those in the caclulator js files, the lines with the placeholder replacement also held javascript variables for stimulus that were case sensitive...

placeholder: 'Length of loan', data: { 'graph-input': 'loan-term', target: 'mortgage-payment.loanTerm',

became:

placeholder: 'Length of loan', data: { 'graph-input': 'loan-term', target: 'mortgage-payment.loanterm',

changing loanTerm to loanterm and causing stimulus to throw a missing target element error and kill js execution. That greedy * ran the sed lower-case and capital-case trickery all the way to the last single quote on the line!

So..... beware the greedy star!

12/31/2018

Xargs -r argument

Xargs can take -r or --no-run-if-empty to avoid failure in a script. I made a little script to remove all curly quote characters, change ndash to mdash, and remove empty whitespace:

#! /bin/bash
set -xe

ag -l '[[:space:]]*$' | xargs -r sed -i 's/[[:space:]]*$//'
ag -l '’' | xargs -r sed -i "s/’/'/g"
ag -l '“' | xargs -r sed -i 's/“/"/g'
ag -l '”' | xargs -r sed -i 's/”/"/g'
ag -l '–' | xargs -r sed -i 's/–/—/g'

Without the -r, if the project doesn't have results from the ag, sed will throw sed: no input files.

note: the set -xe shows the command run to stdout (x) and exits the whole script if a line fails (e).

12/4/2018

Sed sentence case all placeholders across project

ag --haml -l 'placeholder' | xargs sed -i -e "s/placeholder: '\(.*\)'/placeholder: '\L\1'/g" -e "s/placeholder: '\(.*\)'/placeholder: '\u\1'/g"

Goes through and lowercases the submatch (the placeholder text), on a second pass capitalizes the submatch.

example:

.* placeholder: 'My Field'.* -> .* placeholder: 'My field'.*

Sed special backslash sequences

Another more complicated one:

ag --haml -l '%span\.form' | xargs sed -i -e 's/%span\.form\(.*\)field \(.*\)/%span.form\1field \L\2/g' -e 's/%span\.form\(.*\)field \(.*\)/%span.form\1field \u\2/g'

example:

.*%span.form__label-radio-field Old Password -> .*%span.form__label-radio-field Old password, where there can be different class text between form and field.

10/16/2018

Vim reformat already typed text to a given length

  1. Copy Pasta the text into vim.
  2. :set textwidth=80
  3. gggqG

the magic here is gq.

8/13/2018

Running a diff between master and current branch

git diff master...HEAD

or

git diff master..

Small recursive macro detail

In the past I've had problems trying to create a recursive macro and it turns out I'm just a dum dum. If you're creating a macro and you want to call that same macro from within itself during that very definition, the register itself needs to be empty first. I believe you can clear it with:

let @j=''

I've got a habit of using the same couple of registers for my macros, namely j and k because they're nearby. Pasting the contents of the register contianing the macro into a file and then altering the macro to add @register-in-question works as long as you've got a starting macro that doesn't end with a special character. i.e. when I want to expand a line with entities separated by spaces into having each entity on its own line (putting hash k/vs in a single column to sort maybe) I'd have:

f s^M^C^C

(since I'm in the habit of using ctrl-c to exit insert mode) but when I paste that into a file I've got 'f s' and have lost the special characters. My past problems relate to already having something stored in the macro and thus during recording it kicks off the previous version and wreaks havoc. Back to having some macros be one shot wonders.

..... ..... time elapses .... ....

hold the phones, a way around this is to use the vim command-line, which retains special characters. Go into command mode, start out using the let syntax to redefine the register, ctrl-r + register, ctrl-e (if you've got emacs style shortcuts defined) and then add the @register to the end. wooohooo. (looking back at the thoughtbot article where I first saw the paste trick I see that the let style was described also... with a repetitive ctrl-r and without the bit about special characters.

Vim close all buffers but the one you're in

:%bd|e#

Sort of a workaround because it actually closes all, then reopens the last one. The alternatives seemed arduous.

8/1/2018

Vim quit with error code

From Solomon's comment on Skorks shortcuts blog post :cq This quits vim with an error code so that whatever dire consequence may be avoided. For example if you're rebasing and realize you shouldn't be, or if you've ctrl-x ctrl-e and changed your mind about firing the command.

Fuller readline shortcuts

From this stackoverflow post and leading to the aforementioned Skorks blog post are more complete lists of all the readline shortcuts including the new-and-fantastic (to me) alt word based movements. This has prompted me to take a look at redoing my keymap to drop left or right shift in favor of positioning alt in a friendlier place. tbd.

Using find excluding the path

I wanted to find all js files in our project but exclude those in the vendor directory. This SO post describes many ways of doing it. I found the -prune option to be confusing. Further down the clear winner by 1518 votes describes using -not -path and so my final command:

find app/javascript '*.js' -type f -not -path '*vendor*'

The other option that was working intially is to ag or grep the results accordingly ag -v vendor to remove the vendor files from the results of the find

6/5/2018

Macro across many files

vim $(find -iname '*.html' | xargs ag -Ql 'w[l]')

Passing multiple files to vim with the $() instead of piping things into vim via xargs wont wonk the terminal display as mentioned on 4/20. After the files are in vim's buffer, you can record a macro including the :wn to save the file and move to the next file in the buffer (As seen here. In that next file I stopped the macro, opened a newline and pasted from the macro register to add the recursive invocation piece to the end blahblahkkkddd/meowIetc^C^C:wn^M@j (I always record to 'j', cuz its under my finger) (I wasn't able to just add the @j while I was recording the macro, something lacking in my understanding there). Yank that line back into register j, then @j to kick off the recursion! video example.

5/29/2018

Desktop notifications from command line

osx: osascript -e 'display notification "Notification text" with title "Title"'

linux (ubuntu): notify-send <message>

In conjuction with & at the end of a command (sending a long running job to the background), these will alert you as to when the job might be finished.

4/20/2018

Give vim multiple files

You can ls <options> | xargs vim to dump a load of files into vim. :args shows all the filenames (messy). :n goes to the next file and if you edit you can smash the commands together :wn :prev goes to the previous file in the list. It seems like all those arguments are dumped into :buffers (much prettier list of the same files as :args) so I was able to use my custom 'cruise the buffers' key bindings. I just cruised the whole list and :bn seems to satisfy the same criteria as :n in the sense that after you've reached the end of the file set you're able to quit without forcing the issue (no E173).

A big caveat is that terminal rendering gets messed up after you exit vim... in alacritty with tmux, in plain alacritty, and in gnome-terminal...

Grep with filetype

grep -rl --include *.svg <search pattern> I used this with giving vim multiple files.

4/5/2018

Running a rake task from rails c

When I ssh into a qa server, I only get one tmux pane (apprently default ssh allows one connection) unless I fiddle with some ssh config. Saying I only have one, and I was going to be in rails console anyway, I can run Rails.application.load_tasks followed by Rake::Task['some_task_name'].invoke to run the task from the console. If there is a namespace it works the same so Rake::Task['session:log_from_data'].invoke is valid.

Also you can list tasks via Rake::Task.tasks

Vim delete blank lines and delete whitespace

delete blank lines: :g/^\s*$/d delete whitespace: :%s/\s\+//g

4/4/2018

Deleting tags from html in vim

<\([^<]*\)> will match all fully formed html/xml tags, anything in < >. I've found this useful for scraping bits of websites by copying the outer html of an element in the browser inspector and dumping it in vim. With this I can delete all the tags in a single substitue command :%s/<\([^<]*\)>//g

Replacing the flex declaration in a single file in vim rather than with sed

%s/@include justify-content(\(.*\));/justify-content: \1;/g

3/19/2018

Vim macros on visual line selection and vim capture groups example

From SO post here.

Example case: rebase file squashing commits. Changing pick to s on a bunch of lines. I used to have absolute and relative line numbering in vim but on pairing with rtravitz and my lead its much easier to reference lines in absolute. With relative line numbering it was possible to record the macro like ciws^C^Cj where each run of the macro bumps down a line after changing the word, and play it to the end of the commits to be squashed. With absolute line numbering this isn't possible without doing math in my head. nope. so:

Record macro making sure that it doesn't move from one line (this is contra my previous method). In the example case it would be ciws^C^C. Visual Line select the remaining lines and hit :. This produces the command line with selection: :'<,'>. Add the macro play like: :'<,'>norm @j, and there you have it.

In the SO post someone also showed a capture group example that swaps two words separated by a comma. This pertains to an earlier diary post about sed (2/27/2018).

'<,'>s/\(\w\+\), \(\w\+\)/\2, \1/

Let me separate that a bit

'<,'> visual range
s/ substitue
\(\w\+\), \(\w\+\) pattern with the capture groups, first becoming \1 and second becoming \2
/ pattern/replacement separator
\2, \1 the replacement string utilizing the capture groups
/ end of subsitute phrase

3/5/2018

Towards a tmux copy paste solution

The keybindings from vim-tmux-navigator overshadow the default tmux selection mode vi keybindings like here. I ran across a post by Chris Toomey that remaps the start selection keys to mimic vim here. Seems to work.... a little twitchy. Maybe an Alacritty thing? It doesn't seem to work at all in gnome-terminal.

2/27/2018

Running sed with a capture group

Need to replace all instances of sass-flex mixins with regular declarations that in compilation get expanded to the browser flag editions by way of autoprefixer.

ag -Ql "@include justify-content" | xargs sed -i 's/@include justify-content(\([^\)]*\));/justify-content: \1;/'

The escaped parens in the matching expression \( and \) in \([^\)]*\) create a variable \1 that is the contents between the unescaped parens in justify-content(some variable stuff in here). This allows all the different values inside to be output in the replace expression

I've come across perhaps a more general way: ag -Ql "@include transform" | xargs sed -rn 's/@include transform\((.*?)\);/transform: \1;/'

This will handle something like @include transform(translate(-50%, -50%) skewY(-4.2deg));

1/12/2018

Running sed nondestructively first

I want to test my sed string on a file, or many files before doing something wild. sed -n "s/search-term/replacement/p" file-name

Then, when its time to alter sed -i "s/search-term/replacement/" file-name

I also can do a count on occurances across a directory to compare after the change ag -Q "search-term" | wc -l

11/28/2017

Symlinking Dotfiles

I've got a dotfiles folder living in the git dir where I keep my dotfiles for git control purposes.

move the dotfile you want (I'll be referring to my i3 config file): mv ~/.config/i3/config ~/git/dotfiles/config

symlink the dotfile so it can be loaded: ln -s ~/git/dotfiles/config ~/.config/i3/config

I always forget the syntax: ln -s followed by <where the file actually lives> followed by <where I want to file to pretend to live>

11/21/2017

Start command in default editor

ctrl-x ctrl-e

11/20/2017

Open last command in default editor

fc

11/15/2017

Replace a string across files in a directory:

find . -type f | xargs sed -i "s/#{phone}/#{link_to phone, \"tel:#{phone_raw}\", class: 'active-states-state__link'}g"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment