Skip to content

Instantly share code, notes, and snippets.

View clintval's full-sized avatar

Clint Valentine clintval

View GitHub Profile
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lindenb
lindenb / distance.js
Created March 3, 2017 10:55
Distance between variants in a VCF. javascript src for bioalcidae https://github.com/lindenb/jvarkit/wiki/BioAlcidae
/**
distribution of distance between variants. Non-Variant, Filterer or GVCF blocks are ignored
usage:
java -jar jvarkit-git/dist/bioalcidae.jar -f distance.js input.vcf | sort -k1,1g | uniq -c
*/
var prev=null;
@Brainiarc7
Brainiarc7 / transient-clustering-gnu-parallel-sshfs.md
Last active September 2, 2024 13:53
How to set up a transient cluster using GNU parallel and SSHFS for distributed jobs (such as FFmpeg media encodes)

Transient compute clustering with GNU Parallel and sshfs:

GNU Parallel is a multipurpose program for running shell commands in parallel, which can often be used to replace shell script loops,find -exec, and find | xargs. It provides the --sshlogin and --sshloginfile options to farm out jobs to multiple hosts, as well as options for sending and retrieving static resources and and per-job input and output files.

For any particular task, however, keeping track of which files need to pushed to and retrieved from the remote hosts is somewhat of a hassle. Furthermore, cancelled or failed runs can leave garbage on the remote hosts, and if input and output files are large, sending them to local disk on the remote hosts is somewhat inefficient.

In a traditional cluster, this problem would be solved by giving all nodes access to a shared filesystem, usually with NFS or something more exotic. However, NFS doesn't wo

@Brainiarc7
Brainiarc7 / gnu-parallel-bioinformatics.md
Last active September 6, 2023 13:59
GNU Parallel usage in Bioinformatics with examples: A primer.
@bsmith89
bsmith89 / .gitattributes
Last active April 5, 2021 23:37
Example files for http://blog.byronjsmith.com/makefile-shortcuts.html, a default makefile for computational research projects
*.ipynb filter=dropoutput_jupyter
*.[tc]sv diff=daff-csv
*.[tc]sv merge=daff-csv
@dianjuar
dianjuar / a stop job is running for session c2 of user.md
Last active May 15, 2022 12:39
Fixing "a stop job is running for session c2 of user xxx (xs / 1min 30s)"

Fix a stop job is running for session c2 of user xxx (xs / 1min 30s)

enter image description here

This advice is pointing to arch based distributions. Manjaro in special

Install watchdog

# pacman -S watchdog >

@josteink
josteink / on_stateful_code.txt
Last active October 21, 2020 05:37
On why stateful code is bad
On why stateful code is bad
===========================
STUDENT: Sir, can I ask a question?
TEACHER: Yes!
STUDENT: How do you put an elephant inside a fridge?
TEACHER: I don't know.
STUDENT: It's easy, you just open the fridge and put it in. I have another question!
TEACHER: Ok, ask.
STUDENT: How to put a donkey inside the fridge?
@kantale
kantale / example_tool.py
Last active March 8, 2016 04:47
How to setup galaxy with TORQUE in Amazon's cloud
#!/usr/bin/python
import sys
'''
Just a simple tool that adds the line number at
the end of each line
'''
with open(sys.argv[1]) as f_in, open(sys.argv[2], 'w') as f_out:
@anmoljagetia
anmoljagetia / terminalFlatTheme.sh
Created September 13, 2015 17:14
This gist shows how to setup terminal colors in the new gnome-terminal (v 3.14) and above. This was generated using a site called http://terminal.sexy
#!/usr/bin/env bash
# Base16 - Gnome Terminal color scheme install script
[[ -z "$PROFILE_NAME" ]] && PROFILE_NAME="terminal.flat.theme"
[[ -z "$PROFILE_SLUG" ]] && PROFILE_SLUG="terminal-flat-theme"
[[ -z "$DCONF" ]] && DCONF=dconf
[[ -z "$UUIDGEN" ]] && UUIDGEN=uuidgen
dset() {
local key="$1"; shift
@nl5887
nl5887 / transfer.fish
Last active July 30, 2024 09:21
Bash and zsh alias for transfer.sh. Transfers files and directories to transfer.sh.
function transfer
if test (count $argv) -eq 0
echo "No arguments specified. Usage:\necho transfer /tmp/test.md\ncat /tmp/test.md | transfer test.md"
return 1
end
## get temporarily filename, output is written to this file show progress can be showed
set tmpfile ( mktemp -t transferXXX )
## upload stdin or file