Matthew Kweskin mkweskin

Method to randomize column order in a csv using the BASH

I received a question about how to randomize column order in a text file. I came up with the method that using common Unix command line tools (sort, sed, tr, join) and the BASH shell. It has been tested on the BSD command line tools in macOS 12 (running zsh) and gnu command line tools BASH on CentOS (running bash).

NOTE: this does not handle quoted commas in the CSV. The only commas should be the delimiters.

Randomize all columns

NUM_COLS=10
NUM_RANDOMIZED_OUTPUT=5

My preferred settings for PuTTY (0.71)

Terminal
- Bell
  - Action to happen when a bell occurs : None (bell disabled)
Window
- Lines of scrollback: 50000
- Appearance
Font : Courier Std, 12 pt

Linux `find`

find is used to find files based on their name or other attributes of the file.

It does not search inside files like the grep command
To find find examples for a speicific purpose, I do a web search for: linux find ...

Basic find command: search for a filename

find /path/to/dir -iname '*text*'

Dissecting this:

/path/to/dir: where you want to search, I often use . for my current directory

	#!/bin/sh

	# creates a new env for a bioconda package and its dependencies. Then, it display the list
	# of newly installed binaries for the focal program.
	# The directory where the env is created is: ~/share/apps/bioinformatics/[program]/conda/[version]/

	if [ -z $1 ]; then
	echo "error: give the name of the bioconda program to be installed when calling this program"
	echo " example: $0 augustus"
	exit 1

	Bootstrap: docker

	From: continuumio/miniconda3

	%post
	/opt/conda/bin/conda config --add channels defaults
	/opt/conda/bin/conda config --add channels bioconda
	/opt/conda/bin/conda config --add channels conda-forge
	/opt/conda/bin/conda config --add channels omnia
	/opt/conda/bin/conda config --add channels plotly

 #!/usr/bin/env python3
 """
 Author: Matthew Kweskin, github: @mkweskin
 A general utility to read in a delimited translation file with two columns
 and rename any text file with these values.
 """
 import argparse

	#!/usr/bin/env python3

	"""
	Author: Matthew Kweskin, github: @mkweskin

	This script converts the sequence names in a FASTA file
	to the md5 hash of the sequence.

	Notes:
	- Gaps will automatically be removed by biopython

	pandoc -f gfm -w jira -o outfile.jira infile.md

	# To import converted file into Confluence:
	# - Create new page
	# - Click on the body of the page, click on the " + \/" dropdown in toolbar ("Insert more content") and select "Markup"
	# - Paste the contents into the pop-up window (select "Confluence wiki" as the format)
	# - Note: The "Markdown" option in the import pop-up doesn't seem to work for Github flavored markdown (gfm).

	#!/usr/bin/env python3

	import pandas as pd
	from Bio import SeqIO
	import argparse
	import os
	import logging
	import sys
	import datetime