bash looping with regexp

Readme.md

some ideas for looping over dirs, files w/ bash

setup some folders and files

need to have files and folders to play with. here all size 0 files, adapt to your own needs...

# create a directory and go there
cd ~
mkdir tmp_playground && cd tmp_playground

# use bash wildcards to create a directory hierarchy
mkdir -p sub{001,002,005,123,89a}/{raw,recon,analysis}/

# inspect on macos and some linux systems this works
open .

# now touch some files...
touch sub{001,002,005,123,89a}/{raw,recon,analysis}/{abcd,aa_0123,bb_0124,cc_9999,xyz}.{txt,nii.gz}

1. Only directories in top-level, loop over them pushd and popd to get in and out of dirs and run a command locally
2. Stay in top level dir and find relative file names w/ intermediate paths
3. Check whether files have certain properties or match certain patterns:

1 loop over dirs

# create a directory and go there
cd ~/tmp_playground

# loop over raw, recon, analysis in each sub
for d in */*; do
  pushd ${d}
  echo -e "----------------- \n-directory name is ${d}"
  # eg count files using wordcount of ls -1 output
  echo -e "there are $(ls -1 * | wc -l) files"
  # now loop over each file:
  for f in *; do
    echo "the filename is ${f}; in dir: ${d}" ;
  done
    
  popd
done

If you need to also have a number / index, then you need some more bash magic see eg. this stackoverflow thread

2 loop over files w/ subdirs

# loop over files in raw, recon, analysis in each sub

But only use the `nii.gz` files:

echo -e 'what the ${f}'
  
for f in */{raw,recon,analysis}/*.nii.gz; do
  echo -e "${f}" # double quotes!
  # myfunction ${f} 
done
  

3 if you want to check filename properties

worth checking out regexp or case statements in bash. See eg [this stackoverflow answer]`(https://stackoverflow.com/a/39517909/4961292)

for more complicated stuff, awk or sed onliners might help, but they all require knowing something about the structure of the filenames, eg SUB-xxx_CONDITION-blabla.nii.gz, for which you then need to figure out a regular expression. maybe `"SUB-(\d{3})_CONDITION-([a-zA-Z]*)"

You can check out the logic for this particular regular expression at this website - NB... there are different flavours so worth checking this works with your scripting language of choice 😀