Skip to content

Instantly share code, notes, and snippets.

@luis-puhl
Last active April 11, 2019 12:17
Show Gist options
  • Select an option

  • Save luis-puhl/e31b74c80aa12478aa7c0d3690b18a56 to your computer and use it in GitHub Desktop.

Select an option

Save luis-puhl/e31b74c80aa12478aa7c0d3690b18a56 to your computer and use it in GitHub Desktop.
Search, Seek and Destroy: All images with only one color. For the rest, turn white to transparent. Do it in 60 processes.
#!/bin/bash
# start time, to know how long it took
start=`date +%s.%N`;
# counter, to know how many files we processed
i=0;
# Max concurrent processes
waitevery=60;
# find some files ended with '.png'
# Find will be ran syncronous, single thread, be patient
mydir=$(mktemp -d "${TMPDIR:-/tmp/}$(basename $0).XXXXXXXXXXXX")
echo "Using temp file '$mydir/clean-tiles.list'";
find $1 -name '*.png' > $mydir/clean-tiles.list;
# for each file in the `find`
file_count=$(cat $mydir/clean-tiles.list | wc -l)
echo "Processing $file_count files";
for source_png in `cat $mydir/clean-tiles.list`;
do
# (
# (
(
# find the file's color (%k) count
color_count=$(identify -format %k $source_png 2>/dev/null);
identify_sucess=$?;
# if [the above command had sucess] and [the $color_count is more than one]
if [ $identify_sucess -eq 0 ] && [ $color_count -gt 1 ];
then (
# if the image has more colors, transform the `white` to `trasnparent`
# keep the file path
convert $source_png -transparent white $source_png 2>&1 >/dev/null;
echo "[white to alpha] $source_png" >>clean-tiles.log;
);
else (
# trash-it
[ $identify_sucess -ne 0 ] && echo "[not a image] $source_png" >>clean-tiles.log;
[ $identify_sucess -eq 0 ] && [ $color_count -le 1 ] && echo "[one color] $source_png" >>clean-tiles.log;
# find the path, withnout the filemane
trashed_path=trashed/`expr "$source_png" : '\(.*\/\).*\.png'`;
# make the dir with the `trashed/` prefix but keeping the dir structure
mkdir -p $trashed_path;
# move-it to trash
mv --backup=t "$source_png" $trashed_path;
);
fi;
) & # fork this task
# ) # supress fork messages
# ) 2>&1 >/dev/null;
# if the max concurrent processes value has been reach, wait for them all to finish
(( i++%waitevery==0 )) && wait && echo -ne "\r$i/$file_count ";
done;
rm $mydir/clean-tiles.list
# wait the stdout to clear, some processes may still be running
wait;
end=`date +%s.%N`;
elapsed=`echo "$end - $start" | bc -l`;
echo "runtime $elapsed --- processed $i tiles";
echo "waiting to write log";
sleep 5;
echo "runtime $elapsed --- processed $i tiles" >>clean-tiles.log;
echo "" >>clean-tiles.log;
@element-code
Copy link

element-code commented Apr 11, 2019

I changed the code a bit, to trash partially white tiles as well; instead of replacing white with transparent (and allow 2 colors because QGIS sometimes renders light grey instead of plain white).

# find the file's color (%k) count
color_count=$(identify -format %k $source_png 2>/dev/null);
identify_sucess=$?;
width=0;
height=0;

# when [the above command had sucess] and [the $color_count is more than 2] read width and height after trim
if [ $identify_sucess -eq 0 ] && [ $color_count -gt 2 ];
then
  # if the image has more colors check if we can trim whitespace and trash it if
  read width height <<< $(convert $source_png   -flatten -fuzz 1% -trim +repage -format "%w %h" info:)
fi;

# if [less than two colors] OR [more than two colors but trimmed]
if ( [ $color_count -le 2 ] || ( [ $color_count -gt 2 ] && ( [ $width -lt 256 ] || [ $height -lt 256 ] ) ) );
then ( # trash-it
  #it is not a image
  [ $identify_sucess -ne 0 ] && echo "[not a image] $source_png" >> clean-tiles.log;
  # its white(ish)
  [ $identify_sucess -eq 0 ] && [ $color_count -le 2 ] && echo "[withe(ish) tile] $source_png" >> clean-tiles.log;
  # its (partially) trimmed
  [ $identify_sucess -eq 0 ] && [ $color_count -gt 2 ] && printf "[trimmed %03d %03d] %s\n" $width $height $source_png >> clean-tiles.log;

  # find the path, without the filemane
  trashed_path=trashed/`expr "$source_png" : '\(.*\/\).*\.png'`;
  # make the dir with the `trashed/` prefix but keeping the dir structure
  mkdir -p $trashed_path;

  # move-it to trash
  mv --backup=t "$source_png" $trashed_path;
);
fi;
) & # fork this task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment