Skip to content

Instantly share code, notes, and snippets.

@Winterhuman
Last active October 15, 2025 18:49
Show Gist options
  • Save Winterhuman/21d7b148db40ff041f397b07a7aafb83 to your computer and use it in GitHub Desktop.
Save Winterhuman/21d7b148db40ff041f397b07a7aafb83 to your computer and use it in GitHub Desktop.
A script for finding the smallest lossless encoding of a PNG or GIF input image (that I know of).
#!/usr/bin/env -S unshare --pid --mount-proc --kill-child --map-root-user /bin/sh
# Licensed under the Zero-Clause BSD terms: https://opensource.org/license/0bsd
## Requires: oxipng & gifsicle
## Optional: pngquant, cwebp & gif2webp, gif2apng, and (perl-image-)exiftool
## -C: Fail if redirects try to overwrite an existing file.
## -e: Fail if any command fails (with exceptions).
## -u: Fail if an unset variable tries to be expanded.
## -f: No glob expansion.
set -Ceuf
perr() { printf "\a\033[1m\033[31m%b\033[0;39m" "$1" >&2; }
clear_prog() { printf "\033]9;4;0\007"; }
exit_clear() { clear_prog; exit 1; }
osc777() { printf "\a\033]777;notify;Sisyphus;%b\033\\" "$1"; }
nexit() { osc777 "$1"; exit_clear; }
pquit() { perr "$1"; nexit "$1"; }
pstat() { printf "\033[1m\033[34m%b\033[0;39m\033[1m%b\033[0;39m" "$1" "$2"; }
## `kill` handles background jobs, but `exit` is required for normal processes.
### The second trap handles unexpected signals, where a notification IS desired.
#### Trying to trap SIGTERM leads to a "Segmentation fault" error
trap 'kill "$$"; exit_clear' INT
trap 'kill "$$"; nexit "SIGQUIT or SIGABRT received! Was operating on: $1"' QUIT ABRT
help() {
cat << "HELP"
Usage: sisyphus [OPTION]... SRC DEST
Losslessly optimise PNGs & GIFs by all known means.
If DEST is omitted, DEST is set to '{SRC wo/ext}.new.ext'. Possible output formats
include: PNG, GIF, WEBP, and APNG.
Options:
-a, --all-oxi <bool> Optionally takes a boolean value. Controls whether to always,
or never, try all OxiPNG variations. If unset, heuristically
determine whether to try every OxiPNG variation.
-f, --force Overwrite existing destination files.
-m, --max-procs <int> Takes an integer greater than zero. Limits the number of
simultaneous processes. The default is '8'.
-n, --no-webp Do not use CWebP nor GIF2WebP.
-N, --no-apng Do not use APNG, aka. 'gif2apng'.
-q, --quiet Do not print messages to STDOUT, nor send notifications.
-r, --results <bool> Optionally takes a boolean value. Controls whether to print
the list containing all the trial outputs.
-s, --size <int> Takes an integer in bytes. Sets the minimum qualifying size.
If the best encoding is greater or equal to the minimum size,
then DEST is not created.
-h, --help Display this help message, and then exit.
Warning:
HDR SRC images will NOT be losslessly optimised!
HELP
exit 0
}
# Setup a private tmpfs for this script to use, which is what 'unshare' is for.
## 'nr_inodes' must be >= to 'max number of files + 1'
work="/tmp"
mount -t tmpfs -o nosuid,nodev,noexec,size=50%,nr_inodes=1349 sisyphus "$work"/ ||
pquit "Failed to overmount '$work/'!\n"
# Argument parsing
ALL_OXI=""
FORCE=""
MAX_PROCS="8"
NO_WEBP=""
NO_APNG=""
QUIET=""
RESULTS=""
SIZE=""
INTERNAL=""
SKIP=""
ARG_COUNT="0"
while [ "$ARG_COUNT" -lt "$#" ]; do
if [ -z "$SKIP" ]; then
case "$1" in
-a|--all-oxi)
case "${2:-""}" in
0|no|false) ALL_OXI="0"; shift ;;
1|yes|true) ALL_OXI="1"; shift ;;
*) ALL_OXI=1 ;;
esac
;;
-f|--force) FORCE="1" ;;
-m|--max-procs)
case "${2:-""}" in
[1-9]*) MAX_PROCS="$2"; shift ;;
*) pquit "No positive integer greater than zero was given for '-m|--max-procs'!\n" ;;
esac
;;
-n|--no-webp) NO_WEBP="1" ;;
-N|--no-apng) NO_APNG="1" ;;
-q|--quiet)
QUIET="1"
## '--quiet' already inhibits the list, however,
## '--results 0' also saves doing other steps
RESULTS="0"
## Functions referencing each other need to be
## defined again to be updated.
### This `perr()` omits `\a`. "$@" handles args
perr() { printf "\033[1m\033[31m%b\033[0;39m" "$1" >&2; }
nexit() { : "$@"; exit_clear; }
pquit() { perr "$1"; exit_clear; }
pstat() { : "$@"; }
;;
-r|--results)
case "${2:-""}" in
0|no|false|hide) RESULTS="0"; shift ;;
1|yes|true|show) RESULTS="1"; shift ;;
*) RESULTS=1 ;;
esac
;;
-s|--size)
case "${2:-""}" in
[0-9]*) SIZE="$2"; shift ;;
*) pquit "No positive integer was given for '-s|--size'!\n" ;;
esac
;;
-h|--help) help ;;
--_internal)
## DO NOT USE MANUALLY. For the APNG optimising
INTERNAL="1"
;;
--) SKIP="1" ;;
-*) pquit "'$1' is not a known option!\n" ;;
*) set -- "$@" "$1"; ARG_COUNT="$(( ARG_COUNT + 1 ))" ;;
esac
else set -- "$@" "$1"; ARG_COUNT="$(( ARG_COUNT + 1 ))"
fi
shift
done
# Validate arguments
## `[ cond ] && cmd` will carry over non-zero exit codes, so always use `||`
[ "$#" -le 2 ] || pquit "Too many arguments given!\n"
## Don't try WebP if `cwebp` & `gif2webp` aren't available
command -v cwebp >/dev/null || NO_WEBP=1
command -v gif2webp >/dev/null || NO_WEBP=1
## Don't try APNG if `gif2apng` isn't available
command -v gif2apng >/dev/null || NO_APNG=1
## Don't try `pngquant` if it's not available
NO_QUANT=""
command -v pngquant >/dev/null || NO_QUANT=1
## SRC
src_real="${1:?$(pquit "No source image file given!\n")}"
[ -s "$src_real" ] || pquit "'$src_real' is not a non-empty file!\n"
### Assign the SRC a file descriptor
exec 3<"$src_real"
src="/proc/self/fd/3"
src_real="$(realpath -- "$src_real")"
### Size detection
find_size() {
## Handle non-existent files by giving them arbitrarily huge sizes
if [ -f "$1" ]; then
wc -c <"$1" || pquit "Failed to find size of '$1'!\n"
else
printf "99999999999999"
fi
}
src_size="$(find_size "$src")"
### Minimum size target validation
check_number() {
[ "$1" != "" ] || return 0
num="$1"
oldnum="$num"
option="$2"
## Strip leading zeros, and avoid integer overflow
num="$(printf "%s" "$num" | sed "s/^0*//")"
## '0' would be trimmed to nothing by `sed`, so reset it
num="${num:-"0"}"
if ! printf "%d" "$num" >/dev/null 2>&1; then
perr "The value given for '$option' is not an integer, or far beyond the integer limit!\nValue: $oldnum\n"
return 1
fi
if [ "$num" -ne "$(printf "%s" "$num" | cut -c -12)" ]; then
perr "The value given for '$option' is over the integer limit!\nValue: $oldnum\n"
return 1
fi
printf "%d" "$num"
}
SIZE="$(check_number "$SIZE" "-s|--size")"
### Max processes validation
MAX_PROCS="$(check_number "$MAX_PROCS" "-m|--max-procs")"
### Mimetype detection
mime="$(file -Lib -- "$src" ||
pquit "Couldn't determine mimetype of '$src_real'!\n"
)"
mime="${mime%;*}"
## DEST
dest_path="${2:-""}"
dest_path="${dest_path%.*}"
dest_path="${dest_path:-${src_real%.*}.new}"
dest_path="$(realpath -- "$dest_path")"
dest_exists() {
[ -z "$FORCE" ] || return 0
[ -n "$1" ] || pquit "'dest_exists()' requires an argument!\n"
## Allow globbing for just this for-loop
set +f
for similar in "$1"*; do
if [ -e "$similar" ] && [ "${similar%.*}" = "$1" ]; then
perr "'$similar' shares a filename with '$1'!\n"
return 1
fi
done
set -f
}
dest_exists "$dest_path" ||
nexit "'$similar' shares a filename with '$dest_path'!\n"
pstat "Input:\t\t\t" "$src_real \033[37m($mime)\033[0;39m\n"
pstat "Output template:\t" "$dest_path.???\n"
[ -z "$SIZE" ] || pstat "Size target:\t\t" "$SIZE bytes\n"
# Optimisation
## In case of PNG
create_png_list() {
zc="0"
while [ "$zc" -le 12 ]; do
cat <<-ZCLIST
--zc=$zc
--zc=$zc --nb
--zc=$zc --nc
--zc=$zc --nb --nc
--zc=$zc --ng
--zc=$zc --nb --ng
--zc=$zc --nc --ng
--zc=$zc --nb --nc --ng
--zc=$zc --np
--zc=$zc --nb --np
--zc=$zc --nc --np
--zc=$zc --nb --nc --np
--zc=$zc --ng --np
--zc=$zc --nb --ng --np
--zc=$zc --nc --ng --np
--zc=$zc --nb --nc --ng --np
ZCLIST
zc="$(( zc + 1 ))"
done
## Don't duplicate `--zopfli` per `--zc={0..12}` level
cat <<-ZOPLIST
--zopfli --zi=255
--zopfli --zi=255 --nb
--zopfli --zi=255 --nc
--zopfli --zi=255 --nb --nc
--zopfli --zi=255 --ng
--zopfli --zi=255 --nb --ng
--zopfli --zi=255 --nc --ng
--zopfli --zi=255 --nb --nc --ng
--zopfli --zi=255 --np
--zopfli --zi=255 --nb --np
--zopfli --zi=255 --nc --np
--zopfli --zi=255 --nb --nc --np
--zopfli --zi=255 --ng --np
--zopfli --zi=255 --nb --ng --np
--zopfli --zi=255 --nc --ng --np
--zopfli --zi=255 --nb --nc --ng --np
ZOPLIST
}
create_png_oxi() {
cat <<-"OXI"
## `noexec` on `$work` prevents having `#!/bin/sh` here instead
set -Ceuf
## Redirect STDERR to STDOUT; `sh` will flip them around later.
### This doesn't mean things won't still write to STDERR though
2>&1
perr() {
printf "\a\033[1m\033[31m%b\033[0;39m" "$1"
printf "\033]777;notify;Sisyphus;%b\033\\" "$1"
}
## `exit 255` prevents `xargs` from processing any more line batches (the
## batches from `--max-procs`)
pquit() { perr "$1"; exit 255; }
options="$1"
safe="$2"
quant_src="$3"
smallest_known="$4"
total_lines="$5"
work="$6"
oxi_baseline_dest="$7"
src="/proc/self/fd/3"
tmp_list="/proc/self/fd/4"
## If '$quant_src' exists (is set), handle its command variants
if [ -n "${quant_src:-""}" ]; then
## If "quant" is in '$options', use '$quant_src' as '$src'
[ "$options" = "${options#*quant}" ] || src="$quant_src"
fi
## Construct the output path based on the options
out="$(printf "%b" "$options" | sed \
-e "s/--zc=/zc/g" \
-e "s/--zopfli[[:space:]]--zi=255/zop/g" \
-e "s/[[:space:]]--/ \+ /g"
)"
out="$(printf "$work/%s.png" "$out")"
## Check if the output (e.g. '$oxi_baseline_dest') already exists
[ ! -f "$out" ] || exit 0
## Execute `oxipng` with additional options & arguments.
### For whatever reason, `env` doesn't accept `oxipng "$options"`
env --split-string \
"oxipng $options" \
--alpha \
--strip "$safe" \
--out "$out" \
-- "$src" >/dev/null 2>&1 ||
pquit "'oxipng $options --opt max --alpha --strip $safe --out $out -- $src' failed!\n"
[ -f "$out" ] || pquit "'$out' is not a real file, but it should be!\n"
## If the size of '$out' isn't less than '$smallest_known', then fake its size
## so `wc` can read it, but, it'll take no space
discard() {
trial="$(realpath -- "$1")"
baseline="$(realpath -- "$2")"
[ -s "$trial" ] || return 0
[ -s "$baseline" ] ||
perr "'$baseline' does not exist, or has no content, but should!\n"
if cmp "$1" "$2" >/dev/null; then
## This handles the case where the output was truncated,
## but, it'd be ordered before '$baseline' by the end
ln --force --symbolic "$baseline" -- "$trial" \
2>/dev/null ||:
else
trial_size="$(wc -c <"$trial")"
if [ "$smallest_known" -le "$trial_size" ]; then
truncate --size=0 -- "$trial"
truncate --size="$trial_size" -- "$trial"
fi
fi
}
## Compare '$out' with '$quant_src' instead if it was the source
if [ -n "${quant_src:-""}" ]; then
discard "$out" "$quant_src"
else
discard "$out" "$oxi_baseline_dest"
fi
## Print the progress.
### This is an approximation based on the options' position in the command list,
### which means progress may jump backwards occasionally
line_index="$(sed "\|$options|q" "$tmp_list" | wc --lines)"
printf "\033]9;4;1;%d\007" "$(( ( line_index * 100 ) / total_lines ))"
OXI
}
try_oxi_vars() {
# Generate the command list
## `pngquant` strips APNGs, so skip it if '$safe' is being used
quant_src="$work/quant.png"
if [ "$safe" = "all" ] && [ -z "$NO_QUANT" ]; then
## `--quality 100-100` means '$quant_src' won't exist if it can't
## be created losslessly
pngquant --quality 100-100 --speed 1 --strip \
--output "$quant_src" -- "$src" ||
perr "Failed to create '$quant_src'!\n"
## If '$src' was already quantised beforehand, '$quant_src' will
## be identical to '$src', so check & remove any duplicates
! cmp -- "$src" "$quant_src" >/dev/null ||
rm "$quant_src" ||
perr "Failed to remove '$quant_src'!\n"
fi &
## Generate a list of all the commands to be passed to `xargs`.
### This is hidden later; see the comment after `[ -n "$quant_src" ]`
tmp_list="$work/list"
create_png_list >>"$tmp_list"
## Wait for `pngquant` to finish if it's still being executed
wait
## If '$quant_src' wasn't created, unset its variable
[ -f "$quant_src" ] || unset quant_src
## If '$quant_src' is set, append the list with the `pngquant` variants
if [ -n "${quant_src:-""}" ]; then
quant_list="$work/quant_list"
## `sed "..." file >>file` leads to input buffering issues.
### `cat` always prints in argument order; STDIN is appended
sed "s|^|quant > |" "$tmp_list" |
cat "$tmp_list" - >"$quant_list"
mv "$quant_list" "$tmp_list" ||
pquit "Failed to overwrite '$tmp_list' with '$quant_list'!\n"
fi
## `mv` can't handle file descriptors, so hide the file here instead.
### If `exec` is ran before `mv`, the old '$tmp_list' contents are left
### accessible at FD 4
exec 4<"$tmp_list"
rm "$tmp_list" || pquit "Failed to hide (remove) '$work/list'!\n"
tmp_list_real="$tmp_list"
tmp_list="/proc/self/fd/4"
## '$total_lines' is read by `$tmp_oxi`; it saves finding it repeatedly.
### `-gt 12` is just a safe guess; not a magic number
total_lines="$(wc --lines <"$tmp_list")"
[ "$total_lines" -gt 12 ] || pquit "'$tmp_list_real' was truncated!\n"
# Create the `xargs` script
tmp_oxi="$work/oxi"
:>"$tmp_oxi"
exec 5<"$tmp_oxi"
rm "$tmp_oxi" || pquit "Failed to hide (remove) '$tmp_oxi'!\n"
tmp_oxi="/proc/self/fd/5"
## Avoid using `sh -c ''` inside `xargs` by creating an external script.
### Trying to pass a HEREDOC to `sh` leads to `xargs` receiving the
### HEREDOC instead (which it only passes to `sh` once)
create_png_oxi >|"$tmp_oxi"
# Pass each command in the list to '$tmp_oxi'
## `>&2` redirects the STDOUT of the script to STDERR, while redirecting
## the STDERR of `xargs` to `/dev/null`. `exit 255` causes STDERR logs
xargs \
--max-procs "$MAX_PROCS" \
--delimiter "\n" \
--replace="%" \
-- sh "$tmp_oxi" \
"%" \
"$safe" \
"${quant_src:-""}" \
"$smallest_known" \
"$total_lines" \
"$work" \
"$oxi_baseline_dest" \
<"$tmp_list" >&2 2>/dev/null ||
exit_clear
}
src_is_png() {
# Create the CWebP & baseline OxiPNG outputs for the heuristics
cwebp_dest="$work/cwebp.webp"
oxi_baseline_dest="$work/zc12.png"
## `$safe` = "safe" is required when SRC is an APNG image.
### Controlled by the `--_internal` option, as set when self-executing
safe="all"
[ -z "$INTERNAL" ] || safe="safe"
if [ "$NO_WEBP" != 1 ]; then
## Passing `-q` or `-m` disables lossless mode
cwebp -quiet -mt -z 9 -alpha_filter best -o "$cwebp_dest" \
-- "$src" ||
perr "Passing '$src_real' through 'cwebp' failed!\n"
fi &
oxipng --opt max --strip "$safe" --alpha --out "$oxi_baseline_dest" \
-- "$src" >/dev/null 2>&1 ||
perr "Passing '$src_real' through 'oxipng' (zc12) failed!\n"
wait
## Skip trying the OxiPNG variants if explicitly requested
[ "$ALL_OXI" != "0" ] || return 0
# Heuristics
oxi_baseline_size="$(find_size "$oxi_baseline_dest")"
cwebp_size="$(find_size "$cwebp_dest")"
smallest_heuristic="$((
oxi_baseline_size < cwebp_size
? oxi_baseline_size
: cwebp_size
))"
## '$smallest_known' gets read by `try_oxi_vars`
smallest_known="$((
smallest_heuristic < src_size
? smallest_heuristic
: src_size
))"
## Use '$SIZE' as the minimum size to beat if given
[ -z "$SIZE" ] ||
smallest_known="$((
smallest_heuristic < SIZE
? smallest_heuristic
: SIZE
))"
# Try the OxiPNG variants if:
# 1. Explicitly requested.
# 2. Else, if the baseline OxiPNG's size is less or equal to 1KiB.
# 3. Else, if the input image is still the smallest known encoding.
if [ "$ALL_OXI" = 1 ] ||
[ "$oxi_baseline_size" -le 1024 ] ||
[ "$src_size" -le "$smallest_heuristic" ]; then
try_oxi_vars
fi
}
## In case of GIF
create_gif_list() {
## `gif2apng` doesn't have an output option; it has to be handled later
cat <<-GIFLIST
gif2webp -quiet -mt -min_size -m 6 -q 100 -metadata none -o "$gif2webp_dest"
gifsicle --optimize=3 --optimize=keep-empty --threads="$nproc" -o "${gifsicle_prefix}3.gif"
gifsicle --optimize=2 --optimize=keep-empty --threads="$nproc" -o "${gifsicle_prefix}2.gif"
gifsicle --optimize=1 --optimize=keep-empty --threads="$nproc" -o "${gifsicle_prefix}1.gif"
gif2apng -i20 -z0
gif2apng -i20 -z1
gif2apng -i20 -z2
gif2apng -i20 -z0 -kp
gif2apng -i20 -z1 -kp
gif2apng -i20 -z2 -kp
GIFLIST
}
create_gif_oxi() {
cat <<-"OXI"
## `noexec` on '$work' prevents using `#!/bin/sh` here
set -Ceuf
## Redirect STDERR to STDOUT; `xargs` will flip them around
2>&1
perr() {
printf "\a\033[1m\033[31m%b\033[0;39m" "$1"
printf "\033]777;notify;Sisyphus;%b\033\\" "$1"
}
## `exit 255` prevents `xargs` from processing any more line batches (the
## batches from `--max-procs`)
pquit() { perr "$1"; exit 255; }
cmdline="$1"
smallest_known="$2"
total_lines="$3"
work="$4"
src="/proc/self/fd/3"
tmp_list="/proc/self/fd/4"
## If '$cmdline' is for `gif2apng`, specify an output file
out=""
if [ "$cmdline" != "${cmdline%%gif2apng*}" ]; then
out="$(printf "%s.apng" "$cmdline" | sed \
-e "s|.*-z|$work/gif2apng + z|" \
-e "s/[[:space:]]-/ \+ /g"
)"
fi
## Execute '$cmdline' with additional arguments
if [ -z "$out" ]; then
env --split-string "$cmdline" -- "$src" >/dev/null ||
pquit "'$cmdline -- $src' failed!\n"
else
## `gif2apng` needs both SRC & DEST as arguments.
### It also only takes relative paths. Source:
### https://sourceforge.net/p/gif2apng/discussion/1022150/thread/8ec5e7e288
src_rel="$(realpath --relative-to="$PWD" -- "$src")"
out_rel="$(realpath --relative-to="$PWD" -- "$out")"
env --split-string "$cmdline" -- \
"$src_rel" "$out_rel" >/dev/null ||
pquit "'$cmdline -- $src $out' failed!\n"
## Strip leftover metadata from `gif2apng`
exiftool -overwrite_original_in_place -all= -- "$out" \
>/dev/null 2>&1 ||:
fi
## Print the progress.
### This is an approximation based on the options' position in the command list,
### which means progress may jump backwards occasionally
line_index="$(sed "\|$cmdline|q" "$tmp_list" | wc --lines)"
printf "\033]9;4;1;%d\007" "$(( ( line_index * 100 ) / total_lines ))"
OXI
}
src_is_gif() {
# Generate the command list
gif2webp_dest="$work/gif2webp.webp"
gifsicle_prefix="$work/gifsicle + o"
tmp_list="$work/list"
nproc="$(nproc)"
smallest_known="$src_size"
[ -z "$SIZE" ] || smallest_known="$SIZE"
create_gif_list >>"$tmp_list"
## Skip `gif2webp` and or `gif2apng` if requested.
### `sed --in-place` can't operate on file descriptor paths
[ -z "$NO_WEBP" ] || sed --in-place "/^gif2webp/d" "$tmp_list"
[ -z "$NO_APNG" ] || sed --in-place "/^gif2apng/d" "$tmp_list"
## `src_is_png()` override the file descriptors as needed
exec 4<"$tmp_list"
rm "$tmp_list" || pquit "Failed to hide (remove) '$work/list'!\n"
tmp_list_real="$tmp_list"
tmp_list="/proc/self/fd/4"
## '$total_lines' is read by `$work/oxi`; it saves calculating repeatedly
total_lines="$(wc --lines <"$tmp_list")"
## Not `-gt 9`, since WebP and APNG can both be excluded
[ "$total_lines" -gt 2 ] || pquit "'$tmp_list_real' was truncated!\n"
# Create the `xargs` script
tmp_oxi="$work/oxi"
:>"$tmp_oxi"
exec 5<"$tmp_oxi"
rm "$tmp_oxi" || pquit "Failed to hide (remove) '$tmp_oxi'!\n"
tmp_oxi="/proc/self/fd/5"
## Avoid using `sh -c ''` inside `xargs` by creating an external script.
### Trying to pass a HEREDOC to `sh` leads to `xargs` receiving the
### HEREDOC instead (which it only passes to `sh` once)
create_gif_oxi >|"$tmp_oxi"
# Pass each command in the list to '$tmp_oxi'
## `>&2` redirects the STDOUT of the script to STDERR, while redirecting
## the STDERR of `xargs` to `/dev/null`. `exit 255` causes STDERR logs
xargs \
--max-procs "$MAX_PROCS" \
--delimiter "\n" \
--replace="%" \
-- sh "$tmp_oxi" \
"%" \
"$smallest_known" \
"$total_lines" \
"$work" \
<"$tmp_list" >&2 2>/dev/null ||
exit_clear
## Remove the first progress bar now that we have the initial APNG images
clear_prog
# Optimise the `gif2apng` outputs with this script
## ...Unless `gif2apng` was explicitly excluded
[ -z "$NO_APNG" ] || return 0
## Allow globbing for just these for-loops
set +f
## De-duplicate the APNGs beforehand
for dup in /tmp/gif2apng*; do
[ -f "$dup" ] || continue
[ ! -L "$dup" ] || continue
for dup2 in /tmp/gif2apng*; do
[ "$dup" != "$dup2" ] || continue
! cmp "$dup" "$dup2" >/dev/null ||
ln --force --symbolic "$dup" "$dup2" 2>/dev/null ||
perr "Failed to symlink '$dup2' to '$dup'!\n"
done
done
for apngsrc in /tmp/gif2apng*; do
## Don't optimise duplicate SRC images; the output is the same
if [ -L "$apngsrc" ]; then
apngsrc_sym="$(readlink -- "$apngsrc")"
pstat ":: " "'$apngsrc' and '$apngsrc_sym' are identical. Skipping.\n"
continue
fi
[ -f "$apngsrc" ] || continue
apngdest="${apngsrc%.*}.oxipng.apng"
apngsrc_real="$apngsrc"
apngdest_real="$apngdest"
:>"$apngdest"
## Each `sisyphus` instance has its own '$work' directory, so
## pass the input & output as file descriptors
exec 6<"$apngsrc"
exec 7<"$apngdest"
apngsrc="/proc/self/fd/6"
apngdest="/proc/self/fd/7"
pstat ":: " "Optimising '$apngsrc_real'...\n"
## `--force` is required to write to '$apngdest'
"$0" --_internal --quiet --force --max-procs "$MAX_PROCS" \
--no-webp --all-oxi "${ALL_OXI:-"1"}" -- \
"$apngsrc" "$apngdest" ||:
## If '$apngdest' is still empty, '$apngsrc' is equivalent
if [ -f "$apngdest_real" ] && [ ! -s "$apngdest_real" ]; then
ln --force --symbolic "$apngsrc_real" -- "$apngdest_real" \
2>/dev/null ||
perr "Failed to symlink '$apngdest_real' to '$apngsrc_real'!\n"
fi
done
set -f
}
# File selection
case "$mime" in
"image/png") src_is_png ;;
"image/gif") src_is_gif ;;
*) pquit "Mimetype of '$src_real' is neither PNG nor GIF!\n" ;;
esac
## Allow globbing for just this for-loop
set +f
sizepathlist=""
for tmp_file in "$work"/*; do
## Tests for an existing file with a real size, which is false if
## '$apngdest' was left as an empty file by `sisyphus`
[ -s "$tmp_file" ] || continue
sizepathlist="$(find_size "$tmp_file")\t$tmp_file\n$sizepathlist"
done
set -f
## 1. Prefix lines with their length followed by a space (e.g. '20 8 bytes ...').
## 2. Sort the lines by their length values (aka. `sort -n`).
## 3. Use space as the delimiter (aka. '-d " "'), and remove everything from
## before the second field (e.g. '8 bytes ...').
## 4. Sort numerically, but for cases where the byte count is the same, preserve
## the ordering from the previous steps (aka. `sort -sn`).
orderedlist="$(
printf "%b" "$sizepathlist" |
awk '{ print length, $0 }' |
sort -n |
cut -d " " -f 2- |
sort -sn
)"
if [ "$RESULTS" = "1" ]; then
## The second sed pattern uses '|' delimiters since '$work' is a path
fancy_orderedlist="$(
printf "%b" "$orderedlist" |
sed "s/\t/ bytes\t/g;s|$work/||g;s/^/\t/g"
)"
pstat "Size order:\n" "$fancy_orderedlist\n\n"
fi
# Output selection
smallest="$(printf "%b" "$orderedlist" | head -n1 | cut -d " " -f 2-)"
smallest_size="$(find_size "$smallest")"
## Check for an improvement
[ "$smallest_size" -le "${SIZE:-"$smallest_size"}" ] ||
pquit "'$smallest' was equal to, or larger than, the minimum size target ($SIZE bytes).\n"
if [ "$smallest_size" -eq "$src_size" ]; then
pstat ":: " "'$src_real' is already optimal.\n"
[ -n "$QUIET" ] || osc777 "'$src_real' is already optimal."
exit 0
fi
[ "$smallest_size" -le "$src_size" ] ||
pquit "'$smallest' was larger than '$src_real'.\n"
## Symlinks also have an apparent size of '0', so exclude them from this check
if [ "$(stat --format "%b" -- "$smallest")" -le 0 ] && [ ! -L "$smallest" ]; then
pquit "'$smallest' was truncated at some point. Cannot use!\n"; fi
## Set '$final_dest', and check the path for existing files
final_dest="${dest_path}.${smallest##*.}"
if [ -e "$final_dest" ] && [ -z "$FORCE" ]; then
pquit "'$final_dest' already exists!\n"; fi
## If `--_internal` is used, then '$final_dest' needs to be a file descriptor
if [ -z "$INTERNAL" ]; then
cp -- "$smallest" "$final_dest" ||
pquit "Failed to copy '$smallest' to '$final_dest'!\n"
else
[ -L "$2" ] ||
pquit "'--_internal' was given, but the destination ('$2') is not
a file descriptor or doesn't exist!\n"
## You can't `cp` to a file descriptor, so write to it directly
cat "$smallest" >|"$2" ||
pquit "Failed to write '$smallest' to '$2'!\n"
fi
## Print the final statistics.
### "> " as in the "quant > ..." prefix of the filename
method="${smallest##*"> "}"
method="${smallest##*"/"}"
method="${method%.*}"
pstat "Final output:\t" "$final_dest \033[37m($method)\033[0;39m\n"
pstat "Size diff:\t" \
"$src_size bytes \033[37m->\033[0;39m \033[1m$smallest_size bytes\n"
clear_prog
[ -n "$QUIET" ] ||
osc777 "'$final_dest' is finished! Size diff: $src_size bytes -> $smallest_size bytes ($method)"
@TPS
Copy link

TPS commented Feb 26, 2025

Per the current source @ https://sourceforge.net/p/nikkhokkho/code/HEAD/tree/trunk/FileOptimizer/Source/cppMain.cpp#l2087, FO currently uses "apngopt, pngquant, PngOptimizer, TruePNG, pngout, oxipng, pngwolf, Leanify, ect, pingo, advpng, deflopt, defluff, deflopt" (not a typo, runs deflopt 2× based on testing) in the PNG toolchain, w/ configurable options via GUI & INI.

FO reportedly works fine under WINE & emulators. Perhaps you could compare that way?

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Managed to use Bottles to run FO, and... inconclusive? Here's what I'm noticing:

  1. Sisyphus can beat FO, and vice-versa, so neither is guaranteed to win unfortunately.
  2. From that cppMain.cpp file you linked, I'm noticing that oxipng -Z is always being used, however, Zopfli can actually lose to non-Zopfli compression (and also, there's no option permutation like I'm doing here, which can also help).
  3. pingo is able to beat cwebp sometimes, which is unfortunate since pingo is also a Windows-only executable, so this script is definitely missing out in those cases.

I haven't tested GIFs or APNGs yet, and I also need to update this script with my new APNG brute-forcing pipeline, but from the above results, I'm guessing that's going to be a mixed bag as well.

EDIT: I've updated Sisyphus with the new APNG stuff, along with some other changes.

@TPS
Copy link

TPS commented Feb 26, 2025

Thanks for the detailed comparison! If you've test files you'd be willing to make available that Sisyphus beats FO, that'd be much appreciated. 🙇🏾‍♂️

Pinging @javiergutierrezchamorro (the FO dev) to take note of above feedback.

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Here's some samples:


decipher-sisyphus

Sisyphus: 1824 bytes (zop.png. I threw these first two samples in to show that FO can beat Sisyphus)

decipher-fo

FO: 1823 bytes


dd-sisyphus

Sisyphus: 479 bytes (quant > zop + nb.png)

dd-fo

FO: 474 bytes


thisline-sisyphus

[FO >] Sisyphus: 413 bytes (zop.png. It's probably --opt max that makes the difference with these zop.png results)

thisline-fo

FO: 414 bytes


disc-sisyphus

Sisyphus: 534 bytes (zop.png)

disc-fo

FO: 537 bytes

disc-fo-sisyphus

FO > Sisyphus: 536 bytes (zop.png)


seat-sisyphus

[FO >] Sisyphus: 246 bytes (quant > zop.png. Regular zop.png matched FO, so pngquant did something here)

seat-fo

FO: 247 bytes


space-sisyphus

Sisyphus: 3396 bytes (zop.png)

space-fo

FO: 3398 bytes

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

I forgot to set the optimisation level to 9 in FO... one moment.

EDIT: Done

@TPS
Copy link

TPS commented Feb 26, 2025

Just to make sure I understand, just the last example beats FO (by 1 byte)? 🤔

@Winterhuman
Copy link
Author

Winterhuman commented Feb 26, 2025

Yeah, I had some other images going before I realised I forgot the level 9 setting; will edit with some of the other examples when I have time.

EDIT: Okay, so bad news. Pretty much every time Sisyphus yields a smaller result, it's because it uses --zopfli and --opt max, which FO doesn't. Except for that quant > zop.png result, where regular zop.png matched FO, so the extra palette randomisation that pngquant introduces can help on rare occasions.

But yeah, ever since I set the level in FO's settings correctly, I've been struggling to beat it; it's usually a tie or loss save for these samples. Welp.

@TPS
Copy link

TPS commented Feb 27, 2025

1 tip: I think you're pretty used to long processing times b/c of your script, but, if you've the spare processing power, multiple instances of FO work quite well on split file lists to speed up overall processing.

@TPS
Copy link

TPS commented Feb 27, 2025

Also, does running Sisyphus before/after FO-processed images improve anything?

@Winterhuman
Copy link
Author

Winterhuman commented Feb 27, 2025

Found these (I'll edit the previous samples if I find anything for them too):

general-fo

FO: 388 bytes

general-fo-sisyphus

FO > Sisyphus: 387 bytes (zop + nb.png. Regular zop.png matches FO, and every permutation of zop + *.png beats it by a single byte, so it's not just --opt max here)


balloon-fo

FO: 2987 bytes (FO > FO: Same size)

balloon-fo-sisyphus

FO > Sisyphus: 2956 bytes (zop.png. Probably --opt max again)

balloon-fo-sisyphus-fo

FO > Sisyphus > FO: 2955 bytes (???)

@TPS
Copy link

TPS commented Feb 27, 2025

Sometimes running FO multiple times on the same file (Shift+F5,▶️) back-to-back gives better results, too, so that's kind-of expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment