Screenshot OCR for text extraction on Hyprland

ocr.sh

#!/bin/bash 
# Dependencies: tesseract-ocr imagemagick wl-clipboard hyprshot

die(){
  notify-send "$1"
  exit 1
}
cleanup(){
  [[ -n $1 ]] && rm -r "$1"
}

SCR_IMG=$(mktemp -d) || die "failed to create tmpdir"

  # shellcheck disable=SC2064
trap "cleanup '$SCR_IMG'" EXIT

hyprshot -m region -f scr.png --silent -o $SCR_IMG || die "failed to take screenshot"
mogrify -modulate 100,0 -resize 400% "$SCR_IMG/scr.png"  || die "failed to convert image"
tesseract "$SCR_IMG/scr.png" "$SCR_IMG/scr" &> /dev/null || die "failed to extract text"
wl-copy < "$SCR_IMG/scr.txt" || die "failed to copy text to clipboard"
notify-send "Text extracted from image" || die "failed to send notification"
exit

And I spot a race condition where mogrify will mostly process the image before hyprshot even writes it. Causing weird errors like:

~/Bin took 2s 
❯ ./ocr
mogrify: improper image header `/tmp/tmp.eOQUsHKCnU/scr.png' @ error/png.c/ReadPNGImage/3941.

~/Bin took 3s 
❯ ./ocr
mogrify: Expected 8192 bytes; found 8055 bytes `/tmp/tmp.tiJrMETrbm/scr.png' @ warning/png.c/MagickPNGWarningHandler/1525.
mogrify: Read Exception `/tmp/tmp.tiJrMETrbm/scr.png' @ error/png.c/MagickPNGErrorHandler/1491.

Adding a sleep 1 works for me.

Works great, I changed from sleep 1 to sleep 0.1 to make it a bit faster. I also had to remove the || die ... from the hyprshot.

Finally ended up with:

#!/bin/bash 
# Dependencies: tesseract-ocr imagemagick wl-clipboard hyprshot

die(){
  notify-send "$1"
  exit 1
}
cleanup(){
  [[ -n $1 ]] && rm -r "$1"
}

SCR_IMG=$(mktemp -d) || die "failed to create tmpdir"

  # shellcheck disable=SC2064
trap "cleanup '$SCR_IMG'" EXIT

hyprshot -m region -f scr.png --silent -o $SCR_IMG
sleep 0.1
mogrify -modulate 100,0 -resize 400% "$SCR_IMG/scr.png"  || die "failed to convert image"
tesseract "$SCR_IMG/scr.png" "$SCR_IMG/scr" &> /dev/null || die "failed to extract text"
wl-copy < "$SCR_IMG/scr.txt" || die "failed to copy text to clipboard"
notify-send "Text extracted from image" || die "failed to send notification"
exit

@golgor I use an older version of hyprshot, maybe it’s related ?

@CapMousse, might be. I am using 1.3.0. It shouldn't return exit code 1, so might be a bug introduced in later version. It works fine with the minor changes. Thanks a lot for making this public :D

#!/usr/bin/env bash
# Dependencies: tesseract imagemagick wl-clipboard hyprshot

die() {
  notify-send "$1"
  exit 1
}
cleanup() {
  [[ -n $1 ]] && rm -r "$1"
}

SCR_IMG=$(mktemp -d) || die "failed to create tmpdir"

# shellcheck disable=SC2064
trap "cleanup '$SCR_IMG'" EXIT

hyprshot -m region -f scr.png --silent -o $SCR_IMG
sleep 0.1
mogrify -modulate 100,0 -resize 400% "$SCR_IMG/scr.png" || die "failed to convert image"
tesseract "$SCR_IMG/scr.png" "$SCR_IMG/scr" &>/dev/null || die "failed to extract text"
wl-copy <"$SCR_IMG/scr.txt" || die "failed to copy text to clipboard"
dunstify "Text extracted from image" || die "failed to send notification"
exit

Better shbang and changed tesseract-ocr to tesseract to arch and nixos.