Read, store, and reproduce binary data, including nulls, in pure bash with no external executables or even subshells.

binary_read_tty.sh

#!/usr/bin/env bash
# Read binary data, including null bytes, from a serial port,
# in pure bash with no external executables nor even subshells,
# even though it's not possible to store a null in a shell variable.
# Uses globals to avoid forking subshells.

# Example, echo -e "\0\0\0" |read FOO  or  printf -v FOO '%b' "\0\0\0"
# FOO will not contain any 0x00 bytes.
# Same goes for mapfile/readarray.

# What you CAN do is read and store every byte other that null normally,
# and you can DETECT when you read a null and record that fact instead of the null byte itself.

# Read bytes one at a time from a file or device node until the data stops.
# Store each byte as a hex pair in it's own array element.
#
# When "read" gets a null byte, the variable will be empty, but it will
# also be empty when there was no data to read.
#
# The exit value from "read" discloses when a null-byte was consumed as a delimiter
# vs when there was no data and the read command timed out.
# $?=0 we got a non-null byte normally
#    1 we got a 0x00, and ate it as a delimiter, x is empty but we know to use '00' for this byte
#  142 we timed-out, x is empty, and we know to terminate the loop and NOT fill x='00'

read_tty () {
	local -i e ;local LANG=C x ;HEX=()
	while : ;do
		IFS= read -t0.1 -r -d'' -n1 -u3 x ;e=$?
		((e>1)) && break
		((e)) && x='00' || printf -v x '%02x' "'$x"
		HEX+=($x)
	done
}

###############################################################
# main
typeset -a HEX=()
LANG=C PORT="/dev/ttyUSB0"
exec 3<>"$PORT"
stty -F "$PORT" 19200 raw pass8 clocal cread -echo crtscts -ixon -ixoff -ixany

read_tty

# We now have an array HEX[], where each element is a hex pair representing one byte, including nulls.
# The binary data can be reconstituted with printf.

# copy the array to a string plus one leading space
# produces x=" 63 61 71..."
x=" ${HEX[*]}"

# substitute all the spaces with "\x"
# produces "\x63\x61\x71..."
# printf that with \x## interpretation
# produces "cat..."
printf '%b' "${x// /\\x}" >capture.bin

# To reproduce the original binary data, the printf output must go to a file
# or a device node or piped to another process etc.

Instead of reading byte by byte, you can let read read up until the next NUL-byte, consuming it, but obviously not storing it in the variable. This makes it more efficient:

# golfed version
bashcat(){ z=\\0;while [ $z ];do LANG=C IFS= read -t9 -rd '' l||z=;printf "%s$z" "$l";done;}

Albeit faster, on my system, this is 270x slower than plain cat when run through pv and using /dev/urandom as a source and /dev/null as a sink.

Nice.

Though the point was not merely to cat, but to store, examine, manipulate, as well as output.

This optimization could still apply but means storing an array of chunks instead of an array of bytes, and then it becomes a bit more laborious to work on the data. How do you pluck out arbitrary blocks at arbitrary offsets? You have to walk all the chunks from the start and count bytes along the way since they have random lengths. A "read" would start at a random offset within a random chunk number, span a random number of chunks, and end at a random offset within a chunk. It's all doable, not even much code, just inconvenient. Though I guess a little get & put function that wraps up the loop & count would make it not so bad.

Still nice, thanks.