Quick shell script to decrypt client-side encrypted file using QNAP HBS *Sync* jobs. Only work with version 2 files - Doesn't work with QNAP HBS Backup jobs!

qnap-clientenc-decrypt.sh

#!/bin/sh

set -ue

[ "$#" -lt 2 ] && { echo "Syntax: $0 <filename> <key>"; exit 1; } 

filename="$1"
key="$2"

# Compose the key by repeating the user input until we have 32 characters (64 hex digits) 
openssl_key=""
while [ "$(printf '%s' "${openssl_key}" | wc -m)" -lt 64 ]; do
  openssl_key=${openssl_key}$(printf '%s' "${key}" | od -An -tx1 | tr -d '\ \n')
done
openssl_key=$(printf '%s' "${openssl_key}" | cut -c1-64)

# Determine if the cleartext has been compressed
compressed="$(dd if="${filename}" bs=1 skip=9 count=1 2>/dev/null | od -An -tx1 | tr -d '\ \n')"
[ "${compressed}" -ne 0 ] && echo "Compressed, use bzip2 to decompress" >&2

# Decipher the header
header="$(dd if="${filename}" bs=1 skip=16 count=64 2>/dev/null | \
  openssl aes-256-ecb -d -K "${openssl_key}" -nopad | od -An -tx1 | tr -d '\ \n')"

# Extract key and iv from the header
ckey="$(printf '%s' "${header}" | cut -c17-80)"
salt="$(printf '%s' "${header}" | cut -c81-112)"

# Decipher the file
dd if="${filename}" bs=1 skip=80  2>/dev/null | openssl aes-256-cbc -d -K "${ckey}" -iv "${salt}"

Okay, so there is one minor version difference in the OS but as you said that shouldn't make the difference. I checked the settings and I deactivated QDedup. Maybe this is what you meant with compression because I couldn't find any other compression option? Password is for testing just 123456 so it is not exceeding 32 char limit.

I copied your script in a decrypt.sh and did the following with a hello_world.txt with the Content "Hello World!" which is encrypted with 123456 passphrase.

cat hello_world.txt | base64
SGVsbG8gV29ybGQh

# Then Qnap does the Backup with encryption Job

./decrypt.sh hello_world.txt 123456
bad decrypt
4651892396:error:06FFF064:digital envelope routines:CRYPTO_internal:bad decrypt:/System/Volumes/Data/SWE/macOS/BuildRoots/e90674e518/Library/Caches/com.apple.xbs/Sources/libressl/libressl-56.60.2/libressl-2.8/crypto/evp/evp_enc.c:521:

# the encrypted file
cat hello_world.txt 
Kʔr^?1?W?7?b?s??z[)H)Cޠ??????{????}?m?~=??????}{u?A?1V???t??0Sż(?????>g?????

# the encrypted file content base64 encoded
cat hello_world.txt | base64
S8qUcl6DHDEBAAAAAAAAANJX8jemYtdz9MF6WylIKUPeoBOCngSQHT/C2nsF5avdyfN9wG3hfj2AhoG3jsl9e3WuQe8xVpm36nTv9DBTxbwo//YEv6EDnj5npIqrlsMK

I added the base64 stream of the binary encrypted files and also of the source file so to make sure it is reproducable on a binary accurate level on other machines.

Thanks for any hints.

It's very hard to tell what's going on...
I need to do some testing. One thing that looks different is that I am doing a sync job, with encryption, while you are doing a backup.
The "external" header seems to match. But there is an additional header, from which I extract key and iv, which may be different...

Will look into it later :)

@ceelian do you mind trying https://github.com/alexkazik/qnap-decrypt ?

I did some testing.
I think that the encryption key used by QNAP HSB Backup is derived from the user password with a different algorithm.
From Sync jobs, the password used to decrypt the header is a 64 bytes long concatenation of the user password.

Maybe the backup job concatenates something to the user password... but honestly is hard to tell!

@dguerry thank you, I was 2 weeks AFK. Will try soon and let you know. Thank you a lot for the investigations.