Asterisk voicemail mailcmd script for VM transcription

sendmail-bluemix

#!/bin/sh

# sendmail-bluemix
# current verison of this script: https://gist.github.com/lgaetz/2cd9c54fb1714e0d509f5f8215b3f5e6
#
#
# Original source created by N. Bernaerts: https://github.com/NicolasBernaerts/debian-scripts/tree/master/asterisk
# modified per: https://jrklein.com/2015/08/17/asterisk-voicemail-transcription-via-ibm-bluemix-speech-to-text-api/
#
#
# Notes: This is a script modified from the original to work with a FreePBX Distro PBX so that email notifications sent from
#        Asterisk voicemail contain a speech to text trasncription provided by IBM Bluemix
#
# License: There are no explicit license terms on the original script or on the blog post with modifications
#          I'm assumig GNU/GPL2+ unless notified otherwise by copyright holder(s)
#
# Usage: copy this file to /usr/sbin/ set ownership to asterisk:asterisk and make it executable.
#        In the [general] section of /etc/asterisk/voicemail.conf set mailcmd=/usr/sbin/sendmail-bluemix
#        This script also uses dos2unix, ensure it is executable by the asterisk user (chmod 777)
#        Specify your IBM Bluemix API key and URL on the lines noted below
#
# Version History:
#              2017-09-17 Initial commit by lgaetz, working but unpolished
#              2017-09-18 Original attachment added to email
#              2017-09-18 fixed problem with email without attachments
#              2020-04-16 COVID-19 edition - updated to use API key using method from github user @jtsage

# Specify IBM Bluemix API credentials
# follow instructions here to get key and url https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-gettingStarted
# key is long string separated by underscore(s). API_OPTS assumes US english, but there are many other language options available
# see full API reference at: https://cloud.ibm.com/apidocs/speech-to-text
API_KEY="xxxxxxxxx_xxxxxxxxxxxxxxx_xxxxxxxxxxxxxxxxxx"
API_URL="https://api.us-south.speech-to-text.watson.cloud.ibm.com/instances/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"
API_OPTS="/v1/recognize?model=en-US_NarrowbandModel"

# set PATH
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

# save the current directory
pushd .
 
# create a temporary directory and cd to it
TMPDIR=$(mktemp -d)
cd $TMPDIR
 
# dump the stream to a temporary file
cat >> stream.org
 
# get the boundary
BOUNDARY=$(grep "boundary=" stream.org | cut -d'"' -f 2)
 
# if mail has no boundaries, assume no attachment
if [ "$BOUNDARY" = "" ]
	
then
	# send the original stream
	mv stream.org stream.new
else
	# cut the original stream into parts
	# stream.part  - header before the boundary
	# stream.part1 - header after the bounday
	# stream.part2 - body of the message
	# stream.part3 - attachment in base64 (WAV file)
	# stream.part4 - footer of the message
	awk '/'$BOUNDARY'/{i++}{print > "stream.part"i}' stream.org
 
	# cut the attachment into parts
	# stream.part3.head - header of attachment
	# stream.part3.wav.base64 - wav file of attachment (encoded base64)
	sed '7,$d' stream.part3 > stream.part3.wav.head
	sed '1,6d' stream.part3 > stream.part3.wav.base64

	# convert the base64 file to a wav file
	dos2unix -o stream.part3.wav.base64
	base64 -di stream.part3.wav.base64 > stream.part3.wav
 
 	# Send WAV to Watson Speech to Text API. Must use "Narrowband" (aka 8k) model since WAV is 8k sample
	# Send WAV to Watson Speech to Text API. Must use "Narrowband" (aka 8k) model since WAV is 8k sample
	CURL_OPTS=""
	curl -s $CURL_OPTS -k -u "apikey:$API_KEY" -X POST \
		--limit-rate 40000 \
		--header "Content-Type: audio/wav" \
		--data-binary @stream.part3.wav \
		"$API_URL$API_OPTS" 1>audio.txt

	# Extract transcript results from JSON response
	TRANSCRIPT=`cat audio.txt | grep transcript | sed 's#^.*"transcript": "##g' | sed 's# "$##g'`
  
	# generate first part of mail body, converting it to LF only
	mv stream.part stream.new
	cat stream.part1 >> stream.new
	sed '$d' < stream.part2 >> stream.new

	# beginning of transcription section
	echo "--- Automated transcription result ---" >> stream.new

	# append result of transcription
	echo "$TRANSCRIPT" >> stream.new

	# end of message body
	tail -1 stream.part2 >> stream.new

	# add orig attachment  
	cat stream.part3 >> stream.new

	# append end of mail body, converting it to LF only
	echo "" >> stream.tmp
	echo "" >> stream.tmp
	cat stream.part4 >> stream.tmp
	dos2unix -o stream.tmp
	cat stream.tmp >> stream.new
 
fi
 
# send the mail thru sendmail
cat stream.new | sendmail -t
 
# go back to original directory
popd
 
# remove all temporary files and temporary directory
rm -Rf $TMPDIR

I have found this to be an awesome tool, but IBM Watson changed its authentication and I wanted to post somewhere - since this took time to figure out - as to how to modify this code to work with the new authentication.

On line 73, Rather than using
curl -s $CURL_OPTS -k -u $API_USERNAME:$API_PASSWORD -X POST \
change it to this:
curl -s $CURL_OPTS -k -u "apikey:{API KEY}" -X POST \

Your API key is accessible from the Bluemix Console Dashboard.

I modified this script to work with Google Cloud Speech API, and also convert the attachment to mp3 in the process:
https://gist.github.com/tony722/7c6d86be2e74fa10a1f344a4c2b093ea

Following the link above, and pulling in the original for MP3 conversion as well:

https://gist.github.com/jtsage/f38151b43debb6bd90ebaa4f0ab2687f

TODO: difficult to debug at present, need to log everything to /var/log/asterisk/freepbx.log

Recently IBM sent email that with the subject:
"URGENT ACTION REQUIRED: watsonplatform.net endpoint change" after February 12, 2021.

I simply added new script, created credentials, copied them and all the answers were there. I swapped Mail command in Voicemail settings in Freepbx.

Worked like a charm! Thank you so much.

Good luck.

Previously had VM to MP3 working fine -- so decided to try and upgrade to transcription
I followed the instructions and updated the script (as per jtsag) as well as the mailcmd

Issue is that when I receive an email -- VM is attached as MP3 -- however at the bottom I just see --- Automated transcription result ------ but no actual transcription of the message?

Can you help me please

thanks

TODO: difficult to debug at present, need to log everything to /var/log/asterisk/freepbx.log

Because I've not done it in ages, does anybody have a quick howto on dumping shell output to the appropriate log channel for this to happen? Working with @atoporovsky to address his issue, I'd like to hit it in one go.

Not having any luck getting this to work. Can I setup a log file to see what is wrong?

@edlentz, did you ever get anywhere with this? I used @jtsage's sendmail.asterisk which includes an IBM error log with each voicemail. I've not been able to solve the 500 errors that are occurring, however.

May need changes to this scrip to use the new stt models:
https://medium.com/ibm-watson-speech-services/watson-speech-to-text-how-to-plan-your-migration-to-the-next-generation-models-6b10605b3bc5

Haha. good timing, I too just got the annoucement from watson.

Many STT models being deprecated 2022-09-15 including "en-US_NarrowbandModel" which this script currently relies on. Needs to be updated, further reading here.

Just change the API_OPTS="/v1/recognize?model=en-US_NarrowbandModel" line to API_OPTS="/v1/recognize?model=en-US_Telephony", and it works fine.
See here: https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-migrate#models-migrate-step1
Also for the non-English users, here is a list of speech models: https://cloud.ibm.com/docs/speech-to-text?topic=speech-to-text-models-ng#models-ng-telephony