This bash script downloads the current Fritz Box event log as JSON file and as TXT file

download-my-fritz-box-logs.sh

#!/bin/bash

#
# This bash script downloads the current Fritz Box event log as JSON file and as TXT file.
#
# The script was tested successfully on MacOS Sonoma 14.1.1 with Fritz!Box 7590 AX V7.57
# 'jq' and 'curl' were installed via MacPorts
#
# The login procedure was implemented according to https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/AVM_Technical_Note_-_Session_ID.pdf
#   or https://avm.de/fileadmin/user_upload/Global/Service/Schnittstellen/Session-ID_deutsch_13Nov18.pdf
#
# The selected Fritz!Box user needs the permissions to view/edit the configuration.
# To get the user name you can select "use username and password" on the Fritz!Box login page, then you
# can see the user name in the combo box. The best way is to create an own user with a strong password
# and only the needed permissions and no access from internet.
#
# The Password must not use Unicode codepoints >255 because they need to handled specially, see AVM links above for details
# If you use those characters, replace them by '.' in the variable FRITZ_PASSWORD.

FRITZ_USERNAME="<replace-with-your-fritz-box-username>"
FRITZ_PASSWORD="<replace-with-your-fritz-box-password>"
BASEURL="http://fritz.box"

set -e -o pipefail
if ! which >/dev/null 2>&1 iconv ; then
  echo 1>&2 "Error: 'iconv' is not installed"
  exit 1
fi
if ! which >/dev/null 2>&1 curl ; then
  echo 1>&2 "Error: 'curl' is not installed"
  exit 1
fi

# get current session id and challenge
resp=$(curl -s "$BASEURL/login_sid.lua")

if [[ "$resp" =~ \<SID\>(0+)\</SID\> ]] ; then
  # SID=0 => not logged in
  if [[ "$resp" =~ \<BlockTime\>([0-9a-fA-F]+)\</BlockTime\> ]] ; then
    BLOCKTIME="${BASH_REMATCH[1]}"
    if [[ "${BLOCKTIME}" -gt "0" ]] ; then
      echo 1>&2 "BlockTime=${BLOCKTIME}, sleeping until finished"
      sleep $(( ${BLOCKTIME} + 1 ))
    fi
  fi
  if [[ "$resp" =~ \<Challenge\>([0-9a-fA-F]+)\</Challenge\> ]] ; then
    CHALLENGE="${BASH_REMATCH[1]}"
    
    # replace all Unicode codepoints >255 by '.' because of a bug in the Fritz!Box.
    # Newer Fritz!Box OS versions don't allow to enter such characters.
    # This requires that the locale environment is setup to UTF8, but on my Mac this doesn't work
    #FRITZ_PASSWORD=$(export LC_CTYPE=UTF-8 ; echo "${FRITZ_PASSWORD}" | sed $'s/[\u0100-\U0010ffff]/./g')
    #FRITZ_PASSWORD=$(export LC_CTYPE=UTF-8 ; echo "${FRITZ_PASSWORD}" | tr $'\u0100-\U0010ffff' '.')
        
    if which >/dev/null 2>&1 md5 ; then
      MD5=$(echo -n "${CHALLENGE}-${FRITZ_PASSWORD}" | iconv --from-code=UTF-8 --to-code=UTF-16LE | md5 )
    elif which >/dev/null 2>&1 md5sum ; then
      MD5=$(echo -n "${CHALLENGE}-${FRITZ_PASSWORD}" | iconv --from-code=UTF-8 --to-code UTF-16LE | md5sum | cut -f1 -d ' ')
    else
      echo 1>&2 "Error: neither 'md5' nor 'md5sum' are installed"
      exit 1
    fi
    RESPONSE="${CHALLENGE}-${MD5}"
    resp=$(curl -s -G -d "response=${RESPONSE}" -d "username=${FRITZ_USERNAME}" "${BASEURL}/login_sid.lua")
  fi
fi

if ! [[ "$resp" =~ \<SID\>(0+)\</SID\> ]] && [[ "$resp" =~ \<SID\>([0-9a-fA-F]+)\</SID\> ]] ; then
  # either SID was already non-zero (authentication disabled) or login succeeded
  SID="${BASH_REMATCH[1]}"
  #echo 1>&2 "SessionID=$SID"
  
  TIMESTAMP=$(date +"%Y-%m-%d_%H-%M-%S")
  OUTPUT_FILE=log-${TIMESTAMP}
  OUTPUT_FILE_JSON=json/${OUTPUT_FILE}.json
  OUTPUT_FILE_TXT=txt/${OUTPUT_FILE}.txt

  mkdir -p json txt
  curl -s -d "xhr=1&lang=de&page=log&sid=${SID}" -H "Content-Type: application/x-www-form-urlencoded" "${BASEURL}/data.lua" > "${OUTPUT_FILE_JSON}"
  if which >/dev/null 2>&1 jq ; then
    jq -r '.data.log[] | .date + " " + .time + " " + .group + " " + ( .id | tostring) + " " + .msg'  < "${OUTPUT_FILE_JSON}" > "${OUTPUT_FILE_TXT}"
    #cat ${OUTPUT_FILE_TXT}
    echo ${OUTPUT_FILE_TXT}
  else
    echo 1>&2 "Warning: 'jq' is not installed, cannot create text version of logfile"
    echo ${OUTPUT_FILE_JSON}
  fi

else
  echo 1>&2 "ERROR: login failed, response: $resp"
  exit 1
fi

# vim: set ts=2 sw=2 expandtab:

now checking if the required tools are installed. Auto fallback from md5 to md5sum

Thanks a lot for this code, I re-used it in https://github.com/schlomo/fritzbox-ntopng-docker

What I noticed is that set -e -o pipefail made the script abort without error message if one of the curl calls failed

set -e -o pipefail is the safe default to stop the script and not continue with empty results.

Tested with FRITZ!Box 6490 Cable and Cygwin, works! Thx!
Using md5sum.

Required cygwin packages: coreutils, jq, libiconv, curl,

I received an error when running the script on MacOS Sequoia:

./download-my-fritz-box-logs.sh jq: error: syntax error, unexpected ']' (Unix shell quoting issues?) at <top-level>, line 1: .data.log.[] | .date + " " + .time + " " + .group + " " + ( .id | tostring) + " " + .msg jq: 1 compile error

The fix was removing the dot between "log" and the square brackets: .data.log.[] on line 81.

OLD:
jq -r '.data.log.[] | .date + " " + .time + " " + .group + " " + ( .id | tostring) + " " + .msg' < "${OUTPUT_FILE_JSON}" > "${OUTPUT_FILE_TXT}"

NEW:
jq -r '.data.log[] | .date + " " + .time + " " + .group + " " + ( .id | tostring) + " " + .msg' < "${OUTPUT_FILE_JSON}" > "${OUTPUT_FILE_TXT}"

thanks for the hint! I think the .[] syntax was introduced in jq 1.7. I changed it to be jq 1.6 compatible