dpkg-diff-pkg-file - diff between original package file vs. local file

-dpkg-diff-pkg-file.md

First version: 2024.18.1
Current version: 2024.18.4

Run dpkg-diff-pkg-file --help for usage. Also available at the top of the script.

dpkg-diff-pkg-file will attempt to show you a visual difference between an original package file vs. your local file. This can help resolve config drift and conflicts ahead of attended/unattended upgrades.

Intro video: https://www.youtube.com/embed/19wm3gI4LfI

INSTALL

You can get the latest version of dpkg-diff-pkg-file from this gist with the following link:
https://gist.githubusercontent.com/kyle0r/dea6d4acb99c7a2d7e0a84101aee8c86/raw/dpkg-diff-pkg-file.sh

Here are some common ways to download the file via shell

curl -sSL 'https://gist.githubusercontent.com/kyle0r/dea6d4acb99c7a2d7e0a84101aee8c86/raw/dpkg-diff-pkg-file.sh' > dpkg-diff-pkg-file

# OR

wget -qO- 'https://gist.githubusercontent.com/kyle0r/dea6d4acb99c7a2d7e0a84101aee8c86/raw/dpkg-diff-pkg-file.sh' > dpkg-diff-pkg-file

# OR apt-helper if curl and wget are not available

/usr/lib/apt/apt-helper download-file 'https://gist.githubusercontent.com/kyle0r/dea6d4acb99c7a2d7e0a84101aee8c86/raw/dpkg-diff-pkg-file.sh' dpkg-diff-pkg-file

# THEN

chmod +x dpkg-diff-pkg-file

# THEN you can choose to place/link the file to a path in your $PATH

USAGE EXAMPLES

Basic invocation to check for differences between the local version of /etc/sudoers and the original package version:

dpkg-diff-pkg-file /etc/sudoers

TODO

TODO: add invocations and tips regarding dist-upgrade/full-upgrade situations OR backports
TODO: add invocation to change the diff side left/right
TODO: consider posting on https://forums.debian.net/ and mailing list, reddit and proxmox

Q: Is it easy to detect if a system is running release X but the sources are set to target release Y? If so, this could be considered as a state check in the script, and the user could be prompted whether to diff the current release or the target release.

dpkg-diff-pkg-file.sh

#!/bin/sh

# This script was created on Debian and should be POSIX compliant
# shell check OK
version='2024.18.4'
#  version convention: DIN ISO 8601 date +%G\.%V\.1 (YEAR.WEEK.RELEASE)
#+ where 1 is incremented per release within the given week

me="${0##*/}" # basename
me="${me%.sh}" # strip .sh suffix
print_version() { printf "%s version: %s\n" "$me" "$version"; }

# shellcheck disable=SC1111
usage=$(cat <<USAGE
  
NAME

$me - diff between original package file vs. local file


VERSION

$(print_version)


DESCRIPTION

Invocation: $me path

For example: $me /etc/sudoers

path can be a relative or absolute path to the local file you want to
diff. For example, if the working dir is /etc/ then you can invoke:

$me sudoers

$me will attempt to show you a visual difference between
the original package file vs. local file. This can help resolve config
drift and conflicts ahead of attended/unattended upgrades.

It was inspired by a Q&A on Stack Exchange: Unix & Linux
https://unix.stackexchange.com/q/72746/19406

Invocation as root is not recommended or required. sudo will be invoked
as required so that you can read and edit local configs during the
visual diff.

Scenario: checking local config files for drift.
For example: debsums --config --changed

Typically, this need arises during (or just before) a major codename
release upgrade, when it is likely that local configs have drifted from
the source deb versions and will conflict with incoming packages being
upgraded.

During an attended upgrade, the sysop has the opportunity to resolve
these conflicts as part of the upgrade process. The resolution process
is usually graceful, provided inline by dpkg prompting the sysop, but
resolving diffs and conflicts can slow down an upgrade, or perhaps
uncover breaking changes that may require a snapshot rollback, delaying
the upgrade.

So this script can give you a preview of these differences before
actually upgrading, or perhaps you just want to visually inspect the
local config drift vs. the original deb version.

Yes, in an ideal world there would be no config drift and no conflicts!
Every package would support the conf.d pattern or a .local config!
Unfortunately, that's not always the case, so this script tries to
address those concerns.

--

The tmp_dir contents is listed in the script output, so you can always
see which deb version is being compared.

As the script progresses through its various stages it pages output,
simply scroll up to see previous output.

The script aims to cleanup any cruft it generates in /var/tmp. The user
will be prompted to confirm the cleanup, and will also be offered
suggestions for performing a manual cleanup.


THINGS TO KEEP IN MIND

$me relies on dpkg-query and the local package state. It
can only target releases that local dpkg knows about. i.e. sources are
present and apt is up-to-date.

--

Some configs are managed by "maintainer scripts", one example of this is
/etc/ssh/sshd_config which is installed via the packages
openssh-server.postinst script.

cite: https://unix.stackexchange.com/q/346332/19406
postinst script: https://bit.ly/3UHjxI8

From the script its possible to determine the source config is stored:
/usr/share/openssh/sshd_config
So one could run:
1: $me /usr/share/openssh/sshd_config
OR
2: vimdiff /usr/share/openssh/sshd_config /etc/ssh/sshd_config

Keep in mind with invocation 1: any changes made would be to
/usr/share/openssh/sshd_config and not /etc/ssh/sshd_config, so its
probably better to use 2:.

Caveat: during an openssl-server package upgrade the latest sshd_config
will be placed in /usr/share/openssh/sshd_config. Then it will be
copied to a tmp file with some potential modifications performed by the
package maintainer scripts. Essentially the final config is generated.
In the same function, the maintainer script calls ucf to update the
config file and, if necessary, prompt the user to resolve conflicts
between the generated tmp file and the local /etc/ssh/sshd_config.
So, while the source file is /usr/share/openssh/sshd_config, there is a
good chance that the source is modified during an install/upgrade.

There may still be some value in running the diff against
/usr/share/openssh/sshd_config, but take into account the information
mentioned herein.


PORTABILITY / COMPATIBILITY

The script should be POSIX compliant. The script was designed to be used
on systems that use the dpkg package manager.


ENVIRONMENT

VERSION_CODENAME override the detection of os release codename

for example:

VERSION_CODENAME=bullseye $me path

If the system has updated package sources for a given os release, you
can use the VERSION_CODENAME to diff the local files aginst the incoming
os release changes.

You can also do the same for backports if you have the relevant
up-to-date apt sources, for example: bookworm-backports, or consider
this more dynamic approach:

VERSION_CODENAME=\$(lsb_release -sc)-backports


LOCAL_DIFF_SIDE specify which side of the diff the local file should
appear, left or right. For example:

LOCAL_DIFF_SIDE=left $me sudoers


PAGER specify which paging utility to use with fallback to less, and
then cat.


AUTHORS

2024 https://github.com/kyle0r


BUGS

Post on the GitHub gist:
https://gist.github.com/kyle0r/dea6d4acb99c7a2d7e0a84101aee8c86


CONTRIBUTING

Feel free to make requests or fork the gist (see BUGS).


LICENSE

MIT


DISCLAIMER

This script is provided “AS IS” and any express or implied warranties,
including, but not limited to, the implied warranties of
merchantability and fitness for a particular purpose are disclaimed.
See the LICENSE distributed file or for complete details.


TODO / IDEAS

Add some options to skip and perform certain sections or skip certain
prompts. i.e. assume the user wants to press ENTER at a given section
prompt. e.g. --skip-manual-cleanup-info --auto-cleanup-no-prompting or
--no-prompts
This will require a new dependency on GNU getopt.
example: https://stackoverflow.com/a/7948533/490487
example: /usr/share/doc/util-linux/examples/getopt-example.bash

--

The first release of this script supports single file comparisons. It
would be possible to refactor the script to support the original use
case AND also support comparing all files in a specific deb package
that have differing local files. For example:
$me --package apache2

--

In addition, it would be possible to write a function to enumerate all
differences in a debsums invocation, grouped by package and then walk
the user through the local diffs vs. the original deb package versions.
For example:
debsums --config --changed | $me --debsums

--

Add a CHANGES or CHANGELOG or similar file to track changes between
$me versions.

--

A lá apt/dpkg, at the diff stage of the script, it would be possible to
give the user the option of starting a shell to examine the situation
(the Z option).

--

✅ Add -V --version option to show the version of the script.

--

✅ Add a warning when invoked as root

--

Consider which is better/more portable/more available:
realpath vs. readlink
cite: https://unix.stackexchange.com/q/136494/19406

  
USAGE
) # the space in the trailing whitespace prevents command substitution performing an rtrim

#  note to self
#  The sh interpreter parses in a procedural-like manner.
#+ This means that only something that interpreter has seen can be referenced.
#+ i.e. If something is not yet parsed/declared it cannot be referenced.

# handy POSIX pause function https://unix.stackexchange.com/a/293941/19406
pause() { printf "%s" "Press ENTER to continue... or CTRL+C to abort."; read -r _; }

# error handling, the script uses set -e (exit on errors) and an EXIT trap
something_went_wrong() { there_was_an_error="yes"; error_msg="$*"; exit 1; } # invokes EXIT trap
was_there_an_error() { [ "yes" = "$there_was_an_error" ] && return 0 || return 1; }

# specify which side of the diff the local file should appear
: "${LOCAL_DIFF_SIDE:=right}"

# use less as PAGER fallback if no env pager is defined
# https://stackoverflow.com/a/28085062/490487
: "${PAGER:=less}"

# check for PAGER dependencies
if [ ! -x "$(command -v "$PAGER")" ]; then # less not found
  PAGER="cat"
  if [ ! -x "$(command -v "$PAGER")" ]; then # cat not found
    echo "$PAGER and cat not found in PATH. aborting." 1>&2
    exit 1
  fi
fi

# return script version
if [ '-V' = "$1" ] || [ '--version' = "$1" ]; then
  print_version; exit 1
fi

# check $1 to see if there is a value, we expect a string path
[ -z "$1" ] && set -- '--help'

# handle -h | --help invocation
if [ '-h' = "$1" ] || [ '--help' = "$1" ]; then
  printf "%s" "$usage" | "$PAGER"; exit 1
fi

# https://askubuntu.com/a/997893
# equivalent to keystroke CTRL+L
scroll_up() {
  printf '\n\n###### SCRIPT PAUSE ######\n '
  printf '\33[H\33[2J'
}

# define the exit and cleanup trap function
exit_and_cleanup() {
  trap - EXIT INT

  scroll_up

  if was_there_an_error; then
    printf "\n\nWARNING: an error was detected.\nERROR MSG: %s\n\nScrollback may contain more details.\n\n" "$error_msg" 1>&2
  fi

  printf '\n\nEXIT & CLEANUP STAGE\n\n'

  if [ ! -d "$tmp_dir" ]; then

    printf "INFO: tmp_dir does not exist. no cleanup to do. exiting.\n\n"
    exit 0

  else

    cat <<EOM

Your shell PAGER ($PAGER) will now be invoked to preview the list of tmp_dir paths to be cleaned up.
Please check the paths carfully!
Exit the PAGER as normal to continue with cleanup.

If you exit the script now cleanup will NOT be performed and manual steps will be required.

EOM

    pause
    scroll_up

    find "$tmp_dir" -depth -and -print | "$PAGER"

    cat <<EOM
Are you happy with the cleanup preview?

WARNING tmp_dir: $tmp_dir will bebe recursively deleted if you continue!

CMD: find "$tmp_dir" -depth -and -print -and -delete

Press ENTER to cleanup the paths or CTRL+C to skip the cleanup and exit this script.

EOM

    pause
    scroll_up

    # POSIX compliant sanity check
    (echo "$tmp_dir" | grep -Eq ^/var/tmp) || { echo "Assertion failed. Aborting cleanup. $tmp_dir path did not start with /var/tmp! exiting." 1>&2; exit 1; } 

    printf '\n\nThe following paths will be deleted:\n\n'
    
    find "$tmp_dir" -depth -and -print -and -delete

    find_exit_code="$?"

    printf "\n\nfind exit code: %s\n\nCLEANUP EXECUTED." "$find_exit_code"
    
    printf '\n\n###### SCRIPT END ######\n\n'

    exit "$find_exit_code"

  fi
}

# START OF MAIN SCRIPT

# exit on errors
set -e

# set the trap function, we trap EXIT and INTERRUPT (CTRL+C) signals
trap exit_and_cleanup EXIT INT

# id dependency
[ -x "$(command -v id)" ] || something_went_wrong 'id not found in PATH. aborting.'

# running as root check
# Text to ASCII Art Generator (TAAG)
# https://www.patorjk.com/software/taag/#p=display&f=ANSI%20Regular&t=WARNING
scroll_up
if [ 0 -eq "$(id -u)" ]; then

  cat <<EOM


  ██     ██  █████  ██████  ███    ██ ██ ███    ██  ██████  
  ██     ██ ██   ██ ██   ██ ████   ██ ██ ████   ██ ██       
  ██  █  ██ ███████ ██████  ██ ██  ██ ██ ██ ██  ██ ██   ███ 
  ██ ███ ██ ██   ██ ██   ██ ██  ██ ██ ██ ██  ██ ██ ██    ██ 
   ███ ███  ██   ██ ██   ██ ██   ████ ██ ██   ████  ██████  

  It is not necessary or recommended to run this script as root.

  sudo will be invoked as required.

  Would you like to continue as root?

EOM
  pause
fi

scroll_up
printf '\n\n###### SCRIPT START ######\n\n'

# try to determine os release codename
# if var is not in the env
if [ -z "$VERSION_CODENAME" ]; then

  # 1st try to source variables from /etc/os-release
  if [ -e /etc/os-release ]; then
    # shellcheck source=/etc/os-release
    . /etc/os-release
  # 2nd try to use lsb_release
  elif VERSION_CODENAME="$(lsb_release -sc 2>/dev/null)"; then
    : # pass
  else
    something_went_wrong "unable to determine os release codename! exiting."
  fi
fi

printf 'Detected os release codename: %s\n\n' "$VERSION_CODENAME"

# check for script dependencies
[ -x "$(command -v find)" ] || something_went_wrong 'find not found in PATH. aborting.'
[ -x "$(command -v sudo)" ] || something_went_wrong 'sudo not found in PATH. aborting.'
[ -x "$(command -v dpkg-query)" ] || something_went_wrong 'dpkg-query not found in PATH. aborting.'
[ -x "$(command -v realpath)" ] || something_went_wrong 'realpath not found in PATH. aborting.' 
[ -x "$(command -v apt-get)" ] || something_went_wrong 'apt-get not found in PATH. aborting.'
[ -x "$(command -v ar)" ] || something_went_wrong 'ar not found in PATH. aborting.'
[ -x "$(command -v tar)" ] || something_went_wrong 'tar not found in PATH. aborting.'
[ -x "$(command -v unxz)" ] || something_went_wrong 'xz-utils package not found. aborting.'
[ -x "$(command -v gunzip)" ] || something_went_wrong 'gzip package not found. aborting.'

if file="$(sudo realpath -m "$1")"; then
  echo "$1 was resolved to: $file"
  if [ ! -e "$file" ]; then
    something_went_wrong "$1 does not exist. exiting."
  fi
  if [ ! -f "$file" ]; then
    something_went_wrong "$1 does not appear to be a regular file. exiting."
  fi
else
  something_went_wrong "$1 could not be resolved"
fi

if package="$(dpkg-query -S "$file")"; then
    package="$(echo "$package" | cut -d':' -f1)"
    echo "${file} is found in package: ${package}"
  else
    something_went_wrong "unable to determine package for ${file}."
fi

tmp_dir=$(mktemp -dp /var/tmp)
cd "$tmp_dir"

cat <<EOM

The $package package from release $VERSION_CODENAME will be downloaded and extracted to tmp_dir: $tmp_dir

You may wish to review: apt-cache policy $package

EOM

pause

scroll_up

apt-get download "$package" -t "${VERSION_CODENAME}" || something_went_wrong 'apt-get download failed.'

ar vx "${package}"*.deb || something_went_wrong 'extracting deb file failed.'

if [ -e data.tar.gz ]; then
  tar xzf data.tar.gz || something_went_wrong 'extracting tar.gz failed'
elif [ -e data.tar.xz ]; then
  tar xf data.tar.xz || something_went_wrong 'extracting tar.xz failed'
else
  something_went_wrong 'error: not able to extract data.tar - unknown compression.'
fi

cat <<EOM

tmp_dir listing:

$(ls -al "$tmp_dir")

EOM

pause

scroll_up

cat <<EOM

This script will guide you through cleaning up $tmp_dir on exit.

MANUAL CLEANUP REFERENCE
Step 1 (preview): find $tmp_dir -depth -and -print | $PAGER
This invocation will give you a preview of the paths to be deleted in $PAGER (PAGER or less fallback).
If the -print invocation looks safe/expected, then use -print -and -delete option to print and delete recursively.

Step 2 (delete): find $tmp_dir -depth -and -print # -and -delete
CAUTION: remove the # to make the command destructive and delete the given path recursively!

EOM

pause
scroll_up

if [ "left" = "$LOCAL_DIFF_SIDE" ]; then
  left_file="$file"
  right_file="${tmp_dir}${file}"
else
  left_file="${tmp_dir}${file}"
  right_file="$file"
fi

cat <<EOM

COMPARING THE CHANGES
vimdiff suggestion:

sudo vimdiff $left_file $right_file

vimdiff will provide a side-by-side visual diff, with diff highlighting.
vimdiff usage: https://devhints.io/vim-diff

To exit/quit vimdiff use the :qa command to "quit" all windows.
To exit/quit and discard all changes :qa!

At this point, you are welcome to use another shell session to compare the files using your preferred utility.

If you press CTRL+C to abort, vimdiff will be skipped and the cleanup and exit routine is started.

If you continue, vimdiff will be invoked per suggestion.
Exit/quit vimdiff to continue this script and start the cleanup.

EOM

pause

scroll_up

sudo vimdiff "$left_file" "$right_file"

# The EXIT trap is invoked once the script exits

LICENSE

Released under MIT License

Copyright (c) 2024 kyle0r

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.