Update links in the SecureDrop documentation using the output of linkcheck.

updating-links.md

Updating links from linkcheck output

cd securedrop-docs

First collect the output of make docs-linkcheck:

make docs-linkcheck > linkcheck.log

It looks like this (head linkcheck.log and tail linkcheck.log):

rm -rf _build/*
make[1]: Leaving directory '/home/user/src/securedrop-docs/docs'
Running Sphinx v2.3.1
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [linkcheck]: targets for 81 source files that are out of date
updating environment: [new config] 81 added, 0 changed, 0 removed
reading sources... [  1%] admin
reading sources... [  2%] backup_and_restore

[snip]

(line    8) ok        https://blog.torproject.org/v2-deprecation-timeline
writing output... [ 98%] what_makes_securedrop_unique
(line   76) ok        https://www.reuters.com/article/us-media-cybercrime/journalists-media-under-attack-from-hackers-google-researchers-idUSBREA2R0EU20140328
writing output... [100%] yubikey_setup
(line   59) ok        https://www.yubico.com/wp-content/uploads/2015/03/YubiKeyManual_v3.4.pdf
(line   15) ok        https://support.yubico.com/hc/en-us/articles/360016614780-OATH-HOTP-Yubico-Best-Practices-Guide
(line   15) redirect  https://www.yubico.com/products/yubikey-hardware/fido-u2f-security-key - permanently to https://www.yubico.com/authentication-standards/fido-u2f/

Remove all the lines that are not reportin permanent redirects:

cat linkcheck.log | grep redirect | grep permanently > permanent-redirects.log

# grep redirect    # only keeps the lines that contain the word 'redirect'
# grep permanently # only keeps the lines that contain the word 'permanently'

The new file looks like this (tail permanent-redirects.log):

[snip]

(line    1) redirect  https://itunes.apple.com/us/app/freeotp-authenticator/id872559395 - permanently to https://apps.apple.com/us/app/freeotp-authenticator/id872559395
(line    6) redirect  https://itunes.apple.com/us/app/google-authenticator/id388497605 - permanently to https://apps.apple.com/us/app/google-authenticator/id388497605
(line    7) redirect  https://pypi.python.org/pypi/authenticator - permanently to https://pypi.org/project/authenticator/
(line   14) redirect  https://arstechnica.com/security/2013/12/scientist-developed-malware-covertly-jumps-air-gaps-using-inaudible-sound/ - permanently to https://arstechnica.com/information-technology/2013/12/scientist-developed-malware-covertly-jumps-air-gaps-using-inaudible-sound/
(line  647) redirect  https://blog.torproject.org/blog/critique-website-traffic-fingerprinting-attacks - permanently to https://blog.torproject.org/critique-website-traffic-fingerprinting-attacks
(line   55) redirect  https://docs.securedrop.org - permanently to https://docs.securedrop.org/en/stable/
(line   14) redirect  https://tails.boum.org/doc/first_steps/startup_options/administration_password/ - permanently to https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
(line   14) redirect  https://tails.boum.org/doc/first_steps/startup_options/administration_password/ - permanently to https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
(line   15) redirect  https://tails.boum.org/doc/first_steps/startup_options/administration_password/ - permanently to https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
(line   15) redirect  https://www.yubico.com/products/yubikey-hardware/fido-u2f-security-key - permanently to https://www.yubico.com/authentication-standards/fido-u2f/

Edit those lines to only keep one original URL and one updated URL on each of them:

cat permanent-redirects | cut -d')' -f2 | cut -d' ' -f4,8 > old-new-links.log

# cut -d')' -f2   # keeps the entire lines except their start up to the (first) closing bracket
# cut -d' ' -f4,8 # splits the lines on spaces and only keeps the 1st and 8th segments (both URLs!)

The new file looks like this (tail old-new-links.log):

[snip]

https://itunes.apple.com/us/app/freeotp-authenticator/id872559395 https://apps.apple.com/us/app/freeotp-authenticator/id872559395
https://itunes.apple.com/us/app/google-authenticator/id388497605 https://apps.apple.com/us/app/google-authenticator/id388497605
https://pypi.python.org/pypi/authenticator https://pypi.org/project/authenticator/
https://arstechnica.com/security/2013/12/scientist-developed-malware-covertly-jumps-air-gaps-using-inaudible-sound/ https://arstechnica.com/information-technology/2013/12/scientist-developed-malware-covertly-jumps-air-gaps-using-inaudible-sound/
https://blog.torproject.org/blog/critique-website-traffic-fingerprinting-attacks https://blog.torproject.org/critique-website-traffic-fingerprinting-attacks
https://docs.securedrop.org https://docs.securedrop.org/en/stable/
https://tails.boum.org/doc/first_steps/startup_options/administration_password/ https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
https://tails.boum.org/doc/first_steps/startup_options/administration_password/ https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
https://tails.boum.org/doc/first_steps/startup_options/administration_password/ https://tails.boum.org/doc/first_steps/welcome_screen/administration_password/
https://www.yubico.com/products/yubikey-hardware/fido-u2f-security-key https://www.yubico.com/authentication-standards/fido-u2f/