-
-
Save AfroThundr3007730/ba99753dda66fc4abaf30fb5c0e5d012 to your computer and use it in GitHub Desktop.
| #!/bin/bash | |
| # Imports DoD root certificates into Linux CA store | |
| # Version 0.4.2 updated 20250425 by AfroThundr | |
| # SPDX-License-Identifier: GPL-3.0-or-later | |
| # For issues or updated versions of this script, browse to the following URL: | |
| # https://gist.github.com/AfroThundr3007730/ba99753dda66fc4abaf30fb5c0e5d012 | |
| # Dependencies: curl gawk openssl unzip wget | |
| set -euo pipefail | |
| shopt -s extdebug nullglob | |
| add_dod_certs() { | |
| local bundle cert certdir file form tmpdir url update | |
| trap '[[ -d ${tmpdir:-} ]] && rm -fr $tmpdir' EXIT INT TERM | |
| # Location of bundle from DISA site | |
| url='https://public.cyber.mil/pki-pke/pkipke-document-library/' | |
| bundle=$(curl -s $url | awk -F '"' 'tolower($2) ~ /dod.zip/ {print $2}') | |
| # Set cert directory and update command based on OS | |
| [[ -f /etc/os-release ]] && source /etc/os-release | |
| if [[ ${ID:-} =~ (fedora|rhel|centos) || | |
| ${ID_LIKE:-} =~ (fedora|rhel|centos) ]]; then | |
| certdir=/etc/pki/ca-trust/source/anchors | |
| update='update-ca-trust' | |
| elif [[ ${ID:-} =~ (debian|ubuntu|mint) || | |
| ${ID_LIKE:-} =~ (debian|ubuntu|mint) ]]; then | |
| certdir=/usr/local/share/ca-certificates | |
| update='update-ca-certificates' | |
| else | |
| certdir=${1:-} && update=${2:-} | |
| fi | |
| [[ ${certdir:-} && ${update:-} ]] || { | |
| printf 'Unable to autodetect OS using /etc/os-release.\n' | |
| printf 'Please provide CA certificate directory and update command.\n' | |
| printf 'Example: %s /cert/store/location update-cmd\n' "${0##*/}" | |
| exit 1 | |
| } | |
| # Extract the bundle | |
| wget -qP "${tmpdir:=$(mktemp -d)}" "$bundle" | |
| unzip -qj "$tmpdir"/"${bundle##*/}" -d "$tmpdir" | |
| # Check for existence of PEM or DER format p7b. | |
| for file in "$tmpdir"/*_{DoD,dod}{.,_}{pem,der}.p7b; do | |
| # Iterate over glob instead of testing directly (SC2144) | |
| [[ -f ${file:-} ]] && | |
| form=${file%.*} && form=${form##*_} && form=${form##*.} && break | |
| done | |
| [[ ${form:-} && ${file:-} ]] || { printf 'No bundles found!\n' && exit 1; } | |
| # Convert the PKCS#7 bundle into individual PEM files | |
| openssl pkcs7 -print_certs -inform "$form" -in "$file" | | |
| awk -v d="$tmpdir" \ | |
| 'BEGIN {c=0} /subject=/ {c++} {print > d "/cert." c ".pem"}' | |
| # Rename the files based on the CA name | |
| for cert in "$tmpdir"/cert.*.pem; do | |
| mv "$cert" "$certdir"/"$( | |
| openssl x509 -noout -subject -in "$cert" | | |
| awk -F '(=|= )' '{print gensub(/ /, "_", "g", $NF)}' | |
| )".crt | |
| done | |
| # Remove temp files and update certificate stores | |
| rm -fr "$tmpdir" && $update | |
| } | |
| # Only execute if not being sourced | |
| [[ ${BASH_SOURCE[0]} == "$0" ]] || return 0 && add_dod_certs "$@" |
The script uses $1 and $2, are those not 2 arguments that get passed in?
If I recall those reference the tokens that awk produces. I will double check.
@danielcjacobs if you look in the if/else block, you'll see that those args are supposed to be set to the certdir and whatever shellscript/binary is being used to update all the cert bundles. the gist tries to provide good defaults, but you can also overwrite if you so desire.
Will this run on mac?
@senterfd-jrg Thanks for catching that. I've updated the script to incorporate it (and fix various shellcheck warnings).
@njohbillie I haven't researched how a mac imports trusted certificates. If it works similarly to Linux certificate management, then you could adapt the script to make it work, if not then you may need to look at another tool to do it.
For the bots that drive search, this works on WSL - Windows Subsystem for Linux Ubuntu nicely. Thank you @AfroThundr3007730 for saving me a bunch of time.
Looks like the naming convention changed again on the .p7b files. I was able to get it to work by modifying the following lines. This is written in a way that no longer depends on "dod" in the filenames.
From:
# Check for existence of PEM or DER format p7b.
for file in "$tmpdir"/*_dod_{pem,der}.p7b; do
# Iterate over glob instead of testing directly (SC2144)
[[ -f ${file:-} ]] && form=${file%.*} && form=${form##*_} && break
done
[[ ${form:-} && ${file:-} ]] || { printf 'No bundles found!\n' && exit 1; }
To:
# Check for existence of PEM or DER format p7b.
for file in "$tmpdir"/*.p7b; do
[[ -f ${file:-} ]] || continue
if [[ $file =~ (_pem|\.pem)\.p7b$ ]]; then
form="pem"
elif [[ $file =~ (_der|\.der)\.p7b$ ]]; then
form="der"
else
continue
fi
break
done
[[ ${form:-} && ${file:-} ]] || { printf 'No valid bundles found!\n' && exit 1; }
@nathanhoeller - Another good catch for the change to the naming convention. Upgrading to this version with your change also revealed that this new version does not replace all the space chars within the certificate's filename with "_" (underscore). I am not a fan of spaces within filenames, so I changed the 1 to "g" in the following gensub() command.
From:
awk -F '(=|= )' '{print gensub(/ /, "_", 1, $NF)}'To:
awk -F '(=|= )' '{print gensub(/ /, "_", "g", $NF)}'
Public.cyber.mil made another change and now the command to initialize bundle, near line 22, is having problems. The previous calls to initialize bundle would return a single value for the variable bundle. Now we are seeing two values. I have added the grep command to use only the line beginning with the phrase http in my file. See below.
From:
bundle=$(curl -s $url | awk -F '"' 'tolower($2) ~ /dod.zip/ {print $2}')To:
bundle=$(curl -s $url | awk -F '"' 'tolower($2) ~ /dod.zip/ {print $2}' | grep "^http")
@senterfd-jrg - I see the spaces in the filenames in /etc/ssl/certs/. It looks like it automatically added the underscores after running the update command, but I can understand wanting to add this to keep the file names consistent. Thanks for the catch! I'll be adding this to my code.
Also, I verified the issue you mentioned with the bundle URL. That's another helpful addition. Thanks!
Is this currently functional, as-is above? I see lots of suggestions for fixes, but wasn't sure if this is still being maintained.
@allebone - I use the code above with some of the modifications that the users have reported. @AfroThundr3007730 did incorporate a bunch of stuff as he mentioned. His script is a big help. The DoD site continually changes their structures which requires a modification to the script. I have come here to this chat and seen recommended updates within a day to hours of required changes based on DoD site changes.
@senterfd-jrg @AfroThundr3007730 I hope my comment wasn’t taken as a complaint or criticism. TBH, I was a little tired and on a small screen with a half-wit understanding of the GIST revisions tab. (Which I didn’t see until today)
I’m just getting started on a base install of 72 servers, and this problem plagues us! (Times a ton)
I’m not scripting savvy but if there is anything I can do to help…let me know. Or I can go back to lurking until someone wants a layman to test. :)
I've updated the file glob to account for the varying bundle names as per @nathanhoeller and I've adjusted the awk command to replace all spaces properly as per @senterfd-jrg . As of today at least, it appears to be working correctly (again).
The other changes to grab the bundle URL don't appear to be needed for now. If they were needed, you can avoid calling grep like so:
- bundle=$(curl -s $url | awk -F '"' 'tolower($2) ~ /dod.zip/ {print $2}')
+ bundle=$(curl -s $url | awk -F '"' '/https/ && tolower($2) ~ /dod.zip/ {print $2}')
@AfroThundr3007730, thanks for this code, have you considered retrofitting to pull the other DISA cert bundles and cycling through them to do a complete install of:
# unclass-certificates_pkcs7_DoD.zip
# unclass-certificates_pkcs7_ECA.zip
# unclass-certificates_pkcs7_JITC.zip
# unclass-certificates_pkcs7_WCF.zip
# unclass-dod_approved_external_pkis_trust_chains_types_1-2_federal_agencies.zip
# unclass-dod_approved_external_pkis_trust_chains_types_3-4_non_federal_issuers.zip
# unclass-dod_approved_external_pkis_trust_chains_types_5-6_foreign_other.zip
# unclass-dod_approved_external_pkis_trust_chains.zip
I'm frequently needing the WCF bundle and was just considering pulling these all for import, just curious if you have done this already before I start modifying the code.
Just tried to run this on Linux Mint Cinnamon 22.1 (xia) using the instructions found here:
https://community.linuxmint.com/tutorial/view/313
When I attempt to execute the file in my terminal, I'm getting this error:
./add-dod-certs.sh: 11: set: Illegal option -o pipefail
What am I doing wrong?
@R1ceAR0n1e - You need to execute with bash. Either set execution bit with chmod or execute as bash add-dod-certs.sh. Also, need to ensure you have bash installed on your OS.
I've implemented add-dod-certs.sh in an Ansible playbook for Fedora/RHEL systems. It can be easily extended to support Ubuntu/Debian systems. Credit to AfroThundr for the original bash script that guided this.
# vim: ft=yaml.ansible
# ensure community.crypto collection is installed on control node
---
- name: Install DoD certificates system-wide
connection: local
hosts: 127.0.0.1
become: true
vars:
dod_certs_url: "https://dl.dod.cyber.mil/wp-content/uploads/pki-pke/zip/unclass-certificates_pkcs7_DoD.zip"
system_cert_dir: "/etc/pki/ca-trust/source/anchors"
tasks:
- name: Extract DoD certs zip
ansible.builtin.unarchive:
src: "{{ dod_certs_url }}"
dest: "/tmp"
list_files: true
remote_src: true
register: certs
- name: Locate DoD PKCS#7 cert bundle
ansible.builtin.set_fact:
pkcs7_bundle_path: >-
{{ "/tmp/" + (certs.files
| select('match', '.*(?i:dod)\.pem\.p7b$')
| first) }}
- name: Extract certificates from PKCS#7 bundle
ansible.builtin.command: >
openssl pkcs7 -print_certs -inform PEM -in {{ pkcs7_bundle_path }}
register: pkcs7_bundle_output
changed_when: false
- name: Split certs into individual PEMs
ansible.builtin.set_fact:
cert_list: "{{ pkcs7_bundle_output.stdout | community.crypto.split_pem }}"
- name: Get cert info
community.crypto.x509_certificate_info:
content: "{{ item }}"
loop: "{{ cert_list }}"
register: cert_info
- name: Build cert data structure
ansible.builtin.set_fact:
cert_data: >-
{{ cert_data | default([]) + [
{
'content': item.0,
'file_name': (item.1.subject.commonName
| replace(' ', '_')
| regex_replace('$', '.crt'))
}
]
}}
loop: "{{ cert_list | zip(cert_info.results) | list }}"
- name: Install certs to system trust store
ansible.builtin.copy:
content: "{{ item.content }}"
dest: "{{ system_cert_dir }}/{{ item.file_name }}"
mode: '0644'
owner: root
group: root
loop: "{{ cert_data }}"
register: cert_installation
- name: Update CA trust
ansible.builtin.command: update-ca-trust
changed_when: cert_installation.changed
when: cert_installation.changed
edit: improved the pkcs7 bundle regex
It doesn’t need any I don’t think