Fix "W: Possible missing firmware /lib/firmware/i915/kbl_dmc_ver1.bin for module i915_bpo" (Ubuntu 16.04, kernel 4.4)

install_intelr_graphic_linux_firmware.sh

#!/bin/bash

cd

wget https://01.org/sites/default/files/downloads/intelr-graphics-linux/sklgucver61.tar.bz2 && \
tar xvjf sklgucver61.tar.bz2 && cd skl_guc_ver6_1/ && sudo ./install.sh

cd

wget https://01.org/sites/default/files/downloads/intelr-graphics-linux/kbldmcver101.tar.bz2 && \
tar xjvf kbldmcver101.tar.bz2 && cd kbl_dmc_ver1_01/ && sudo ./install.sh 

cd

sudo update-initramfs -u -k all

@UrbanVampire, as you mainly did the whole magic, I was wondering to first, ask you to make a new project on github and host the code of this script for better maintenance and probably access, to others willing to improve it. what do you think about that?

Here is a slightly modified version just to keep track of everything you've added in the script:

#!/bin/bash
#
# Script to download and install missing Intel kernel drivers
# Made by Jiab77 <[email protected]>, 2020
# Improved by Mikko Rantalainen <[email protected]>, 2020
# Automatic missing FW detection added by UrbanVampire <[email protected]>, 2021
# Improved script colors and suggested handling missing nVidia firmware files by Jiab77 <[email protected]>, 2021
# Automagical missing nVidia firmware installation added by UrbanVampire <[email protected]>, 2021
# Minor spellchecking and fixing, minor display fixes and credits added by Jiab77 <[email protected]>, 2021
#
# Some definitions
# Config
DEBUG=0     # Debug flag
FWTotal=0   # Total missing FWs counter
FWSuccs=0   # Succesful installed FWs counter
FWError=0   # Failed FWs
declare -a InstalledFWs	# Array to store and check alredy installed FWs
# Colors:
RED="\033[1;31m"
GREEN="\033[1;32m"
YELLOW="\033[1;33m"
BLUE="\033[1;34m"
PURPLE="\033[1;35m"
CYAN="\033[1;36m"
WHITE="\033[1;37m"
NC="\033[0m"
NL="\n"
TAB="\t"
#
# Function to execute a comand and get it's output
function DebuggedExec(){
    #
    # Make sure that we have an parameter
    if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}DebuggedExec function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
        #
        OUTPUT=$(eval $1 2>&1)			# Let's execute the given command
    if [[ $? -ne 0 ]]; then			# Do we have an error?
        ((FWError++))			# Yes, we have.
        echo -e "${RED}Failed${NC}"
        if [[ $DEBUG -eq 1 ]]; then	# Are we in debugging mode?
            echo -e "${RED}Executed command was:${TAB}${YELLOW}$1${NC}"
            echo -e "${RED}Error message was:${TAB}${YELLOW}$OUTPUT${NC}"
        fi
        return 1
    fi
    return 0			# No errors, everything went fine
}
#
# Function to process single missing FW
function ProcessFirmware(){
    #
    # Make sure that we have an parameter
    if [[ $# -eq 0 ]]; then echo -e "${NL}${RED}ProcessFirmware function called w/o parameters. Something went really wrong...${NC}${NL}"; return 1; fi
    #
    ((FWTotal++))		# We got a missing FW, let's count it.
    #
    # Let's extract FW name and full path:
    FWFullPath=$(echo $1 | sed -n 's/^.*firmware //p')
    FWFileName=$(echo $1 | sed -n 's/^.*firmware//p')
    #
    if [[ $FWTotal -eq 1 ]]; then	# Is it first line? If so let's calculate tabulation offset
        let TABoff=$MaxLength-${#1}+${#FWFileName}+2
    fi
    #
    # Time to check if this file alredy installed
    if [[ " ${InstalledFWs[*]} " == *"$FWFileName"* ]]; then
        ((FWSuccs++))			# Everything went Ok
        echo -e "${CYAN}Alredy installed, Skipping${NC}"
        return 0
    else
        # Moved this line here to avoid printing blank '$FWFileName' value when already installed
        # And then output something like ": Alredy installed, Skipping", with an extra ": "
        # Because '$FWFileName' is not set as nothing is missing.
        #
        # Maybe we should move the text "Alredy installed, Skipping" somewhere else?
        echo -ne "${GREEN}$FWFileName${BLUE}:${TAB}${NC}\033[50D\033[${TABoff}C"
    fi	
    echo -ne "${BLUE}Downloading... ${NC}"
    local TEMPFILE="$(tempfile)"		# Get a temporary filename
    #
    # Is it nVidia?
    if [[ "$FWFileName" == *"nvidia"* ]]; then
        echo -ne "${CYAN}nVidia FW detected. It could take some time.${NL}\033[50D\033[${TABoff}CPlease wait... ${NC}"
        NVVersion=$(echo $FWFileName | sed -n 's/^.*nvidia\///p'| sed -n 's/\/.*$//p') # Extract version
        FolderNam="NVIDIA-Linux-x86_64-$NVVersion"
        RunFlName="$FolderNam.run"
        NVFWFlNme=$(echo $FWFileName | sed -n 's/^.*\///p') # Extract nV FW filename
        URL="https://download.nvidia.com/XFree86/Linux-x86_64/$NVVersion/$RunFlName"
        DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
        echo -ne "${BLUE}Extracting... ${NC}"
        DebuggedExec "sh \"$TEMPFILE\" -x --target /tmp/$FolderNam"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE"; rm -drf /tmp/$FolderNam ; return 1; fi
        rm "$TEMPFILE"
        DebuggedExec "mv /tmp/$FolderNam/firmware/$NVFWFlNme \"$TEMPFILE\""
        if [[ $? -ne 0 ]]; then rm -drf /tmp/$FolderNam ; return 1; fi
        # file is extracted and ready to go
        rm -drf /tmp/$FolderNam			# Some cleanup
    else # No, this is not nVidia, let's try to download FW from git.kernel.org
        local URL="https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain"
        DebuggedExec "wget -nv -O \"$TEMPFILE\" $URL$FWFileName"
        if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    fi
    # Now let's try to move downloaded file to /lib/firmware
    echo -ne "${BLUE}Installing... ${NC}"
    # First we need to check if path is exists
    DebuggedExec "mkdir -p ${FWFullPath%/*}"
    if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    # Moving the file to it's place
    DebuggedExec "mv -v \"$TEMPFILE\" $FWFullPath"
    if [[ $? -ne 0 ]]; then rm "$TEMPFILE" ; return 1; fi
    #
    ((FWSuccs++))				# Everything went Ok
    echo -e "${GREEN}Ok${NC}"
    InstalledFWs[$FWSuccs]=$FWFileName	# Store FW in list of installed FWs
    return 0
}
#
#################################
#
# Here's the main body
#
# Let's make sure that we are superuser
if [[ $EUID -ne 0 ]]; then echo -e "${NL}${RED}Must be run with superuser privileges:${NC} sudo $0 [--debug]${NL}"; exit 1; fi
#
# Is there some parameters?
if [[ $# -ne 0 ]] ; then
    if [[ $1 = "--debug" ]] ; then
        DEBUG=1; echo -e "${NL}${YELLOW}Debugging enabled.${NC}${NL}"
    else
        echo -e "${NL}${RED}Usage:${NC} sudo $0 [--debug]${NL}"; exit 1
    fi
else
    echo -e "${NL}${CYAN}Hint: ${WHITE}You can use the${GREEN} --debug ${WHITE}option for extended error info.${NC}${NL}"
fi
#
# Here we call update-initramfs, grep-search it's output for "missing HW" message
echo -e "${BLUE}Detecting missing FWs. It could take some time, please wait...${NC}${NL}"
MFWs=$(update-initramfs -u 2>&1 >/dev/null | grep 'Possible missing firmware' | sed -n 's/ for.*$//p') 
#
MaxLength=0	# Get longest string length - we'll need it later to calculate tabulation
while IFS= read -r line; do if [[ ${#line} -gt $MaxLength ]]; then MaxLength=${#line}; fi; done < <(echo "$MFWs")
#
# Let's process missing FWs one by one
while IFS= read -r line; do ProcessFirmware "$line"; done < <(echo "$MFWs")
#
# Did we found some missing FWs?
if [[ $FWTotal -eq 0 ]]; then echo -e "$${BLUE}No missing FWs found. Nothing to do. Exiting...${NC}${NL}"; exit 0; fi
# Is there some successful FWs?
if [[ $FWSuccs -eq 0 ]]; then		# Nope, no luck
    echo -ne "${NL}${YELLOW}WARNING: No FWs found or downloaded. See messages above"
    [[ $DEBUG -eq 0 ]] && echo -ne " or try --debug option for more info"
    echo -ne ". Exiting...${NC}${NL}"
    exit 1
fi
# Maybe ALL FWs downloaded with success?
if [[ $FWSuccs -ne $FWTotal ]]; then	# Nope
    echo -ne "${NL}${YELLOW}WARNING: Some FWs was not found or not downloaded. See messages above"
    [[ $DEBUG -eq 1 ]] || echo -ne " or try --debug option for more info"
    echo -ne ". But You still can regenerate kernels.${NC}${NL}"
fi
# Now we need to re-generate all kernels
echo -ne "${NL}${BLUE}It's time to re-generate kernels. Press ${GREEN}Enter ${BLUE}to continue or ${RED}CTRL+C ${BLUE}to skip:${NC}"
read
echo -e "${NL}${BLUE}Generating kernels. It could take some time, please wait...${NC}${NL}"
sudo update-initramfs -u -k all | grep 'Generating'
echo -e "${NL}${GREEN}Finished.${NC}${NL}"

I'm also wondering if we should move to case / esac instead of multiplying if [[ "$FWFileName" == *"firmware"* ]]; then to manage many other missing firmware as you did for nVidia, like for your missing rtl* firmware files?

This way we could manage several missing ones like that:

case "$FWFileName" in
*"nvidia"*)
    # code block
;;
*"rtl"*)
    # code block
;;
*)
    # code block for all other not supported cases, like for displaying a message
    # if all previous tests failed.
;;
esac

I don't have your bash scripting skills so I'm not really sure that is possible anyway 😅
oh, and for someone that claims to be a Linux beginner, you're just impressive! I'm feeling like knowing nothing 😆

@UrbanVampire, about the architecture detection we could try something simple like this:

if [[ $(lscpu | grep -i x86_64 | wc -l) -eq 1 ]]; then
    # This architecture is 64 bit
else
    # This architecture is 32 bit
fi

I was pretty sure that I could get this kind information from the file /proc/cpuinfo also but I could not find any relevant and precise values that we could use to detect the architecture better. I've also looked at some other files in /proc but could not anything else that could do the job so maybe lscpu is the simplest way.

I can try to run strace lscpu and see if it query some files that we could use instead of calling lscpu directly.

Ok I've tried with strace lscpu 2>&1 | grep -vi "No such" | grep -i "openat" and the file /proc/cpuinfo is used but also many other ones:

$ strace lscpu 2>&1 | grep -vi "No such" | grep -i "openat"
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libsmartcols.so.1", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/cpuinfo", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/kernel_max", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/possible", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/present", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/online", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/thread_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/core_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/core_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/topology/physical_package_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index0/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index1/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index2/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/type", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/level", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/size", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cache/index3/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_max_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu0/cpufreq/cpuinfo_min_freq", O_RDONLY|O_CLOEXEC) = 3
... (removed to reduce redondant lines due to multi cores)
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/thread_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/core_siblings", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/core_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/topology/physical_package_id", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index0/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index1/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index2/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cache/index3/shared_cpu_map", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cpufreq/cpuinfo_max_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/cpu/cpu7/cpufreq/cpuinfo_min_freq", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/sys/devices/system/node", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
openat(AT_FDCWD, "/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/usr/share/locale-langpack/fr/LC_MESSAGES/util-linux.mo", O_RDONLY) = 4
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache", O_RDONLY) = 4
openat(AT_FDCWD, "/sys/devices/system/node/node0/cpumap", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/proc/sys/kernel/osrelease", O_RDONLY) = 3
openat(AT_FDCWD, "/sys/firmware/dmi/tables/DMI", O_RDONLY) = -1 EACCES (Permission denied)
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/bus/pci/devices", O_RDONLY) = 3
openat(AT_FDCWD, "/proc/self/status", O_RDONLY) = 3

Yes, I'm having a cpu with 8 cores, that's why the list of open files goes from 0 to 7 😅

@Jiab77, Sorry for late answer, it was a hard week on work.

No, you did it

Oh, c'mon, it was you who found nvidia workaround. Not to mention that I wouldn't even start working without your original script.

I got both on my system for example but the lib64 folder is almost empty and just contains a symlink to a file stored in /lib sweat_smile but I'm using an Ubuntu based Linux distrib so maybe it can be different on original Ubuntu or any other Linux distribs.

AFAIK, the "firmware" folder is placed in "/lib" on both 32 and 64 systems. But after recent patches "/lib" is moved to "/usr/lib" on some systems. So, maybe it's better to extract destination path from original "Possible missing hardware" message.

maybe lscpu is the simplest way.

I think the arch command is the simplest way. But there's other thing: latest 32-bit drivers on nvidia site is 390.143. I'm not really sure what FW version is needed for 32-bit kernels. Anyway we need some experiments to understand:

1. Are the firmware files the same in 32 and 64 nvidia driver versions?
2. Will 64-bit nvidia 'run' script unpack files on a 32-bit system?
3. Can nvidia 'run' script be unpacked with a regular untar?

I'm also wondering if we should move to case / esac instead of multiplying if

Now we have only 2 options, nvidia and NOT nvidia,, but i'll think about it.

ask you to make a new project on github

I have zero experience with github but i'll try :)

@Jiab77
https://github.com/UrbanVampire/missing-firmware-fix
Just fixed false "Already installed" for now.

@Jiab77

1. Are the firmware files the same in 32 and 64 nvidia driver versions?
2. Will 64-bit nvidia 'run' script unpack files on a 32-bit system?
3. Can nvidia 'run' script be unpacked with a regular untar?

Well, answer to "2" and "3" is "no". At least I haven't found a way to extract files from the installer using standard archivers. And the 64-bit installer uses 64-bit binary stub, so it won't run on a 32-bit systems. It means that "1" is doesn't matter cos' we cannot use 64 installer on 32 system.
The rest is easy: just download installer from "Linux-x86" or "Linux-x86_64" folder of nVidia site according to system architecture.
The code is updated.
I think for now my job is done here.
BTW, don't You wanna add German of French README translation? Sorry, don't know what Your native language is.

@Jiab77, Sorry for late answer, it was a hard week on work.

@UrbanVampire, No worries, it's pretty same on my side, I'm also overloaded of work... that's why I'm replying you just now 😅

Oh, c'mon, it was you who found nvidia workaround. Not to mention that I wouldn't even start working without your original script.

Well, it was pretty simple to be honest but do the job to automate the whole process as you did is not the same at all!

AFAIK, the "firmware" folder is placed in "/lib" on both 32 and 64 systems. But after recent patches "/lib" is moved to "/usr/lib" on some systems. So, maybe it's better to extract destination path from original "Possible missing hardware" message.

Yes you're right and I saw in the code that you managed it very well. 🤘

I think the arch command is the simplest way. But there's other thing: latest 32-bit drivers on nvidia site is 390.143. I'm not really sure what FW version is needed for 32-bit kernels. Anyway we need some experiments to understand:

Thanks a lot for that, I was not aware about the arch command and it's much better than parsing lscpu output 😁

I have zero experience with github but i'll try :)

You did it very well for the less I could see 👍 I've stared the project already and added on my watch list 😜

I think for now my job is done here.
BTW, don't You wanna add German of French README translation? Sorry, don't know what Your native language is.

Well, I'm a native French speaker but I'm not really sure that would be useful to translate it in French honestly as many French native speakers that work in the computer domain knows reading English already and if they don't then they should learn it, it will give them a much wider source of knowledge 😜

BTW, let's keep in touch somewhere else, what do you think? write me an email if interested 😉

Thanks a lot guys. Great script!!!

Thank you Gentlemen.....excellent