WRF 4.1.2 installation with netCDF4, HDF5 in ubuntu 18.04 LTS

install_wrf41.sh

#!/bin/bash
## WRF installation with parallel process.
# Download and install required library and data files for WRF.
# License: LGPL
# Jamal Khan <[email protected]>
# Tested in Ubuntu 18.04 LTS

# basic package managment
sudo apt update
sudo apt upgrade
sudo apt install gcc gfortran g++ libtool automake autoconf make m4 grads default-jre csh

## Directory Listing
export HOME=`cd;pwd`
mkdir $HOME/WRF
cd $HOME/WRF
mkdir Downloads
mkdir Library

## Downloading Libraries
cd Downloads
wget -c https://www.zlib.net/zlib-1.2.13.tar.gz
wget -c https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.10/hdf5-1.10.5/src/hdf5-1.10.5.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-c/4.9.0/netcdf-c-4.9.0.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-fortran/4.6.0/netcdf-fortran-4.6.0.tar.gz
wget -c http://www.mpich.org/static/downloads/3.3.1/mpich-3.3.1.tar.gz
wget -c https://download.sourceforge.net/libpng/libpng-1.6.37.tar.gz
wget -c https://www.ece.uvic.ca/~frodo/jasper/software/jasper-1.900.1.zip

# Compilers
export DIR=$HOME/WRF/Library
export CC=gcc
export CXX=g++
export FC=gfortran
export F77=gfortran

# zlib
cd $HOME/WRF/Downloads
tar -xvzf zlib-1.2.13.tar.gz
cd zlib-1.2.13/
./configure --prefix=$DIR
make
make install

# hdf5 library for netcdf4 functionality
cd $HOME/WRF/Downloads
tar -xvzf hdf5-1.10.5.tar.gz
cd hdf5-1.10.5
./configure --prefix=$DIR --with-zlib=$DIR --enable-hl --enable-fortran
make check
make install

export HDF5=$DIR
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH

## Install NETCDF C Library
cd $HOME/WRF/Downloads
tar -xvzf netcdf-c-4.9.0.tar.gz
cd netcdf-c-4.9.0/
export CPPFLAGS=-I$DIR/include 
export LDFLAGS=-L$DIR/lib
./configure --prefix=$DIR --disable-dap
make check
make install

export PATH=$DIR/bin:$PATH
export NETCDF=$DIR

## NetCDF fortran library
cd $HOME/WRF/Downloads
tar -xvzf netcdf-fortran-4.6.0.tar.gz
cd netcdf-fortran-4.6.0/
export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export CPPFLAGS=-I$DIR/include 
export LDFLAGS=-L$DIR/lib
export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz" 
./configure --prefix=$DIR --disable-shared
make check
make install

## MPICH
cd $HOME/WRF/Downloads
tar -xvzf mpich-3.3.1.tar.gz
cd mpich-3.3.1/
./configure --prefix=$DIR
make
make install

export PATH=$DIR/bin:$PATH

# libpng
cd $HOME/WRF/Downloads
export LDFLAGS=-L$DIR/lib
export CPPFLAGS=-I$DIR/include
tar -xvzf libpng-1.6.37.tar.gz
cd libpng-1.6.37/
./configure --prefix=$DIR
make
make install

# JasPer
cd $HOME/WRF/Downloads
unzip jasper-1.900.1.zip
cd jasper-1.900.1/
autoreconf -i
./configure --prefix=$DIR
make
make install
export JASPERLIB=$DIR/lib
export JASPERINC=$DIR/include

############################ WRF 4.1.2 #################################
## WRF v4.1.2
## Downloaded from git tagged releases
########################################################################
cd $HOME/WRF/Downloads
wget -c https://github.com/wrf-model/WRF/archive/v4.1.2.tar.gz
tar -xvzf v4.1.2.tar.gz -C $HOME/WRF
cd $HOME/WRF/WRF-4.1.2
./clean
./configure # 34, 1 for gfortran and distributed memory
./compile em_real

export WRF_DIR=$HOME/WRF/WRF-4.1.2

## WPSV4.1
cd $HOME/WRF/Downloads
wget -c https://github.com/wrf-model/WPS/archive/v4.1.tar.gz
tar -xvzf v4.1.tar.gz -C $HOME/WRF
cd $HOME/WRF/WPS-4.1
./configure #3
./compile

######################## Post-Processing Tools ####################
## ARWpost
cd $HOME/WRF/Downloads
wget -c http://www2.mmm.ucar.edu/wrf/src/ARWpost_V3.tar.gz
tar -xvzf ARWpost_V3.tar.gz -C $HOME/WRF
cd $HOME/WRF/ARWpost
./clean
sed -i -e 's/-lnetcdf/-lnetcdff -lnetcdf/g' $HOME/WRF/ARWpost/src/Makefile
./configure #3
sed -i -e 's/-C -P/-P/g' $HOME/WRF/ARWpost/configure.arwp
./compile

######################## Model Setup Tools ########################
## DomainWizard
cd $HOME/WRF/Downloads
wget -c http://esrl.noaa.gov/gsd/wrfportal/domainwizard/WRFDomainWizard.zip
mkdir $HOME/WRF/WRFDomainWizard
unzip WRFDomainWizard.zip -d $HOME/WRF/WRFDomainWizard
chmod +x $HOME/WRF/WRFDomainWizard/run_DomainWizard

######################## Static Geography Data ####################
# http://www2.mmm.ucar.edu/wrf/users/download/get_sources_wps_geog.html
cd $HOME/WRF/Downloads
wget -c https://www2.mmm.ucar.edu/wrf/src/wps_files/geog_high_res_mandatory.tar.gz
tar -xvzf geog_high_res_mandatory.tar.gz -C $HOME/WRF


## export PATH and LD_LIBRARY_PATH
echo "export PATH=$DIR/bin:$PATH" >> ~/.bashrc
echo "export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH" >> ~/.bashrc

Can you help me?
I installed NVIDIA HPC SDK to use pgf90. And I got the following error when installing WRF.
./compile em_real

Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
/lib/cpp -P -nostdinc -traditional-cpp -DUSE_NETCDF4_FEATURES -DWRFIO_NCD_LARGE_FILE_SUPPORT -DEM_CORE=1 -DNMM_CORE=0 -DNMM_MAX_DIM=2600 -DDA_CORE=0 -DWRFPLUS=0 -DIWORDSIZE=4 -DDWORDSIZE=8 -DRWORDSIZE=4 -DLWORDSIZE=4 -DNONSTANDARD_SYSTEM_SUBR -DWRF_USE_CLM -DUSE_NETCDF4_FEATURES -DWRFIO_NCD_LARGE_FILE_SUPPORT -DDM_PARALLEL -DNETCDF -DHDF5 -DLANDREAD_STUB=1 -DUSE_ALLOCATABLES -Dwrfmodel -DGRIB1 -DINTIO -DKEEP_INT_AROUND -DLIMIT_ARGS -DBUILD_RRTMG_FAST=1 -DSHOW_ALL_VARS_USED=0 -DCONFIG_BUF_LEN=65536 -DMAX_DOMAINS_F=21 -DMAX_HISTORY=25 -DNMM_NEST=0 test_io_idx.F90 > test_io_idx.f
Diffwrf io_int will be built later on in this compile. No need to rerun compile.
mpif90 -r4 -i4 -w -Mfree -byteswapio -mp -Minfo=mp -Mrecursive -mp -Minfo=mp -Mrecursive -o module_internal_header_util.o -c module_internal_header_util.f
mpicc -DFSEEKO64_OK -o io_int_idx.o -c -w -O3 io_int_idx.c
gfortran: error: unrecognized command line option '-r4'; did you mean '-r'?
gfortran: error: unrecognized command line option '-i4'
gfortran: error: unrecognized command line option '-Mfree'; did you mean '-free'?
gfortran: error: unrecognized command line option '-byteswapio'
gfortran: error: unrecognized command line option '-mp'; did you mean '-p'?
gfortran: error: unrecognized command line option '-Minfo=mp'
gfortran: error: unrecognized command line option '-Mrecursive'; did you mean '-frecursive'?
gfortran: error: unrecognized command line option '-mp'; did you mean '-p'?
gfortran: error: unrecognized command line option '-Minfo=mp'
gfortran: error: unrecognized command line option '-Mrecursive'; did you mean '-frecursive'?
makefile:62: recipe for target 'module_internal_header_util.o' failed
make[3]: [module_internal_header_util.o] Error 1 (ignored)
mpif90 -r4 -i4 -w -Mfree -byteswapio -mp -Minfo=mp -Mrecursi

Hi @kcj3784, sorry but I have no experience with pg compilers - only intel and gnu. You have to select carefully on the compiler choice screen. Goodluck.

@whatheway Thanks for giving a helping hand to @alycest. If you do not mind, I would be interested to know also if you had diagnosed and found a solution to the problem faced by @alycest

Cheers!

I believe it has to do with the libraries website path changing. NCAR and a lot of the other libraries website links changed. I had to modify my scripts to reflect that change. I recommend using github links as much as possible.

@jamal919 Hello Sir your Email please .

Hi, why do we want to add export LIBS="-lnetcdf -lhdf5_hl -lhdf5 -lz" in the script?

once we configure libraries using this, am getting an error while configuring flex library for compiling WRF chem as follows..
gcc -g -O -o flex ccl.o dfa.o ecs.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o libfl.a -lnetcdf -lhdf5_hl -lhdf5 -lz
/usr/bin/ld: cannot find -lnetcdf: No such file or directory
/usr/bin/ld: cannot find -lhdf5_hl: No such file or directory
/usr/bin/ld: cannot find -lhdf5: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [Makefile:97: flex] Error 1

@musaidpp in your command, something is not right, it should be prefixed with a -L/directory/to/libraries/lib. This is where you should have libnetcdf.so libnetcdff.so etc. e.g. your command should look something like the following -

gcc -g -O -o flex ccl.o dfa.o ecs.o gen.o main.o misc.o nfa.o parse.o scan.o skel.o sym.o tblcmp.o yylex.o libfl.a -L/path/to/lib -lnetcdf -lhdf5_hl -lhdf5 -lz

@jamal919 i have given all the path as it is in ur script. Bt I don't understand why it's not showing in the error message. Anyhow, without the LIBS="" , the chem was compiled and this can you please tell me, why should we need that line in our bashrc or in the script?

@jamal919 i have given all the path as it is in ur script. Bt I don't understand why it's not showing in the error message. Anyhow, without the LIBS="" , the chem was compiled and this can you please tell me, why should we need that line in our bashrc or in the script?

It was needed to compile netcdf-fortran, as netcdf-fortran links to netcdf-c, and it links to hdf. But you do not need to add this line to bashrc.

hello sir @jamal919 i need some information about WRF-Ocean . please

The zlib download link needs to be updated to https://www.zlib.net/zlib-1.2.13.tar.gz. https://www.zlib.net/zlib-1.2.12.tar.gz is not available any more.

The NETCDF download links are also broken. Replace with
wget -c https://downloads.unidata.ucar.edu/netcdf-c/4.9.0/netcdf-c-4.9.0.tar.gz
wget -c https://downloads.unidata.ucar.edu/netcdf-fortran/4.6.0/netcdf-fortran-4.6.0.tar.gz

Thanks @demiangomez. Updated.

Hello, Sir

Thank you for the script, but while installing the HDF5 i got the following errors:
***** 12 FAILURES! *****
Command exited with non-zero status 1
0.50user 0.10system 0:00.61elapsed 98%CPU (0avgtext+0avgdata 4068maxresident)k
0inputs+0outputs (0major+18193minor)pagefaults 0swaps
Makefile:3143: recipe for target 'dt_arith.chkexe_' failed
make[4]: *** [dt_arith.chkexe_] Error 1
make[4]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:3128: recipe for target 'build-check-s' failed
make[3]: *** [build-check-s] Error 2
make[3]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:3123: recipe for target 'test' failed
make[2]: *** [test] Error 2
make[2]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:2905: recipe for target 'check-am' failed
make[1]: *** [check-am] Error 2
make[1]: Leaving directory '/home/brillianzha/WRF/Downloads/hdf5-1.10.5/test'
Makefile:654: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1

Thanks in advance

@Brillianzha, You have to check the whole output and figure out where the failure happens. Please check if the output of the ./configure script L49 to see if all the dependencies are picked up correctly.

Sir, for setting paths for libraries do we need to type the export commands in bashrc sourcefile or can we type them in WRF/Downloads directory too. please clarify

Sir, for setting paths for libraries do we need to type the export commands in bashrc sourcefile or can we type them in WRF/Downloads directory too. please clarify

If you follow the script line-by-line, it is probably better to run the export commands in the terminal directly.

However, you will notice that at the end of the above script the PATH and LD_LIBRARY_PATH is updated in the .bashrc file.

I am thankful jamal919. It took me maybe 3 weeks struggeling trying to install it myself based on the website, and having problem with export and variables and now I just launched your script and it compiles, the ideal example worked pretty fine! I can start learning now! THANKS!

I am thankful jamal919. It took me maybe 3 weeks struggeling trying to install it myself based on the website, and having problem with export and variables and now I just launched your script and it compiles, the ideal example worked pretty fine! I can start learning now! THANKS!

Hi @kermarrecg , thanks for your feedback and I am glad to know that this helped you. Happy exploring WRF. Cheers!

Prio Jamal Bhai,
I use Ubuntu 20.04 LS version from USB. I am trying to compile the WRF in my system from several weeks but have been unfortunate every time. This time I tried your processes. I followed all of your codes serially. Compilation of LIBRARIES are okay. Configuration of WRF-4.1.2 is also okay. But whenever I tried to compile the WRF, every time it returns with errors that some module files were not found, some argument not matching, and the .exe files were not created. In fact, I repeatedly got the same results from my earlier efforts with instructions from other engineers and with other WRF versions (from 3.8 to 4.1.2). Can you please knock me a little to find where the problem is?

Thanks a lot for the information. I followed your protocols for installing WRF. I installed WRF on Ubuntu 22.04 Server edition (Server Name: HPE ProLiant DL385 Gen10 Plus )

I faced some issues while following your steps, but I resolved those issues.

[1] Along with sudo apt install gcc gfortran g++ libtool automake autoconf make m4 grads default-jre csh we need to install sudo apt install unzip also

[2] WRF 4.1.2 throws some physics errors, but WRF 4.4.2 will work.

[3]For ARW Post, we need to do the following steps:

cd $HOME/WRF/Downloads
wget -c http://www2.mmm.ucar.edu/wrf/src/ARWpost_V3.tar.gz
tar -xvzf ARWpost_V3.tar.gz -C $HOME/WRF_DIR
cd $HOME/WRF/ARWpost
./clean
sed -i -e 's/-lnetcdf/-lnetcdff -lnetcdf/g' $HOME/WRF/ARWpost/src/Makefile
export NETCDF=$DIR
./configure
export GCC_VERSION=$(/usr/bin/gcc -dumpfullversion | awk '{print$1}')
export GFORTRAN_VERSION=$(/usr/bin/gfortran -dumpfullversion | awk '{print$1}')
export GPLUSPLUS_VERSION=$(/usr/bin/g++ -dumpfullversion | awk '{print$1}')
export GCC_VERSION_MAJOR_VERSION=$(echo $GCC_VERSION | awk -F. '{print $1}')
export GFORTRAN_VERSION_MAJOR_VERSION=$(echo $GFORTRAN_VERSION | awk -F. '{print $1}')
export GPLUSPLUS_VERSION_MAJOR_VERSION=$(echo $GPLUSPLUS_VERSION | awk -F. '{print $1}')
export version_10="10"
if [ $GCC_VERSION_MAJOR_VERSION -ge $version_10 ] || [ $GFORTRAN_VERSION_MAJOR_VERSION -ge $version_10 ] || [ $GPLUSPLUS_VERSION_MAJOR_VERSION -ge $version_10 ]
then
sed -i '32s/-ffree-form -O -fno-second-underscore -fconvert=big-endian -frecord-marker=4/-ffree-form -O -fno-second-underscore -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch /g' configure.arwp
fi
sed -i -e 's/-C -P -traditional/-P -traditional/g' $HOME/WRF/ARWpost/configure.arwp
./compile
export PATH=$HOME/WRF/ARWpost/ARWpost.exe:$PATH

Hi @jamal919 , do you have the updated list of libraries for installing the version 4.4 of WPS and WRF. Kindly share them if present, cause I am encountering errors while installing

I followed the same step as suggested by @jamal919 .. and installed the following WRF and WPS versions:

For WRF: git clone --recurse-submodule https://github.com/wrf-model/WRF.git
For WPS: git clone --recurse-submodule https://github.com/wrf-model/WPS.git

Thank you, @jamal919 for saving my day!!! It worked perfectly for compiling WRFv4.6. =)

The only part that I modified is to allow for mismatched argument when using higher version of gfortran --
According to https://forum.mmm.ucar.edu/threads/how-to-fix-rank-mismatch-between-actual-argument-at-1-and-actual-argument-at-2-scalar-and-rank-1.18391/#post-44568. When using higher levels of gfortran (say 11.3) to compile MPICH and WRF-Chem, it will cause a rank mismatch as shown below:

checking whether gfortran allows mismatched arguments... no
configure: error: The Fortran compiler gfortran will not compile files that call

allowing for mismatched arguments and then re-configure MPICH:

export fallow_argument=-fallow-argument-mismatch
export boz_argument=-fallow-invalid-boz
export FFLAGS="$fallow_argument $boz_argument -m64"
export FCFLAGS="$fallow_argument $boz_argument -m64"

Hi @wde0924, I am glad that it helped, and thanks for documenting this argument-mismatch issue that was introduced in GCC 9/10.

@ everyone who found this helpful and commented, I thank you too. I am seeing quite a few comments about the problem faced, I am sorry that it did not work for you, and I could not be of help when you commented.

It has been a long time, and is in my to-do list to revise this guide - I should do that at some point. Particularly, there are some extra environment variable (cflags, libs) often needed with the latest version of netcdf-c and netcdf-fortran. I will update that too - hopefully sooner than later.

Cheers!