Astropy has a nice description how arbitrary packages can be installed into the bundled version of CASA (all versions before 6): https://docs.astropy.org/en/stable/install.html#installing-astropy-into-casa
This works well but can be a bit annoying to restart CASA multiple times and wait in between. The following simple bash script solves this:
# packages to be installed
####################################################################################################
# Not all python packages work. Just try it and if using the package fails, the installation will
# only caused CASA to take up a few bytes more storage but not cause harm.
PACKAGES=(astropy aplpy=1.1.1)
# install pip into CASA
####################################################################################################
casa --nologger --log2term <<-casaINPUT
from setuptools.command import easy_install
easy_install.main(['--user', 'pip'])
casaINPUT
# install the packages
####################################################################################################
FormattedPackageList="["
for i in "${PACKAGES[@]}"
do
FormattedPackageList=$FormattedPackageList"'$i', "
done
FormattedPackageList=${FormattedPackageList::-2}"]"
casa --nologger --log2term <<-casaINPUT
import pip
for package in $FormattedPackageList: pip.main(['install', package, '--user'])
casaINPUT