Astropy has a nice description how arbitrary packages can be installed into the bundled version of CASA (all versions before 6): https://docs.astropy.org/en/stable/install.html#installing-astropy-into-casa
This works well but can be a bit annoying to restart CASA multiple times and wait in between. The following simple bash script solves this:
# packages to be installed
####################################################################################################