Bootstrap example to get around bugs with panda
and rucio
on CVMFS and PYTHONPATH
pollution as described in the March 2023 PATHelp email thread "ROOT undefined symbol error when setting up ROOT and Rucio together after setupATLAS".
The PYTHONPATH
is polluted with Python 3.6, so make an environment with the last pip
and pandas
releases that were compatible with Python 3.6.
The boostrap.sh
Bash script sets up a Python virtual environment that has a version of pip
that won't break with Python 3.6 which is further used in example.sh
.
After the first time the environment has been setup it can be setup without the bootstrap step by sourcing example.sh
again.
$ ssh <somewhere with CVMFS>
# $ git clone https://gist.github.com/b3c1815b06a972582716ef358619b982.git bootstrap-example # if you don't have SSH keys setup
$ git clone [email protected]:b3c1815b06a972582716ef358619b982.git bootstrap-example # if you do have SSH keys
$ cd bootstrap-example
$ . example.sh
relevant output:
...
************************************************************************
Requested: rucio xrootd panda pyami ...
Setting up python 3.9.14-x86_64-centos7 ...
Setting up emi 23.02.22-snapshot-x86_64-centos7 ...
Setting up rucio 1.30.3 ...
Info: Setting compatibility to centos7
Info: Set RUCIO_AUTH_TYPE to x509_proxy
Setting up xrootd 5.5.3-x86_64-centos7 ...
Setting up panda 1.5.40 ...
Setting up pyami pyAMI-5.1.3 ...
>>>>>>>>>>>>>>>>>>>>>>>>> Information for user <<<<<<<<<<<<<<<<<<<<<<<<<
emi:
Your proxy has 10h:28m:55s remaining
pyami:
Using voms proxy for authentication.
************************************************************************
...
Python 3.9.14 (main, Sep 21 2022, 20:28:08)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
PYTHONPATH=/home/feickert/bootstrap-example/romain-example/lib/python3.9/site-packages:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/pyAmi/pyAMI-5.1.3/lib:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/PandaClient/1.5.40/lib/python3.6/site-packages:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/xrootd/5.5.3-x86_64-centos7/lib/python3.9/site-packages:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/rucio-clients/1.30.3/lib/python3.6/site-packages:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/emi/23.02.22-snapshot-x86_64-centos7/usr/lib64/python3.6/site-packages:/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/emi/23.02.22-snapshot-x86_64-centos7/usr/lib/python3.6/site-packages
# python -m pip list
Package Version
------------------ ---------
argcomplete 1.12.3
attrs 22.2.0
awkward 2.1.1
awkward-cpp 12
certifi 2022.12.7
charset-normalizer 2.0.12
dataclasses 0.8
decorator 5.1.1
dogpile.cache 1.1.5
gfal2_util 1.8.0
idna 3.4
importlib-metadata 4.8.3
jsonschema 3.2.0
numpy 1.24.2
packaging 23.0
panda-client 1.5.40
pandas 1.1.5
pbr 5.11.1
pip 21.3.1
python-dateutil 2.8.2
pytz 2022.7.1
requests 2.27.1
rucio-clients 1.30.3
setuptools 67.6.0
six 1.16.0
stevedore 3.5.2
tabulate 0.8.10
typing_extensions 4.1.1
uproot 5.0.5
urllib3 1.26.8
wheel 0.40.0
xrootd 0.0.0
zipp 3.6.0
WARNING: You are using pip version 21.3.1; however, version 23.0.1 is available.
You should consider upgrading via the '/home/feickert/bootstrap-example/romain-example/bin/python -m pip install --upgrade pip' command.
# python -m pip list --local
Package Version
--------------- --------
awkward 2.1.1
awkward-cpp 12
numpy 1.24.2
packaging 23.0
pandas 1.1.5
pip 21.3.1
python-dateutil 2.8.2
pytz 2022.7.1
setuptools 67.6.0
uproot 5.0.5
wheel 0.40.0
WARNING: You are using pip version 21.3.1; however, version 23.0.1 is available.
You should consider upgrading via the '/home/feickert/bootstrap-example/romain-example/bin/python -m pip install --upgrade pip' command.
command -v rucio: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/rucio-clients/1.30.3/bin/rucio
command -v xrootd: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/xrootd/5.5.3-x86_64-centos7/bin/xrootd
command -v prun: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/PandaClient/1.5.40/bin/prun
command -v ami: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/pyAmi/pyAMI-5.1.3/bin/ami
<module 'rucio' from '/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/rucio-clients/1.30.3/lib/python3.6/site-packages/rucio/__init__.py'>
<module 'pyAMI' from '/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/pyAmi/pyAMI-5.1.3/lib/pyAMI/__init__.py'>
<module 'XRootD' from '/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/xrootd/5.5.3-x86_64-centos7/lib/python3.9/site-packages/XRootD/__init__.py'>
<module 'pandaclient' from '/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/PandaClient/1.5.40/lib/python3.6/site-packages/pandaclient/__init__.py'>
<module 'uproot' from '/home/feickert/bootstrap-example/romain-example/lib/python3.9/site-packages/uproot/__init__.py'>
<module 'pandas' from '/home/feickert/bootstrap-example/romain-example/lib/python3.9/site-packages/pandas/__init__.py'>