This page presents a project idea for a Python framework on job submission, with the intent to trigger collaboration on the topics.
- Kenneth Hoste, Stijn De Weirdt (HPC-UGent)
Kenneth Hoste boegel
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ eb --search Python/Python-3.2.3-goolf --robot ~/TMP | |
| == temporary log file in case of crash /var/folders/6y/x4gmwgjn5qz63b7ftg4j_40m0000gn/T/easybuild-_veog_.log | |
| == Searching (case-insensitive) for 'Python/Python-3.2.3-goolf' in /Users/kehoste/TMP | |
| == Searching (case-insensitive) for 'python/python-3.2.3-goolf' in /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs | |
| == * /Users/kehoste/TMP/EasyBuild-v1.8.2/easybuild-easyconfigs/easybuild/easyconfigs/p/Python/Python-3.2.3-goolf-1.4.10.eb | |
| == * /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/p/Python/Python-3.2.3-goolf-1.4.10.eb | |
| == temporary log file /var/folders/6y/x4gmwgjn5qz63b7ftg4j_40m0000gn/T/easybuild-_veog_.log has been removed. | |
| $ eb -S Python/Python-3.2.3-goolf --robot ~/TMP | |
| == temporary log file in case of crash /var/folders/6y/x4gmwgjn5qz63b7ftg4j_40m0000gn/T/easybuild-Wrx3ub.log |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ eb HPL-2.0-goolf-1.4.10.eb --dry-run -r | |
| == temporary log file in case of crash /var/folders/6y/x4gmwgjn5qz63b7ftg4j_40m0000gn/T/easybuild-hrxcCr.log | |
| == Dry run: printing build status of easyconfigs and dependencies | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/g/GCC/GCC-4.7.2.eb (module: GCC/4.7.2) | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/h/hwloc/hwloc-1.6.2-GCC-4.7.2.eb (module: hwloc/1.6.2-GCC-4.7.2) | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenMPI/OpenMPI-1.6.4-GCC-4.7.2.eb (module: OpenMPI/1.6.4-GCC-4.7.2) | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/g/gompi/gompi-1.4.10.eb (module: gompi/1.4.10) | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/o/OpenBLAS/OpenBLAS-0.2.6-gompi-1.4.10-LAPACK-3.4.2.eb (module: OpenBLAS/0.2.6-gompi-1.4.10-LAPACK-3.4.2) | |
| * [x] /Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs/f/FFTW/FFTW-3.3.3-gompi-1.4.10.eb (module: FFTW/3.3. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| == 2013-12-16 17:20:09,107 main.EnvironmentModulesC INFO Prepended list of module paths with path used by EasyBuild: /Users/kehoste/.local/easybuild/modules/all | |
| == 2013-12-16 17:20:09,107 main.EnvironmentModulesC INFO $MODULEPATH set based on list of module paths: /Users/kehoste/.local/easybuild/modules/all | |
| == 2013-12-16 17:20:09,107 main.filetools INFO Command modulecmd found at /Users/kehoste/Modules/3.2.10/bin/modulecmd | |
| == 2013-12-16 17:20:09,108 main.EnvironmentModulesC INFO Full path for module command is /Users/kehoste/Modules/3.2.10/bin/modulecmd, so using it | |
| == 2013-12-16 17:20:09,378 main INFO Dry run: printing build status of easyconfigs and dependencies | |
| CFGS=/Users/kehoste/work/easybuild-easyconfigs/easybuild/easyconfigs | |
| * [ ] $CFGS/i/icc/icc-2013.5.192.eb (module: icc/2013.5.192) | |
| * [ ] $CFGS/i/ifort/ifort-2013.5.192.eb (module: ifort/2013.5.192) | |
| * [ ] $CFGS/i/impi/impi-4.1.1.036.eb (module: impi/4.1.1.036) | |
| * [ ] $CFGS/i/imkl/imkl-11.0.5.192.eb (module: imkl/11.0.5.192) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Lightning tutorials | |
| - Using OpenMP to simply parallelize cpu-intensive C code | |
| Configuration Managment | |
| - Kadeploy: from scalable and reliable bare-metal provisioning to a reconfigurable experimental testbed | |
| - Quattor - configuration and fabric management done right | |
| - An Overview of Aquilon |
boegel
/ gist:8062785
Created
December 20, 2013 22:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ## | |
| # This file is an EasyBuild reciPY as per https://github.com/hpcugent/easybuild | |
| # | |
| # Copyright:: Copyright 2012-2013 University of Luxembourg / LCSB, Cyprus Institute / CaSToRC, Ghent University | |
| # Authors:: Wiktor Jurkowski <[email protected]>, Fotis Georgatos <[email protected]>, \ | |
| # George Tsouloupas <[email protected]>, Kenneth Hoste | |
| # License:: MIT/GPL | |
| # $Id$ | |
| # | |
| # This work implements a part of the HPCBIOS project and is a component of the policy: |
boegel
/ gist:8114456
Created
December 24, 2013 15:01
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| == 2013-12-24 15:58:37,334 main.EnvironmentModulesC DEBUG Current MODULEPATH: /Users/kehoste/.local/easybuild/modules/all:/tmp/EBv1.10/modules/all | |
| == 2013-12-24 15:58:37,334 main.EnvironmentModulesC DEBUG Adjusted LD_LIBRARY_PATH from '/tmp/EBv1.10/software/EasyBuild/1.10.0/lib' to '/tmp/EBv1.10/software/EasyBuild/1.10.0/lib' | |
| == 2013-12-24 15:58:37,334 main.EnvironmentModulesC DEBUG Running module command '/Users/kehoste/Modules/3.2.10/bin/modulecmd python --terse avail EasyBuild/1.0.0' from /private/tmp | |
| == 2013-12-24 15:58:37,357 main.EnvironmentModulesC DEBUG Correcting paths in LD_LIBRARY_PATH from ['/tmp/EBv1.10/software/EasyBuild/1.10.0/lib'] to ['/tmp/EBv1.10/software/EasyBuild/1.10.0/lib'] | |
| == 2013-12-24 15:58:37,358 main.EnvironmentModulesC DEBUG 'module available EasyBuild/1.0.0' gave 1 answers: ['EasyBuild/1.0.0'] | |
| == 2013-12-24 15:58:37,358 main INFO EasyBuild/1.0.0 is already installed (module found), skipping | |
| == EasyBuild/1.0.0 is already installed (module found), skipping | |
| == 2013-12-24 15:58:37,35 |
boegel
/ gist:8483302
Created
January 17, 2014 23:02
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| -bash-4.1$ ./bdist_rpm.sh &> /dev/null | |
| -bash-4.1$ rpm -q --filesbypkg -p dist/python-vsc-mympirun-3.2.3-1.ug.noarch.rpm | grep vsc/__init__.py | |
| -bash-4.1$ rm -rf /tmp/vsc40023/vsc-mympirun-test/*; mkdir -p /tmp/vsc40023/vsc-mympirun-test/lib/python2.6/site-packages; export PYTHONPATH=/tmp/vsc40023/vsc-mympirun-test/lib/python2.6/site-packages:$PYTHONPATH; python setup.py install --prefix=/tmp/$USER/vsc-mympirun-test --single-version-externally-managed --record /tmp/$USER/foo --no-compile &> /dev/null-bash-4.1$ | |
| -bash-4.1$ | |
| -bash-4.1$ | |
| -bash-4.1$ | |
| -bash-4.1$ TESTDIR=/tmp/vsc40023/vsc-mympirun-test; rm -rf $TESTDIR/*; mkdir -p $TESTDIR/lib/python2.6/site-packages; export PYTHONPATH=$TESTDIR/lib/python2.6/site-packages:$PYTHONPATH; python setup.py install --prefix=$TESTDIR --single-version-externally-managed --record /tmp/$USER/foo --no-compile &> /dev/null-bash-4.1$ ls $TESTDIR/lib/python2.6/site-packages/vsc/__init__.py | |
| /tmp/vsc40023/vsc-mympirun-test/lib/python2.6/site-packages/vsc/__init__.py | |
| -bash-4.1$ | |
| -ba |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # EASYCONFIGFORMAT 2.0 | |
| # this is a version test | |
| """ | |
| docstring test | |
| @author: Stijn De Weirdt (UGent) | |
| @maintainer: Kenneth Hoste (UGent) | |
| """ | |
| name = "toy" | |
| homepage = 'http://hpcugent.github.com/easybuild' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| dependency_1-gcc = (GCC, 4.7.2) | |
| dependency_2-openmpi = (OpenMPI, 1.6.4, GCC == 4.7.2) | |
| dependency_3-openblas = (OpenBLAS, >= 0.2.6, suffix:-LAPACK-3.4.2, gompi == 1.1.10 suffix:-no-OFED) | |
| dependency_4-fftw = (FFTW, 3.3.3, gompi == 1.1.10 suffix:-no-OFED) | |
| dependency_5-scalapack = (ScaLAPACK, 2.0.2, suffix:-OpenBLAS-0.2.6-LAPACK-3.4.2, gompi == 1.1.10 suffix:-no-OFED) | |
| # or |
boegel
/ python_framework_resource_managers.md
Last active
August 29, 2015 13:56
Python framework for job submission - project idea