Python
In this system, you can install pyenv in the user's environment by executing the following script.
Python version control is possible with pyenv. Please see below for details.
Setup pyenv environment
Execute the following command to install pyenv in your home directory on super and gpu.
$ bash /work/app/pyenv/pyenv-setup-20210617.bash
After executing the installation script, a file called bash_env will be generated in the current directory.
To load pyenv by default, copy the contents to ~/.bash_profile with the following command.
$ cat bash_env >> ~/.bash_profile
After installing pyenv, log out from the front end node once, and then log in again.
Basic usage of pyenv
Install Python
$ pyenv install --list ←View available Python versions ・・・・ $ pyenv install 3.7.8←Install Python 3.7.8
Switch python versions
$ pyenv versions ←Check the installed version * system (set by /home/userA/.pyenv/version) 3.7.8 $ pyenv global 3.7.8 ←Switch to Python 3.7.8 $ python --version
You can also add packages using pip.
For more detailed usage, please check the pyenv documentation etc.
Simple Python version management
https://github.com/pyenv/pyenv
How to run Python
When you execute Python programs with high load, please use computing nodes instead of front end nodes.
Example job scripts
Large-Scale Parallel Computing Server
#!/bin/sh #PBS -l select=1 #PBS -q P_016 #PBS -N sample DIRNAME=`basename $PBS_O_WORKDIR` WORKDIR=/work/$USER/$PBS_JOBID mkdir -p $WORKDIR cp -raf $PBS_O_WORKDIR $WORKDIR cd $WORKDIR/$DIRNAME aprun python program.py cd; if cp -raf $WORKDIR/$DIRNAME $PBS_O_WORKDIR/.. ; then rm -rf $WORKDIR; fi
Accelerator Server
(Running in interactive mode)
userA@gpu2:~> qsub -I -q CA_001 qsub: waiting for job 70568.gpu1 to start qsub: job 70568.gpu1 ready -bash-4.2$ python Python 3.7.8 (default, Mar 25 2021, 09:54:46) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux Type "help", "copyright", "credits" or "license" for more information. >>>
You can also run python script as a job.
(Example)
#!/bin/sh #PBS -l select=1 #PBS -q CA_001 #PBS -N sample DIRNAME=`basename $PBS_O_WORKDIR` WORKDIR=/work/$USER/$PBS_JOBID mkdir -p $WORKDIR cp -raf $PBS_O_WORKDIR $WORKDIR cd $WORKDIR/$DIRNAME python program.py cd; if cp -raf $WORKDIR/$DIRNAME $PBS_O_WORKDIR/.. ; then rm -rf $WORKDIR; fi