Python

In this system, you can install pyenv in the user's environment by executing the following script.
Python version control is possible with pyenv. Please see below for details.

Execute the following command to install pyenv in your home directory on super and gpu.

$ bash /work/app/pyenv/pyenv-setup-20210617.bash


After executing the installation script, a file called bash_env will be generated in the current directory.
To load pyenv by default, copy the contents to ~/.bash_profile with the following command.

$ cat bash_env >> ~/.bash_profile


After installing pyenv, log out from the front end node once, and then log in again.

Install Python

$ pyenv install --list ←View available Python versions
・・・・
$ pyenv install 3.7.8←Install Python 3.7.8


Switch python versions

$ pyenv versions ←Check the installed version
* system (set by /home/userA/.pyenv/version)
  3.7.8
$ pyenv global 3.7.8 ←Switch to Python 3.7.8
$ python --version 


You can also add packages using pip.
For more detailed usage, please check the pyenv documentation etc.

Simple Python version management
https://github.com/pyenv/pyenv

When you execute Python programs with high load, please use computing nodes instead of front end nodes.

Example job scripts

Large-Scale Parallel Computing Server

#!/bin/sh
#PBS -l select=1
#PBS -q P_016
#PBS -N sample
DIRNAME=`basename $PBS_O_WORKDIR` 
WORKDIR=/work/$USER/$PBS_JOBID 
mkdir -p $WORKDIR
cp -raf $PBS_O_WORKDIR $WORKDIR 
cd $WORKDIR/$DIRNAME

aprun python program.py

cd; if cp -raf $WORKDIR/$DIRNAME $PBS_O_WORKDIR/.. ; then rm -rf $WORKDIR; fi


Accelerator Server

(Running in interactive mode)

userA@gpu2:~> qsub -I -q CA_001
qsub: waiting for job 70568.gpu1 to start
qsub: job 70568.gpu1 ready

-bash-4.2$ python
Python 3.7.8 (default, Mar 25 2021, 09:54:46) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>


You can also run python script as a job.
(Example)

#!/bin/sh
#PBS -l select=1
#PBS -q CA_001
#PBS -N sample
DIRNAME=`basename $PBS_O_WORKDIR`
WORKDIR=/work/$USER/$PBS_JOBID
mkdir -p $WORKDIR
cp -raf $PBS_O_WORKDIR $WORKDIR
cd $WORKDIR/$DIRNAME

python program.py

cd; if cp -raf $WORKDIR/$DIRNAME $PBS_O_WORKDIR/.. ; then rm -rf $WORKDIR; fi
  • application/python.txt
  • Last modified: 2021/06/17 23:52
  • by ccms