====== Compilers and Libraries ====== Describes how to use the compiler and libraries on a Large-Scale Parallel Computing Server. ===== Compiler ===== Following compilers are available on the supercomputer.\\ ^Compiler ^Version ^Notes ^ |Cray Compiler Fortran/C/C++ |8.7.10\\ 8.6.5 |default: 8.6.5 | |Intel Compiler Fortran/C/C++ |19.1.3.304\\ 19.1.0.166\\ 19.0.2.187\\ 18.0.2.199\\ 17.0.4.196 |default: 18.0.2.199 | |PGI Compiler Fortran/C/C++ |19.1-0\\ 18.5-0 |default: 19.1-0 | |GNU Compiler Fortran/C/C++ |4.9.3\\ 5.3.0\\ 6.1.0\\ 7.3.0\\ 8.3.0 |default: 7.3.0 | ------ ==== Programming environment ==== \\ **Command**\\ The command names of the Fortran, C, C ++ compiler are unified to ftn, cc, and CC.\\ The compiler called by the command will switch automatically by switching the programming environment.\\ ^Compiler ^Command ^Cray Compiler ^Intel Compiler ^PGI Compiler ^gnu compiler ^ |Fortran |ftn |crayftn |ifort |pgf90 |gfortran | |C |cc |craycc |icc |pgcc |gcc | |C++ |CC |crayCC |icpc |pgc++ |g++ | Use ftn, cc, CC command in the case compiling MPI programs.\\ MPI libraries are linked automatically. \\ **Switching programming environment**\\ The module files is as follows.\\ ^Cray compiler ^Intel compiler ^PGI compiler ^gnu compiler ^ |PrgEnv-cray |PrgEnv-Intel |PrgEnv-pgi |PrgEnv-gnu | Switch the programming environment with the module switch command.\\ PrgEnv-cray is default.\\ To switch to the Intel compiler, execute as follows. $ module switch PrgEnv-cray PrgEnv-intel \\ **Common options**\\ The following options are common to all programming environments.\\ ^Option ^Description ^ |-craype-verbose |Display set options. | |-static |Set static linking. | |-dynamic |Set dynamic linking. | |-shared |Make shared library. | |-help |Display available options in the loaded programming environment. | ------ ==== Cray Compiler ==== \\ **Setting of program environment**\\ Cray compiler is the default one on super.\\ If you want to change the compiling environment to other, execute a following command.\\ \\ ・Example change to the environment of Intel compiler. $ module switch PrgEnv-intel PrgEnv-cray \\ If you want to change the compiler version, execute a following command.\\ \\ ・Example change the compiler version $ module avail cce cce/8.6.5(default) cce/8.7.10 $ module switch cce cce/8.7.10 \\ **Compile Options**\\ \\ ・Optimization Options ^Option ^Description ^ |-o outfile |Set output file. If ‘-o’ option is not specified, the default output file is a.out.| |-llibrary_name |Specify libraries to be linked. | |-Llibrary_path |Specify paths to be searched for the libraries. | |-h autothread |Enables auto threading (default: disabled) | |-h omp/noomp |Enable / disables OpenMP directives. (default: enabled) | |-h thread[0-3] |Specify optimization level of OpenMP (3:Full optimization, default is 2) | |-O [0-3] |Specify automatic threading level. (3:Highest level, default is 2) | |-h ipa[0-5] |Specify the level of interprocedural optimization. | \\ ・Options for Fortran ^Option ^Description ^ |-e0 |Initialize undefined local stack variables to 0. | |-ev |Assign variables to stack (There are exceptions.) | |-ez |Initialize the memory allocated by allocate() statements to zero. | |-f [free|fixed] |Set the format of source files. | \\ ・Options for C ^Option ^Description ^ |-h c99 |Compile with C99 standard manner. | |-h noc99 |Compile without C99 standard manner. | |-h zero |Initialize undefined local stack variables to 0. | \\ ・Debugging options ^Option ^Description ^ |-g |Output debugging information. | |-G [0-2] |Manage debugging information. (0: Most detailed. -G0=-g） | |-K trap=... |Trap exception handling. [fp | divz | denorm | ... ] | \\ **Example**\\ \\ ・Make the execution module ’hello.out’ from the Fortran source file ’hello.f.’ in fixed form. $ ftn -f fixed -o hello.out hello.f \\ ・Make the execution module ’hello.out’ from the Fortran source file ‘hello.f90’ in free form. $ ftn -f free -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' auto-threaded from the Fortran source file 'hello.f' in fixed form. $ ftn -h autothread -f fixed -o hello.out hello.f \\ ・Make the execution module 'hello.out' auto-threaded from the Fortran source file 'hello.f90' in free form. $ ftn -h autothread -f free -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' from the C source file 'hello.c'. $ cc hello.c -o hello.out \\ ・Make the execution module 'hello.out' auto-threaded from the C source file 'hello.c'. $ cc -h autothread hello.c -o hello.out \\ ・Make the execution module 'hello.out' from the C++ source file 'hello.cpp'. $ CC hello.cpp -o hello.out \\ ・Make the execution module 'hello.out' auto-threaded from the C++ source file 'hello.cpp'. $ CC -h autothread hello.cpp -o hello.out ------ ==== Intel Compiler ==== \\ **Setting of program environment**\\ Change the program environment by the following line, because Cray compiler is default on super.\\ \\ ・Example change to the environment of Intel compiler. $ module switch PrgEnv-cray PrgEnv-intel \\ If you want to change the compiler version, execute a following command.\\ \\ ・Example change the compiler version $ module avail intel intel/17.0.4.196 intel/18.0.2.199(default) intel/19.0.2.187 intel/19.1.0.166 $ module switch intel intel/19.0.2.187 \\ **Compile Options**\\ \\ ・Optimization options and others ^Option ^Description ^ |-o outfile |Set output file. If ‘-o’ option is not specified, the default output file is a.out. | |-llibrary_name |Specify libraries to be linked. | |-Llibrary_path |Specify paths to be searched for the libraries. | |-O0|-O1|-O2|-O3 |Set optimizing options. -O2 is default. | |-fast |Set the whole program execution speed maximum. The following options are set automatically.\\ -ipo、-O3、-no-prec-div、-static、-fp-model fast=2、-xHost | |-parallel |Compile the input source file enabling auto-threading. | |-openmp |Compile the input source file enabling OpenMP. | \\ ・Options for Fortran ^Option ^Description ^ |-free|-fixed |Set the format of source files. | \\ ・Debugging Options ^Option ^Description ^ |-g |Output debugging information. | |-traceback |Output traceback when an error occurs. | |-fpe[0-3] |Trap exception handling. (-fpe0 is most detailed) | \\ **Example**\\ \\ ・Make the execution module 'hello.out' from Fortran source file 'hello.f' in fixed form. $ ftn -fixed -o hello.out hello.f \\ ・Make the execution module ’hello.out’ from the Fortran source file ‘hello.f90’ in free form. $ ftn -free -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' auto-threaded from the Fortran source file 'hello.f' in free form. $ ftn -fixed -parallel -o hello.out hello.f \\ ・Make the execution module 'hello.out' auto-threaded from the Fortran source file 'hello.f90' in free form. $ ftn -free -parallel -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' from C source file 'hello.c'. $ cc -o hello.out hello.c \\ ・Make the execution module 'hello.out' auto-threaded from C source file 'hello.c'. $ cc -parallel -o hello.out hello.c \\ ・Make the execution module 'hello.out' from the C++ source file 'hello.cpp'. $ CC -o hello.out hello.cpp \\ ・Make the execution module 'hello.out' auto-threaded from the C++ source file 'hello.cpp'. $ CC -parallel -o hello.out hello.cpp ------ ==== PGI Compiler ==== \\ **Setting of program environment**\\ Change the environment by the following line, because Cray compiler is default on super.\\ \\ ・Example change from environment of Cray compiler $ module switch PrgEnv-cray PrgEnv-pgi \\ If you want to change the compiler version, execute a following command.\\ \\ ・Example change the compiler version $ module avail pgi pgi/18.5 pgi/19.1(default) $ module switch pgi pgi/18.5 \\ **Compile Options**\\ \\ ・Optimization Options and Others ^Option ^Description ^ |-o outfile |Set output file. If ‘-o’ option is not specified, the default output file is a.out. | |-llibrary_name |Specify libraries to be linked. | |-Llibrary_path |Specify paths to be searched for the libraries. | |-O0|-O1|-O2|-O3|-O4 |Set optimizing options. -O2 is default. | |-fast |Enable general optimization flag. | |-Mconcur |Compile the input source file enabling auto threading. | |-mp |Compile the input source file enabling OpenMP. | \\ ・Option for Fortran ^Option ^Description ^ |-Mfree|-Mfixed |Set Specify program format of free or fixed. | \\ ・Debugging options ^Option ^Description ^ |-g|-gopt |Output debugging information. | \\ **Example**\\ \\ ・make the execution module 'hello.out' from the Fortran source file 'hello.f' in fixed form. $ ftn -Mfixed -o hello.out hello.f \\ ・make the execution module 'hello.out' which is autothreaded from the Fortran source file 'hello.f' in fixed form. $ ftn -Mfixed -Mconcur -o hello.out hello.f \\ ・make the execution module 'hello.out' using OpenMP from the Fortran source file 'hello.f' in fixed form. $ ftn -mp -Mfixed -o hello.out hello.f \\ ・Make the execution module ’hello.out’ from the Fortran source file ‘hello.f90’ in free form. $ ftn -Mfree -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' auto threaded from the Fortran source file 'hello.f90' in free form. $ ftn -Mfree -Mconcur -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' using OpenMP from the Fortran source file 'hello.f90' in free form. $ ftn -mp -Mfree -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' from C source file 'hello.c'. $ cc -o hello.out hello.c \\ ・Make the execution module 'hello.out' which is auto-threaded from C source file 'hello.c'. $ cc -Mconcur -o hello.out hello.c \\ ・Make the execution module 'hello.out' using OpenMP from C source file 'hello.c'. $ cc -mp -o hello.out hello.c \\ ・Make the execution module 'hello.out' from the C++ source file 'hello.cpp' $ CC -o hello.out hello.cpp \\ ・Make the execution module 'hello.out' auto threaded from the C++ source file 'hello.cpp'. $ CC -Mconcur -o hello.out hello.cpp \\ ・Make the execution module 'hello.out' using OpenMP from the C++ source file 'hello.cpp'. $ CC -mp -o hello.out hello.cpp ------ ==== GNU Compiler ==== \\ **Setting of program environment**\\ Change the program environment by the following line, because Cray compiler is default on super.\\ \\ ・Example change to the environment of Intel compiler. $ module switch PrgEnv-cray PrgEnv-gnu \\ If you want to change the compiler version, execute a following command.\\ \\ ・Example change the compiler version $ module avail gcc gcc/4.9.3 gcc/5.3.0 gcc/6.1.0 gcc/7.3.0(default) gcc/8.3.0 $ module switch gcc gcc/8.3.0 \\ **Compile Options**\\ \\ ・Optimization options and others ^Option ^Description ^ |-o outfile |Set output file. If ‘-o’ option is not specified, the default output file is a.out. | |-llibrary_name |Specify libraries to be linked. | |-Llibrary_path |Specify paths to be searched for the libraries. | |-O0|-O1|-O2|-O3|-O4 |Set optimizing options. -O2 is default. | |-fopenmp |Compile the input source file enabling OpenMP. | \\ ・Options for Fortran ^Option ^Description ^ |-ffree-form|-ffixed-form |Set the format of source files. | \\ ・Debugging options ^Option ^Description ^ |-g |Output debugging information. | |-g0|-g1|-g2|-g3 |Manage debugging information.(-g2 = -g) | \\ **Example**\\ \\ ・Make the execution module 'hello.out' from Fortran source file 'hello.f' in fixed form. $ ftn -ffixed-form -o hello.out hello.f \\ ・Make the execution module ’hello.out’ from the Fortran source file ‘hello.f90’ in free form. $ ftn -ffree-form -o hello.out hello.f90 \\ ・Make the execution module 'hello.out' from C source file 'hello.c'. $ cc -o hello.out hello.c \\ ・Make the execution module 'hello.out' from the C++ source file 'hello.cpp'. $ CC -o hello.out hello.cpp \\ ===== Libraries ===== Following libraries are available for compiling and linking. ^Library ^Version ^Compiler ^ |CSML(Cray Scientific and Math Libraries) |19.05.5\\ 18.03.1 |all compilers | |Intel MKL (Intel Math Kernel Library) |19.1.3.304\\ 19.1.0.166\\ 19.0.2.187\\ 18.0.2.199\\ 17.0.4.196 |Intel compiler | |Third Party Products |- |all compilers | ------ ==== CSML(Cray Scientific and Math Libraries) ==== \\ CSML (Cray Scientific and Math Libraries) is a set of libraries provide by Cray.\\ Following libraries are available. ^Library ^Description ^Included libraries ^Module ^Option ^ |Cray LibSci |Scientific library optimized for XC50 system\\ Default |BLAS, \\ LAPACK, \\ BLACS, \\ ScaLAPCK, \\ IRT, etc |cray-libsci | | |Cray PETSc(Portable, Extensible Toolkit for Scientific Computation) |Linear / nonlinear equation parallel solver library |MUMPS, \\ SuperLU, \\ SuperLU_dist, \\ ParMETIS, \\ HYPRE, etc |cray-petsc | | |Cray Trilinos Packages |Object-oriented interface of scientific library\\ Load cray-petsc in advance. |PETSc, \\ Metis/ParMetis, \\ SuperLU, \\ Aztec, \\ BLAS, \\ LAPACK |cray-trilinos | | |TPSL(Third Party Scientific Libraries) |Math library available with PETSc/Trilinos |MUMPS, \\ Super_LU, \\ Super_LU_dist, \\ ParMetis, \\ Hypre, \\ Sumdials, \\ Scotch, etc|cray-tpsl | | |FFTW3.3 Library |FFTW version 3.3 |FFTW3.3 |cray-fftw |-lfftw3\\ (MPI parallel)\\ -lfftw3_mpi\\ (thread parallel)\\ -lfftw3_threads| |FFTW2.1 Library |FFTW version2.1\\ Library name depends on single precision and double precision.|FFTW2.1 |fftw |(single precision)\\ -lsrfftw_mpi\\ -lsfftw_mpi\\ -lsrfftw\\ -lsfftw\\ (double precision)\\ -ldrfftw_mpi\\ -ldfftw_mpi\\ -ldrfftw\\ -ldfftw| \\ **Setting of program environment**\\ Change the program environment by the following line, because Cray compiler is default on super.\\ \\ ・Example change to environment of Intel compiler. $ module switch PrgEnv-cray PrgEnv-intel \\ **Example**\\ \\ ・ex1： Using FFTW 3.3 Library from MPI parallel program with Cray compiler $ module load cray-fftw $ cc main.c -L${FFTW_DIR} -lfftw3_mpi -lfftw3 \\ ・ex2： Using FFTW 3.3 Library from MPI and thread parallel program with Intel compiler $ module switch PrgEnv-cray PrgEnv-intel $ module load cray-fftw $ cc -qopenmp main.c -L${FFTW_DIR} -lfftw3_mpi -lfftw3_threads -lfftw3 \\ ・ex3： Using FFTW 2.1 Library with single precision $ module load fftw $ cc main.c -lsrfftw_mpi -lsfftw_mpi -lsrfftw -lsfftw \\ ・ex4： Using FFTW 2.1 Library with double precision $ module load fftw $ cc main.c -ldrfftw_mpi -ldfftw_mpi -ldrfftw -ldfftw ------ ==== Intel MKL ==== \\ Intel MKL (Intel Math Kernel Library) contains BLAS, LAPACK, SparseBLAS, PARDISO, Iterative Sparse Solver, FFT, random number generation and so on.\\ \\ **Setting of program environment**\\ In order to use Intel MKL, it is necessary to load the Intel program environment(PrgEnv-intel) and unload the Cray-libsci to avoid contention. \\ ・Example change from environment of Intel compiler. $ module switch PrgEnv-cray PrgEnv-intel $ module unload cray-libsci \\ **Example**\\ \\ ・Make the execution module 'hello.out' from the Fortran source file 'hello.f' using BLAS in fixed form. $ ftn -mkl -o hello.out -fixed hello.f \\ ・Make the execution module 'hello.out' from C source file 'hello.c' using BLAS. $ cc -mkl -o hello.out hello.c ------ ==== Third Party Products ==== \\ The following libraries are available. ^Library ^Module ^Note ^ |NetCDF(Unidata’s Network Common Data Format) Library |cray-netcdf |serial version | |::: |cray-parallel-netcdf |parallel version| |HDF5(Hierarchical Data Format 5) Libraries & Utilities |cray-hdf5 |serial version | |::: |cray-hdf5-parallel |parallel version | \\ ~~NOCACHE~~