Intel® Math Kernel Library (Intel® MKL) provides highly optimized, extensively threaded computing math routines for applications that require performance on latest Intel processors, including Intel® Xeon Phi coprocessor. See [1] [2]. This article gives an introduction on how to use Intel ® MKL ScaLAPACK library on Intel Xeon Phi coprocessors. A test code is attached.
Preparation
- Intel® MKL 11. 0 or higher
ScaLAPACK routines are tuned for the Intel® Xeon Phi Architecture based on the Intel® MIC Architecture for Intel since Intel® MKL 11.0 or higher.
- Intel® C++ or Fortran compilers version 12.1 or higher
Needed for compiling applications for Intel® Xeon Phi -coprocessor
- Intel® MPI 4.1 or higher [3]
Intel® MKL ScaLAPACK supports only Intel® MPI Library on Intel® Xeon Phi coprocessors.
- Test platform (include hardware and software)
Intel® MKL support host processors based on Intel® 64 or compatible architectures and Intel® Xeon Phi™ Coprocessors. For example, for this article we use:
- Host machine: Intel(R) Xeon(R) CPU E5-2680 @ 2.70GHz, 2 CPU x 8core = 16 threads available, Red Hat Enterprise Linux Server release 6.2 64 bits, kernel 2.6.32-220.el6.x86_64.
- Intel® Xeon Phi™ Coprocessor 7110P, 61 cores, 1.09GHz, 8GB GDDR5 memory with MPSS 3.1
On the host machine: Intel® MKL 11. 1, Intel® MPI 4. 1.2, Intel® Fortran Compiler 14.0, Intel® C++ Compiler 14.0.
Please note the mkl libraries are installed to directory /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib and it includes 3 subdirectories.
$ ls /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib
ia32 intel64 mic
Content
Step 1. Prepare ScaLapack example code
Here is a C example code pblas3_d_example.c, which is from previous MKL version. It calls PDGEMM (PBLAS) and PDLANGE (ScaLAPACK). Please copy it to your working directory.
$ mkdir ~/scalapack_mic
$ cp pblas3_d_example.c ~/scalapack_mic/.
Please note Intel® MKL provides code examples in install directory. You can find one example code there freely.
Step2. Compiling the example code
Differing from other MKL routines, Intel® MKL Scalapack Library supports two programming models on Intel® Xeon Phi Coprocessors:
·Native model: in this native mode, the application runs solely on the coprocessors. The application can be launched from the host or the coprocessor.
·Symmetric model: in this mode, the application runs on both the host and the coprocessors simultaneously. Coprocessor worked as a regular node of the heterogeneous cluster consisted of the host and coprocessors. Sometimes the mode is named Hybrid model
Building steps for the native model:
Set up the proper environment settings for the compiler and for the Intel® MPI Library for Intel® Xeon Phi™ Coprocessor
$ source /opt/intel/composer_xe_2013_sp1/bin/compilervars.sh intel64
$ source /opt/intel/impi/4.1.2/bin64/mpivars.sh
Build the application for the coprocessor:
$ mpiicc -mmic pblas3_d_example.c -o pblas3.mic -I/opt/intel/composer_xe_2013_sp1.1.106/mkl/include /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/mic/libmkl_scalapack_lp64.a -Wl,--start-group /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/mic/libmkl_intel_lp64.a /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/mic/libmkl_intel_thread.a /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/mic/libmkl_core.a /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/mic/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -openmp
The pblas3.mic executable will be produced.
For native mode, skip to Step 3.
Build the application for the host
The pblas3.host executable will be produced.
Step 3: Run the application
Before run the application, we need to set up run-time environment as Ref[3] on coprocessors
scp /opt/intel/impi/4.1.2.040/mic/bin/* mic0:/bin/
scp /opt/intel/impi/4.1.2.040/mic/lib/* mic0:/lib64/
scp /opt/intel/composer_xe_2013_sp1.1.106/compiler/lib/mic/ mic0:/lib64/
Native mode:
Upload the application pblas3.mic to the /tmp directory on the coprocessors using the scp command.
$ scp pblas3.mic mic0:/tmp/.
$ ssh mic0
$ cd /tmp
$ mpirun -n 4 ./pblas3.mic
The command spawns 4 MPI processes on a coprocessor.
Symmetric model:
Upload the application pblas3.mic to the /tmp directory on the coprocessors.
Enable the MPI communication between host and coprocessors and use the appropriate script to set up the runtime environment:
$ source /opt/intel/composer_xe_2013_sp1/bin/compilervars.sh intel64
$ source /opt/intel/impi/4.1.2/bin64/mpivars.sh
$mpirun -n 2 -host localhost ./pblas3.host : -n 2 -host mic0 /tmp/pblas3.mic
The command spawns 2 MPI processes on host machine and 2 MPI processes on a coprocessor.
$mpirun -n 2 -genv I_MPI_DEBUG 2 -genv I_MPI_FABRICS tcp -host localhost ./pblas3.host : -n 2 -genv I_MPI_DEBUG 2 -host mic0 /tmp/pblas3.mic
Summary
This article shows how to compile and run a simple Intel MKL Scalapack application on Intel® Xeon Phi coprocessor. Besides the native model, Intel MKL ScaLAPACK can benefit from another usage model offered by the Intel® MPI Library. We call it as symmetric model or hybrid model, which treats each Intel Xeon Phi coprocessor as a regular node in a heterogeneous cluster consisted of Intel® Xeon® processors and Intel® Xeon Phi coprocessors. Symmetric model allows developers to run ScaLapack on both host and coprocessors. Check more information on Intel MKL ScaLapack in the MKL User guide and MPI User guide.
[1] Online articles: Using Intel® MKL on Intel® Xeon Phi http://software.intel.com/en-us/articles/intel-mkl-on-the-intel-xeon-phi-coprocessors
[2] MKL User Guide http://software.intel.com/sites/products/documentation/doclib/mkl_sa/11/mkl_userguide_lnx/GUID-108D4F3E-D299-4D35-BAED-ECC5B48DA57E.htm
[3] Intel® MPI Library reference and online doc.
http://software.intel.com/articles/using-the-intel-mpi-library-on-intel-xeon-phi-coprocessor-systems
Contratto di licenza:
Allegati protetti:
| Allegato | Dimensione |
|---|---|
Scarica pblas3_d_example.c | 16.8 KB |