Linux High performance Computing Implementation Guide. By Syed Asim Abbas with help MR Kamarn Azeem CBTs.
For comments and suggestions : This e-mail address is being protected from spambots. You need JavaScript enabled to view it
HPCC Installation :
--------------------
I have 4 node for HPCC. 2 nodes are HP ML 360 servers with 2 x dual core processor. One is 1.7 mhz processor and other is AMD Sempron(tm) Processor 2600+
1) Centos 5.2 Installation on all nodes
2) Stop unnecessary services
3) /etc/hosts
------------
vi /etc/hosts (hpcc2)
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.1.180 hpcc1.isb.ots.com hpcc1
192.168.1.49 hpcc2.isb.ots.com hpcc2
192.168.1.69 test.isb.com test
192.168.1.40 nworkhorse.isb.ots.com nworkhorse
vi /etc/hosts (test)
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.1.180 hpcc1.isb.ots.com hpcc1
192.168.1.49 hpcc2.isb.ots.com hpcc2
192.168.1.69 test.isb.com test
192.168.1.40 nworkhorse.isb.ots.com nworkhorse
vi /etc/hosts (nworkhorse)
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.1.180 hpcc1.isb.ots.com hpcc1
192.168.1.49 hpcc2.isb.ots.com hpcc2
192.168.1.69 test.isb.com test
192.168.1.40 nworkhorse.isb.ots.com nworkhorse
root@hpcc1 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
/root/.ssh/id_dsa already exists.
Overwrite (y/n)?
[root@hpcc1 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
/root/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is: [root@hpcc2 ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa):
/root/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
3a:c0:dc:ad:aa:52:07:f8:43:b2:cb:ee:81:c8:6a:8f This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[root@hpcc2 ~]# scp ~/.ssh/id_dsa.pub root@hpcc1:/root/.ssh/authorized_keys
root@hpcc1's password:
id_dsa.pub 100% 612 0.6KB/s 00:00
[root@hpcc2 ~]#
[root@hpcc2 ~]#
[root@hpcc2 ~]#
[root@hpcc2 ~]#
[root@hpcc2 ~]# ssh hpcc1
Last login: Tue May 5 11:02:26 2009 from 192.168.1.19
[root@hpcc1 ~]# logout
7b:a6:db:3e:55:ce:da:6b:ce:48:28:d3:9a:29:84:82 This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[root@hpcc1 ~]# scp ~/.ssh/id_dsa.pub root@hpcc2:/root/.ssh/authorized_keys
root@hpcc2's password:
id_dsa.pub 100% 612 0.6KB/s 00:00
[root@hpcc1 ~]# ssh hpcc2
Last login: Wed May 13 11:03:40 2009 from hpcc1.isb.ots.com
[root@hpcc2 ~]# logout
Connection to hpcc2 closed.
NTP: for all nodes :
-----
Node 1
[root@hpcc2 ~]# vi /etc/ntp.conf
[root@hpcc2 ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [FAILED]
Starting ntpd: [ OK ]
[root@hpcc2 ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 .LOCL. 10 l 5 64 1 0.000 0.000 0.001
[root@hpcc2 ~]#
Node 2
[root@hpcc2 ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@hpcc2 ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001
[root@hpcc2 ~]# vi /etc/ntp.conf
[root@hpcc2 ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@hpcc2 ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
10.0.0.1 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001
127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001
[root@hpcc2 ~]#
Node 3
[root@test ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@test ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001
[root@test ~]# vi /etc/ntp.conf
[root@test ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@test ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
192.168.1.180 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001
127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001
[root@test ~]#
[root@nworkhorse ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@nworkhorse ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
127.127.1.0 .LOCL. 10 l 1 64 1 0.000 0.000 0.001
[root@nworkhorse ~]# vi /etc/ntp.conf
[root@nworkhorse ~]# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
Starting ntpd: [ OK ]
[root@nworkhorse ~]# ntpq -p -n
remote refid st t when poll reach delay offset jitter
==============================================================================
192.168.1.180 LOCAL(0) 11 u - 64 1 0.155 -691264 0.001
127.127.1.0 .LOCL. 10 l - 64 0 0.000 0.000 0.001
[root@nworkhorse ~]#
PDSH:
-----
[root@hpcc1 ~]# wget http://nchc.dl.sourceforge.net/sourceforge/pdsh/pdsh-2.18-1.src.rpm
--11:52:48-- http://nchc.dl.sourceforge.net/sourceforge/pdsh/pdsh-2.18-1.src.rpm
Resolving nchc.dl.sourceforge.net... 211.79.60.17, 2001:e10:ffff:1f02::17
Connecting to nchc.dl.sourceforge.net|211.79.60.17|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 462342 (452K) [application/x-wais-source]
Saving to: `pdsh-2.18-1.src.rpm'
100%[======================================================================================================>] 462,342 6.43K/s in 16s
11:53:05 (28.9 KB/s) - `pdsh-2.18-1.src.rpm' saved [462342/462342]
[root@hpcc1 ~]# scp pdsh-2.18-1.src.rpm hpcc2:/root
pdsh-2.18-1.src.rpm 100% 452KB 451.5KB/s 00:00
[root@hpcc1 ~]#
[root@hpcc1 ~]# rpmbuild --rebuild pdsh-2.18-1.src.rpm
Installing pdsh-2.18-1.src.rpm
warning: user grondo does not exist - using root
warning: group grondo does not exist - using root
warning: user grondo does not exist - using root
warning: group grondo does not exist - using root
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.4710
+ umask 022
+ cd /usr/src/redhat/BUILD
+ LANG=C
[root@hpcc2 ~]# rpmbuild --rebuild pdsh-2.18-1.src.rpm
Installing pdsh-2.18-1.src.rpm
warning: user grondo does not exist - using root
warning: group grondo does not exist - using root
warning: user grondo does not exist - using root
warning: group grondo does not exist - using root
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.4710
+ umask 022
+ cd /usr/src/redhat/BUILD
+ LANG=C
[root@hpcc1 i386]# rpm -ivh pdsh-*
Preparing... ########################################### [100%]
1:pdsh-rcmd-ssh ########################################### [ 14%]
2:pdsh ########################################### [ 29%]
3:pdsh-debuginfo ########################################### [ 43%]
4:pdsh-mod-dshgroup ########################################### [ 57%]
5:pdsh-mod-machines ########################################### [ 71%]
6:pdsh-mod-netgroup ########################################### [ 86%]
7:pdsh-rcmd-exec ########################################### [100%]
[root@hpcc2 i386]# rpm -ivh pdsh-*
Preparing... ########################################### [100%]
1:pdsh-rcmd-ssh ########################################### [ 14%]
2:pdsh ########################################### [ 29%]
3:pdsh-debuginfo ########################################### [ 43%]
4:pdsh-mod-dshgroup ########################################### [ 57%]
5:pdsh-mod-machines ########################################### [ 71%]
6:pdsh-mod-netgroup ########################################### [ 86%]
7:pdsh-rcmd-exec ########################################### [100%]
[root@test ~]# rpm -ivh /usr/src/redhat/RPMS/i386/pdsh-*
Preparing... ##### ( 14%)
########################################### [100%]
1:pdsh-rcmd-ssh ########################################### [ 14%]
2:pdsh ########################################### [ 29%]
3:pdsh-debuginfo ########################################### [ 43%]
4:pdsh-mod-dshgroup ########################################### [ 57%]
5:pdsh-mod-machines ########################################### [ 71%]
6:pdsh-mod-netgroup ########################################### [ 86%]
7:pdsh-rcmd-exec ########################################### [100%]
[root@ nworkhorse i386]# rpm -ivh pdsh-*
Preparing... ########################################### [100%]
1:pdsh-rcmd-ssh ########################################### [ 14%]
2:pdsh ########################################### [ 29%]
3:pdsh-debuginfo ########################################### [ 43%]
4:pdsh-mod-dshgroup ########################################### [ 57%]
5:pdsh-mod-machines ########################################### [ 71%]
6:pdsh-mod-netgroup ########################################### [ 86%]
7:pdsh-rcmd-exec ########################################### [100%]
NFS Setup
[root@hpcc1 .ssh]# vi /etc/exports
/cluster *(rw,sync)
[root@hpcc1 .ssh]# pdsh -a mkdir /cluster
failed to install module options for "misc/dshgroup"
[root@hpcc1 .ssh]#
[root@hpcc1 .ssh]#
[root@hpcc1 .ssh]# /etc/init.d/nfs restart
Shutting down NFS mountd: [FAILED]
Shutting down NFS daemon: [FAILED]
Shutting down NFS quotas: [FAILED]
Shutting down NFS services: [ OK ]
Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
[root@hpcc1 .ssh]#
[root@hpcc2 .ssh]# mount -t nfs hpcc1:/cluster/ /cluster/
[root@hpcc2 .ssh]#
[root@hpcc2 .ssh]#
[root@hpcc2 .ssh]# df -
df: `-': No such file or directory
[root@hpcc2 .ssh]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/cciss/c0d0p2 29G 5.4G 22G 20% /
tmpfs 1.7G 0 1.7G 0% /dev/shm
hpcc1:/cluster/ 19G 6.3G 12G 35% /cluster
[root@hpcc2 .ssh]#
Node 1
[root@hpcc1 /]# groupadd -g 604 power
[root@hpcc1 /]# useradd -u 700 -g 604 -d /cluster/power hpower
Node 2
[root@hpcc2 ~]# groupadd -g 604 power
[root@hpcc2 ~]# useradd -u 700 -g 604 -d /cluster/power hpower
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
[root@hpcc2 ~]#
Node 3
[root@test]# mount -t nfs hpcc1:/cluster/ /cluster/
[root@test ~]# mkdir /cluster
[root@test ~]# groupadd -g 604 power
[root@test ~]# useradd -u 700 -g 604 -d /cluster/power hpower
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
Node 4
[root@nworkhorse ~]# mkdir /cluster
[root@nworkhorse ~]# mount -t nfs hpcc1:/cluster/ /cluster/
[root@nworkhorse ~]# groupadd -g 604 power
[root@nworkhorse ~]# useradd -u 700 -g 604 -d /cluster/power hpower
useradd: warning: the home directory already exists.
Not copying any file from skel directory into it.
Node 1
[hpower@hpcc1 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/cluster/power/.ssh/id_dsa):
Created directory '/cluster/power/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /cluster/power/.ssh/id_dsa.
Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.
The key fingerprint is:
79:d4:a0:4e:2e:e5:b1:05:5e:9c:93:56:06:0f:d4:1f This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[hpower@hpcc1 ~]$
[hpower@hpcc1 ~]$
[hpower@hpcc1 ~]$ ls
[hpower@hpcc1 ~]$ cd .ssh/
[hpower@hpcc1 .ssh]$ ls
id_dsa id_dsa.pub
[hpower@hpcc1 .ssh]$ vi authorized
[hpower@hpcc1 .ssh]$ vi id_dsa.pub
[hpower@hpcc1 .ssh]$ cat id_dsa.pub > authorized_keys
Node 2
[hpower@hpcc2 ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/cluster/power/.ssh/id_dsa):
/cluster/power/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /cluster/power/.ssh/id_dsa.
Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.
The key fingerprint is:
85:5a:0f:c2:d6:21:6e:55:38:24:5d:60:02:28:16:7b This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[hpower@hpcc2 ~]$
[hpower@hpcc2 ~]$ cd .ssh/
[hpower@hpcc2 .ssh]$ cat id_dsa.pub >> authorized_keys
Node 3
[hpower@test ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/cluster/power/.ssh/id_dsa):
/cluster/power/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /cluster/power/.ssh/id_dsa.
Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.
The key fingerprint is:
82:d9:8e:34:48:a7:f8:32:21:8d:e6:17:3d:2b:dd:21 This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[hpower@test .ssh]$ cat id_dsa.pub >> authorized_keys
Node 4
[hpower@nworkhorse ~]$ ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/cluster/power/.ssh/id_dsa):
/cluster/power/.ssh/id_dsa already exists.
Overwrite (y/n)? y
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /cluster/power/.ssh/id_dsa.
Your public key has been saved in /cluster/power/.ssh/id_dsa.pub.
The key fingerprint is:
c1:a6:01:ec:1e:a6:e9:8b:5e:89:2d:33:06:8b:24:cb This e-mail address is being protected from spambots. You need JavaScript enabled to view it
[hpower@nworkhorse ~]$ cat id_dsa.pub >> authorized_keys
cat: id_dsa.pub: No such file or directory
[hpower@nworkhorse ~]$ cd .ssh/
[hpower@nworkhorse .ssh]$ cat id_dsa.pub >> authorized_keys
[hpower@hpcc1 .ssh]$ chmod 644 authorized_keys
[hpower@hpcc1 .ssh]$ ssh hpcc2
The authenticity of host 'hpcc2 (192.168.1.49)' can't be established.
RSA key fingerprint is 99:6f:8d:b7:8b:89:ae:63:cf:e8:3f:3e:60:d1:bb:c3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hpcc2,192.168.1.49' (RSA) to the list of known hosts.
[hpower@hpcc2 ~]$ logout
Connection to hpcc2 closed.
[hpower@hpcc1 .ssh]$ ssh hpcc1
Last login: Sat Jun 6 14:25:48 2009 from hpcc2.isb.ots.com
[hpower@hpcc1 ~]$ logout
[hpower@hpcc2 .ssh]$ ssh hpcc1
The authenticity of host 'hpcc1 (192.168.1.180)' can't be established.
RSA key fingerprint is a2:04:92:48:b8:58:56:b7:24:9d:e5:da:30:75:92:b6.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hpcc1,192.168.1.180' (RSA) to the list of known hosts.
[hpower@hpcc1 ~]$ logout
Connection to hpcc1 closed.
[hpower@hpcc2 .ssh]$ ssh hpcc2
Last login: Sat Jun 6 14:25:41 2009 from hpcc1.isb.ots.com
[hpower@hpcc2 ~]$ logout
[root@hpcc1 ~]# cp mpich2-1.0.8p1.tar.gz /cluster/
[root@hpcc1 ~]#
[root@hpcc1 /]# chown hpower.power /cluster/ -R
hpower@hpcc1 ~]$ cd ..
[hpower@hpcc1 cluster]$ ls
mpich2-1.0.8p1.tar.gz power
[hpower@hpcc1 cluster]$ tar zxvf mpich2-1.0.8p1.tar.gz
mpich2-1.0.8p1/
mpich2-1.0.8p1/mpich2.def
mpich2-1.0.8p1/test/
mpich2-1.0.8p1/test/.codingcheck
[hpower@hpcc1 mpich2-1.0.8p1]$ ./configure --prefix=/cluster/mpich2
config.status: executing default-3 commands
Configuration completed.
If done with out any error then
[hpower@hpcc1 mpich2-1.0.8p1]$ make
[hpower@hpcc1 mpich2-1.0.8p1]$ make install
[hpower@hpcc1 ~]$ vi .bash_profile
# .bash_profile
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi
# User specific environment and startup programs
PATH=$PATH:$HOME/bin:/cluster/mpich2/bin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/cluster/mpich2/lib
#PATH=$PATH:$HOME/bin
export PATH
export LD_LIBRARY_PATH
[mpiuser@node2 ~]$ source .bash_profile
[mpiuser@node1 ~]$ echo $PATH
/usr/lib/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/cluster/mpiuser/bin:/cluster/mpiuser/bin:/cluster/mpich2/bin
Create new file :
[hpower@hpcc1 ~]$ vi mpd.hosts
hpcc1
hpcc2
Create new file :
[hpower@hpcc1 ~]$ vi mpd.hosts
secretword=asim
[mpiuser@node1 ~]$ chmod 600 .mpd.conf
run
[hpower@hpcc1 ~]$ mpd &
[hpower@hpcc1 ~]$ mpd &
[1] 13499
[hpower@hpcc1 ~]$ mpdtrace
hpcc1
[hpower@hpcc1 ~]$ mpdtrace
hpcc1
[hpower@hpcc1 ~]$ mpdallexit
[hpower@hpcc1 ~]$ mpdboot -n 2
[hpower@hpcc1 ~]$ mpdtrace
hpcc1
hpcc2
[hpower@hpcc1 ~]$
Both nodes added Good
For Checkup
[hpower@hpcc1 ~]$ mpdboot -n 2 --chkuponly
checking hpcc2
there are 2 hosts up (counting local)
Now let's start some becnhmarking
[hpower@hpcc1 cluster]$ cd mpich2-1.0.8p1/examples/
cpi file already complied and executeable
-rw-r--r-- 1 hpower power 678 Nov 3 2007 child.c
-rwxr-xr-x 1 hpower power 576450 Jun 6 14:34 cpi
-rw-r--r-- 1 hpower power 1515 Nov 3 2007 cpi.c
-rw-r--r-- 1 hpower power 1964 Jun 6 14:34 cpi.o
-rw-r--r-- 1 hpower power 4469 Nov 3 2007 cpi.vcproj
drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 cxx
drwxr-xr-x 2 hpower power 4096 Mar 27 01:40 developers
-rw-r--r-- 1 hpower power 10446 Nov 3 2007 examples.sln
drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 f77
drwxr-xr-x 2 hpower power 4096 Jun 6 14:31 f90
-rw-r--r-- 1 hpower power 455 Nov 3 2007 hellow.c
-rw-r--r-- 1 hpower power 1892 Nov 3 2007 icpi.c
-rw-r--r-- 1 hpower power 6802 Jun 6 14:31 Makefile
-rw-r--r-- 1 hpower power 6767 Mar 27 01:40 Makefile.in
-rw-r--r-- 1 hpower power 1490 Mar 12 2008 Makefile.sm
drwxr-xr-x 2 hpower power 4096 Mar 27 01:39 mpiexec
-rw-r--r-- 1 hpower power 1049 Nov 3 2007 parent.c
-rw-r--r-- 1 hpower power 46399 Nov 3 2007 pmandel.c
-rw-r--r-- 1 hpower power 47798 Nov 3 2007 pmandel_fence.c
-rw-r--r-- 1 hpower power 4522 Nov 3 2007 pmandel_fence.vcproj
-rw-r--r-- 1 hpower power 45576 Nov 3 2007 pmandel_service.c
-rw-r--r-- 1 hpower power 4532 Nov 3 2007 pmandel_service.vcproj
-rw-r--r-- 1 hpower power 47510 Nov 3 2007 pmandel_spaserv.c
-rw-r--r-- 1 hpower power 4510 Nov 3 2007 pmandel_spaserv.vcproj
-rw-r--r-- 1 hpower power 49858 Nov 3 2007 pmandel_spawn.c
-rw-r--r-- 1 hpower power 4522 Nov 3 2007 pmandel_spawn.vcproj
-rw-r--r-- 1 hpower power 4492 Nov 3 2007 pmandel.vcproj
-rw-r--r-- 1 hpower power 2175 Nov 3 2007 README
-rw-r--r-- 1 hpower power 1116 Nov 3 2007 spawn_merge_child1.c
-rw-r--r-- 1 hpower power 694 Nov 3 2007 spawn_merge_child2.c
-rw-r--r-- 1 hpower power 1795 Nov 3 2007 spawn_merge_parent.c
-rw-r--r-- 1 hpower power 3554 Nov 3 2007 spawntest.vcproj
-rw-r--r-- 1 hpower power 1642 Nov 3 2007 srtest.c
drwxr-xr-x 7 hpower power 4096 Mar 27 01:40 windows
[hpower@hpcc1 ~]$ cd -
/cluster/mpich2-1.0.8p1/examples
[hpower@hpcc1 examples]$ mpiexec -n 1 ./cpi
Lamnodes Failed!
Check if you had booted lam before calling mpiexec else use -machinefile to pass host file to mpiexec
LAM Error but i m not using LAM ...........
After 20 mins fiding finally i found the reason mpiexec was running from /usr/bin/mpiexec
[hpower@hpcc1 ~]$ which mpiexec
/usr/bin/mpiexec
If run this using absolute path it is working. Means there is some problem with our .bash_profile
[hpower@hpcc1 ~]$ /cluster/mpich2/bin/mpiexec -n 1 /cluster/mpich2-1.0.8p1/examples/cpi
Process 0 of 1 is on hpcc1.isb.ots.com
pi is approximately 3.1415926544231341, Error is 0.0000000008333410
wall clock time = 0.000317
I found only one solution to solve
[root@hpcc1 ~]# mv /usr/bin/mpiexec /tmp/ :)
Its Working Now
[hpower@hpcc1 ~]$ which mpiexec
/cluster/mpich2/bin/mpiexec
[hpower@hpcc1 ~]$
[hpower@hpcc1 ~]$ mpiexec -n 1 /cluster/mpich2-1.0.8p1/examples/cpi
Process 0 of 1 is on hpcc1.isb.ots.com
pi is approximately 3.1415926544231341, Error is 0.0000000008333410
wall clock time = 0.000297
[hpower@hpcc1 ~]$ mpiexec -n 2 /cluster/mpich2-1.0.8p1/examples/cpi
Process 0 of 2 is on hpcc1.isb.ots.com
Process 1 of 2 is on hpcc2.isb.ots.com
pi is approximately 3.1415926544231318, Error is 0.0000000008333387
wall clock time = 0.001234
[hpower@hpcc1 ~]$
There is an other program available for testing lets compile it first
/cluster/mpich2-1.0.8p1/examples
[hpower@hpcc1 examples]$ mpicc -o icpi icpi.c
Now execute it and also open top -c to check the processes.
First we will run with single node
[hpower@hpcc1 examples]$ mpiexec -n 1 ./icpi
Enter the number of intervals: (0 quits) 1000000000
pi is approximately 3.1415926535921401, Error is 0.0000000000023470
wall clock time = 22.537397
Enter the number of intervals: (0 quits)
top -c out put :
[root@hpcc1 ~]# top -c
top - 15:53:08 up 1:57, 3 users, load average: 0.21, 0.11, 0.03
Tasks: 188 total, 2 running, 186 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.1%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3368196k total, 936760k used, 2431436k free, 140504k buffers
Swap: 12289716k total, 0k used, 12289716k free, 634932k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14289 hpower 25 0 2332 812 676 R 100 0.0 0:07.80./icpi
1 root 15 0 2032 640 552 S 0 0.0 0:01.82 init [5]
Lets run with both nodes
[hpower@hpcc1 examples]$ mpiexec -n 2 ./icpi
Enter the number of intervals: (0 quits) 1000000000
pi is approximately 3.1415926535905170, Error is 0.0000000000007239
wall clock time = 11.285166
Enter the number of intervals: (0 quits)
HPCC2 top -c output
top - 15:56:47 up 2:01, 5 users, load average: 0.27, 0.08, 0.06
Tasks: 205 total, 2 running, 203 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3368120k total, 1858180k used, 1509940k free, 197464k buffers
Swap: 13309844k total, 0k used, 13309844k free, 1488212k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4615 hpower 25 0 2356 812 676 R 100 0.0 0:10.26 ./icpi
4593 root 15 0 2320 1080 800 R 0 0.0 0:01.06 top -c
HPCC1 top -c Out put
[root@hpcc1 ~]# top -c
top - 15:56:46 up 2:01, 3 users, load average: 0.14, 0.10, 0.04
Tasks: 188 total, 2 running, 186 sleeping, 0 stopped, 0 zombie
Cpu(s): 12.5%us, 0.0%sy, 0.0%ni, 87.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3368196k total, 941472k used, 2426724k free, 140656k buffers
Swap: 12289716k total, 0k used, 12289716k free, 634876k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14312 hpower 25 0 2328 848 712 R 100 0.0 0:08.84 ./icpi
1 root 15 0 2032 640 552 S 0 0.0 0:01.82 init [5]
Now I am going to add nodes 3 on /etc/mpd.hosts
[hpower@hpcc1 ~]$ vi mpd.hosts
hpcc1
hpcc2
test
[hpower@hpcc1 ~]$ mpdallexit
[hpower@hpcc1 ~]$ mpdboot -n 3
[hpower@hpcc1 ~]$ mpdtrace
hpcc1
test
hpcc2
All 3 nodes addes successfully
Now lets run scripts again for benchmarking
[hpower@hpcc1 ~]$ cd /cluster/mpich2-1.0.8p1/examples/
[hpower@hpcc1 examples]$ mpiexec -n 2 ./icpi
Enter the number of intervals: (0 quits) 1000000000
pi is approximately 3.1415926535905170, Error is 0.0000000000007239
wall clock time = 43.697841
[hpower@hpcc1 examples]$ mpiexec -n 3 ./icpi
Enter the number of intervals: (0 quits) 1000000000
pi is approximately 3.1415926535899761, Error is 0.0000000000001830
wall clock time = 29.853977
Hmmmm better then two nodes add another 0 :) It will take some time .
Add node 4
there are not enough hosts on which to start all processes
[hpower@hpcc1 ~]$ vi mpd.hosts
[hpower@hpcc1 ~]$ mpdboot -n 4
[hpower@hpcc1 ~]$ mpdtrace
hpcc1
test
nworkhorse
hpcc2
[hpower@hpcc1 examples]$ mpiexec -n 4 ./icpi
Enter the number of intervals: (0 quits) 1000000000
pi is approximately 3.1415926535896128, Error is 0.0000000000001803
wall clock time = 21.904199
Lets Test GoTo Blast.
Download Location
http://www.tacc.utexas.edu/resources/software/login/gotoblas/GotoBLAS-1.26.tar.gz
[root@hpcc1 ~]# cp GotoBLAS-1.26.tar.gz /cluster/
[root@hpcc1 ~]# cd /cluster/
[root@hpcc1 cluster]# ls
GotoBLAS-1.26.tar.gz mpich2 mpich2-1.0.8p1 mpich2-1.0.8p1.tar.gz power
[root@hpcc1 cluster]# tar zxvf GotoBLAS-1.26.tar.gz
[root@hpcc1 cluster]# su - hpower
[hpower@hpcc1 cluster]$ cd GotoBLAS
[hpower@hpcc1 GotoBLAS]$ ./quickbuild.32bit
[hpower@hpcc1 GotoBLAS]$ make
It takes some time
Now download hpl
Download Link
http://www.netlib.org/benchmark/hpl/hpl-2.0.tar.gz
http://www.netlib.org/benchmark/hpl/hpl.tgz
root@hpcc1 cluster]# cd
[root@hpcc1 ~]# ls
@ Desktop hpl.tgz ipscan-3.0_beta4-1.i386.rpm mpich2-1.0.8p1.tar.gz snapshot1.jpg snapshot4.jpg
abc.sh GotoBLAS-1.26.tar.gz install.log login.php?product=server20 pdsh-2.10-4.src.rpm snapshot2.jpg tm
anaconda-ks.cfg hpl-2.0.tar.gz install.log.syslog mac.txt
[hpower@hpcc1 cluster]$ ls
GotoBLAS GotoBLAS-1.26.tar.gz hpl-2.0.tar.gz hpl.tgz mpich2 mpich2-1.0.8p1 mpich2-1.0.8p1.tar.gz power
[hpower@hpcc1 cluster]$ tar zxf hpl-2.0.tar.gz
[hpower@hpcc1 cluster]$ tar zxf hpl.tgz
[hpower@hpcc1 hpl]$ cp setup/Make.Linux_PII_FBLAS_gm .
First check GCC version/Path
[hpower@hpcc1 hpl]$ gcc -v
Using built-in specs.
Target: i386-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-libgcj-multifile --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic --host=i386-redhat-linux
Thread model: posix
/usr/lib/gcc/i386-redhat-linux/4.1.1/
Now edit Make.Linux_PII_FBLAS_gm file
[hpower@hpcc1 hpl]$ vi Make.Linux_PII_FBLAS_gm
Edit the following lines :
ARCH = Linux_PII_FBLAS_gm
TOPdir = $(HOME)/hpl
INCdir = $(TOPdir)/include
BINdir = $(TOPdir)/bin/$(ARCH)
LIBdir = $(TOPdir)/lib/$(ARCH)
#
HPLlib = $(LIBdir)/libhpl.a
LAdir = /cluster/GotoBLAS
LAlib = $(LAdir)/libgoto.a -1m -L/usr/lib/gcc/i386-redhat-linux/4.1.1/
CC = mpicc
CCFLAGS = $(HPL_DEFS) -O3
LINKER = mpicc
Ok Done now compile it .
[hpower@hpcc1 hpl]$ make arch=Linux_PII_FBLAS_gm
[hpower@test hpl]$ cd /cluster/power/hpl/bin/Linux_PII_FBLAS_gm/
[hpower@test Linux_PII_FBLAS_gm]$ ls
HPL.dat xhpl
[hpower@test Linux_PII_FBLAS_gm]$ cp HPL.dat HPL.dat.orig
MY HPL parameter First testing with 2 nodes
Innovative Computing Laboratory, University of Tennessee
HPL.out output file name (if any)
6 device out (6=stdout,7=stderr,file)
2 # of problems sizes (N)
7000 Ns
4 # of NBs
100 NBs
0 PMAP process mapping (0=Row-,1=Column-major)
3 # of process grids (P x Q)
1 Ps
2 Qs
16.0 threshold
3 # of panel fact
0 1 2 PFACTs (0=left, 1=Crout, 2=Right)
2 # of recursive stopping criterium
2 4 NBMINs (>= 1)
1 # of panels in recursion
2 NDIVs
3 # of recursive panel fact.
0 1 2 RFACTs (0=left, 1=Crout, 2=Right)
1 # of broadcast
0 BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1 # of lookahead depth
0 DEPTHs (>=0)
2 SWAP (0=bin-exch,1=long,2=mix)
64 swapping threshold
0 L1 in (0=transposed,1=no-transposed) form
0 U in (0=transposed,1=no-transposed) form
1 Equilibration (0=no,1=yes)
8 memory alignment in double (> 0)
~
~
[hpower@hpcc1 Linux_PII_FBLAS_gm]$ mpiexec -n 2 ./xhpl
The following parameter values will be used:
N : 6000
NB : 100
PMAP : Row-major process mapping
P : 1
Q : 2
PFACT : Left Crout Right
NBMIN : 2 4
NDIV : 2
RFACT : Left Crout Right
BCAST : 1ring
DEPTH : 0
SWAP : Mix (threshold = 64)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
top -c on node 1 (hpcc1) Procssor speed 3.0 dual core
Cpu(s): 6.2%us, 1.5%sy, 0.0%ni, 92.1%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 3368196k total, 1229388k used, 2138808k free, 215208k buffers
Swap: 12289716k total, 0k used, 12289716k free, 695944k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16856 hpower 15 0 172m 141m 1016 S 62 4.3 0:55.41 ./xhpl
top -c on node 3 (test ) 1.7 MHZ PC class machine
Cpu(s): 78.1%us, 1.3%sy, 0.0%ni, 19.6%id, 1.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 443392k total, 437712k used, 5680k free, 21312k buffers
Swap: 1052248k total, 0k used, 1052248k free, 189308k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14055 hpower 25 0 175m 144m 960 R 78.6 33.4 0:59.60 ./xhpl
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be 1.110223e-16
- Computational tests pass if scaled residuals are less than 16.0
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2L2 6000 100 1 2 46.28 3.113e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0034911 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0072807 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0014339 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2L4 6000 100 1 2 49.01 2.939e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0038909 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0081145 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0015981 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2C2 6000 100 1 2 45.78 3.146e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0034911 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0072807 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0014339 ...... PASSED
Again RUN now incrase the size of NS.
The following parameter values will be used:
N : 7000
NB : 100
PMAP : Row-major process mapping
P : 1
Q : 2
PFACT : Left Crout Right
NBMIN : 2 4
NDIV : 2
RFACT : Left Crout Right
BCAST : 1ring
DEPTH : 0
SWAP : Mix (threshold = 64)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
- Computational tests pass if scaled residuals are less than 16.0
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2L2 7000 100 1 2 72.10 3.173e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.1808081 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091439 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0016514 ...... PASSED
With 4 node
Cpu(s): 2.9%us, 0.7%sy, 0.0%ni, 96.3%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 3368196k total, 1120648k used, 2247548k free, 215220k buffers
Swap: 12289716k total, 0k used, 12289716k free, 695976k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17007 hpower 15 0 68892 34m 1016 S 28 1.0 0:28.31 ./xhpl
Cpu(s): 62.2%us, 20.4%sy, 0.0%ni, 8.6%id, 8.6%wa, 0.0%hi, 0.3%si, 0.0%st
Mem: 443392k total, 175608k used, 267784k free, 2532k buffers
Swap: 1052248k total, 41448k used, 1010800k free, 86836k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14144 hpower 15 0 70652 36m 960 R 63.1 8.5 0:10.83 ./xhpl
14066 root 15 0 2212 948 752 R 0.3 0.2 0:01.75 top -c
Cpu(s): 36.0%us, 10.3%sy, 0.0%ni, 50.0%id, 0.0%wa, 1.7%hi, 2.0%si, 0.0%st
Mem: 969896k total, 602732k used, 367164k free, 31900k buffers
Swap: 2096408k total, 0k used, 2096408k free, 374868k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4803 hpower 16 0 70416 36m 960 R 46.5 3.9 0:30.35 ./xhpl
============================================================================
HPLinpack 1.0a -- High-Performance Linpack benchmark -- January 20, 2004
Written by A. Petitet and R. Clint Whaley, Innovative Computing Labs., UTK
============================================================================
An explanation of the input/output parameters follows:
T/V : Wall time / encoded variant.
N : The order of the coefficient matrix A.
NB : The partitioning blocking factor.
P : The number of process rows.
Q : The number of process columns.
Time : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.
The following parameter values will be used:
N : 4000
NB : 100
PMAP : Row-major process mapping
P : 2
Q : 2
PFACT : Left Crout Right
NBMIN : 2 4
NDIV : 2
RFACT : Left Crout Right
BCAST : 1ring
DEPTH : 0
SWAP : Mix (threshold = 64)
L1 : transposed form
U : transposed form
EQUIL : yes
ALIGN : 8 double precision words
----------------------------------------------------------------------------
- The matrix A is randomly generated for each test.
- The following scaled residual checks will be computed:
1) ||Ax-b||_oo / ( eps * ||A||_1 * N )
2) ||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 )
3) ||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo )
- The relative machine precision (eps) is taken to be 1.110223e-16
- Computational tests pass if scaled residuals are less than 16.0
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2L2 4000 100 2 2 46.16 9.248e-01
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2L4 4000 100 2 2 18.48 2.310e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2C2 4000 100 2 2 21.46 1.990e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2C4 4000 100 2 2 18.39 2.322e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2R2 4000 100 2 2 18.68 2.285e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00L2R4 4000 100 2 2 17.58 2.428e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2L2 4000 100 2 2 19.05 2.241e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2L4 4000 100 2 2 17.81 2.398e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2C2 4000 100 2 2 17.51 2.438e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2C4 4000 100 2 2 18.25 2.340e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2R2 4000 100 2 2 18.62 2.293e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00C2R4 4000 100 2 2 17.84 2.393e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2L2 4000 100 2 2 18.31 2.331e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2L4 4000 100 2 2 17.48 2.442e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2C2 4000 100 2 2 18.03 2.368e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2C4 4000 100 2 2 18.90 2.259e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0086843 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0080198 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0018228 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2R2 4000 100 2 2 18.64 2.290e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098900 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091332 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020759 ...... PASSED
============================================================================
T/V N NB P Q Time Gflops
----------------------------------------------------------------------------
WR00R2R4 4000 100 2 2 18.59 2.296e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1 * N ) = 0.0098668 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1 * ||x||_1 ) = 0.0091118 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) = 0.0020710 ...... PASSED
============================================================================
Finished 18 tests with the following results:
18 tests completed and passed residual checks,
0 tests completed and failed residual checks,
0 tests skipped because of illegal input values.
----------------------------------------------------------------------------
End of Tests.
============================================================================
