MPI教程

0HVVDJH󰀏3DVVLQJ󰀏3URJUDPPLQJ

03,󰀏8VHUV·󰀏*XLGH󰀏LQ

)2575$1

,QWURGXFWLRQWR0HVVDJHSDVVLQJSURJUDPPLQJ

03,󰀄8VHU󰀄*XLGH󰀄LQ󰀄)2575$1

'U3HWHU6󰀇3DFKHFR'HSDUWPHQWRI0DWKHPDWLFV8QLYHUVLW\\RI6DQ)UDQFLVFR6DQ)UDQFLVFR󰀗&$󰀚󰀛󰀜󰀜󰀝

0DUFK󰀞󰀟󰀗󰀜󰀚󰀚󰀠:RR&KDW0LQJ&RPSXWHU&HQWUH8QLYHUVLW\\RI+RQJ.RQJ

+RQJ.RQJ0DUFK󰀜󰀝󰀗󰀜󰀚󰀚󰀝

1. Introduction2. Greetings !

2.1 General MPI Programs

2.2 Finding out About the Rest of the World2.3 Message : Data + Envelope2.4 MPI_Send and MPI_Recv3. An Application3.1 Serial program

3.2 Parallelizing the Trapezoid Rule3.3 I/O on Parallel Processors4. Collective Communication4.1 Tree-Structured Communication4.2 Broadcast4.3 Reduce

4.4 Other Collective Communication Functions5. Grouping Data for Communication5.1 The Count Parameter

5.2 Derived Types and MPI_Type_struct5.3 Other Derived Datatype Constructors5.4 Pack/Unpack

5.5 Deciding Which Method to Use6. Communicators and Topologies6.1 Fox's Algorithm6.2 Communicators

6.3 Working with Groups, Contexts, and Communicators6.4 MPI_Comm_split6.5 Topologies6.6 MPI_Cart_sub

6.7 Implementation of Fox's Algorithm7. Where To Go From Here7.1 What We Haven't Discussed7.2 Implementations of MPI7.3 More Information on MPI7.4 The Future of MPI

8. Compiling and Running MPI Programs9. Reference

󰀔󰀕 ,QWURGXFWLRQ

KH󰀏0HVVDJH󰀜3DVVLQJ󰀏,QWHUIDFH󰀏RU󰀏03,󰀏LV󰀏D󰀏OLEUDU\\󰀏RI󰀏IXQFWLRQV󰀏DQG󰀏PDFURV󰀏WKDW󰀏FDQ󰀏EH󰀏XVHG󰀏LQ&󰀢󰀏)2575$1󰀢󰀏DQG󰀏&󰀣󰀣󰀏SURJUDPV󰀢󰀏$V󰀏LWV󰀏QDPH󰀏LPSOLHV󰀢󰀏03,󰀏LV󰀏LQWHQGHG󰀏IRU󰀏XVH󰀏LQSURJUDPV󰀏WKDW󰀏H[SORLW󰀏WKH󰀏H[LVWHQFH󰀏RI󰀏PXOWLSOH󰀏SURFHVVRUV󰀏E\\󰀏PHVVDJH󰀜SDVVLQJ󰀦

03,󰀏ZDV󰀏GHYHORSHG󰀏LQ󰀏󰀩󰀪󰀪󰀫󰀜󰀩󰀪󰀪󰀬󰀏E\\󰀏D󰀏JURXS󰀏RI󰀏UHVHDUFKHUV󰀏IURP󰀏LQGXVWU\\󰀢󰀏JRYHUQPHQW󰀢󰀏DQGDFDGHPLD󰀦󰀏$V󰀏VXFK󰀢󰀏LW󰀏LV󰀏RQH󰀏RI󰀏WKH󰀏ILUVW󰀏VWDQGDUGV󰀏IRU󰀏SURJUDPPLQJ󰀏SDUDOOHO󰀏SURFHVVRUV󰀢󰀏DQG󰀏LWLV󰀏WKH󰀏ILUVW󰀏WKDW󰀏LV󰀏EDVHG󰀏RQ󰀏PHVVDJH󰀜SDVVLQJ󰀦

,Q󰀏󰀩󰀪󰀪󰀭󰀢󰀏$󰀂8VHU·V󰀂*XLGH󰀂WR󰀂03,󰀂KDV󰀏EHHQ󰀏ZULWWHQ󰀏E\\󰀏'U󰀏3HWHU󰀏6󰀦󰀏3DFKHFR󰀦󰀏7KLV󰀏LV󰀏D󰀏EULHIWXWRULDO󰀏LQWURGXFWLRQ󰀏WR󰀏VRPH󰀏RI󰀏WKH󰀏PRUH󰀏LPSRUWDQW󰀏IHDWXUH󰀏RI󰀏WKH󰀏03,󰀏IRU󰀏&󰀏SURJUDPPHUV󰀦,W󰀏LV󰀏D󰀏QLFHO\\󰀏ZULWWHQ󰀏GRFXPHQWDWLRQ󰀏DQG󰀏XVHUV󰀏LQ󰀏RXU󰀏XQLYHUVLW\\󰀏ILQG󰀏LW󰀏YHU\\󰀏FRQFLVH󰀏DQG󰀏HDV\\WR󰀏UHDG󰀦

+RZHYHU󰀢󰀏PDQ\\󰀏XVHUV󰀏RI󰀏SDUDOOHO󰀏FRPSXWHU󰀏DUH󰀏LQ󰀏WKH󰀏VFLHQWLILF󰀏DQG󰀏HQJLQHHUV󰀏FRPPXQLW\\󰀏DQGPRVW󰀏RI󰀏WKHP󰀏XVH󰀏)2575$1󰀏DV󰀏WKHLU󰀏SULPDU\\󰀏FRPSXWHU󰀏ODQJXDJH󰀦󰀏0RVW󰀏RI󰀏WKHP󰀏GRQ·W󰀏XVH&󰀏ODQJXDJH󰀏SURILFLHQWO\\󰀦󰀏7KLV󰀏VLWXDWLRQ󰀏RFFXUV󰀏YHU\\󰀏IUHTXHQWO\\󰀏LQ󰀏+RQJ󰀏.RQJ󰀦󰀏$󰀏D󰀏UHVXOW󰀢󰀏WKH󰀳$󰀂8VHU·V󰀂*XLGH󰀂WR󰀂03,󰀑󰀂LV󰀏WUDQVODWHG󰀏WR󰀏WKLV󰀏JXLGH󰀏LQ󰀏)RUWUDQ󰀏WR󰀏DGGUHVV󰀏IRU󰀏WKH󰀏QHHG󰀏RIVFLHQWLILF󰀏SURJUDPPHUV󰀦

$FNQRZOHGJPHQWV󰀦󰀏,󰀏JUDWHIXOO\\󰀏DFNQRZOHGJH󰀏'U󰀏3HWHU󰀏6󰀦󰀏3DFKHFR󰀏IRU󰀏WKH󰀏XVH󰀏RI󰀏&󰀏YHUVLRQ󰀏RI󰀏WKHXVHU󰀏JXLGH󰀏RQ󰀏ZKLFK󰀏WKLV󰀏JXLGH󰀏LV󰀏EDVHG󰀦󰀏,󰀏ZRXOG󰀏DOVR󰀏JUDWHIXOO\\󰀏WKDQNV󰀏WR󰀏WKH󰀏&RPSXWHU&HQWUH󰀏RI󰀏WKH󰀏8QLYHUVLW\\󰀏RI󰀏+RQJ󰀏.RQJ󰀏IRU󰀏WKHLU󰀏KXPDQ󰀏UHVRXUFH󰀏VXSSRUW󰀏RI󰀏WKLV󰀏ZRUN󰀦󰀏$QG,󰀏DOVR󰀏WKDQNV󰀏WR󰀏DOO󰀏WKH󰀏UHVHDUFK󰀏LQVWLWXWLRQ󰀏ZKLFK󰀏VXSSRUWHG󰀏WKH󰀏RULJLQDO󰀏ZRUN󰀏E\\󰀏'U3DFKHFR󰀦

󰀙󰀕 *UHHWLQJV󰀄󰀛

7KH󰀏ILUVW󰀏SURJUDP󰀏WKDW󰀏PRVW󰀏RI󰀏XV󰀏VDZ󰀏ZDV󰀏WKH󰀏󰀳+HOOR󰀢󰀏ZRUOG󰀵󰀳󰀏SURJUDP󰀏LQ󰀏PRVW󰀏RI󰀏LQWURGXFWRU\\SURJUDPPLQJ󰀏ERRNV󰀦󰀏,W󰀏VLPSO\\󰀏SULQWV󰀏WKH󰀏PHVVDJH󰀏󰀳+HOOR󰀢󰀏ZRUOG󰀵󰀳󰀦󰀏$󰀏YDULDQW󰀏WKDW󰀏PDNHV󰀏VRPH󰀏XVH󰀏RIPXOWLSOH󰀏SURFHVVHV󰀏LV󰀏WR󰀏KDYH󰀏HDFK󰀏SURFHVV󰀏VHQG󰀏D󰀏JUHHWLQJ󰀏WR󰀏DQRWKHU󰀏SURFHVV󰀦

,Q󰀏03,󰀢󰀏WKH󰀏SURFHVV󰀏LQYROYHG󰀏LQ󰀏WKH󰀏H[HFXWLRQ󰀏RI󰀏D󰀏SDUDOOHO󰀏SURJUDP󰀏DUH󰀏LGHQWLILHG󰀏E\\󰀏D󰀏VHTXHQFH󰀏RIQRQ󰀜QHJDWLYH󰀏LQWHJHUV󰀦󰀏,I󰀏WKHUH󰀏DUH󰀏S󰀏SURFHVVHV󰀏H[HFXWLQJ󰀏D󰀏SURJUDP󰀢󰀏WKH\\󰀏ZLOO󰀏KDYH󰀏UDQNV󰀏󰀶󰀢󰀏󰀩󰀢󰀦󰀦󰀦󰀢󰀏S󰀜󰀩󰀦7KH󰀏IROORZLQJ󰀏SURJUDP󰀏KDV󰀏HDFK󰀏SURFHVV󰀏RWKHU󰀏WKDQ󰀏󰀶󰀏VHQG󰀏D󰀏PHVVDJH󰀏WR󰀏SURFHVV󰀏󰀶󰀢󰀏DQG󰀏SURFHVV󰀏󰀶SULQWV󰀏RXW󰀏WKH󰀏PHVVDJHV󰀏LW󰀏UHFHLYHG󰀦program greetingsinclude 'mpif.h'integer my_rankinteger pinteger sourceinteger destinteger tag

character*100 message

character*10 digit_stringinteger size

integer status(MPI_STATUS_SIZE)integer ierr

call MPI_Init(ierr)

call MPI_Comm_rank(MPI_COMM_WORLD, my_rank, ierr)call MPI_Comm_size(MPI_COMM_WORLD, p, ierr)if (my_rank .NE. 0) then

write(digit_string,FMT=\"(I3)\") my_rankmessage = 'Greetings from process '+ // trim(digit_string) // ' !'dest = 0tag = 0

call MPI_Send(message, len_trim(message),

+ MPI_CHARACTER, dest, tag, MPI_COMM_WORLD, ierr)else

do source = 1, p-1tag = 0

call MPI_Recv(message, 100, MPI_CHARACTER,+ source, tag, MPI_COMM_WORLD, status, ierr)write(6,FMT=\"(A)\") messageenddoendif

call MPI_Finalize(ierr)end program greetings

7KH󰀏GHWDLOV󰀏RI󰀏FRPSLOLQJ󰀏DQG󰀏H[HFXWLQJ󰀏WKLV󰀏SURJUDP󰀏LV󰀏LQ󰀏FKDSWHU󰀏󰀷󰀦

:KHQ󰀏WKH󰀏SURJUDP󰀏LV󰀏FRPSLOHG󰀏DQG󰀏UXQ󰀏ZLWK󰀏WZR󰀏SURFHVVHV󰀢󰀏WKH󰀏RXWSXW󰀏VKRXOG󰀏EHGreetings from process 1!

,I󰀏LW·V󰀏UXQ󰀏ZLWK󰀏IRXU󰀏SURFHVVHV󰀢󰀏WKH󰀏RXWSXW󰀏VKRXOG󰀏EHGreetings from process 1!

Greetings from process 2!Greetings from process 3!

$OWKRXJK󰀏WKH󰀏GHWDLOV󰀏RI󰀏ZKDW󰀏KDSSHQV󰀏ZKHQ󰀏WKH󰀏SURJUDP󰀏LV󰀏H[HFXWHG󰀏YDU\\󰀏IURP󰀏PDFKLQH󰀏WR󰀏PDFKLQH󰀢WKH󰀏HVVHQWLDOV󰀏DUH󰀏WKH󰀏VDPH󰀏RQ󰀏DOO󰀏PDFKLQHV󰀦󰀏3URYLGHG󰀏ZH󰀏UXQ󰀏RQH󰀏SURFHVV󰀏RQ󰀏HDFK󰀏SURFHVVRU󰀦

󰀩󰀦 7KH󰀏XVHU󰀏LVVXHV󰀏D󰀏GLUHFWLYH󰀏WR󰀏WKH󰀏RSHUDWLQJ󰀏V\\VWHP󰀏ZKLFK󰀏KDV󰀏WKH󰀏HIIHFW󰀏RI󰀏SODFLQJ󰀏DFRS\\󰀏RI󰀏WKH󰀏H[HFXWDEOH󰀏SURJUDP󰀏RQ󰀏HDFK󰀏SURFHVVRU󰀦󰀩󰀦 (DFK󰀏SURFHVVRU󰀏EHJLQV󰀏H[HFXWLRQ󰀏RI󰀏LWV󰀏FRS\\󰀏RI󰀏WKH󰀏H[HFXWDEOH󰀦

󰀩󰀦 'LIIHUHQW󰀏SURFHVVHV󰀏FDQ󰀏H[HFXWH󰀏GLIIHUHQW󰀏VWDWHPHQWV󰀏E\\󰀏EUDQFKLQJ󰀏ZLWKLQ󰀏WKH

SURJUDP󰀦󰀏7\\SLFDOO\\󰀏WKH󰀏EUDQFKLQJ󰀏ZLOO󰀏EH󰀏EDVHG󰀏RQ󰀏SURFHVV󰀏UDQNV󰀦

6R󰀏WKH󰀏Greetings󰀏SURJUDP󰀏XVHV󰀏WKH󰀏6LQJOH󰀂3URJUDP󰀂0XOWLSOH󰀂'DWD󰀂RU󰀏630'󰀏SDUDGLJP󰀦󰀏7KDW󰀏LV󰀢󰀏ZHREWDLQ󰀏WKH󰀏HIIHFW󰀏RI󰀏GLIIHUHQW󰀏SURJUDPV󰀏UXQQLQJ󰀏RQ󰀏GLIIHUHQW󰀏SURFHVVRUV󰀏E\\󰀏WDNLQJ󰀏EUDQFKHV󰀏ZLWKLQ󰀏DVLQJOH󰀏SURJUDP󰀏RQ󰀏WKH󰀏EDVLV󰀏RI󰀏SURFHVV󰀏UDQN󰀏󰀺󰀏WKH󰀏VWDWHPHQWV󰀏H[HFXWHG󰀏E\\󰀏SURFHVV󰀏󰀶󰀏DUH󰀏GLIIHUHQWIURP󰀏WKRVH󰀏H[HFXWHG󰀏E\\󰀏WKH󰀏RWKHU󰀏SURFHVVHV󰀢󰀏HYHQ󰀏WKRXJK󰀏DOO󰀏SURFHVVHV󰀏DUH󰀏UXQQLQJ󰀏WKH󰀏VDPHSURJUDP󰀦󰀏7KLV󰀏LV󰀏WKH󰀏PRVW󰀏FRPPRQO\\󰀏XVHG󰀏PHWKRG󰀏IRU󰀏ZULWLQJ󰀏0,0'󰀏SURJUDPV󰀢󰀏DQG󰀏ZH·OO󰀏XVH󰀏LWH[FOXVLYHO\\󰀏LQ󰀏WKLV󰀏*XLGH󰀦

󰀔󰀕 *HQHUDO󰀄03,󰀄3URJUDPV

(YHU\\󰀏03,󰀏SURJUDP󰀏PXVW󰀏FRQWDLQ󰀏WKH󰀏SUHSUHFHVVRU󰀏GLUHFWLYHinclude ‘mpif.h’

7KLV󰀏ILOH󰀢󰀏PSLI󰀦K󰀢󰀏FRQWDLQV󰀏WKH󰀏GHILQLWLRQV󰀢󰀏PDFURV󰀏DQG󰀏IXQFWLRQ󰀏SURWRW\\SHV󰀏QHFHVVDU\\

IRU󰀏FRPSLOLQJ󰀏DQ󰀏03,󰀏SURJUDP󰀦

%HIRUH󰀏DQ\\󰀏RWKHU󰀏03,󰀏IXQFWLRQV󰀏FDQ󰀏EH󰀏FDOOHG󰀢󰀏WKH󰀏IXQFWLRQ󰀏MPI_Init󰀏PXVW󰀏EH󰀏FDOOHG󰀢DQG󰀏LW󰀏VKRXOG󰀏RQO\\󰀏EH󰀏FDOOHG󰀏RQFH󰀦󰀏)RUWUDQ󰀏03,󰀏URXWLQHV󰀏KDYH󰀏DQ󰀏IERROR DUJXPHQW󰀜󰀏WKLV󰀏FRQWDLQV󰀏WKH󰀏HUURU󰀏FRGH󰀦󰀏$IWHU󰀏D󰀏SURJUDP󰀏KDV󰀏ILQLVKHG󰀏XVLQJ󰀏03,󰀏OLEUDU\\󰀢󰀏LW󰀏PXVWFDOO󰀏MPI_Finialize󰀦󰀏7KLV󰀏FOHDQV󰀏XS󰀏DQ\\󰀏󰀳XQILQLVKHG󰀏EXVLQHVV󰀳󰀏OHIW󰀏E\\󰀏03,󰀏󰀜󰀏H󰀦J󰀦SHQGLQJ󰀏UHFHLYHV󰀏WKDW󰀏ZHUH󰀏QHYHU󰀏FRPSOHWHG󰀦󰀏6R󰀏D󰀏W\\SLFDO󰀏03,󰀏SURJUDP󰀏KDV󰀏WKHIROORZLQJ󰀏OD\\RXW󰀦...

include 'mpif.h'...

call MPI_Init(ierr)..

.

call MPI_Finialize(ierr)...

end program

󰀙󰀕 )LQGLQJ󰀄RXW󰀄$ERXW󰀄WKH󰀄5HVW󰀄RI󰀄WKH󰀄:RUOG

03,󰀏SURYLGHV󰀏WKH󰀏IXQFWLRQ󰀏MPI_Comm_rank󰀢󰀏ZKLFK󰀏UHWXUQV󰀏WKH󰀏UDQN󰀏RI󰀏D󰀏SURFHVVLQ󰀏LWV󰀏VHFRQG󰀏LQ󰀏LWV󰀏VHFRQG󰀏DUJXPHQW󰀢󰀏,WV󰀏V\\QWD[󰀏LV

call MPI_Comm_rank(COMM, RANK, IERROR)integer COMM, RANK, IERROR

7KH󰀏ILUVW󰀏DUJXPHQW󰀏LV󰀏D󰀏FRPPXQLFDWRU󰀦󰀏(VVHQWLDOO\\󰀏D󰀏FRPPXQLFDWRU󰀏LV󰀏D󰀏FROOHFWLRQ󰀏RISURFHVVHV󰀏WKDW󰀏FDQ󰀏VHQG󰀏PHVVDJH󰀏WR󰀏HDFK󰀏RWKHU󰀦󰀏)RU󰀏EDVLF󰀏SURJUDPV󰀢󰀏WKH󰀏RQO\\FRPPXQLFDWRU󰀏QHHGHG󰀏LV󰀏MPI_COMM_WORLD󰀦󰀏,W󰀏LV󰀏SUHGHILQHG󰀏LQ󰀏03,󰀏DQGFRQVLVWV󰀏RI󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏UXQQLQJ󰀏ZKHQ󰀏SURJUDP󰀏H[HFXWLRQ󰀏EHJLQV󰀦

0DQ\\󰀏RI󰀏WKH󰀏FRQVWUXFWV󰀏LQ󰀏RXU󰀏SURJUDPV󰀏DOVR󰀏GHSHQG󰀏RQ󰀏WKH󰀏QXPEHU󰀏RI󰀏SURFHVVHVH[HFXWLQJ󰀏WKH󰀏SURJUDP󰀦󰀏6R󰀏03,󰀏SURYLGHV󰀏WKH󰀏IXQFWLRQV󰀏MPI_Comm_size󰀏IRUGHWHUPLQLQJ󰀏WKLV󰀦󰀏,WV󰀏ILUVW󰀏DUJXPHQW󰀏LV󰀏D󰀏FRPPXQLFDWRU󰀦󰀏,W󰀏UHWXUQV󰀏WKH󰀏QXPEHU󰀏RISURFHVVHV󰀏LQ󰀏D󰀏FRPPXQLFDWRU󰀏LQ󰀏LWV󰀏VHFRQG󰀏DUJXPHQW󰀦󰀏,WV󰀏V\\QWD[󰀏LVcall MPI_Comm_size(COMM, P, IERROR)integer COMM, P, IERROR

󰀣󰀕 0HVVDJH󰀄󰀤󰀄'DWD󰀄󰀦󰀄(QYHORSH

7KH󰀏DFWXDO󰀏PHVVDJH󰀜SDVVLQJ󰀏LQ󰀏RXU󰀏SURJUDP󰀏LV󰀏FDUULHG󰀏RXW󰀏E\\󰀏WKH󰀏03,󰀏IXQFWLRQV󰀏MPI_Send DQG

MPI_Recv󰀦󰀏7KH󰀏ILUVW󰀏FRPPDQG󰀏VHQGV󰀏D󰀏PHVVDJH󰀏WR󰀏D󰀏GHVLJQDWHG󰀏SURFHVV󰀦󰀏7KH󰀏VHFRQG󰀏UHFHLYHV󰀏DPHVVDJH󰀏IURP󰀏D󰀏SURFHVV󰀦󰀏7KHVH󰀏DUH󰀏WKH󰀏PRVW󰀏EDVLF󰀏PHVVDJH󰀜SDVVLQJ󰀏FRPPDQGV󰀏LQ󰀏03,󰀦󰀏,Q󰀏RUGHU󰀏IRUWKH󰀏PHVVDJH󰀏WR󰀏EH󰀏VXFFHVVIXOO\\󰀏FRPPXQLFDWHG󰀏WKH󰀏V\\VWHP󰀏PXVW󰀏DSSHQG󰀏VRPH󰀏LQIRUPDWLRQ󰀏WR󰀏WKH󰀏GDWDWKDW󰀏WKH󰀏DSSOLFDWLRQ󰀏SURJUDP󰀏ZLVKHV󰀏WR󰀏WUDQVPLW󰀦󰀏7KLV󰀏DGGLWLRQDO󰀏LQIRUPDWLRQ󰀏IRUPV󰀏WKH󰀏HQYHORSH󰀏RI󰀏WKHPHVVDJH󰀦󰀏,Q󰀏03,󰀏LW󰀏FRQWDLQV󰀏WKH󰀏IROORZLQJ󰀏LQIRUPDWLRQ󰀦

󰀩󰀦 7KH󰀏UDQN󰀏RI󰀏WKH󰀏UHFHLYHU󰀦󰀩󰀦 7KH󰀏UDQN󰀏RI󰀏WKH󰀏VHQGHU󰀦󰀩󰀦 $󰀏WDJ󰀦

󰀩󰀦 $󰀏FRPPXQLFDWRU󰀦

7KHVH󰀏LWHPV󰀏FDQ󰀏EH󰀏XVHG󰀏E\\󰀏WKH󰀏UHFHLYHU󰀏WR󰀏GLVWLQJXLVK󰀏DPRQJ󰀏LQFRPLQJ󰀏PHVVDJHV󰀦󰀏7KH󰀏VRXUFHDUJXPHQW󰀏FDQ󰀏EH󰀏XVHG󰀏WR󰀏GLVWLQJXLVK󰀏PHVVDJHV󰀏UHFHLYHG󰀏IURP󰀏GLIIHUHQW󰀏SURFHVVHV󰀦󰀏7KH󰀏WDJ󰀏LV󰀏D󰀏XVHU󰀜VSHFLILHG󰀏integer󰀏WKDW󰀏FDQ󰀏EH󰀏XVHG󰀏WR󰀏GLVWLQJXLVK󰀏PHVVDJHV󰀏UHFHLYHG󰀏IRUP󰀏D󰀏VLQJOH󰀏SURFHVV󰀦󰀏)RU

H[DPSOH󰀢󰀏VXSSRVH󰀏SURFHVV󰀏$󰀏LV󰀏VHQGLQJ󰀏WZR󰀏PHVVDJHV󰀏WR󰀏SURFHVV󰀏%󰀼󰀏ERWK󰀏PHVVDJHV󰀏FRQWDLQV󰀏D󰀏VLQJOHUHDO󰀏QXPEHU󰀦󰀏2QH󰀏RI󰀏WKH󰀏UHDO󰀏QXPEHU󰀏LV󰀏WR󰀏EH󰀏XVHG󰀏LQ󰀏D󰀏FDOFXODWLRQ󰀢󰀏ZKLOH󰀏WKH󰀏RWKHU󰀏LV󰀏WR󰀏EH󰀏SULQWHG󰀦,Q󰀏RUGHU󰀏WR󰀏GHWHUPLQH󰀏ZKLFK󰀏LV󰀏ZKLFK󰀢󰀏$󰀏FDQ󰀏XVH󰀏GLIIHUHQW󰀏WDJV󰀏IRU󰀏WKH󰀏WZR󰀏PHVVDJHV󰀦󰀏,I󰀏%󰀏XVHV󰀏WKHVDPH󰀏WZR󰀏WDJV󰀏LQ󰀏WKH󰀏FRUUHVSRQGLQJ󰀏UHFHLYHV󰀢󰀏ZKHQ󰀏LW󰀏UHFHLYHV󰀏WKH󰀏PHVVDJHV󰀢󰀏LW󰀏ZLOO󰀏󰀳NQRZ󰀳󰀏ZKDW󰀏WRGR󰀏ZLWK󰀏WKHP󰀦󰀏03,󰀏JXDUDQWHHV󰀏WKDW󰀏WKH󰀏LQWHJHUV󰀏󰀶󰀜󰀫󰀽󰀾󰀿󰀾󰀏FDQ󰀏EH󰀏XVHG󰀏DV󰀏WDJV󰀦󰀏0RVW󰀏LPSOHPHQWDWLRQVDOORZ󰀏PXFK󰀏ODUJHU󰀏YDOXHV󰀦

$V󰀏ZH󰀏QRWHG󰀏DERYH󰀢󰀏D󰀏FRPPXQLFDWRU󰀏LV󰀏EDVLFDOO\\󰀏D󰀏FROOHFWLRQ󰀏RI󰀏SURFHVVHV󰀏WKDW󰀏FDQ󰀏VHQG󰀏PHVVDJHV󰀏WRHDFK󰀏RWKHU󰀦󰀏:KHQ󰀏WZR󰀏SURFHVVHV󰀏DUH󰀏FRPPXQLFDWLQJ󰀏XVLQJ󰀏MPI_Send󰀏DQG󰀏MPI_Recv󰀢󰀏LWVLPSRUWDQFH󰀏DULVHV󰀏ZKHQ󰀏VHSDUDWH󰀏PRGXOHV󰀏RI󰀏D󰀏SURJUDP󰀏KDYH󰀏EHHQ󰀏ZULWWHQ󰀏LQGHSHQGHQWO\\󰀏RI󰀏HDFKRWKHU󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏VXSSRVH󰀏ZH󰀏ZLVK󰀏WR󰀏VROYH󰀏D󰀏V\\VWHP󰀏RI󰀏GLIIHUHQWLDO󰀏HTXDWLRQV󰀢󰀏DQG󰀢󰀏LQ󰀏WKH󰀏FRXUVHRI󰀏VROYLQJ󰀏WKH󰀏V\\VWHP󰀢󰀏ZH󰀏QHHG󰀏WR󰀏VROYH󰀏D󰀏V\\VWHP󰀏RI󰀏OLQHDU󰀏HTXDWLRQ󰀦󰀏5DWKHU󰀏WKDQ󰀏ZULWLQJ󰀏WKH󰀏OLQHDUV\\VWHP󰀏VROYHU󰀏IURP󰀏VFUDWFK󰀢󰀏ZH󰀏PLJKW󰀏ZDQW󰀏WR󰀏XVH󰀏D󰀏OLEUDU\\󰀏RI󰀏IXQFWLRQV󰀏IRU󰀏VROYLQJ󰀏OLQHDU󰀏V\\VWHPV󰀏WKDWZDV󰀏ZULWWHQ󰀏E\\󰀏VRPHRQH󰀏HOVH󰀏DQG󰀏WKDW󰀏KDV󰀏EHHQ󰀏KLJKO\\󰀏RSWLPL]HG󰀏IRU󰀏WKH󰀏V\\VWHP󰀏ZH·UH󰀏XVLQJ󰀦󰀏+RZ󰀏GRZH󰀏DYRLG󰀏FRQIXVLQJ󰀏WKH󰀏PHVVDJHV󰀏ZH󰀏VHQG󰀏IURP󰀏SURFHVV󰀏$󰀏WR󰀏SURFHVV󰀏%󰀏ZLWK󰀏WKRVH󰀏VHQW󰀏E\\󰀏WKH󰀏OLEUDU\\IXQFWLRQV󰀏\"󰀏%HIRUH󰀏WKH󰀏DGYHQW󰀏RI󰀏FRPPXQLFDWRUV󰀢󰀏ZH󰀏ZRXOG󰀏SUREDEO\\󰀏KDYH󰀏WR󰀏SDUWLWLRQ󰀏WKH󰀏VHW󰀏RIYDOLG󰀏WDJV󰀢󰀏VHWWLQJ󰀏DVLGH󰀏VRPH󰀏RI󰀏WKHP󰀏IRU󰀏H[FOXVLYH󰀏XVH󰀏E\\󰀏WKH󰀏OLEUDU\\󰀏IXQFWLRQV󰀦󰀏7KLV󰀏LV󰀏WHGLRXV󰀏DQG󰀏LWZLOO󰀏FDXVH󰀏SUREOHPV󰀏LI󰀏ZH󰀏WU\\󰀏WR󰀏UXQ󰀏RXU󰀏SURJUDP󰀏RQ󰀏DQRWKHU󰀏V\\VWHP󰀏󰀺󰀏WKH󰀏RWKHU󰀏V\\VWHP·V󰀏OLQHDU󰀏VROYHUPD\\󰀏QRW󰀏󰁂SUREDEO\\󰀏ZRQ·W󰁃󰀏UHTXLUH󰀏WKH󰀏VDPH󰀏VHW󰀏RI󰀏WDJV󰀦󰀏:LWK󰀏WKH󰀏DGYHQW󰀏RI󰀏FRPPXQLFDWRUV󰀢󰀏ZHVLPSO\\󰀏FUHDWH󰀏D󰀏FRPPXQLFDWRU󰀏WKDW󰀏FDQ󰀏EH󰀏XVHG󰀏H[FOXVLYHO\\󰀏E\\󰀏WKH󰀏OLQHDU󰀏VROYHU󰀢󰀏DQG󰀏SDVV󰀏LW󰀏DV󰀏DQDUJXPHQW󰀏LQ󰀏FDOOV󰀏WR󰀏WKH󰀏VROYHU󰀦󰀏:H·OO󰀏GLVFXVV󰀏WKH󰀏GHWDLOV󰀏RI󰀏WKLV󰀏ODWHU󰀦󰀏)RU󰀏QRZ󰀢󰀏ZH󰀏FDQ󰀏JHW󰀏DZD\\󰀏ZLWKXVLQJ󰀏WKH󰀏SUHGHILQHG󰀏FRPPXQLFDWRU󰀏MPI_COMM_WORLD󰀦󰀏,W󰀏FRQVLVWV󰀏RI󰀏DOO󰀏WKH󰀏SURFHVVHVUXQQLQJ󰀏WKH󰀏SURJUDP󰀏ZKHQ󰀏H[HFXWLRQ󰀏EHJLQV󰀦

󰀔󰀕 03,B6HQG󰀄DQG󰀄03,B5HFY

7R󰀏VXPPDUL]H󰀢󰀏OHW·V󰀏GHWDLO󰀏WKH󰀏V\\QWD[󰀏RI󰀏MPI_Send󰀏DQG󰀏MPI_Recv󰀦

MPI_Send( Message, count, datatype, dest, tag, comm, ierror) message(*)

INTEGER count, datatype, dest, tag, comm, ierrorMPI_Recv( message, count, datatype, source, tag,comm, status, ierror) message(*)

INTEGER count, datatype, dest, tag, commINTEGER status(MPI_STATUS_SIZE),ierror

0RVW󰀏03,󰀏IXQFWLRQV󰀏VWRUHV󰀏DQ󰀏LQWHJHU󰀏HUURU󰀏FRGH󰀏LQ󰀏WKH󰀏DUJXPHQW󰀏LHUURU󰀦󰀏+RZHYHU󰀢󰀏ZH󰀏ZLOO󰀏LJQRUH

WKHVH󰀏UHWXUQ󰀏YDOXHV󰀏LQ󰀏PRVW󰀏FDVHV󰀦

7KH󰀏FRQWHQWV󰀏RI󰀏WKH󰀏PHVVDJH󰀏DUH󰀏VWRUHG󰀏LQ󰀏D󰀏EORFN󰀏RI󰀏PHPRU\\󰀏UHIHUHQFHG󰀏E\\󰀏WKH󰀏DUJXPHQWmessage󰀦󰀏7KH󰀏QH[W󰀏WZR󰀏DUJXPHQWV󰀢󰀏count󰀏DQG󰀏datatype󰀢󰀏DOORZ󰀏WKH󰀏V\\VWHP󰀏WR󰀏LGHQWLI\\󰀏WKH󰀏HQG󰀏RIWKH󰀏PHVVDJH󰀏󰀺󰀏LW󰀏FRQWDLQV󰀏D󰀏VHTXHQFH󰀏RI󰀏count󰀏YDOXHV󰀢󰀏HDFK󰀏KDYLQJ󰀏03,󰀏W\\SH󰀏datatype󰀦󰀏7KLV󰀏W\\SH󰀏LVQRW󰀏D󰀏)RUWUDQ󰀏W\\SH󰀢󰀏DOWKRXJK󰀏PRVW󰀏RI󰀏WKH󰀏SUHGHILQHG󰀏W\\SHV󰀏FRUUHVSRQG󰀏)RUWUDQ󰀏W\\SHV󰀦󰀏7KH󰀏SUHGHILQHG03,󰀏W\\SHV󰀏DQG󰀏WKH󰀏FRUUHVSRQGLQJ󰀏)2575$1󰀏W\\SHV󰀏󰁂LI󰀏WKH\\󰀏H[LVW󰁃󰀏DUH󰀏OLVWHG󰀏LQ󰀏WKH󰀏IROORZLQJ󰀏WDEOH󰀦

03,󰀏GDWDW\\SH

)2575$1󰀏GDWDW\\SH

03,B,17(*(503,B5($/

03,B'28%/(B35(&,6,2103,B&203/(;03,B/2*,&$/03,B&+$5$&7(503,B%<7(03,B3$&.('

,17(*(55($/

'28%/(󰀏35(&,6,21&203/(;/2*,&$/&+$5$&7(5󰁂󰀩󰁃

7KH󰀏ODVW󰀏WZR󰀏W\\SHV󰀢󰀏MPI_BYTE󰀏DQG󰀏MPI_PACKED󰀢󰀏GRQ·W󰀏FRUUHVSRQG󰀏WR󰀏VWDQGDUG󰀏)RUWUDQ󰀏W\\SHV󰀦7KH󰀏03,B%<7(󰀏W\\SH󰀏FDQ󰀏EH󰀏XVHG󰀏LI󰀏\\RX󰀏ZLVK󰀏WR󰀏IRUFH󰀏WKH󰀏V\\VWHP󰀏WR󰀏SHUIRUP󰀏QR󰀏FRQYHUVLRQEHWZHHQ󰀏GLIIHUHQW󰀏GDWD󰀏UHSUHVHQWDWLRQV󰀏󰁂󰀏H󰀦J󰀦󰀏RQ󰀏D󰀏KHWHURJHQHRXV󰀏QHWZRUN󰀏RI󰀏ZRUNVWDWLRQV󰀏XVLQJGLIIHUHQW󰀏UHSUHVHQWDWLRQV󰀏RI󰀏GDWD󰁃󰀦󰀏:H·OO󰀏GLVFXVV󰀏WKH󰀏W\\SH󰀏MPI_PACKED󰀏ODWHU󰀦

1RWH󰀏WKDW󰀏WKH󰀏DPRXQW󰀏RI󰀏VSDFH󰀏DOORFDWHG󰀏IRU󰀏WKH󰀏UHFHLYLQJ󰀏EXIIHU󰀏GRHV󰀏QRW󰀏KDYH󰀏WR󰀏PDWFK󰀏WKH󰀏H[DFWDPRXQW󰀏RI󰀏VSDFH󰀏LQ󰀏WKH󰀏PHVVDJH󰀏EHLQJ󰀏UHFHLYHG󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏ZKHQ󰀏RXU󰀏SURJUDP󰀏LV󰀏UXQ󰀢󰀏WKH󰀏VL]H󰀏RIWKH󰀏PHVVDJH󰀏WKDW󰀏SURFHVV󰀏󰀩󰀏VHQGV󰀢󰀏len_trim(message)󰀢󰀏LV󰀏󰀽󰀷󰀏FKDUDFWHUV󰀢󰀏EXW󰀏SURFHVV󰀏󰀶󰀏UHFHLYHV󰀏WKHPHVVDJH󰀏LQ󰀏D󰀏EXIIHU󰀏WKDW󰀏KDV󰀏VWRUDJH󰀏IRU󰀏󰀩󰀶󰀶󰀏FKDUDFWHUV󰀦󰀏7KVL󰀏PDNHV󰀏VHQVH󰀦󰀏,Q󰀏JHQHUDO󰀢󰀏WKH󰀏UHFHLYLQJSURFHVV󰀏PD\\󰀏QRW󰀏NQRZ󰀏WKH󰀏H[DFW󰀏VL]H󰀏RI󰀏WKH󰀏PHVVDJH󰀏EHLQJ󰀏VHQW󰀦󰀏6R󰀏03,󰀏DOORZV󰀏D󰀏PHVVDJH󰀏WR󰀏EHUHFHLYHG󰀏DV󰀏ORQJ󰀏DV󰀏WKHUH󰀏LV󰀏VXIILFLHQW󰀏VWRUDJH󰀏DOORFDWHG󰀦󰀏,I󰀏WKHUH󰀏LVQ·W󰀏VXIILFLHQW󰀏VWRUDJH󰀢󰀏DQ󰀏RYHUIORZHUURU󰀏RFFXUV󰀏>󰀬@󰀦

7KH󰀏DUJXPHQWV󰀏dest󰀏DQG󰀏source󰀏DUH󰀢󰀏UHVSHFWLYHO\\󰀢󰀏WKH󰀏UDQNV󰀏RI󰀏WKH󰀏UHFHLYLQJ󰀏DQG󰀏WKH󰀏VHQGLQJSURFHVVHV󰀦󰀏03,󰀏DOORZV󰀏source󰀏WR󰀏EH󰀏󰀳ZLOGFDUG󰀳󰀦󰀏7KHUH󰀏LV󰀏D󰀏SUHGHILQHG󰀏FRQVWDQWMPI_ANY_SOURCE󰀏WKDW󰀏FDQ󰀏EH󰀏XVHG󰀏LI󰀏D󰀏SURFHVV󰀏LV󰀏UHDG\\󰀏WR󰀏UHFHLYH󰀏D󰀏PHVVDJH󰀏IURP󰀏DQ\\󰀏VHQGLQJSURFHVV󰀏UDWKHU󰀏WKDQ󰀏D󰀏SDUWLFXODU󰀏VHQGLQJ󰀏SURFHVV󰀦󰀏7KHUH󰀏LV󰀏QRW󰀏D󰀏ZLOGFDUG󰀏IRU󰀏dest󰀦

$V󰀏ZH󰀏QRWHG󰀏HUOLHU󰀢󰀏03,󰀏KDV󰀏WZR󰀏PHFKDQLVPV󰀏VSHFLILFDOO\\󰀏GHVLJQHG󰀏IRU󰀏󰀳SDUWLWLRQLQJ󰀏WKH󰀏PHVVDJHVSDFH󰀳󰀏󰀺󰀏WDJV󰀏DQG󰀏FRPPXQLFDWRUV󰀦󰀏7KH󰀏DUJXPHQWV󰀏tag󰀏DQG󰀏comm󰀏DUH󰀢󰀏UHVSHFWLYHO\\󰀢󰀏WKH󰀏WDJ󰀏DQGFRPPXQLFDWRU󰀦󰀏7KH󰀏tag󰀏LV󰀏DQ󰀏LQWHJHU󰀢󰀏DQG󰀏IRU󰀏QRZ󰀢󰀏RXU󰀏RQO\\󰀏FRPPXQLFDWRU󰀏LVMPI_COMM_WORLD󰀢󰀏ZKLFK󰀢󰀏DV󰀏ZH󰀏QRWHG󰀏HDUOLHU󰀏LV󰀏SUHGHILQHG󰀏RQ󰀏DOO󰀏03,󰀏V\\VWHPV󰀏DQG󰀏FRQVLVWVRI󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏UXQQLQJ󰀏ZKHQ󰀏H[HFXWLRQ󰀏RI󰀏WKH󰀏SURJUDP󰀏EHJLQV󰀦󰀏7KHUH󰀏LV󰀏D󰀏ZLOGFDUG󰀢MPI_ANY_TAG󰀢󰀏WKDW󰀏MPI_Recv󰀏FDQ󰀏XVH󰀏IRU󰀏WKH󰀏WDJ󰀦󰀏7KHUH󰀏LV󰀏QR󰀏ZLOGFDUG󰀏IRU󰀏WKH󰀏FRPPXQLFDWRU󰀦,Q󰀏RWKHU󰀏ZRUGV󰀢󰀏LQ󰀏RUGHU󰀏IRU󰀏SURFHVV󰀏$󰀏WR󰀏VHQG󰀏D󰀏PHVVDJH󰀏WR󰀏SURFHVV󰀏%󰀢󰀏WKH󰀏DUJXPHQW󰀏comm󰀏WKDW󰀏$XVHV󰀏LQ󰀏MPI_Send󰀏PXVW󰀏EH󰀏LGHQWLFDO󰀏WR󰀏WKH󰀏DUJXPHQW󰀏WKDW󰀏%󰀏XVHV󰀏LQ󰀏MPI_Recv󰀦

7KH󰀏ODVW󰀏DUJXPHQW󰀏RI󰀏MPI_Recv󰀢󰀏status󰀢󰀏UHWXUQV󰀏LQIRUPDWLRQ󰀏RQ󰀏WKH󰀏GDWD󰀏WKDW󰀏ZDV󰀏DFWXDOO\\󰀏UHFHLYHG󰀦,W󰀏UHIHUHQFHV󰀏D󰀏DUUD\\󰀏ZLWK󰀏WZR󰀏HOHPHQWV󰀏󰀜󰀏RQH󰀏IRU󰀏WKH󰀏VRXUFH󰀏DQG󰀏RQH󰀏IRU󰀏WKH󰀏WDJV󰀦󰀏6R󰀏LI󰀢󰀏IRU󰀏H[DPSOH󰀢WKH󰀏source󰀏RI󰀏WKH󰀏UHFHLYH󰀏ZDV󰀏MPI_ANY_SOURCE󰀢󰀏WKHQ󰀏status󰀏ZLOO󰀏FRQWDLQ󰀏WKH󰀏UDQN󰀏RI󰀏WKHSURFHVV󰀏WKDW󰀏VHQW󰀏WKH󰀏PHVVDJH󰀦

󰀔󰀕 $Q󰀄$SSOLFDWLRQ

1RZ󰀏WKDW󰀏ZH󰀏NQRZ󰀏KRZ󰀏WR󰀏VHQG󰀏PHVVDJH󰀏ZLWK󰀏03,󰀢󰀏OHW·V󰀏ZULWH󰀏D󰀏SURJUDP󰀏WKDW󰀏XVHV󰀏PHVVDJH󰀜

SDVVLQJ󰀏WR󰀏FDOFXODWH󰀏D󰀏GHILQLWH󰀏LQWHJUDO󰀏ZLWK󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀦

󰀔󰀕 6HULDO󰀄SURJUDP

5HFDOO󰀏WKDW󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀏HVWLPDWHV󰀏E\\󰀏GLYLGLQJ󰀏WKH󰀏LQWHUYDO󰀏>D󰀢E@󰀏LQWR󰀏Q

VHJPHQWV󰀏RI󰀏HTXDO󰀏DQG󰀏FDOFXODWLQJ󰀏WKH󰀏IROORZLQJ󰀏VXP󰀦

+HUH󰀢󰀏K󰀂 󰀂󰀥E󰀂󰀦󰀂D󰀧󰀨Q󰀢󰀏DQG󰀏[L󰀂 󰀂D󰀂󰀪󰀂LK󰀢󰀏L󰀏 󰀏󰀶󰀢󰀦󰀦󰀦󰀢Q󰀦

%\\󰀏SXWWLQJ󰀏I󰁂[󰁃󰀏LQWR󰀏D󰀏VXESURJUDP󰀢󰀏ZH󰀏FDQ󰀏ZULWH󰀏D󰀏VHULDO󰀏SURJUDP󰀏IRU󰀏FDOFXODWLQJ󰀏DQLQWHJUDO󰀏XVLQJ󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀦

C serial.f -- calculate definite integral usingtrapezoidalC rule.C

C The function f(x) is hardwired.C Input: a, b, n.

C Output: estimate of integral from a to b of f(x)C using n trapezoids.PROGRAM serialIMPLICIT NONEreal integralreal areal binteger nreal hreal xinteger ireal fexternal f

print *, 'Enter a, b, and n'read *, a, b, n

h = (b-a)/n

integral = (f(a) + f(b))/2.0x = a

do i = 1 , n-1x = x + h

integral = integral + f(x)enddo

integral = integral*h

print *,'With n =', n,' trapezoids, our estimate'print *,'of the integral from ', a, ' to ',b, ' = ' ,+integralend

C******************************************************

real function f(x)IMPLICIT NONEreal x

C Calculate f(x).f = x*xreturnend

C******************************************************

󰀙󰀕 3DUDOOHOL]LQJ󰀄WKH󰀄7UDSH]RLG󰀄5XOH

2QH󰀏DSSURDFK󰀏WR󰀏SDUDOOHOL]LQJ󰀏WKLV󰀏SURJUDP󰀏LV󰀏WR󰀏VLPSO\\󰀏VSOLW󰀏WKH󰀏LQWHUYDO󰀏>D󰀢E@󰀏XS󰀏DPRQJ󰀏WKH

SURFHVVHV󰀢󰀏DQG󰀏HDFK󰀏SURFHVV󰀏FDQ󰀏HVWLPDWH󰀏WKH󰀏LQWHJUDO󰀏RI󰀏I󰁂[󰁃󰀏RYHU󰀏LWV󰀏VXELQWHUYDO󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏HVWLPDWHWKH󰀏WRWDO󰀏LQWHJUDO󰀢󰀏WKH󰀏SURFHVVHV·󰀏ORFDO󰀏FDOFXODWLRQV󰀏DUH󰀏DGGHG󰀦

6XSSRVH󰀏WKHUH󰀏DUH󰀏S󰀏SURFHVVHV󰀏DQG󰀏Q󰀏WUDSH]RLGV󰀢󰀏DQG󰀢󰀏LQ󰀏RUGHU󰀏WR󰀏VLPSOLI\\󰀏WKH󰀏GLVFXVVLRQ󰀢󰀏DOVRVXSSRVH󰀏WKDW󰀏Q󰀏LV󰀏HYHQO\\󰀏GLYLVLEOH󰀏E\\󰀏S󰀦󰀏7KHQ󰀏LW󰀏LV󰀏QDWXUDO󰀏IRU󰀏WKH󰀏ILUVW󰀏SURFHVV󰀏WR󰀏FDOFXODWH󰀏WKH󰀏DUHD󰀏RIWKH󰀏ILUVW󰀏Q󰁋S󰀏WUDSH]RLGV󰀢󰀏WKH󰀏VHFRQG󰀏SURFHVV󰀏WR󰀏FDOFXODWH󰀏WKH󰀏DUHD󰀏RI󰀏WKH󰀏QH[W󰀏Q󰁋S󰀢󰀏HWF󰀦󰀏6R󰀏SURFHVV󰀏TZLOO󰀏HVWLPDWH󰀏WKH󰀏LQWHJUDO󰀏RYHU󰀏WKH󰀏LQWHUYDO

7KXV󰀏HDFK󰀏SURFHVV󰀏QHHGV󰀏WKH󰀏IROORZLQJ󰀏LQIRUPDWLRQ󰀦

• 7KH󰀏QXPEHU󰀏RI󰀏SURFHVVHV󰀢󰀏S󰀦• ,WV󰀏UDQN󰀦

• 7KH󰀏HQWLUH󰀏LQWHUYDO󰀏RI󰀏LQWHJUDWLRQ󰀢󰀏>D󰀢E@󰀦• 7KH󰀏QXPEHU󰀏RI󰀏VXELQWHUYDOV󰀢󰀏Q󰀦

5HFDOO󰀏WKDW󰀏WKH󰀏ILUVW󰀏WZR󰀏LWHPV󰀏FDQ󰀏EH󰀏IRXQG󰀏E\\󰀏FDOOLQJ󰀏WKH󰀏03,󰀏IXQFWLRQV󰀏MPI_Comm_size󰀏DQGMPI_Comm_Rank󰀦󰀏7KH󰀏ODWWHU󰀏WZR󰀏LWHPV󰀏VKRXOG󰀏SUREDEO\\󰀏EH󰀏LQSXW󰀏E\\󰀏WKH󰀏XVHU󰀦󰀏%XW󰀏WKLV󰀏FDQ󰀏UDLVHVRPH󰀏GLIILFXOW󰀏SUREOHPV󰀦󰀏6R󰀏IRU󰀏RXU󰀏ILUVW󰀏DWWHPSW󰀏DW󰀏FDOFXODWLQJ󰀏WKH󰀏LQWHJUDO󰀢󰀏OHW·V󰀏󰀳KDUGZLUH󰀳󰀏WKHVHYDOXHV󰀏E\\󰀏VLPSO\\󰀏VHWWLQJ󰀏WKHLU󰀏YDOXHV󰀏ZLWK󰀏DVVLJQPHQW󰀏VWDWHPHQWV󰀦

$󰀏VWUDLJKWIRUZDUG󰀏DSSURDFK󰀏WR󰀏VXPPLQJ󰀏WKH󰀏SURFHVVHV·󰀏LQGLYLGXDO󰀏FDOFXODWLRQV󰀏LV󰀏WR󰀏KDYH󰀏HDFKSURFHVV󰀏VHQG󰀏LWV󰀏ORFDO󰀏FDOFXODWLRQ󰀏WR󰀏SURFHVV󰀏󰀶󰀏DQG󰀏KDYH󰀏SURFHVV󰀏󰀶󰀏GR󰀏WKH󰀏ILQDO󰀏DGGLWLRQ󰀦:LWK󰀏WKHVH󰀏DVVXPSWLRQV󰀏ZH󰀏FDQ󰀏ZULWH󰀏D󰀏SDUDOOHO󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDP󰀦

c trap.f -- Parallel Trapezoidal Rule, first versionc

c Input: None.

c Output: Estimate of the integral from a to b of f(x)c using the trapezoidal rule and n trapezoids.c

c Algorithm:

c 1. Each process calculates \"its\" interval ofc integration.

c 2. Each process estimates the integral of f(x)c over its interval using the trapezoidal rule.c 3a. Each process != 0 sends its integral to 0.c 3b. Process 0 sums the calculations received fromc the individual processes and prints the result.c

c Note: f(x), a, b, and n are all hardwired.c

program trapezoidal

c

IMPLICIT NONEinclude 'mpif.h'c

integer my_rank ! My process rank.integer p ! The number of processes.real a ! Left endpoint.real b ! Right endpoint.

integer n ! Number of trapezoids.real h ! Trapezoid base length.

real local_a ! Left endpoint for my process.real local_b ! Right endpoint my process.integer local_n ! Number of trapezoids for my! calculation.

real integral ! Integral over my interval.real total ! Total integral.

integer source ! Process sending integal.integer dest ! All messages go to 0.integer tag

integer status(MPI_STATUS_SIZE)integer ierrreal Trap

data a, b, n, dest, tag /0.0, 1.0, 1024, 0, 50/C Let the system do what it needs to start up MPI.call MPI_INIT(ierr)C Get my process rank.

call MPI_COMM_RANK(MPI_COMM_WORLD, my_rank, ierr)C Find out how many processes are being used.call MPI_COMM_SIZE(MPI_COMM_WORLD, p, ierr)

h = (b-a)/n ! h is the same for all processes.local_n = n/p ! So is the number of trapezoids.

C Length of each process' interval of integration = local_n*h.C So my interval starts at :local_a = a + my_rank*local_n*hlocal_b = local_a + local_n*h

integral = Trap(local_a, local_b, local_n, h)C Add up the integals calculated by each process.if (my_rank .EQ. 0) thentotal = integraldo source = 1, p-1

call MPI_RECV(integral, 1, MPI_REAL, source, tag,+ MPI_COMM_WORLD, status, ierr)total = total + integralenddoelse

call MPI_SEND(integral, 1, MPI_REAL, dest,+ tag, MPI_COMM_WORLD, ierr)endif

C Print the result.if (my_rank .EQ. 0) thenwrite(6,200) n

200 format(' ','With n = ',I4,' trapezoids, our estimate')write(6,300) a, b, total

300 format(' ','of the integral from ',f6.2,' to ',f6.2,+ ' = ',f11.5)endif

C Shut down MPI.

call MPI_FINALIZE(ierr)end program trapezoidal

real function Trap(local_a, local_b, local_n, h)

IMPLICIT NONEreal local_areal local_binteger local_nreal h

real integral ! Store result in integal.real xreal ireal f

integral = (f(local_a) + f(local_b))/2.0x = local_a

do i = 1, local_n - 1x = x + h

integral = integral + f(x)

enddo

integal = integral*hTrap = integralreturnend

real function f(x)IMPLICIT NONEreal xf = x*xend󰀏

2EVHUYH󰀏WKDW󰀏WKLV󰀏SURJUDP󰀏DOVR󰀏XVHV󰀏WKH󰀏630'󰀏SDUDGLJP󰀦󰀏(YHQ󰀏WKRXJK󰀏SURFHVV󰀏󰀶󰀏H[HFXWHV󰀏DQHVVHQWLDOO\\󰀏GLIIHUHQW󰀏VHW󰀏RI󰀏FRPPDQGV󰀏IURP󰀏WKH󰀏UHPDLQLQJ󰀏SURFHVVHV󰀢󰀏LW󰀏VWLOO󰀏UXQV󰀏WKH󰀏VDPH󰀏SURJUDP󰀦7KH󰀏GLIIHUHQW󰀏FRPPDQGV󰀏DUH󰀏H[HFXWHG󰀏E\\󰀏EUDQFKLQJ󰀏EDVHG󰀏RQ󰀏WKH󰀏SURFHVV󰀏UDQN󰀦

󰀔󰀕 ,󰀭2󰀄RQ󰀄3DUDOOHO󰀄3URFHVVRUV

2QH󰀏REYLRXV󰀏SUREOHP󰀏ZLWK󰀏RXU󰀏SURJUDP󰀏LV󰀏LWV󰀏ODFN󰀏RI󰀏JHQHUDOLW\\󰀏󰀺󰀏WKH󰀏GDWD󰀢󰀏D󰀢󰀏E󰀢󰀏DQG󰀏Q󰀢󰀏DUH󰀏KDUGZLUHG󰀦

7KH󰀏XVHU󰀏VKRXOG󰀏EH󰀏DEOH󰀏WR󰀏HQWHU󰀏WKHVH󰀏YDOXHV󰀏GXULQJ󰀏H[HFXWLRQ󰀦󰀏/HW·V󰀏ORRN󰀏PRUH󰀏FDUHIXOO\\󰀏DW󰀏WKHSUREOHP󰀏RI󰀏,󰁋2󰀏RQ󰀏SDUDOOHO󰀏PDFKLQHV󰀦

,Q󰀏RXU󰀏greetings󰀏DQG󰀏trapezoid󰀏SURJUDPV󰀏ZH󰀏DVVXPHG󰀏WKDW󰀏SURFHVV󰀏󰀶󰀏FRXOG󰀏ZULWH󰀏WR󰀏VWDQGDUGRXWSXW󰀏󰁂󰀏WKH󰀏WHUPLQDO󰀏VFUHHQ󰁃󰀦󰀏0RVW󰀏SDUDOOHO󰀏SURFHVVRUV󰀏SURYLGH󰀏WKLV󰀏PXFK󰀏,󰁋2󰀦󰀏,Q󰀏IDFW󰀢󰀏PRVW󰀏SDUDOOHOSURFHVVRUV󰀏DOORZ󰀏DOO󰀏SURFHVVRUV󰀏WR󰀏ERWK󰀏UHDG󰀏IURP󰀏VWDQGDUG󰀏LQSXW󰀏DQG󰀏ZULWH󰀏WR󰀏VWDQGDUG󰀏RXWSXW󰀦+RZHYHU󰀏GLIILFXOW󰀏DULVH󰀏ZKHQ󰀏VHYHUDO󰀏SURFHVVHV󰀏DUH󰀏VLPXOWDQHRXVO\\󰀏WU\\LQJ󰀏WR󰀏H[HFXWH󰀏,󰁋2󰀏IXQFWLRQV󰀦,Q󰀏RUGHU󰀏WR󰀏XQGHUVWDQG󰀏WKLV󰀢󰀏OHW·V󰀏ORRN󰀏DW󰀏DQ󰀏H[DPSOH󰀦

6XSSRVH󰀏ZH󰀏PRGLI\\󰀏WKH󰀏trapezoid󰀏SURJUDP󰀏VR󰀏WKDW󰀏HDFK󰀏SURFHVV󰀏DWWHPSWV󰀏WR󰀏UHDG󰀏WKH󰀏YDOXHV󰀏D󰀢󰀏E󰀢DQG󰀏Q󰀏E\\󰀏DGGLQJ󰀏WKH󰀏VWDWHPHQWread *, a , b, n

6XSSRVH󰀏DOVR󰀏WKDW󰀏ZH󰀏UXQ󰀏WKH󰀏SURJUDP󰀏ZLWK󰀏WZR󰀏SURFHVVHV󰀏DQG󰀏WKH󰀏XVHU󰀏W\\SHV󰀏LQ0 1 1024

:KDW󰀏KDSSHQ󰀏\"󰀏'R󰀏ERWK󰀏SURFHVVHV󰀏JHW󰀏WKH󰀏GDWD󰀏\"󰀏'RHV󰀏RQO\\󰀏RQH󰀏\"󰀏2U󰀢󰀏HYHQ󰀏ZRUVH󰀢󰀏GRHV󰀏󰁂VD\\󰁃󰀏SURFHVV󰀶󰀏JHW󰀏WKH󰀏󰀶󰀏DQG󰀏󰀩󰀢󰀏ZKLOH󰀏SURFHVV󰀏󰀩󰀏JHWV󰀏WKH󰀏󰀩󰀶󰀽󰀬󰀏\"󰀏,I󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏JHW󰀏WKH󰀏GDWD󰀢󰀏ZKDW󰀏KDSSHQVZKHQ󰀏ZH󰀏ZULWH󰀏D󰀏SURJUDP󰀢󰀏ZKHUH󰀏ZH󰀏ZDQW󰀏SURFHVV󰀏󰀶󰀏JHWV󰀏WKH󰀏GDWD󰀢󰀏ZKDW󰀏KDSSHQV󰀏WR󰀏WKH󰀏RWKHUV󰀏\"󰀏,V󰀏LWHYHQ󰀏UHDVRQDEOH󰀏WR󰀏KDYH󰀏PXOWLSOH󰀏SURFHVVHV󰀏UHDGLQJ󰀏GDWD󰀏IURP󰀏D󰀏VLQJOH󰀏WHUPLQDO󰀏\"

2Q󰀏WKH󰀏RWKHU󰀏KDQG󰀢󰀏ZKDW󰀏KDSSHQV󰀏LI󰀏VHYHUDO󰀏SURFHVVHV󰀏DWWHPSW󰀏WR󰀏VLPXOWDQHRXVO\\󰀏ZULWH󰀏GDWD󰀏WR󰀏WKHWHUPLQDO󰀏VFUHHQ󰀦󰀏'RHV󰀏WKH󰀏GDWD󰀏IURP󰀏SURFHVV󰀏󰀶󰀏JHW󰀏SULQWHG󰀏ILUVW󰀢󰀏WKHQ󰀏WKH󰀏GDWD󰀏IRUP󰀏SURFHVV󰀏󰀩󰀢󰀏HWF󰀏\"2U󰀏GRHV󰀏WKH󰀏GDWD󰀏DSSHDU󰀏LQ󰀏VRPH󰀏UDQGRP󰀏RUGHU󰀏\"󰀏2U󰀢󰀏HYHQ󰀏ZRUVH󰀢󰀏GRHV󰀏WKH󰀏GDWD󰀏IURP󰀏WKH󰀏GLIIHUHQWSURFHVVHV󰀏JHW󰀏DOO󰀏PL[HG󰀏XS󰀏󰀜󰀏VD\\󰀢󰀏KDOI󰀏D󰀏OLQH󰀏IURP󰀏󰀶󰀢󰀏WZR󰀏FKDUDFWHUV󰀏IURP󰀏󰀩󰀢󰀏󰀫󰀏FKDUDFWHUV󰀏IURP󰀏󰀶󰀢󰀏WZROLQHV󰀏IURP󰀏󰀽󰀢󰀏HWF󰀏\"

7KH󰀏VWDQGDUG󰀏,󰁋2󰀏FRPPDQGV󰀏DYDLODEOH󰀏LQ󰀏)RUWUDQ󰀏󰁂DQG󰀏PRVW󰀏RWKHU󰀏ODQJXDJHV󰁃󰀏GRQ·W󰀏SURYLGH󰀏VLPSOHVROXWLRQV󰀏WR󰀏WKHVH󰀏SUREOHPV󰀢󰀏DQG󰀏,󰁋2󰀏FRQWLQXHV󰀏WR󰀏EH󰀏WKH󰀏VXEMHFW󰀏RI󰀏FRQVLGHUDEOH󰀏UHVHDUFK󰀏LQ󰀏WKHSDUDOOHO󰀏SURFHVVLQJ󰀏FRPPXQLW\\󰀦󰀏6R󰀏OHW·V󰀏ORRN󰀏DW󰀏VRPH󰀏QRW󰀏VR󰀏VLPSOH󰀏VROXWLRQV󰀏WR󰀏WKHVH󰀏SUREOHPV󰀦7KXV󰀏IDU󰀢󰀏ZH󰀏KDYH󰀏DVVXPHG󰀏WKDW󰀏SURFHVV󰀏󰀶󰀏FDQ󰀏DW󰀏OHDVW󰀏ZULWH󰀏WR󰀏VWDQGDUG󰀏RXWSXW󰀦󰀏:H󰀏ZLOO󰀏DOVR󰀏DVVXPHWKDW󰀏LW󰀏FDQ󰀏UHDG󰀏IURP󰀏VWDQGDUG󰀏LQSXW󰀦󰀏,Q󰀏PRVW󰀏FDVHV󰀢󰀏ZH󰀏ZLOO󰀏RQO\\󰀏DVVXPH󰀏WKDW󰀏SURFHVV󰀏󰀶󰀏FDQ󰀏GR󰀏,󰁋2󰀦,W󰀏VKRXOG󰀏EH󰀏QRWHG󰀏WKDW󰀏WKLV󰀏LV󰀏D󰀏YHU\\󰀏ZHDN󰀏DVVXPSWLRQ󰀢󰀏VLQFH󰀢󰀏DV󰀏ZH󰀏QRWHG󰀏PRVW󰀏SDUDOOHO󰀏PDFKLQHVDOORZ󰀏PXOWLSOH󰀏SURFHVVHV󰀏WR󰀏FDUU\\󰀏RXW󰀏,󰁋2󰀦󰀏,I󰀏RQO\\󰀏SURFHVV󰀏󰀶󰀏FDQ󰀏GR󰀏,󰁋2󰀢󰀏WKHQ󰀏ZH󰀏QHHG󰀏IRU󰀏SURFHVV󰀏󰀶󰀏WR󰀏VHQG󰀏WKH󰀏XVHU󰀏LQSXW󰀏WR󰀏WKH󰀏RWKHUSURFHVVHV󰀦󰀏7KLV󰀏LV󰀏UHDGLO\\󰀏DFFRPSOLVKHG󰀏ZLWK󰀏D󰀏VKRUW󰀏,󰁋2󰀏IXQFWLRQ󰀏WKDW󰀏XVHV󰀏MPI_Send DQGMPI_Recv󰀦

C

*****************************************************************

C Function Get_data

C Reads in the user input a, b, and n.C Input arguments:

C 1. integer my_rank: rank of current process.C 2. integer p: number of processes.C Output parameters:

C 1. real a: left endpoint a.C 2. real b: right endpoint b.

C 3. integer n: number of trapezoids.C Algorithm:

C 1. Process 0 prompts user for input andC reads in the values.

C 2. Process 0 sends input values to otherC processes.C

subroutine Get_data(a, b, n, my_rank, p)IMPLICIT NONEreal areal binteger ninteger my_rankinteger p

INCLUDE 'mpif.h'C

integer sourceinteger destinteger tag

integer status(MPI_STATUS_SIZE)integer ierrdata source /0/C

if (my_rank == 0) thenprint *, 'Enter a, b and n'

read *, a, b, nCC

do dest = 1 , p-1

tag = 0

call MPI_SEND(a, 1, MPI_REAL , dest, tag,+ MPI_COMM_WORLD, ierr )tag = 1

call MPI_SEND(b, 1, MPI_REAL , dest, tag,+ MPI_COMM_WORLD, ierr )

tag = 2

call MPI_SEND(n, 1, MPI_INTEGER, dest,+ tag, MPI_COMM_WORLD, ierr )enddoelsetag = 0

call MPI_RECV(a, 1, MPI_REAL , source, tag,+ MPI_COMM_WORLD, status, ierr )tag = 1

call MPI_RECV(b, 1, MPI_REAL , source, tag,+ MPI_COMM_WORLD, status, ierr )tag = 2

call MPI_RECV(n, 1, MPI_INTEGER, source, tag,+ MPI_COMM_WORLD, status, ierr )endifreturnendCC

C

****************************************************************

**󰀏

󰀔󰀕 &ROOHFWLYH󰀄&RPPXQLFDWLRQ

7KHUH󰀏DUH󰀏SUREDEO\\󰀏D󰀏IHZ󰀏WKLQJV󰀏LQ󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDP󰀏WKDW󰀏ZH󰀏FDQ󰀏LPSURYH󰀏RQ󰀦󰀏)RUH[DPSOH󰀢󰀏WKHUH󰀏LV󰀏WKH󰀏,󰁋2󰀏LVVXH󰀦󰀏7KHUH󰀏DUH󰀏DOVR󰀏D󰀏FRXSOH󰀏RI󰀏SUREOHPV󰀏ZH󰀏KDYHQ·W󰀏GLVFXVVHG\\HW󰀦󰀏/HW·V󰀏ORRN󰀏DW󰀏ZKDW󰀏KDSSHQV󰀏ZKHQ󰀏WKH󰀏SURJUDP󰀏LV󰀏UXQ󰀏ZLWK󰀏HLJKW󰀏SURFHVVHV󰀦

$OO󰀏WKH󰀏SURFHVVHV󰀏EHJLQ󰀏H[HFXWLQJ󰀏WKH󰀏SURJUDP󰀏󰁂󰀏PRUH󰀏RU󰀏OHVV󰀏󰁃󰀏VLPXODWDQHRXVO\\󰀦󰀏+RZHYHU󰀢DIWHU󰀏FDUU\\LQJ󰀏RXW󰀏WKH󰀏EDVLF󰀏VHW󰀜XS󰀏WDVNV󰀏󰁂󰀏FDOOV󰀏WR󰀏03,B,QLW󰀢󰀏03,B&RPPBVL]H󰀢󰀏DQG03,B&RPPBUDQN󰁃󰀢󰀏SURFHVVHV󰀏󰀩󰀜󰀾󰀏DUH󰀏LGOH󰀏ZKLOH󰀏SURFHVV󰀏󰀶󰀏FROOHFWV󰀏WKH󰀏LQSXW󰀏GDWD󰀦󰀏:H󰀏GRQ·WZDQW󰀏WR󰀏KDYH󰀏LGOH󰀏SURFHVVHV󰀢󰀏EXW󰀏LQ󰀏YLHZ󰀏RI󰀏RXU󰀏UHVWULFWLRQV󰀏RQ󰀏ZKLFK󰀏SURFHVVHV󰀏FDQ󰀏UHDGLQSXW󰀏GDWD󰀢󰀏WKH󰀏KLJKHU󰀏UDQN󰀏SURFHVVHV󰀏PXVW󰀏FRQWLQXH󰀏WR󰀏ZDLW󰀏ZKLOH󰀏󰀶󰀏VHQGV󰀏WKH󰀏QSXW󰀏GDWD󰀏WRWKH󰀏ORZHU󰀏UDQN󰀏SURFHVVHV󰀦󰀏7KLV󰀏LVQ·W󰀏MXVW󰀏DQ󰀏,󰁋2󰀏LVVXH󰀦󰀏1RWLFH󰀏WKDW󰀏WKHUH󰀏LV󰀏D󰀏VLPLODULQHIILFLHQF\\󰀏DW󰀏WKH󰀏HQG󰀏RI󰀏WKH󰀏SURJUDP󰀢󰀏ZKHQ󰀏SURFHVV󰀏󰀶󰀏GRHV󰀏DOO󰀏WKH󰀏ZRUN󰀏RI󰀏FROOHFWLQJ󰀏DQGGDGGLQJ󰀏WKH󰀏ORFDO󰀏LQWHJUDOV󰀦

2I󰀏FRXUVH󰀢󰀏WKLV󰀏LV󰀏KLJKO\\󰀏XQGHVLUDEOH󰀏󰀺󰀏WKH󰀏PDLQ󰀏SRLQW󰀏RI󰀏SDUDOOHO󰀏SURFHVVLQJ󰀏LV󰀏WR󰀏JHW󰀏PXOWLSOHSURFHVVHV󰀏WR󰀏FROODERUDWH󰀏RQ󰀏VROYLQJ󰀏D󰀏SUREOHP󰀦󰀏,I󰀏RQH󰀏RI󰀏WKH󰀏SURFHVVHV󰀏LV󰀏GRLQJ󰀏PRVW󰀏RI󰀏WKHZRUN󰀢󰀏ZH󰀏PLJKW󰀏DV󰀏ZHOO󰀏XVH󰀏D󰀏FRQYHQWLRQDO󰀢󰀏VLQJOH󰀜SURFHVVRU󰀏PDFKLQH󰀦

󰀔󰀕 7UHH󰀯6WUXFWXUHG󰀄&RPPXQLFDWLRQ

/HW·V󰀏WU\\󰀏WR󰀏LPSURYH󰀏RXU󰀏FRGH󰀦󰀏:H·OO󰀏EHJLQ󰀏E\\󰀏IRFXVLQJ󰀏RQ󰀏WKH󰀏GLVWULEXWLRQ󰀏RI󰀏WKH󰀏LQSXW󰀏GDWD󰀦󰀏+RZ

FDQ󰀏ZH󰀏GLYLGH󰀏WKH󰀏ZRUN󰀏PRUH󰀏HYHQO\\󰀏DPRQJ󰀏WKH󰀏SURFHVVHV󰀏\"󰀏$󰀏QDWXUDO󰀏VROXWLRQ󰀏LV󰀏WR󰀏LPDJLQH󰀏WKDW󰀏ZHKDYH󰀏D󰀏WUHH󰀏RI󰀏SURFHVVHV󰀢󰀏ZLWK󰀏󰀶󰀏DW󰀏WKH󰀏URRW󰀦

'XULQJ󰀏WKH󰀏ILUVW󰀏VWDJH󰀏RI󰀏GDWD󰀏GLVWULEXWLRQ󰀢󰀏󰀶󰀏VHQGV󰀏WKH󰀏GDWD󰀏WR󰀏󰁂VD\\󰁃󰀏󰀬󰀦󰀏'XULQJ󰀏WKH󰀏QH[W󰀏VWDJH󰀢󰀏󰀶󰀏VHQGVWKH󰀏GDWD󰀏WR󰀏󰀽󰀢󰀏ZKLOH󰀏󰀬󰀏VHQGV󰀏LW󰀏WR󰀏󰀿󰀦󰀏'XULQJ󰀏WKH󰀏ODVW󰀏VWDJH󰀢󰀏󰀶󰀏VHQGV󰀏WR󰀏󰀩󰀢󰀏ZKLOH󰀏󰀽󰀏VHQGV󰀏WR󰀏󰀫󰀢󰀏󰀬󰀏VHQGV󰀏WR󰀭󰀢󰀏DQG󰀏󰀿󰀏VHQGV󰀏WR󰀏󰀾󰀦󰀏󰁂VHH󰀏ILJXUH󰀏󰀬󰀦󰀩󰁃󰀏6R󰀏ZH󰀏KDYH󰀏UHGXFHG󰀏RXU󰀏LQSXW󰀏GLVWULEXWLRQ󰀏ORRS󰀏IURP󰀏󰀾󰀏VWDJHV󰀏WR󰀫󰀏VWDJHV󰀦󰀏0RUH󰀏JHQHUDOO\\󰀢󰀏LI󰀏ZH󰀏KDYH󰀏S󰀏SURFHVVHV󰀢󰀏WKLV󰀏SURFHGXUH󰀏DOORZV󰀏XV󰀏WR󰀏GLVWULEXWH󰀏WKH󰀏LQSXW󰀏GDWDLQ󰀏

VWDJHV󰀢󰀏UDWKHU󰀏WKDQ󰀏S󰀜󰀩󰀏VWDJHV󰀢󰀏ZKLFK󰀢󰀏LI󰀏S󰀏LV󰀏ODUJH󰀢󰀏LV󰀏D󰀏KXJH󰀏VDYLQJV󰀦

)LJXUH󰀇3URFHVVRUVFRQILJXUHGDVDWUHH

,Q󰀏RUGHU󰀏WR󰀏PRGLI\\󰀏WKH󰀏*HWBGDWD󰀏IXQFWLRQ󰀏WR󰀏XVH󰀏D󰀏WUHH󰀜VWUXFWXUHG󰀏GLVWULEXWLRQ󰀏VFKHPH󰀢󰀏ZH󰀏QHHG󰀏WRLQWURGXFH󰀏D󰀏ORRS󰀏ZLWK󰀏FDOFXODWH󰀏DW󰀏HDFK󰀏VWDJH󰀦

VWDJHV󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏LPSOHPHQW󰀏WKH󰀏ORRS󰀢󰀏HDFK󰀏SURFHVV󰀏QHHGV󰀏WR

• ZKHWKHU󰀏LW󰀏UHFHLYHV󰀢󰀏DQG󰀢󰀏LI󰀏VR󰀢󰀏WKH󰀏VRXUFH󰀏󰀼󰀏DQG• ZKHWKHU󰀏LW󰀏VHQGV󰀢󰀏DQG󰀢󰀏LI󰀏VR󰀢󰀏WKH󰀏GHVWLQDWLRQ󰀦

$V󰀏\\RX󰀏FDQ󰀏SUREDEO\\󰀏JXHVV󰀢󰀏WKHVH󰀏FDOFXODWLRQV󰀏FDQ󰀏EH󰀏D󰀏ELW󰀏FRPSOLFDWHG󰀢󰀏HVSHFLDOO\\󰀏VLQFH󰀏WKHUH󰀏LV󰀏QRFDQRQLFDO󰀏FKRLFH󰀏RI󰀏RUGHULQJ󰀦󰀏,Q󰀏RXU󰀏H[DPSOH󰀢󰀏ZH󰀏FKRVH󰀏󰀺󰀶VHQGV󰀏WR󰀏󰀬󰀦

󰀶󰀏VHQGV󰀏WR󰀏󰀽󰀢󰀏󰀬󰀏VHQGV󰀏WR󰀏󰀿󰀦

󰀶󰀏VHQGV󰀏WR󰀏󰀩󰀢󰀏󰀽󰀏VHQGV󰀏WR󰀏󰀫󰀢󰀏󰀬󰀏VHQGV󰀏WR󰀏󰀭󰀢󰀏󰀿󰀏VHQGV󰀏WR󰀏󰀾󰀦:H󰀏PLJKW󰀏DOVR󰀏KDYH󰀏FKRVHQ󰀏󰁂󰀏IRU󰀏H[DPSOH󰀏󰁃󰀏󰀺󰀶󰀏VHQGV󰀏WR󰀏󰀩󰀦

󰀶󰀏VHQGV󰀏WR󰀏󰀽󰀢󰀏󰀩󰀏VHQGV󰀏WR󰀏󰀫󰀦

󰀶󰀏VHQGV󰀏WR󰀏󰀬󰀢󰀏󰀩󰀏VHQGV󰀏WR󰀏󰀭󰀢󰀏󰀽󰀏VHQGV󰀏WR󰀏󰀿󰀢󰀏󰀫󰀏VHQGV󰀏WR󰀏󰀾󰀦

,QGHHG󰀢󰀏XQOHVV󰀏ZH󰀏NQRZ󰀏VRPHWKLQJ󰀏DERXW󰀏WKH󰀏XQGHUO\\LQJ󰀏WRSRORJ\\󰀏RI󰀏RXU󰀏PDFKLQH󰀢󰀏ZH󰀏FDQ·W󰀏UHDOO\\GHFLGH󰀏ZKLFK󰀏VFKHPH󰀏LV󰀏EHWWHU󰀦

6R󰀏LGHDOO\\󰀏ZH󰀏ZRXOG󰀏SUHIHU󰀏WR󰀏XVH󰀏D󰀏IXQFWLRQ󰀏WKDW󰀏KDV󰀏EHHQ󰀏VSHFLILFDOO\\󰀏WDLORUHG󰀏WR󰀏WKH󰀏PDFKLQH󰀏ZH·UHXVLQJ󰀏VR󰀏WKDW󰀏ZH󰀏ZRQ·W󰀏KDYH󰀏WR󰀏ZRUU\\󰀏DERXW󰀏DOO󰀏WKHVH󰀏WHGLRXV󰀏GHWDLOV󰀢󰀏DQG󰀏ZH󰀏ZRQ·W󰀏KDYH󰀏WR󰀏PRGLI\\RXU󰀏FRGH󰀏HYHU\\󰀏WLPH󰀏ZH󰀏FKDQJH󰀏PDFKLQHV󰀦󰀏$V󰀏\\RX󰀏PD\\󰀏KDYH󰀏JXHVV󰀢󰀏03,󰀏SURYLGHV󰀏VXFK󰀏D󰀏IXQFWLRQ󰀦

󰀔󰀕 %URDGFDVW

$󰀏FRPPXQLFDWLRQ󰀏SDWWHUQ󰀏WKDW󰀏LQYROYHV󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏D󰀏FRPPXQLFDWRU󰀏LV󰀏D

FROOHFWLYH󰀂FRPPXQLFDWLRQ󰀦󰀏$V󰀏D󰀏FRQVHTXHQFH󰀢󰀏D󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏XVXDOO\\󰀏LQYROYHVPRUH󰀏WKDQ󰀏WZR󰀏SURFHVVHV󰀦󰀏$󰀏EURDGFDVW󰀏LV󰀏D󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏LQ󰀏ZKLFK󰀏D󰀏VLQJOHSURFHVV󰀏VHQGV󰀏WKH󰀏VDPH󰀏GDWD󰀏WRS󰀏HYHU\\󰀏SURFHVV󰀦󰀏,Q󰀏03,󰀏WKH󰀏IXQFWLRQ󰀏IRU󰀏EURDGFDVWLQJGDWD󰀏LV󰀏03,B%FDVW󰀏󰀺

MPI_BCAST( BUFFER, COUNT, DATATYPE, ROOT, COMM, IERROR)

BUFFER(*)

INTEGER COUNT, DATA, ROOT, COMM, IERROR

,W󰀏VLPSO\\󰀏VHQGV󰀏D󰀏FRS\\󰀏RI󰀏WKH󰀏GDWD󰀏LQ󰀏%8))(5󰀏RQ󰀏SURFHVV󰀏5227󰀏WR󰀏HDFK󰀏SURFHVV󰀏LQWKH󰀏FRPPXQLFDWRU󰀏&200󰀦󰀏,7󰀏VKRXOG󰀏EH󰀏FDOOHG󰀏E\\󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏WKHFRPPXQLFDWRU󰀏ZLWK󰀏WKH󰀏VDPH󰀏DUJXPHQWV󰀏IRU󰀏5227󰀏DQG󰀏&200󰀦󰀏+HQFH󰀏D󰀏EURDGFDVWPHVVDJH󰀏FDQQRW󰀏EH󰀏UHFHLYHG󰀏ZLWK󰀏03,B5HFY󰀦󰀏7KH󰀏SDUDPHWHUV󰀏&2817󰀏DQG'$7$7<3(󰀏KDYH󰀏WKH󰀏VDPH󰀏IXQFWLRQ󰀏WKDW󰀏WKH\\󰀏KDYH󰀏LQ󰀏03,B6HQG󰀏DQG󰀏03B5HFY󰀏󰀺WKH\\󰀏VSHFLI\\󰀏WKH󰀏H[WHQW󰀏RI󰀏WKH󰀏PHVVDJH󰀦󰀏+RZHYHU󰀢󰀏XQOLNH󰀏WKH󰀏SRLQW󰀜WR󰀜SRLQW󰀏IXQFWLRQV󰀢03,󰀏LQVLVWV󰀏WKDW󰀏LQ󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏&2817󰀏DQG󰀏'$7$7<3(󰀏EH󰀏WKHVDPH󰀏RQ󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏FRPPXQLFDWRU󰀏>󰀬@󰀦󰀏7KH󰀏UHDVRQ󰀏IRU󰀏WKLV󰀏LV󰀏WKDW󰀏LQVRPH󰀏FROOHFWLYH󰀏RSHUDWLRQV󰀏󰁂VHH󰀏EHORZ󰁃󰀢󰀏D󰀏VLQJOH󰀏SURFHVV󰀏ZLOO󰀏UHFHLYH󰀏GDWD󰀏IURP󰀏PDQ\\RWKHU󰀏SURFHVVHV󰀢󰀏DQG󰀏LQ󰀏RUGHU󰀏IRU󰀏D󰀏SURJUDP󰀏WR󰀏GHWHUPLQH󰀏KRZ󰀏PXFK󰀏GDWD󰀏KDV󰀏EHHQUHFHLYHG󰀢󰀏LW󰀏ZRXOG󰀏QHHG󰀏DQ󰀏HQWLUH󰀏DUUD\\󰀏RI󰀏UHWXUQ󰀏VWDWXVHV󰀦:H󰀏FDQ󰀏UHZULWH󰀏WKH󰀏*HWBGDWD󰀏IXQFWLRQ󰀏XVLQJ󰀏03,B%FDVW󰀏DV󰀏IROORZV󰀦subroutine Get_data2(a, b, n, my_rank)real areal binteger ninteger my_rankinteger ierrinclude 'mpif.h'CC

if (my_rank .EQ. 0) thenprint *, 'Enter a, b, and n'

read *, a, b, n

endifC

call MPI_BCAST(a, 1, MPI_REAL , 0, MPI_COMM_WORLD,ierr )

call MPI_BCAST(b, 1, MPI_REAL , 0, MPI_COMM_WORLD,ierr )

call MPI_BCAST(n, 1, MPI_INTEGER, 0, MPI_COMM_WORLD,ierr )

end subroutine Get_data2CC

C

******************************************************************

&HUWDLQO\\󰀏WKLV󰀏YHUVLRQ󰀏RI󰀏*HWBGDWD󰀏LV󰀏PXFK󰀏PRUH󰀏FRPSDFW󰀏DQG󰀏UHDGLO\\FRPSUHKHQVLEOH󰀏WKDQ󰀏WKH󰀏RULJLQDO󰀢󰀏DQG󰀏LI󰀏03,B%FDVW󰀏KDV󰀏EHHQ󰀏RSWLPL]HG󰀏IRU󰀏\\RXUV\\VWHP󰀢󰀏LW󰀏ZLOO󰀏DOVR󰀏EH󰀏D󰀏JRRG󰀏GHDO󰀏IDVWHU󰀦

󰀙󰀕 5HGXFH

,Q󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDP󰀏DIWHU󰀏WKH󰀏LQSXW󰀏SKDVH󰀢󰀏HYHU\\󰀏SURFHVVRU󰀏H[HFXWHV󰀏HVVHQWLDOO\\󰀏WKH󰀏VDPH

FRPPDQGV󰀏XQWLO󰀏WKH󰀏ILQDO󰀏VXPPDWLRQ󰀏SKDVH󰀦󰀏6R󰀏XQOHVV󰀏RXU󰀏IXQFWLRQ󰀏I󰁂[󰁃󰀏LV󰀏IDLUO\\󰀏FRPSOLFDWHG󰀏󰁂󰀏L󰀦H󰀦󰀢󰀏LWUHTXLUHV󰀏FRQVLGHUDEO\\󰀏PRUH󰀏ZRUN󰀏WR󰀏HYDOXDWH󰀏RYHU󰀏FHUWDLQ󰀏SDUWV󰀏RI󰀏>D󰀢E@󰁃󰀢󰀏WKLV󰀏SDUW󰀏RI󰀏WKH󰀏SURJUDPGLVWULEXWHV󰀏WKH󰀏ZRUN󰀏HTXDOO\\󰀏DPRQJ󰀏WKH󰀏SURFHVVRUV󰀦󰀏$V󰀏ZH󰀏KDYH󰀏DOUHDG\\󰀏QRWHG󰀢󰀏WKLV󰀏LV󰀏QRW󰀏WKH󰀏FDVHZLWK󰀏WKH󰀏ILQDO󰀏VXPPDWLRQ󰀏SKDVH󰀢󰀏ZKHQ󰀢󰀏RQFH󰀏DJDLQ󰀢󰀏SURFHVV󰀏󰀶󰀏JHWV󰀏D󰀏GLVSURSRUWLRQDWH󰀏DPRXQW󰀏RI󰀏WKHZRUN󰀦󰀏+RZHYHU󰀢󰀏\\RX󰀏KDYH󰀏SUREDEO\\󰀏DOUHDG\\󰀏QRWLFHG󰀏WKDW󰀏E\\󰀏UHYHUVLQJ󰀏WKH󰀏DUURZV󰀏LQ󰀏ILJXUH󰀏󰀬󰀦󰀩󰀢󰀏ZH󰀏FDQXVH󰀏WKH󰀏VDPH󰀏LGHD󰀏ZH󰀏XVHG󰀏LQ󰀏VHFWLRQ󰀏󰀬󰀦󰀩󰀦󰀏7KDW󰀏LV󰀢󰀏ZH󰀏FDQ󰀏GLVWULEXWH󰀏WKH󰀏ZRUN󰀏RI󰀏FDOFXODWLQJ󰀏WKH󰀏VXPDPRQJ󰀏WKH󰀏SURFHVVRUV󰀏DV󰀏IROORZV󰀦󰀩󰀦

󰁂D󰁃󰀏󰀩󰀏VHQGV󰀏WR󰀏󰀶󰀢󰀏󰀫󰀏VHQGV󰀏WR󰀏󰀽󰀢󰀏󰀭󰀏VHQGV󰀏WR󰀏󰀬󰀢󰀏󰀾󰀏VHQGV󰀏WR󰀏󰀿󰀦

󰁂E󰁃󰀏󰀶󰀏DGGV󰀏LWV󰀏LQWHJUDO󰀏WR󰀏WKDW󰀏RI󰀏󰀩󰀢󰀏󰀽󰀏DGGV󰀏LWV󰀏LQWHJUDO󰀏WR󰀏WKDW󰀏RI󰀏󰀫󰀢󰀏HWF󰀦󰀽󰀦󰀏

󰁂D󰁃󰀏󰀽󰀏VHQGV󰀏WR󰀏󰀶󰀢󰀏󰀿󰀏VHQGV󰀏WR󰀏󰀬󰀦

󰁂E󰁃󰀏󰀶󰀏DGGV󰀢󰀏󰀬󰀏DGGV󰀦󰀫󰀦

󰁂D󰁃󰀏󰀬󰀏VHQGV󰀏WR󰀏󰀶󰀦D󰀦 󰀶󰀏DGGV󰀦

2I󰀏FRXUVH󰀢󰀏ZH󰀏UXQ󰀏LQWR󰀏WKH󰀏VDPH󰀏TXHVWLRQ󰀏WKDW󰀏RFFXUUHG󰀏ZKHQ󰀏ZH󰀏ZHUH󰀏ZULWLQJ󰀏RXU󰀏RZQ󰀏EURDGFDVW󰀏󰀺LV󰀏WKLV󰀏WUHH󰀏VWUXFWXUH󰀏PDNLQJ󰀏RSWLPDO󰀏XVH󰀏RI󰀏WKH󰀏WRSRORJ\\󰀏RI󰀏RXU󰀏PDFKLQH󰀏\"󰀏2QFH󰀏DJDLQ󰀢󰀏ZH󰀏KDYH󰀏WRDQVZHU󰀏WKDW󰀏WKLV󰀏GHSHQGV󰀏RQ󰀏WKH󰀏PDFKLQH󰀦󰀏6R󰀢󰀏DV󰀏EHIRUH󰀢󰀏ZH󰀏VKRXOG󰀏OHW󰀏03,󰀏GR󰀏WKH󰀏ZRUN󰀢󰀏E\\󰀏XVLQJ󰀏DQRSWLPL]HG󰀏IXQFWLRQ󰀦

7KH󰀏󰀳JOREDO󰀏VXP󰀳󰀏WKDW󰀏ZH󰀏ZLVK󰀏WR󰀏FDOFXODWH󰀏LV󰀏DQ󰀏H[DPSOH󰀏RI󰀏D󰀏JHQHUDO󰀏FODVV󰀏RI󰀏FROOHFWLYHFRPPXQLFDWLRQ󰀏RSHUDWLRQV󰀏FDOOHG󰀏UHGXFWLRQ󰀂RSHUDWLRQV󰀦󰀏,Q󰀏D󰀏JOREDO󰀏UHGXFWLRQ󰀏RSHUDWLRQ󰀢󰀏DOO󰀏WKH󰀏SURFHVVHV󰁂󰀏LQ󰀏D󰀏FRPPXQLFDWRU󰀏󰁃󰀏FRQWULEXWH󰀏GDWD󰀏ZKLFK󰀏LV󰀏FRPELQHG󰀏XVLQJ󰀏D󰀏ELQDU\\󰀏RSHUDWLRQ󰀦󰀏7\\SLFDO󰀏ELQDU\\RSHUDWLRQV󰀏DUH󰀏DGGLWLRQ󰀢󰀏PD[󰀢󰀏PLQ󰀢󰀏ORJLFDO󰀏DQG󰀢󰀏HWF󰀦󰀏7KH󰀏03,󰀏IXQFWLRQ󰀏IRU󰀏SHUIRUPLQJ󰀏D󰀏UHGXFWLRQRSHUDWLRQ󰀏LV

MPI_Reduce(OPERAND, RESULT, COUNT, DATATYEP, OP, ROOT, COMM,IERROR)

OPERAND(*), RESULT(*)

INTEGER COUNT, DATATYPE, OP, ROOT, COMM, IERROR

03,B5HGXFH󰀏FRPELQHV󰀏WKH󰀏RSHUDQGV󰀏VWRUHG󰀏LQ󰀏23(5$1'󰀏XVLQJ󰀏RSHUDWLRQ󰀏23󰀏DQG󰀏VWRUHV󰀏WKHUHVXOW󰀏LQ󰀏5(68/7󰀏RQ󰀏SURFHVV󰀏5227󰀦󰀏%RWK󰀏23(5$1'󰀏DQG󰀏5(68/7󰀏UHIHU󰀏WR󰀏&2817󰀏PHPRU\\ORFDWLRQV󰀏ZLWK󰀏W\\SH󰀏'$7$7<3(󰀦󰀏03,B5HGXFH󰀏PXVW󰀏EH󰀏FDOOHG󰀏E\\󰀏DOO󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏FRPPXQLFDWRU&200󰀢󰀏DQG󰀏&2817󰀢󰀏'$7$7<3(󰀢󰀏DQG󰀏23󰀏PXVW󰀏EH󰀏WKH󰀏VDPH󰀏RQ󰀏HDFK󰀏SURFHVV󰀦7KH󰀏DUJXPHQW󰀏23󰀏FDQ󰀏WDNH󰀏RQ󰀏RQH󰀏RI󰀏WKH󰀏IROORZLQJ󰀏SUHGHILQHG󰀏YDOXHV󰀦

2SHUDWLRQ󰀊1DPH03,B0$;03,B0,103,B68003,B352'03,B/$1'03,B%$1'03,B/2503,B%2503,B/;2503,B%;2503,B0$;/2&03,B0,1/2&

0HDQLQJ0D[LPXP0LQLPXP6XP3URGXFW/RJLFDO󰀏$QG%LWZLVH󰀏$QG/RJLFDO󰀏2U%LWZLVW󰀏2U

/RJLFDO󰀏([FOXVLYH󰀏2U%LWZLVH󰀏([FOXVLYH󰀏2U

0D[LPXP󰀏DQG󰀏/RFDWLRQ󰀏RI󰀏0D[LPXP0LQLPXP󰀏DQG󰀏/RFDWLRQ󰀏RI󰀏0LQLPXP

,W󰀏LV󰀏DOVR󰀏SRVVLEOH󰀏WR󰀏GHILQH󰀏DGGLWLRQDO󰀏RSHUDWLRQV󰀦󰀏)RU󰀏GHWDLOV󰀏VHH󰀏>󰀬@󰀦

$V󰀏DQ󰀏H[DPSOH󰀢󰀏OHW·V󰀏UHZULWH󰀏WKH󰀏ODVW󰀏IHZ󰀏OLQHV󰀏RI󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDP󰀦C Add up the integrals calculated by each process.MPI_Reduce( INTEGRAL, TOTAL, 1, MPI_REAL, MPI_SUM, 0,+ MPI_COMM_WORLD, ierr)C Print the result.

1RWH󰀏WKDW󰀏HDFK󰀏SURFHVVRU󰀏FDOOV󰀏03,B5HGXFH󰀏ZLWK󰀏WKH󰀏VDPH󰀏DUJXPHQWV󰀦󰀏,Q󰀏SDUWLFXODU󰀢󰀏HYHQ󰀏WKRXJKWRWDO󰀏RQO\\󰀏KDV󰀏VLJQLILFDQFH󰀏RQ󰀏SURFHVV󰀏󰀶󰀢󰀏HDFK󰀏SURFHVV󰀏PXVW󰀏VXSSO\\󰀏DQ󰀏DUJXPHQW󰀦

󰀔󰀕 2WKHU󰀄&ROOHFWLYH󰀄&RPPXQLFDWLRQ)XQFWLRQV03,󰀏VXSSOLHV󰀏PDQ\\󰀏RWKHU󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏IXQFWLRQV󰀦󰀏:H󰀏EULHIO\\󰀏HQXPHUDWH󰀏VRPH󰀏RI󰀏WKHVHKHUH󰀦󰀏)RU󰀏IXOO󰀏GHWDLOV󰀢󰀏VHH󰀏>󰀬@󰀦

• MPI_Barrier( COMM, IERROR)INTEGER COMM, IERROR

03,B%DUULHU󰀏SURYLGHV󰀏D󰀏PHFKDQLVP󰀏IRU󰀏V\\QFKURQL]LQJ󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏FRPPXQLFDWRU󰀏FRPP󰀦(DFK󰀏SURFHVV󰀏EORFNV󰀏󰁂L󰀦H󰀦󰀢󰀏SDXVHV󰁃󰀏XQWLO󰀏HYHU\\󰀏SURFHVV󰀏LQ󰀏FRPP󰀏KDV󰀏FDOOHG󰀏03,B%DUULHU󰀦MPI_Gather(SEND_BUF, SEND_COUNT, SEND_TYPE, RECV_BUF,RECV_COUNT, RECV_TYPE, ROOT, COMM, IERROR )SEND_BUF(*), RECV_BUF(*)

INTEGER SEND_COUNT,SEND_TYPE,RECV_COUNT,RECV_TYPE,ROOT,COMM,IERROR

(DFK󰀏SURFHVV󰀏LQ󰀏FRPP󰀏VHQGV󰀏WKH󰀏FRQWHQWV󰀏RI󰀏VHQGBEXI󰀏WR󰀏SURFHVV󰀏ZLWK󰀏UDQN󰀏URRW󰀦󰀏7KH󰀏USRFHVV󰀏URRWFRQFDWHQDWHV󰀏WKH󰀏UHFHLYHG󰀏GDWD󰀏LQ󰀏SURFHVV󰀏UDQN󰀏RUGHU󰀏LQ󰀏UHFYBEXI󰀦󰀏7KDW󰀏LV󰀢󰀏WKH󰀏GDWD󰀏IURP󰀏SURFHVV󰀏󰀶󰀏LVIROORZHG󰀏E\\󰀏WKH󰀏GDWD󰀏IURP󰀏SURFHVV󰀏󰀩󰀢󰀏ZKLFK󰀏LV󰀏IROORZHG󰀏E\\󰀏WKH󰀏GDWD󰀏IURP󰀏SURFHVV󰀏󰀽󰀢󰀏HWF󰀦󰀏7KH󰀏UHFYDUJXPHQWV󰀏DUH󰀏VLJQLILFDQW󰀏RQO\\󰀏RQ󰀏WKH󰀏SURFHVV󰀏ZLWK󰀏UDQN󰀏URRW󰀦󰀏7KH󰀏DUJXPHQW󰀏UHFYBFRXQW󰀏LQGLFDWHV󰀏WKHQXPEHU󰀏RI󰀏LWHPV󰀏UHFHLYHG󰀏IURP󰀏HDFK󰀏SURFHVV󰀏󰀜󰀏QRW󰀏WKH󰀏WRWDO󰀏QXPEHU󰀏UHFHLYHG󰀦MPI_Scatter(SEND_BUF, SEND_COUNT, SEND_TYPE, RECV_BUF,RECV_COUNT, RECV_TYPE, ROOT, COMM, IERROR )SEND_BUF(*), RECV_BUF(*)

INTEGER SEND_COUNT,SEND_TYPE,RECV_COUNT,RECV_TYPE,ROOT,COMM,IERROR

7KH󰀏SURFHVV󰀏ZLWK󰀏UDQN󰀏URRW󰀏GLVWULEXWHV󰀏WKH󰀏FRQWHQWV󰀏RI󰀏VHQGBEXI󰀏DPRQJ󰀏WKH󰀏SURFHVVHV󰀦󰀏7KH󰀏FRQWHQWVRI󰀏VHQGBEXI󰀏DUH󰀏VSOLW󰀏LQWR󰀏S󰀏VHJPHQWV󰀏HDFK󰀏FRQVLVWLQJ󰀏RI󰀏VHQGBFRXQW󰀏LWHPV󰀦󰀏7KH󰀏ILUVW󰀏VHJPHQW󰀏JRHV󰀏WRSURFHVV󰀏󰀶󰀢󰀏WKH󰀏VHFRQG󰀏WR󰀏SURFHVV󰀏󰀩󰀢󰀏HWF󰀦󰀏7KH󰀏VHQG󰀏DUJXPHQWV󰀏DUH󰀏VLJQLILFDQW󰀏RQO\\󰀏RQ󰀏SURFHVV󰀏URRW󰀦MPI_Allgather(SEND_BUF, SEND_COUNT, SEND_TYPE, RECV_BUF,RECV_COUNT, RECV_TYPE, ROOT, COMM, IERROR )SEND_BUF(*), RECV_BUF(*)

INTEGER SEND_COUNT,SEND_TYPE,RECV_COUNT,RECV_TYPE,ROOT,COMM,IERROR󰀏

03,B$OOJDWKHU󰀏JDWKHUV󰀏WKH󰀏FRQWHQWV󰀏RI󰀏HDFK󰀏VHQGBEXI󰀏RQ󰀏HDFK󰀏SURFHVV󰀦󰀏,WV󰀏HIIHFW󰀏LV󰀏WKH󰀏VDPH󰀏DV󰀏LIWKHUH󰀏ZHUH󰀏D󰀏VHTXHQFH󰀏RI󰀏S󰀏FDOOV󰀏WR󰀏03,B*DWKHU󰀢󰀏HDFK󰀏RI󰀏ZKLFK󰀏KDV󰀏D󰀏GLIIHUHQW󰀏SURFHVV󰀏DFWLQJ󰀏DVURRW󰀦

MPI_AllReduce(OPERAND, RESULT, COUNT, DATATYPE, OP, COMM, IERROR)

OPERAND(*), RESULT(*)

INTEGER COUNT, DATATYPE, OP ,COMM,IERROR

03,B$OOUHGXFH󰀏VWRUHV󰀏WKH󰀏UHVXOW󰀏RI󰀏WKH󰀏UHGXFH󰀏RSHUDWLRQ󰀏23󰀏LQ󰀏HDFK󰀏SURFHVV·󰀏UHVXOW󰀏EXIIHU󰀦

󰀔󰀕 *URXSLQJ󰀄'DWD󰀄IRU&RPPXQLFDWLRQ

:LWK󰀏FXUUHQW󰀏JHQHUDWLRQ󰀏PDFKLQHV󰀏VHQGLQJ󰀏D󰀏PHVVDJH󰀏LV󰀏DQ󰀏H[SHQVLYH󰀏RSHUDWLRQ󰀦󰀏6R󰀏DV󰀏D󰀏UXOHRI󰀏WKXPE󰀢󰀏WKH󰀏IHZHU󰀏PHVVDJHV󰀏VHQW󰀢󰀏WKH󰀏EHWWHU󰀏WKH󰀏RYHUDOO󰀏SHUIRUPDQFH󰀏RI󰀏WKH󰀏SURJUDP󰀦+RZHYHU󰀢󰀏LQ󰀏HDFK󰀏RI󰀏RXU󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDPV󰀢󰀏ZKHQ󰀏ZH󰀏GLVWULEXWHG󰀏WKH󰀏LQSXW󰀏GDWD󰀢󰀏ZHVHQW󰀏D󰀢󰀏E󰀏DQG󰀏Q󰀏LQ󰀏VHSDUDWH󰀏PHVVDJHV󰀏󰀜󰀏ZKHWKHU󰀏ZH󰀏XVHG󰀏03,B6HQG󰀏DQG󰀏03,B5HFY󰀏RU03,B%FDVW󰀦󰀏6R󰀏ZH󰀏VKRXOG󰀏EH󰀏DEOH󰀏WR󰀏LPSURYH󰀏WKH󰀏SHUIRUPDQFH󰀏RI󰀏WKH󰀏SURJUDP󰀏E\\󰀏VHQGLQJWKH󰀏WKUHH󰀏LQSXW󰀏YDOXHV󰀏LQ󰀏D󰀏VLQJOH󰀏PHVVDJH󰀦󰀏03,󰀏SURYLGHV󰀏WKUHH󰀏PHFKDQLVPV󰀏IRU󰀏JURXSLQJLQGLYLGXDO󰀏GDWD󰀏LWHPV󰀏LQWR󰀏D󰀏VLQJOH󰀏PHVVDJH󰀏󰀺󰀏WKH󰀏FRXQW󰀏SDUDPHWHU󰀏WR󰀏WKH󰀏YDULRXVFRPPXQLFDWLRQ󰀏URXWLQHV󰀢󰀏GHULYHG󰀏GDWDW\\SHV󰀢󰀏DQG󰀏03,B3DFN󰀏󰁋󰀏03,B8QSDFN󰀦󰀏:H󰀏H[DPLQHHDFK󰀏RI󰀏WKHVH󰀏RSWLRQV󰀏LQ󰀏WXUQ󰀦

󰀔󰀕 7KH󰀄&RXQW󰀄3DUDPHWHU

5HFDOO󰀏WKDW󰀏03,B6HQG󰀢󰀏03,B5HFY󰀢󰀏03,B%FDVW󰀢󰀏DQG󰀏03,B5HGXFH󰀏DOO󰀏KDYH󰀏D󰀏FRXQW

DQG󰀏D󰀏datatype󰀏DUJXPHQW󰀦󰀏7KHVH󰀏WZR󰀏SDUDPHWHUV󰀏DOORZ󰀏WKH󰀏XVHU󰀏WR󰀏JURXS󰀏GDWD󰀏LWHPVKDYLQJ󰀏WKH󰀏VDPH󰀏EDVLF󰀏W\\SH󰀏LQWR󰀏D󰀏VLQJOH󰀏PHVVDJH󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏XVH󰀏WKLV󰀢󰀏WKH󰀏JURXSHGGDWD󰀏LWHPV󰀏PXVW󰀏EH󰀏VWRUHG󰀏LQ󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀦󰀏6LQFH󰀏)RUWUDQ󰀏JXDUDQWHHVWKDW󰀏DUUD\\󰀏HOHPHQWV󰀏DUH󰀏VWRUHG󰀏LQ󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀢󰀏LI󰀏ZH󰀏ZLVK󰀏WR󰀏VHQGWKH󰀏HOHPHQWV󰀏RI󰀏DQ󰀏DUUD\\󰀢󰀏RU󰀏D󰀏VXEVHW󰀏RI󰀏DQ󰀏DUUD\\󰀢󰀏ZH󰀏FDQ󰀏GR󰀏VR󰀏LQ󰀏D󰀏VLQJOH󰀏PHVVDJH󰀦󰀏,QIDFW󰀢󰀏ZH·YH󰀏DOUHDG\\󰀏GRQH󰀏WKLV󰀏LQ󰀏VHFWLRQ󰀏󰀽󰀢󰀏ZKHQ󰀏ZH󰀏VHQW󰀏DQ󰀏DUUD\\󰀏RI󰀏character󰀦$V󰀏DQRWKHU󰀏H[DPSOH󰀢󰀏VXSSRVH󰀏ZH󰀏ZLVK󰀏WR󰀏VHQG󰀏WKH󰀏VHFRQG󰀏KDOI󰀏RI󰀏D󰀏YHFWRU󰀏FRQWDLQLQJ󰀩󰀶󰀶󰀏UHDO󰀏QXPEHUV󰀏IURP󰀏SURFHVV󰀏󰀶󰀏WR󰀏SURFHVV󰀏󰀩󰀦

real vector(100)

integer status(MPI_STATUS_SIZE)integer p, my_rank, ierrinteger i...

if (my_rank == 0) then

C Initialize vector and send.tag = 47count = 50dest = 1

call MPI_SEND(vector(51), count, MPI_REAL, dest, tag,+ MPI_COMM_WORLD, ierr)elsetag = 47count = 50source = 0

call MPI_RECV(vector(51), count, MPI_REAL, source,tag,

+ MPI_COMM_WORLD, status, ierr)endif

8QIRUWXQDWHO\\󰀢󰀏WKLV󰀏GRHVQ·W󰀏KHOS󰀏XV󰀏ZLWK󰀏WKH󰀏WUDSH]RLG󰀏UXOH󰀏SURJUDP󰀦󰀏7KH󰀏GDWD󰀏ZHZLVK󰀏WR󰀏GLVWULEXWH󰀏WR󰀏WKH󰀏RWKHU󰀏SURFHVVHV󰀢󰀏D󰀢󰀏E󰀢󰀏DQG󰀏Q󰀢󰀏DUH󰀏VWRUHG󰀏LQ󰀏DQ󰀏DUUD\\󰀦󰀏6R󰀏HYHQ󰀏LIZH󰀏GHFODUHG󰀏WKHP󰀏RQH󰀏DIWHU󰀏WKH󰀏RWKHU󰀏LQ󰀏RXU󰀏SURJUDP󰀢real areal binteger n

)RUWUDQ󰀏GRHV󰀏QRW󰀏JXDUDQWHH󰀏WKDW󰀏WKH\\󰀏DUH󰀏VWRUHG󰀏LQ󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀦󰀏2QHPLJKW󰀏EH󰀏WHPSWHG󰀏WR󰀏VWRUH󰀏Q󰀏DV󰀏D󰀏IORDW󰀏DQG󰀏SXW󰀏WKH󰀏WKUHH󰀏YDOXHV󰀏LQ󰀏DQ󰀏DUUD\\󰀢󰀏EXW󰀏WKLVZRXOG󰀏EH󰀏SRRU󰀏SURJUDPPLQJ󰀏VW\\OH󰀏DQG󰀏LW󰀏ZRXOGQ·W󰀏DGGUHVV󰀏WKH󰀏IXQGDPHQWDO󰀏LVVXH󰀦󰀏,QRUGHU󰀏WR󰀏VROYH󰀏WKH󰀏SUREOHP󰀏ZH󰀏QHHG󰀏WR󰀏XVH󰀏RQH󰀏RI󰀏03,·V󰀏RWKHU󰀏IDFLOLWLHV󰀏IRU󰀏JURXSLQJGDWD󰀦

󰀙󰀕 'HULYHG󰀄7\\SHV󰀄DQG󰀄03,B7\\SHBVWUXFW

,W󰀏PLJKW󰀏VHHP󰀏WKDW󰀏DQRWKHU󰀏RSWLRQ󰀏ZRXOG󰀏EH󰀏WR󰀏VWRUH󰀏D󰀢󰀏E󰀢󰀏DQG󰀏Q󰀏LQ󰀏D󰀏GHULYHG󰀏W\\SH󰀏ZLWK

WKUHH󰀏PHPEHUV󰀏󰀜󰀏WZR󰀏UHDOV󰀏DQG󰀏DQ󰀏LQWHJHU󰀏󰀜󰀏DQG󰀏WU\\󰀏WR󰀏XVH󰀏WKH󰀏GDWDW\\SH󰀏DUJXPHQW󰀏WR03,B%FDVW󰀦󰀏7KH󰀏GLIILFXOW\\󰀏KHUH󰀏LV󰀏WKDW󰀏WKH󰀏W\\SH󰀏RI󰀏GDWDW\\SH󰀏LV󰀏03,B'DWDW\\SH󰀢󰀏ZKLFK󰀏LVDQ󰀏DFWXDO󰀏W\\SH󰀏LWVHOI󰀏󰀜󰀏QRW󰀏WKH󰀏VDPH󰀏WKLQJ󰀏DV󰀏D󰀏XVHU󰀜GHILQHG󰀏W\\SH󰀏LQ󰀏)RUWUDQ󰀏󰀪󰀶󰀦󰀏)RUH[DPSOH󰀢󰀏VXSSRVH󰀏ZH󰀏LQFOXGHG󰀏WKH󰀏W\\SH󰀏GHILQLWLRQtype INDATA_TYPEreal areal binteger nend type

DQG󰀏WKH󰀏YDULDEOH󰀏GHILQLWLRQ

type (INDATA_TYPE) indata1RZ󰀏LI󰀏ZH󰀏FDOO󰀏03,B%FDVW

call MPI_Bcast(indata, 1, INDATA_TYPE, 0,MPI_COMM_WORLD,+ ierror)

WKH󰀏SURJUDP󰀏ZLOO󰀏IDLO󰀦󰀏7KH󰀏GHWDLO󰀏GHSHQG󰀏RQ󰀏WKH󰀏LPSOHPHQWDWLRQ󰀏RI󰀏03,󰀏WKDW󰀏\\RX·UHXVLQJ󰀦󰀏7KH󰀏SUREOHP󰀏KHUH󰀏LV󰀏WKDW󰀏03,󰀏LV󰀏D󰀏SUH󰀦H[LVWLQJ󰀏OLEUDU\\󰀏RI󰀏IXQFWLRQV󰀦󰀏7KDW󰀏LV󰀢󰀏WKH03,󰀏IXQFWLRQV󰀏ZHUH󰀏ZULWWHQ󰀏ZLWKRXW󰀏NQRZOHGJH󰀏RI󰀏WKH󰀏GDWDW\\SHV󰀏WKDW󰀏\\RX󰀏GHILQH󰀏LQ\\RXU󰀏SURJUDP󰀦󰀏,Q󰀏SDUWLFXODU󰀢󰀏QRQH󰀏RI󰀏WKH󰀏03,󰀏IXQFWLRQV󰀏󰀳NQRZV󰀳󰀏DERXW,1'$7$B7<3(󰀦

03,󰀏SURYLGHV󰀏D󰀏SDUWLDO󰀏VROXWLRQ󰀏WR󰀏WKLV󰀏SUREOHP󰀢󰀏E\\󰀏DOORZLQJ󰀏WKH󰀏XVHU󰀏WR󰀏EXLOG󰀏03,GDWDW\\SHV󰀏DW󰀏H[HFXWLRQ󰀏WLPH󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏EXLOG󰀏DQ󰀏03,󰀏GDWDW\\SH󰀢󰀏RQH󰀏HVVHQWLDOO\\VSHFLILHV󰀏WKH󰀏OD\\RXW󰀏RI󰀏WKH󰀏GDWD󰀏LQ󰀏WKH󰀏W\\SH󰀏󰀜󰀏WKH󰀏PHPEHU󰀏W\\SHV󰀏DQG󰀏WKHLU󰀏UHODWLYHORFDWLRQV󰀏LQ󰀏PHPRU\\󰀦󰀏6XFK󰀏D󰀏W\\SH󰀏LV󰀏FDOOHG󰀏D󰀏03,󰀂GHULYHG󰀂GDWD󰀂W\\SH󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏VHH󰀏KRZWKLV󰀏ZRUNV󰀢󰀏OHW·V󰀏ZULWH󰀏D󰀏IXQFWLRQ󰀏WKDW󰀏ZLOO󰀏EXLOG󰀏D󰀏03,󰀏GHULYHG󰀏W\\SH󰀦MODULE GLOBALtype INDATA_TYPEreal areal binteger n

end type INDATA_TYPEEND MODULE GLOBAL

subroutine Build_derived_type(indata, mesg_mpi_t)use GLOBALINCLUDE 'mpif.h'IMPLICIT NONE

type(INDATA_TYPE) indatainteger mesg_mpi_tinteger ierr

integer block_lengths(3)integer displacements(3)integer address(4)integer typelist(3)

C Build a derived datatype consisting of two real andan integer.

C First specify the types.typelist(1) = MPI_REALtypelist(2) = MPI_REALtypelist(3) = MPI_INTEGER

C Specify the number of elements of each type.block_lengths(1) = 1block_lengths(2) = 1block_lengths(3) = 1

C Calculate the displacements of the members relativeto indata.

call MPI_Address(indata, address(1), ierr)call MPI_Address(indata%a, address(2), ierr)call MPI_Address(indata%b, address(3), ierr)call MPI_Address(indata%n, address(4), ierr)displacements(1) = address(2) - address(1)displacements(2) = address(3) - address(1)displacements(3) = address(4) - address(1)C Build the derived datatype

call MPI_TYPE_STRUCT(3, block_lengths, displacements,+ typelist, mesg_mpi_t, ierr)

C Commit it -- tell system we'll be using it forcommunication.

call MPI_TYPE_COMMIT(mesg_mpi_t, ierr)returnend

7KH󰀏ILUVW󰀏WKUHH󰀏VWDWHPHQWV󰀏VSHFLI\\󰀏WKH󰀏W\\SHV󰀏RI󰀏WKH󰀏PHPEHUV󰀏RI󰀏WKH󰀏03,󰀏GHULYHG󰀏W\\SH󰀢DQG󰀏WKH󰀏QH[W󰀏WKUHH󰀏VSHFLILHV󰀏WKH󰀏QXPEHU󰀏RI󰀏HOHPHQWV󰀏RI󰀏HDFK󰀏W\\SH󰀦󰀏7KH󰀏QH[W󰀏IRXUFDOFXODWH󰀏WKH󰀏DGGUHVVHV󰀏RI󰀏WKH󰀏WKUHH󰀏PHPEHUV󰀏RI󰀏indata󰀦󰀏7KH󰀏QH[W󰀏WKUHH󰀏VWDWHPHQWVXVH󰀏WKH󰀏FDOFXODWHG󰀏DGGUHVVHV󰀏WR󰀏GHWHUPLQH󰀏WKH󰀏GLVSODFHPHQWV󰀏RI󰀏WKH󰀏WKUHH󰀏PHPEHUVUHODWLYH󰀏WR󰀏WKH󰀏DGGUHVV󰀏RI󰀏WKH󰀏ILUVW󰀏󰀜󰀏ZKLFK󰀏LV󰀏JLYHQ󰀏GLVSODFHPHQW󰀏󰀶󰀦󰀏:LWK󰀏WKLVLQIRUPDWLRQ󰀢󰀏ZH󰀏NQRZ󰀏WKH󰀏W\\SHV󰀢󰀏VL]HV󰀏DQG󰀏UHODWLYH󰀏ORFDWLRQV󰀏RI󰀏WKH󰀏PHPEHUV󰀏RI󰀏DYDULDEOH󰀏KDYLQJ󰀏)RUWUDQ󰀏󰀪󰀶󰀏W\\SH󰀏,1'$7$B7<3(󰀢󰀏DQG󰀏KHQFH󰀏ZH󰀏FDQ󰀏GHILQH󰀏D󰀏GHULYHGGDWD󰀏W\\SH󰀏WKDW󰀏FRUUHVSRQGV󰀏WR󰀏WKH󰀏)RUWUDQ󰀏W\\SH󰀦󰀏7KLV󰀏LV󰀏GRQH󰀏E\\󰀏FDOOLQJ󰀏WKH󰀏IXQFWLRQVMPI_Type_struct󰀏DQG󰀏MPI_Type_commit󰀦

7KH󰀏QHZO\\󰀏FUHDWHG󰀏03,󰀏GDWDW\\SH󰀏FDQ󰀏EH󰀏XVHG󰀏LQ󰀏DQ\\󰀏RI󰀏WKH󰀏03,󰀏FRPPXQLFDWLRQIXQFWLRQV󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏XVH󰀏LW󰀢󰀏ZH󰀏VLPSO\\󰀏XVH󰀏WKH󰀏VWDUWLQJ󰀏DGGUHVV󰀏RI󰀏D󰀏YDULDEOH󰀏RI󰀏W\\SH,1'$7$B7<3(󰀏DV󰀏WKH󰀏ILUVW󰀏DUJXPHQW󰀢󰀏DQG󰀏WKH󰀏GHULYHG󰀏W\\SH󰀏LQ󰀏WKH󰀏GDWDW\\SHDUJXPHQW󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏ZH󰀏FRXOG󰀏UHZULWH󰀏WKH󰀏*HWBGDWD󰀏IXQFWLRQ󰀏DV󰀏IROORZV󰀦subroutine Get_data3(indata, my_rank)use global

type(INDATA_TYPE) indatainteger my_rankinteger mesg_mpi_tinteger ierrinclude 'mpif.h'if (my_rank == 0) thenprint *, 'Enter a, b, and n'

read *, indata%a, indata%b, indata%nendif

call Build_derived_type(indata, mesg_mpi_t)call MPI_BCAST(indata, 1, mesg_mpi_t, 0,MPI_COMM_WORLD,+ierr )

returnend

7R󰀏VXPPDUL]H󰀢󰀏WKHQ󰀢󰀏ZH󰀏FDQ󰀏EXLOG󰀏JHQHUDO󰀏03,󰀏GHULYHG󰀏GDWDW\\SHV󰀏E\\󰀏FDOOLQJMPI_Type_struct󰀦󰀏7KH󰀏V\\QWD[󰀏LV

call MPI_TYPE_STRUCT(count, array_of_block_lengths,+array_of_displacements, array_of_types, newtype,ierror)

integer count, array_of_block_lengths(*),

integer array_of_displacements(*) , array_of_types(*)integer array_of_types(*), newtype, ierror

7KH󰀏DUJXPHQW󰀏count󰀏LV󰀏WKH󰀏QXPEHU󰀏RI󰀏HOHPHQWV󰀏LQ󰀏WKH󰀏GHULYHG󰀏W\\SH󰀦󰀏,W󰀏LV󰀏DOVR󰀏WKH󰀏VL]HRI󰀏WKH󰀏WKUHH󰀏DUUD\\V󰀢󰀏array_of_block_lengths󰀢󰀏array_of_displacements󰀢󰀏DQGarray_of_types󰀦󰀏7KH󰀏DUUD\\󰀏array_of_block_lengths󰀏FRQWDLQV󰀏WKH󰀏QXPEHU󰀏RIHQWULHV󰀏LQ󰀏HDFK󰀏HOHPHQW󰀏RI󰀏WKH󰀏W\\SH󰀦󰀏6R󰀏LI󰀏DQ󰀏HOHPHQW󰀏RI󰀏WKH󰀏W\\SH󰀏LV󰀏DQ󰀏DUUD\\󰀏RI󰀏PYDOXHV󰀢󰀏WKHQ󰀏WKH󰀏FRUUHVSRQGLQJ󰀏HQWU\\󰀏LQ󰀏array_of_block_lengths󰀏LV󰀏P󰀦󰀏7KH󰀏DUUD\\array_of_displacements󰀏FRQWDLQV󰀏WKH󰀏GLVSODFHPHQW󰀏RI󰀏HDFK󰀏HOHPHQW󰀏IURP󰀏WKHEHJLQQLQJ󰀏RI󰀏WKH󰀏PHVVDJH󰀢󰀏DQG󰀏WKH󰀏DUUD\\󰀏array_of_types󰀏FRQWDLQV󰀏WKH󰀏03,󰀏GDWDW\\SHRI󰀏HDFK󰀏HQWU\\󰀦󰀏7KH󰀏DUJXPHQW󰀏newtype󰀏UHWXUQV󰀏D󰀏SRLQWHU󰀏WR󰀏WKH󰀏03,󰀏GDWDW\\SH󰀏FUHDWHGE\\󰀏WKH󰀏FDOO󰀏WR󰀏MPI_Type_struct󰀦

1RWH󰀏DOVR󰀏WKDW󰀏newtype󰀏DQG󰀏WKH󰀏HQWULHV󰀏LQ󰀏array_of_types󰀏DOO󰀏KDYH󰀏WSHMPI_Datatype󰀦󰀏6R󰀏MPI_Type_struct󰀏FDQ󰀏EH󰀏FDOOHG󰀏UHFXUVLYHO\\󰀏WR󰀏EXLOG󰀏PRUHFRPSOH[󰀏GHULYHG󰀏GDWDW\\SHV󰀦

󰀣󰀕 2WKHU󰀄'HULYHG󰀄'DWDW\\SH󰀄&RQVWUXFWRUV

MPI_Type_struct󰀏LV󰀏WKH󰀏PRVW󰀏JHQHUDO󰀏GDWDW\\SH󰀏FRQVWUXFWRU󰀏LQ󰀏03,󰀢󰀏DQG󰀏DV󰀏D

FRQVHTXHQFH󰀢󰀏WKH󰀏XVHU󰀏PXVW󰀏SURYLGH󰀏D󰀏FRPSOHWH󰀏GHVFULSWLRQ󰀏RI󰀏HDFK󰀏HOHPHQW󰀏RI󰀏WKHW\\SH󰀦󰀏,I󰀏WKH󰀏GDWD󰀏WR󰀏EH󰀏WUDQVPLWWHG󰀏FRQVLVWV󰀏RI󰀏D󰀏VXEVHW󰀏RI󰀏WKH󰀏HQWULHV󰀏LQ󰀏DQ󰀏DUUD\\󰀢󰀏ZHVKRXOGQ󰁍W󰀏QHHG󰀏WR󰀏SURYLGH󰀏VXFK󰀏GHWDLOHG󰀏LQIRUPDWLRQ󰀢󰀏VLQFH󰀏DOO󰀏WKH󰀏HOHPHQWV󰀏KDYH󰀏WKHVDPH󰀏EDVLF󰀏W\\SH󰀦󰀏03,󰀏SURYLGHV󰀏WKUHH󰀏GHULYHG󰀏GDWDW\\SH󰀏FRQVWUXFWRUV󰀏IRU󰀏GHDOLQJ󰀏ZLWKWKLV󰀏VLWXDWLRQ󰀺󰀏MPI_Type_Contiguous󰀢󰀏MPI_Type_vector󰀏DQGMPI_Type_indexed󰀦󰀏7KH󰀏ILUVW󰀏FRQVWUXFWRU󰀏EXLOGV󰀏D󰀏GHULYHG󰀏W\\SH󰀏ZKRVH󰀏HOHPHQWVDUH󰀏FRQWLJXRXV󰀏HQWULHV󰀏LQ󰀏DQ󰀏DUUD\\󰀦󰀏7KH󰀏VHFRQG󰀏EXLOGV󰀏D󰀏W\\SH󰀏ZKRVH󰀏HOHPHQWV󰀏DUHHTXDOO\\󰀏VSDFHG󰀏HQWULHV󰀏RI󰀏DQ󰀏DUUD\\󰀢󰀏DQG󰀏WKH󰀏WKLUG󰀏EXLOGV󰀏D󰀏W\\SH󰀏ZKRVH󰀏HOHPHQWV󰀏DUHDUELWUDU\\󰀏HQWULHV󰀏RI󰀏DQ󰀏DUUD\\󰀦󰀏1RWH󰀏WKDW󰀏EHIRUH󰀏DQ\\󰀏GHULYHG󰀏W\\SH󰀏FDQ󰀏EH󰀏XVHG󰀏LQFRPPXQLFDWLRQ󰀏LW󰀏PXVW󰀏EH󰀏FRPPLWWHG󰀏ZLWK󰀏D󰀏FDOO󰀏WR󰀏MPI_Type_commit󰀦'HWDLOV󰀏RI󰀏WKH󰀏V\\QWD[󰀏RI󰀏WKH󰀏DGGLWLRQDO󰀏W\\SH󰀏FRQVWUXFWRUV󰀏IROORZ󰀦

MPI_Type_contiguous(count, oldtype, newtype, ierror)integer count, oldtype, newtype, ierror

MPI_Type_contiguous󰀏FUHDWHV󰀏D󰀏GHULYHG󰀏GDWDW\\SH󰀏FRQVLVWLQJ󰀏RI󰀏count󰀏HOHPHQWV󰀏RIW\\SH󰀏oldtype󰀦󰀏7KH󰀏HOHPHQWV󰀏EHORQJ󰀏WR󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀦

MPI_Type_vector(count, block_length, stride,+element_type, newtype, ierror)

integer count, blocklength, stride oldtype, newtype,ierror

MPI_Type_vector󰀏FUHDWHV󰀏D󰀏GHULYHG󰀏W\\SH󰀏FRQVLVWLQJ󰀏RI󰀏count󰀏HOHPHQWV󰀦󰀏(DFKHOHPHQW󰀏FRQWDLQV󰀏block_length󰀏HQWULHV󰀏RI󰀏W\\SH󰀏element_type󰀦󰀏Stride󰀏LV󰀏WKHQXPEHU󰀏RI󰀏HOHPHQWV󰀏RI󰀏W\\SH󰀏element_type󰀏EHWZHHQ󰀏VXFFHVVLYH󰀏HOHPHQWV󰀏RInew_type󰀦

MPI_Type_indexed(count, array_of_block_lengths,

+array_of_displacements, element_type, newtype, ierrorinteger count, array_of_block_lengths(*),

+array_of_displacements, element_type, newtype, ierrorMPI_Type_indexed󰀏FUHDWHV󰀏D󰀏GHULYHG󰀏W\\SH󰀏FRQVLVWLQJ󰀏RI󰀏count󰀏HOHPHQWV󰀦󰀏7KH󰀏LWKHOHPHQW󰀏󰁂L󰀏 󰀏󰀩󰀢󰀏󰀦󰀦󰀦󰀢󰀏count󰁃󰀢󰀏FRQVLVWV󰀏RI󰀏array_of_block_lengths[i]󰀏HQWULHV󰀏RI󰀏W\\SHelement_type󰀢󰀏DQG󰀏LW󰀏LV󰀏GLVSODFHG󰀏array_of_displacements[i]󰀏XQLWV󰀏RI󰀏W\\SHelement_type󰀏IURP󰀏WKH󰀏EHJLQQLQJ󰀏RI󰀏newtype󰀦

󰀲󰀕 3DFN󰀭8QSDFN

$Q󰀏DOWHUQDWLYH󰀏DSSURDFK󰀏WR󰀏JURXSLQJ󰀏GDWD󰀏LV󰀏SURYLGHG󰀏E\\󰀏WKH󰀏03,󰀏IXQFWLRQV

MPI_Pack󰀏DQG󰀏MPI_Unpack󰀦󰀏MPI_Pack󰀏DOORZV󰀏RQH󰀏WR󰀏H[SOLFLWO\\󰀏VWRUHQRQFRQWLJXRXV󰀏GDWD󰀏LQ󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀢󰀏DQG󰀏MPI_Unpack󰀏FDQ󰀏EH󰀏XVHGWR󰀏FRS\\󰀏GDWD󰀏IURP󰀏D󰀏FRQWLJXRXV󰀏EXIIHU󰀏LQWR󰀏QRQFRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀦󰀏,QRUGHU󰀏WR󰀏VHH󰀏KRZ󰀏WKH\\󰀏DUH󰀏XVHG󰀢󰀏OHW󰁍V󰀏UHZULWH󰀏Get_data󰀏RQH󰀏ODVW󰀏WLPH󰀦subroutine Get_data4(a, b, n, my_rank)real areal binteger ninteger my_rankC

INCLUDE 'mpif.h'integer ierr

character buffer(100)integer positionC in the bufferC

if (my_rank .EQ. 0) thenprint *,'Enter a, b, and n'read *, a, b, nC

C Now pack the data into buffer. Position = 0C says start at beginning of buffer.position = 0C

C Position is in/out

call MPI_PACK(a, 1, MPI_REAL , buffer, 100,+ position, MPI_COMM_WORLD, ierr )

C Position has been incremented: it now refer-C ences the first free location in buffer.C

call MPI_PACK(b, 1, MPI_REAL , buffer, 100,+ position, MPI_COMM_WORLD, ierr )C Position has been incremented again.C

call MPI_PACK(n, 1, MPI_INTEGER, buffer, 100,+ position, MPI_COMM_WORLD, ierr )C Position has been incremented again.C

C Now broadcast contents of buffer

call MPI_BCAST(buffer, 100, MPI_PACKED, 0,+ MPI_COMM_WORLD, ierr )else

call MPI_BCAST(buffer, 100, MPI_PACKED, 0,+ MPI_COMM_WORLD, ierr )C

C Now unpack the contents of buffer

position = 0

call MPI_UNPACK(buffer, 100, position, a, 1,+ MPI_REAL , MPI_COMM_WORLD, ierr )

C Once again position has been incremented:C it now references the beginning of b.C

call MPI_UNPACK(buffer, 100, position, b, 1,+ MPI_REAL , MPI_COMM_WORLD, ierr )

call MPI_UNPACK(buffer, 100, position, n, 1,+ MPI_INTEGER, MPI_COMM_WORLD, ierr )endifreturnendC

,Q󰀏WKLV󰀏YHUVLRQ󰀏RI󰀏Get_data󰀏SURFHVV󰀏󰀶󰀏XVHV󰀏MPI_Pack󰀏WR󰀏FRS\\󰀏a󰀏WR󰀏buffer󰀏DQGWKHQ󰀏DSSHQG󰀏b󰀏DQG󰀏n󰀦󰀏$IWHU󰀏WKH󰀏EURDGFDVW󰀏RI󰀏buffer󰀢󰀏WKH󰀏UHPDLQLQJ󰀏SURFHVVHV󰀏XVHMPI_Unpack󰀏WR󰀏VXFFHVVLYHO\\󰀏H[WUDFW󰀏a󰀢󰀏b󰀢󰀏DQG󰀏n󰀏IURP󰀏buffer󰀦󰀏1RWH󰀏WKDW󰀏WKHGDWDW\\SH󰀏IRU󰀏WKH󰀏FDOOV󰀏WR󰀏MPI_Bcast󰀏LV󰀏MPI_PACKED󰀦7KH󰀏V\\QWD[󰀏RI󰀏MPI_Pack󰀏LV

MPI_Pack( pack_data, in_count, datatype, buffer, size,+position_ptr, comm, ierror) pack_data(*), buffer(*)

integer in_count, datatype, size, position_ptr, comm,ierror

7KH󰀏SDUDPHWHU󰀏pack_data󰀏UHIHUHQFHV󰀏WKH󰀏GDWD󰀏WR󰀏EH󰀏EXIIHUHG󰀦󰀏,W󰀏VKRXOG󰀏FRQVLVW󰀏RIin_count󰀏HOHPHQWV󰀢󰀏HDFK󰀏KDYLQJ󰀏W\\SH󰀏datatype󰀦󰀏7KH󰀏SDUDPHWHU󰀏position_ptr LV󰀏DQLQ󰀨RXW󰀏SDUDPHWHU󰀦󰀏2Q󰀏LQSXW󰀢󰀏WKH󰀏GDWD󰀏UHIHUHQFHG󰀏E\\󰀏pack_data󰀏LV󰀏FRSLHG󰀏LQWRPHPRU\\󰀏VWDUWLQJ󰀏DW󰀏DGGUHVV󰀏buffer + position_ptr󰀦󰀏2Q󰀏UHWXUQ󰀢󰀏position_ptrUHIHUHQFHV󰀏WKH󰀏ILUVW󰀏ORFDWLRQ󰀏LQ󰀏buffer󰀏DIWHU󰀏WKH󰀏GDWD󰀏WKDW󰀏ZDV󰀏FRSLHG󰀦󰀏7KH󰀏SDUDPHWHUsize󰀏FRQWDLQV󰀏WKH󰀏VL]H󰀏LQ󰀂E\\WHV󰀏RI󰀏WKH󰀏PHPRU\\󰀏UHIHUHQFHG󰀏E\\󰀏buffer󰀢󰀏DQG󰀏comm󰀏LV󰀏WKHFRPPXQLFDWRU󰀏WKDW󰀏ZLOO󰀏EH󰀏XVLQJ󰀏buffer󰀦7KH󰀏V\\QWD[󰀏RI󰀏MPI_Unpack󰀏LV

MPI_Unpack(buffer, size, position_ptr, unpack_data,+count, datatype, comm, ierror)

inbuf(*), outbuf(*)

integer insize, position, outcount, datatype, comm,ierror

7KH󰀏SDUDPHWHU󰀏buffer󰀏UHIHUHQFHV󰀏WKH󰀏GDWD󰀏WR󰀏EH󰀏XQSDFNHG󰀦󰀏,W󰀏FRQVLVWV󰀏RI󰀏size󰀏E\\WHV󰀦7KH󰀏SDUDPHWHU󰀏position_ptr󰀏LV󰀏RQFH󰀏DJDLQ󰀏DQ󰀏LQ󰀨RXW󰀏SDUDPHWHU󰀦󰀏:KHQ󰀏MPI_UnpackLV󰀏FDOOHG󰀢󰀏WKH󰀏GDWD󰀏VWDUWLQJ󰀏DW󰀏DGGUHVV󰀏buffer + position_ptr󰀏LV󰀏FRSLHG󰀏LQWR󰀏WKHPHPRU\\󰀏UHIHUHQFHG󰀏E\\󰀏unpack_data󰀦󰀏2Q󰀏UHWXUQ󰀢󰀏position_ptr󰀏UHIHUHQFHV󰀏WKH󰀏ILUVWORFDWLRQ󰀏LQ󰀏buffer󰀏DIWHU󰀏WKH󰀏GDWD󰀏WKDW󰀏ZDV󰀏MXVW󰀏FRSLHG󰀦󰀏MPI_Unpack󰀏ZLOO󰀏FRS\\󰀏countHOHPHQWV󰀏KDYLQJ󰀏W\\SH󰀏datatype󰀏LQWR󰀏unpack_data󰀦󰀏7KH󰀏FRPPXQLFDWRU󰀏DVVRFLDWHGZLWK󰀏buffer󰀏LV󰀏comm󰀦

󰀴󰀕 'HFLGLQJ󰀄:KLFK󰀄0HWKRG󰀄WR󰀄8VH

,I󰀏WKH󰀏GDWD󰀏WR󰀏EH󰀏VHQW󰀏LV󰀏VWRUHG󰀏LQ󰀏FRQVHFXWLYH󰀏HQWULHV󰀏RI󰀏DQ󰀏DUUD\\󰀢󰀏WKHQ󰀏RQH󰀏VKRXOG󰀏VLPSO\\󰀏XVHWKH󰀏count󰀏DQG󰀏datatype󰀏DUJXPHQWV󰀏WR󰀏WKH󰀏FRPPXQLFDWLRQ󰀏IXQFWLRQ󰁂V󰁃󰀦󰀏7KLV󰀏DSSURDFKLQYROYHV󰀏QR󰀏DGGLWLRQDO󰀏RYHUKHDG󰀏LQ󰀏WKH󰀏IRUP󰀏RI󰀏FDOOV󰀏WR󰀏GHULYHG󰀏GDWDW\\SH󰀏FUHDWLRQ󰀏IXQFWLRQV󰀏RUFDOOV󰀏WR󰀏MPI_Pack/MPI_Unpack󰀦

,I󰀏WKHUH󰀏DUH󰀏D󰀏ODUJH󰀏QXPEHU󰀏RI󰀏HOHPHQWV󰀏WKDW󰀏DUH󰀏QRW󰀏LQ󰀏FRQWLJXRXV󰀏PHPRU\\󰀏ORFDWLRQV󰀢󰀏WKHQEXLOGLQJ󰀏D󰀏GHULYHG󰀏W\\SH󰀏ZLOO󰀏SUREDEO\\󰀏LQYROYH󰀏OHVV󰀏RYHUKHDG󰀏WKDQ󰀏D󰀏ODUJH󰀏QXPEHU󰀏RI󰀏FDOOV󰀏WRMPI_Pack/MPI_Unpack󰀦

,I󰀏WKH󰀏GDWD󰀏DOO󰀏KDYH󰀏WKH󰀏VDPH󰀏W\\SH󰀏DQG󰀏DUH󰀏VWRUHG󰀏DW󰀏UHJXODU󰀏LQWHUYDOV󰀏LQ󰀏PHPRU\\󰀏󰁂H󰀦J󰀦󰀢󰀏DFROXPQ󰀏RI󰀏D󰀏PDWUL[󰁃󰀢󰀏WKHQ󰀏LW󰀏ZLOO󰀏DOPRVW󰀏FHUWDLQO\\󰀏EH󰀏PXFK󰀏HDVLHU󰀏DQG󰀏IDVWHU󰀏WR󰀏XVH󰀏D󰀏GHULYHGGDWDW\\SH󰀏WKDQ󰀏LW󰀏ZLOO󰀏EH󰀏WR󰀏XVH󰀏MPI_Pack/MPI_Unpack󰀦󰀏)XUWKHUPRUH󰀢󰀏LI󰀏WKH󰀏GDWD󰀏DOO󰀏KDYHWKH󰀏VDPH󰀏W\\SH󰀢󰀏EXW󰀏DUH󰀏VWRUHG󰀏LQ󰀏LUUHJXODUO\\󰀏VSDFHG󰀏ORFDWLRQV󰀏LQ󰀏PHPRU\\󰀢󰀏LW󰀏ZLOO󰀏VWLOO󰀏SUREDEO\\EH󰀏HDVLHU󰀏DQG󰀏PRUH󰀏HIILFLHQW󰀏WR󰀏FUHDWH󰀏D󰀏GHULYHG󰀏W\\SH󰀏XVLQJ󰀏MPI_Type_indexed󰀦󰀏)LQDOO\\󰀢󰀏LIWKH󰀏GDWD󰀏DUH󰀏KHWHURJHQHRXV󰀏DQG󰀏RQH󰀏LV󰀏UHSHDWHGO\\󰀏VHQGLQJ󰀏WKH󰀏VDPH󰀏FROOHFWLRQ󰀏RI󰀏GDWD󰀏󰁂H󰀦J󰀦󰀢URZ󰀏QXPEHU󰀢󰀏FROXPQ󰀏QXPEHU󰀢󰀏PDWUL[󰀏HQWU\\󰁃󰀢󰀏WKHQ󰀏LW󰀏ZLOO󰀏EH󰀏EHWWHU󰀏WR󰀏XVH󰀏D󰀏GHULYHG󰀏W\\SH󰀢󰀏VLQFHWKH󰀏RYHUKHDG󰀏RI󰀏FUHDWLQJ󰀏WKH󰀏GHULYHG󰀏W\\SH󰀏LV󰀏LQFXUUHG󰀏RQO\\󰀏RQFH󰀢󰀏ZKLOH󰀏WKH󰀏RYHUKHDG󰀏RI󰀏FDOOLQJMPI_Pack/MPI_Unpack󰀏PXVW󰀏EH󰀏LQFXUUHG󰀏HYHU\\󰀏WLPH󰀏WKH󰀏GDWD󰀏LV󰀏FRPPXQLFDWHG󰀦7KLV󰀏OHDYHV󰀏WKH󰀏FDVH󰀏ZKHUH󰀏RQH󰀏LV󰀏VHQGLQJ󰀏KHWHURJHQHRXV󰀏GDWD󰀏RQO\\󰀏RQFH󰀢󰀏RU󰀏YHU\\󰀏IHZ󰀏WLPHV󰀦,Q󰀏WKLV󰀏FDVH󰀢󰀏LW󰀏PD\\󰀏EH󰀏D󰀏JRRG󰀏LGHD󰀏WR󰀏FROOHFW󰀏VRPH󰀏LQIRUPDWLRQ󰀏RQ󰀏WKH󰀏FRVW󰀏RI󰀏GHULYHG󰀏W\\SHFUHDWLRQ󰀏DQG󰀏SDFNLQJ󰁋XQSDFNLQJ󰀏WKH󰀏GDWD󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏RQ󰀏DQ󰀏Q&8%(󰀏󰀽󰀏UXQQLQJ󰀏WKH03,&+󰀏LPSOHPHQWDWLRQ󰀏RI󰀏03,󰀢󰀏LW󰀏WDNHV󰀏DERXW󰀏󰀩󰀽󰀏PLOOLVHFRQGV󰀏WR󰀏FUHDWH󰀏WKH󰀏GHULYHG󰀏W\\SHXVHG󰀏LQ󰀏Get_data3󰀢󰀏ZKLOH󰀏LW󰀏RQO\\󰀏WDNHV󰀏DERXW󰀏󰀽󰀏PLOOLVHFRQGV󰀏WR󰀏SDFN󰀏RU󰀏XQSDFN󰀏WKH󰀏GDWD󰀏LQGet_data4󰀦󰀏2I󰀏FRXUVH󰀢󰀏WKH󰀏VDYLQJ󰀏LVQ󰁍W󰀏DV󰀏JUHDW󰀏DV󰀏LW󰀏VHHPV󰀏EHFDXVH󰀏RI󰀏WKH󰀏DV\\PPHWU\\󰀏LQ󰀏WKHSDFN󰁋XQSDFN󰀏SURFHGXUH󰀦󰀏7KDW󰀏LV󰀢󰀏ZKLOH󰀏SURFHVV󰀏󰀶󰀏SDFNV󰀏WKH󰀏GDWD󰀢󰀏WKH󰀏RWKHU󰀏SURFHVVHV󰀏DUH󰀏LGOH󰀢DQG󰀏WKH󰀏HQWLUH󰀏IXQFWLRQ󰀏ZRQ󰁍W󰀏FRPSOHWH󰀏XQWLO󰀏ERWK󰀏WKH󰀏SDFN󰀏DQG󰀏XQSDFN󰀏DUH󰀏H[HFXWHG󰀦󰀏6R󰀏WKHFRVW󰀏UDWLR󰀏LV󰀏SUREDEO\\󰀏PRUH󰀏OLNH󰀏󰀫󰀺󰀩󰀏WKDQ󰀏󰀿󰀺󰀩󰀦

7KHUH󰀏DUH󰀏DOVR󰀏D󰀏FRXSOH󰀏RI󰀏VLWXDWLRQV󰀏LQ󰀏ZKLFK󰀏WKH󰀏XVH󰀏RI󰀏MPI_Pack󰀏DQG󰀏MPI_Unpack󰀏LVSUHIHUUHG󰀦󰀏1RWH󰀏ILUVW󰀏WKDW󰀏LW󰀏PD\\󰀏EH󰀏SRVVLEOH󰀏WR󰀏DYRLG󰀏WKH󰀏XVH󰀏RI󰀏V\\VWHP󰀏EXIIHULQJ󰀏ZLWK󰀏SDFN󰀢VLQFH󰀏WKH󰀏GDWD󰀏LV󰀏H[SOLFLWO\\󰀏VWRUHG󰀏LQ󰀏D󰀏XVHU󰀜GHILQHG󰀏EXIIHU󰀦󰀏7KH󰀏V\\VWHP󰀏FDQ󰀏H[SORLW󰀏WKLV󰀏E\\QRWLQJ󰀏WKDW󰀏WKH󰀏PHVVDJH󰀏GDWDW\\SH󰀏LV󰀏MPI_PACKED󰀦󰀏$OVR󰀏QRWH󰀏WKDW󰀏WKH󰀏XVHU󰀏FDQ󰀏VHQG󰀳YDULDEOH󰀜OHQJWK󰀳󰁍󰀏PHVVDJHV󰀏E\\󰀏SDFNLQJ󰀏WKH󰀏QXPEHU󰀏RI󰀏HOHPHQWV󰀏DW󰀏WKH󰀏EHJLQQLQJ󰀏RI󰀏WKHEXIIHU󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏VXSSRVH󰀏ZH󰀏ZDQW󰀏WR󰀏VHQG󰀏URZV󰀏RI󰀏D󰀏VSDUVH󰀏PDWUL[󰀦󰀏,I󰀏ZH󰀏KDYH󰀏VWRUHG󰀏DURZ󰀏DV󰀏D󰀏SDLU󰀏RI󰀏DUUD\\V󰀏󰀜󰀜󰀜󰀏RQH󰀏FRQWDLQLQJ󰀏WKH󰀏FROXPQ󰀏VXEVFULSWV󰀢󰀏DQG󰀏RQH󰀏FRQWDLQLQJ󰀏WKHFRUUHVSRQGLQJ󰀏PDWUL[󰀏HQWULHV󰀏󰀜󰀜󰀜󰀏ZH󰀏FRXOG󰀏VHQG󰀏D󰀏URZ󰀏IURP󰀏SURFHVV󰀏󰀶󰀏WR󰀏SURFHVV󰀏󰀩󰀏DV󰀏IROORZV󰀦

PROGRAM SpaRowINCLUDE 'mpif.h'integer HUGE

parameter (HUGE = 100)integer pinteger my_rankreal entries(10)

integer column_subscripts(10)integer nonzeroesinteger positioninteger row_numbercharacter buffer *100

integer status(MPI_STATUS_SIZE)integer ierrinteger i

data nonzeroes /10/C

call MPI_INIT( ierr)

call MPI_COMM_SIZE(MPI_COMM_WORLD, p, ierr )call MPI_COMM_RANK(MPI_COMM_WORLD, my_rank, ierr )C

if (my_rank .EQ. 0) then

C Get the number of nonzeros in the row.C Initialize entries and column_subscriptsdo i = 1, nonzeroesentries(i) = 2*i

column_subscripts(i) = 3*ienddoC

C Now pack the data and send

position = 1

call MPI_PACK( nonzeroes, 1, MPI_INTEGER, buffer, HUGE,+ position, MPI_COMM_WORLD, ierr )

call MPI_PACK( row_number, 1, MPI_INTEGER, buffer, HUGE,+ position, MPI_COMM_WORLD, ierr )

call MPI_PACK(entries, nonzeroes, MPI_REAL , buffer,+ HUGE, position, MPI_COMM_WORLD, ierr )

call MPI_PACK(column_subscripts,nonzeroes,MPI_INTEGER,+ buffer, HUGE, position, MPI_COMM_WORLD, ierr )call MPI_SEND(buffer, position, MPI_PACKED, 1, 0,+ MPI_COMM_WORLD, ierr )else

call MPI_RECV(buffer, HUGE, MPI_PACKED, 0, 0,+ MPI_COMM_WORLD, status, ierr )position = 1

call MPI_UNPACK(buffer, HUGE, position, nonzeroes,+ 1, MPI_INTEGER, MPI_COMM_WORLD, ierr )

call MPI_UNPACK(buffer, HUGE, position, row_number,+ 1, MPI_INTEGER, MPI_COMM_WORLD, ierr )

call MPI_UNPACK(buffer,HUGE, position, entries,+ nonzeroes, MPI_REAL , MPI_COMM_WORLD, ierr )call MPI_UNPACK(buffer, HUGE, position,+ column_subscripts,

+ nonzeroes, MPI_INTEGER, MPI_COMM_WORLD, ierr )do i = 1, nonzeroes

print *, entries(i), column_subscripts(i)enddoendifC

call MPI_FINALIZE(ierr)

end󰀏

󰀙󰀕 &RPPXQLFDWRUV󰀄DQG7RSRORJLHV

7KH󰀏XVH󰀏RI󰀏FRPPXQLFDWRUV󰀏DQG󰀏WRSRORJLHV󰀏PDNHV󰀏03,󰀏GLIIHUHQW󰀏IURP󰀏PRVW󰀏RWKHU󰀏PHVVDJH󰀜SDVVLQJ󰀏V\\VWHPV󰀦󰀏5HFROOHFW󰀏WKDW󰀢󰀏ORRVHO\\󰀏VSHDNLQJ󰀢󰀏D󰀏FRPPXQLFDWRU󰀏LV󰀏D󰀏FROOHFWLRQ󰀏RI󰀏SURFHVVHVWKDW󰀏FDQ󰀏VHQG󰀏PHVVDJHV󰀏WR󰀏HDFK󰀏RWKHU󰀦󰀏$󰀏WRSRORJ\\󰀏LV󰀏D󰀏VWUXFWXUH󰀏LPSRVHG󰀏RQ󰀏WKH󰀏SURFHVVHV󰀏LQD󰀏FRPPXQLFDWRU󰀏WKDW󰀏DOORZV󰀏WKH󰀏SURFHVVHV󰀏WR󰀏EH󰀏DGGUHVVHG󰀏LQ󰀏GLIIHUHQW󰀏ZD\\V󰀦󰀏,Q󰀏RUGHU󰀏WRLOOXVWUDWH󰀏WKHVH󰀏LGHDV󰀢󰀏ZH󰀏ZLOO󰀏GHYHORS󰀏FRGH󰀏WR󰀏LPSOHPHQW󰀏)R[󰁍V󰀏DOJRULWKP󰀏IRU󰀏PXOWLSO\\LQJ󰀏WZRVTXDUH󰀏PDWULFHV󰀦

󰀔󰀕 )R[󰀶V󰀄$OJRULWKP

:H󰀏DVVXPH󰀏WKDW󰀏WKH󰀏IDFWRU󰀏PDWULFHV󰀏$󰀏 󰀏󰁂DLM󰁃󰀏DQG󰀏%󰀏 󰀏󰁂ELM󰁃󰀏KDYH󰀏RUGHU󰀏Q󰀦󰀏:H󰀏DOVR󰀏DVVXPH󰀏WKDW󰀏WKHQXPEHU󰀏RI󰀏SURFHVVHV󰀢󰀏S󰀢󰀏LV󰀏D󰀏SHUIHFW󰀏VTXDUH󰀢󰀏ZKRVH󰀏VTXDUH󰀏URRW󰀏HYHQO\\󰀏GLYLGHV󰀏Q󰀦󰀏6D\\󰀏S󰀏 󰀏T󰀂󰀢󰀏DQG󰀏

Q󰁋T󰀦󰀏,Q󰀏)R[󰁍V󰀏DOJRULWKP󰀏WKH󰀏IDFWRU󰀏PDWULFHV󰀏DUH󰀏SDUWLWLRQHG󰀏DPRQJ󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏D󰀏EORFN󰀂FKHFNHUERDUGIDVKLRQ󰀦󰀏6R󰀏ZH󰀏YLHZ󰀏RXU󰀏SURFHVVHV󰀏DV󰀏D󰀏YLUWXDO󰀏WZR󰀜GLPHQVLRQDO󰀏T󰀏[󰀏T󰀏JULG󰀢󰀏DQG󰀏HDFK󰀏SURFHVV󰀏LVDVVLJQHG󰀏DQ󰀏[󰀏VXEPDWUL[󰀏RI󰀏HDFK󰀏RI󰀏WKH󰀏IDFWRU󰀏PDWULFHV󰀦󰀏0RUH󰀏IRUPDOO\\󰀢󰀏ZH󰀏KDYH󰀏D󰀏PDSSLQJ

WKDW󰀏LV󰀏ERWK󰀏RQH󰀜WR󰀜RQH󰀏DQG󰀏RQWR󰀦󰀏7KLV󰀏GHILQHV󰀏RXU󰀏JULG󰀏RI󰀏SURFHVVHV󰀺󰀏SURFHVV󰀏L󰀏EHORQJV󰀏WR󰀏WKH󰀏URZDQG󰀏FROXPQ󰀏JLYHQ󰀏E\\󰀏

󰀦󰀏)XUWKHU󰀢󰀏WKH󰀏SURFHVV󰀏ZLWK󰀏UDQN󰀏

LV󰀏DVVLJQHG󰀏WKH󰀏VXEPDWULFHV

󰀢

DQG

󰀦

)RU󰀏H[DPSOH󰀢󰀏LI󰀏S󰀏 󰀏󰀪󰀢󰀏

3URFHVV󰀏󰀶

󰀏󰁂[󰁋󰀫󰀢󰀏[󰀏PRG󰀏󰀫󰁃󰀢󰀏DQG󰀏Q󰀏 󰀏󰀿󰀢󰀏WKHQ󰀏$󰀏ZRXOG󰀏EH󰀏SDUWLWLRQHG󰀏DV󰀏IROORZV󰀦

3URFHVV󰀏󰀩

3URFHVV󰀏󰀽

3URFHVV󰀏󰀫3URFHVV󰀏󰀬3URFHVV󰀏󰀭

3URFHVV󰀏󰀿3URFHVV󰀏󰀾3URFHVV󰀏󰀷

,Q󰀏)R[󰁍V󰀏DOJRULWKP󰀢󰀏WKH󰀏EORFN󰀏VXEPDWULFHV󰀢󰀏$UV󰀏DQG󰀏%VW󰀢󰀏V󰀏 󰀏󰀶󰀢󰀏󰀩󰀢󰀏󰀦󰀦󰀦󰀦󰀏󰀢󰀏T󰀏󰀜󰀏󰀩󰀢󰀏DUH󰀏PXOWLSOLHG󰀏DQGDFFXPXODWHG󰀏RQ󰀏SURFHVV󰀏do step = 0, q - 1

1. Choose a submatrix of A from each row of processes.2. In each row of processes broadcast the submatrix

chosen in that row to the other processes in that row.3. On each process, multiply the newly received submatrixof A by the submatrix of B currently residing on theprocess.4. On each process, send the submatrix of B to theprocess directly above. (On processes in the firstrow, send the submatrix to the last row.)

enddo

7KH󰀏VXEPDWUL[󰀏FKRVHQ󰀏LQ󰀏WKH󰀏UWK󰀏URZ󰀏LV󰀏$U󰀇X󰀢󰀏󰀢󰀏ZKHUH

X󰀏 󰀏󰁂U󰀏󰀣󰀏step󰁃󰀏PRG󰀏T󰀦󰀦󰀏7KH󰀏EDVLF󰀏DOJRULWKP󰀏LV󰀺

󰀔󰀕 &RPPXQLFDWRUV

,I󰀏ZH󰀏WU\\󰀏WR󰀏LPSOHPHQW󰀏)R[󰁍V󰀏DOJRULWKP󰀢󰀏LW󰀏EHFRPHV󰀏DSSDUHQW󰀏WKDW󰀏RXU󰀏ZRUN󰀏ZLOO󰀏EH󰀏JUHDWO\\󰀏IDFLOLWDWHG

LI󰀏ZH󰀏FDQ󰀏WUHDW󰀏FHUWDLQ󰀏VXEVHWV󰀏RI󰀏SURFHVVHV󰀏DV󰀏D󰀏FRPPXQLFDWLRQ󰀏XQLYHUVH󰀏󰀜󰀜󰀜󰀏DW󰀏OHDVW󰀏RQ󰀏D󰀏WHPSRUDU\\EDVLV󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏LQ󰀏WKH󰀏SVHXGR󰀜FRGH

1. In each row of processes broadcast the submatrix

chosen in that row to the other processes in that row.

LW󰀏ZRXOG󰀏EH󰀏XVHIXO󰀏WR󰀏WUHDW󰀏HDFK󰀏URZ󰀏RI󰀏SURFHVVHV󰀏DV󰀏D󰀏FRPPXQLFDWLRQ󰀏XQLYHUVH󰀢󰀏ZKLOH󰀏LQ󰀏WKHVWDWHPHQW

1. On each process, send the submatrix of B to theprocess directly above. (On processes in the firstrow, send the submatrix to the last row.)

LW󰀏ZRXOG󰀏EH󰀏XVHIXO󰀏WR󰀏WUHDW󰀏HDFK󰀏FROXPQ󰀏RI󰀏SURFHVVHV󰀏DV󰀏D󰀏FRPPXQLFDWLRQ󰀏XQLYHUVH󰀦

7KH󰀏PHFKDQLVP󰀏WKDW󰀏03,󰀏SURYLGHV󰀏IRU󰀏WUHDWLQJ󰀏D󰀏VXEVHW󰀏RI󰀏SURFHVVHV󰀏DV󰀏D󰀏󰀳FRPPXQLFDWLRQ󰁍󰀳󰀏XQLYHUVHLV󰀏WKH󰀏FRPPXQLFDWRU󰀦󰀏8S󰀏WR󰀏QRZ󰀢󰀏ZH󰁍YH󰀏EHHQ󰀏ORRVHO\\󰀏GHILQLQJ󰀏D󰀏FRPPXQLFDWRU󰀏DV󰀏D󰀏FROOHFWLRQ󰀏RISURFHVVHV󰀏WKDW󰀏FDQ󰀏VHQG󰀏PHVVDJHV󰀏WR󰀏HDFK󰀏RWKHU󰀦󰀏+RZHYHU󰀢󰀏QRZ󰀏WKDW󰀏ZH󰀏ZDQW󰀏WR󰀏FRQVWUXFW󰀏RXU󰀏RZQ

FRPPXQLFDWRUV󰀢󰀏ZH󰀏ZLOO󰀏QHHG󰀏D󰀏PRUH󰀏FDUHIXO󰀏GLVFXVVLRQ󰀦

,Q󰀏03,󰀢󰀏WKHUH󰀏DUH󰀏WZR󰀏W\\SHV󰀏RI󰀏FRPPXQLFDWRUV󰀺󰀏LQWUD󰀦FRPPXQLFDWRUV󰀏DQG󰀏LQWHU󰀦FRPPXQLFDWRUV󰀦

,QWUD󰀜FRPPXQLFDWRUV󰀏DUH󰀏HVVHQWLDOO\\󰀏D󰀏FROOHFWLRQ󰀏RI󰀏SURFHVVHV󰀏WKDW󰀏FDQ󰀏VHQG󰀏PHVVDJHV󰀏WR󰀏HDFK󰀏RWKHUDQG󰀏HQJDJH󰀏LQ󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏RSHUDWLRQV󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏MPI_COMM_WORLD󰀏LV󰀏DQLQWUD󰀜FRPPXQLFDWRU󰀢󰀏DQG󰀏ZH󰀏ZRXOG󰀏OLNH󰀏IRU󰀏HDFK󰀏URZ󰀏DQG󰀏HDFK󰀏FROXPQ󰀏RI󰀏SURFHVVHV󰀏LQ󰀏)R[󰁍VDOJRULWKP󰀏WR󰀏IRUP󰀏DQ󰀏LQWUD󰀜FRPPXQLFDWRU󰀦󰀏,QWHU󰀜FRPPXQLFDWRUV󰀢󰀏DV󰀏WKH󰀏QDPH󰀏LPSOLHV󰀢󰀏DUH󰀏XVHG󰀏IRUVHQGLQJ󰀏PHVVDJHV󰀏EHWZHHQ󰀏SURFHVVHV󰀏EHORQJLQJ󰀏WR󰀏GLVMRLQW󰀏LQWUD󰀜FRPPXQLFDWRUV󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏DQLQWHU󰀜FRPPXQLFDWRU󰀏ZRXOG󰀏EH󰀏XVHIXO󰀏LQ󰀏DQ󰀏HQYLURQPHQW󰀏WKDW󰀏DOORZHG󰀏RQH󰀏WR󰀏G\\QDPLFDOO\\󰀏FUHDWHSURFHVVHV󰀺󰀏D󰀏QHZO\\󰀏FUHDWHG󰀏VHW󰀏RI󰀏SURFHVVHV󰀏WKDW󰀏IRUPHG󰀏DQ󰀏LQWUD󰀜FRPPXQLFDWRU󰀏FRXOG󰀏EH󰀏OLQNHG󰀏WRWKH󰀏RULJLQDO󰀏VHW󰀏RI󰀏SURFHVVHV󰀏󰁂H󰀦J󰀦󰀢󰀏MPI_COMM_WORLD󰁃󰀏E\\󰀏DQ󰀏LQWHU󰀜FRPPXQLFDWRU󰀦󰀏:H󰀏ZLOO󰀏RQO\\GLVFXVV󰀏LQWUD󰀜FRPPXQLFDWRUV󰀦󰀏7KH󰀏LQWHUHVWHG󰀏UHDGHU󰀏LV󰀏UHIHUUHG󰀏WR󰀏>󰀬@󰀏IRU󰀏GHWDLOV󰀏RQ󰀏WKH󰀏XVH󰀏RI󰀏LQWHU󰀜FRPPXQLFDWRUV󰀦

$󰀏PLQLPDO󰀏󰁂LQWUD󰀜󰁃FRPPXQLFDWRU󰀏LV󰀏FRPSRVHG󰀏RI

• D󰀏*URXS󰀢󰀏DQG• D󰀏&RQWH[W󰀦

$󰀏JURXS󰀏LV󰀏DQ󰀏RUGHUHG󰀏FROOHFWLRQ󰀏RI󰀏SURFHVVHV󰀦󰀏,I󰀏D󰀏JURXS󰀏FRQVLVWV󰀏RI󰀏S󰀏SURFHVVHV󰀢󰀏HDFK󰀏SURFHVV󰀏LQ󰀏WKHJURXS󰀏LV󰀏DVVLJQHG󰀏D󰀏XQLTXH󰀏UDQN󰀢󰀏ZKLFK󰀏LV󰀏MXVW󰀏D󰀏QRQQHJDWLYH󰀏LQWHJHU󰀏LQ󰀏WKH󰀏UDQJH󰀏󰀶󰀢󰀏󰀩󰀢󰀏󰀦󰀦󰀦󰀦󰀏󰀢󰀏S󰀂󰀜󰀏󰀩󰀦󰀏$FRQWH[W󰀏FDQ󰀏EH󰀏WKRXJKW󰀏RI󰀏DV󰀏D󰀏V\\VWHP󰀜GHILQHG󰀏WDJ󰀏WKDW󰀏LV󰀏DWWDFKHG󰀏WR󰀏D󰀏JURXS󰀦󰀏6R󰀏WZR󰀏SURFHVVHV󰀏WKDWEHORQJ󰀏WR󰀏WKH󰀏VDPH󰀏JURXS󰀏DQG󰀏WKDW󰀏XVH󰀏WKH󰀏VDPH󰀏FRQWH[W󰀏FDQ󰀏FRPPXQLFDWH󰀦󰀏7KLV󰀏SDLULQJ󰀏RI󰀏D󰀏JURXSZLWK󰀏D󰀏FRQWH[W󰀏LV󰀏WKH󰀏PRVW󰀏EDVLF󰀏IRUP󰀏RI󰀏D󰀏FRPPXQLFDWRU󰀦󰀏2WKHU󰀏GDWD󰀏FDQ󰀏EH󰀏DVVRFLDWHG󰀏WR󰀏DFRPPXQLFDWRU󰀦󰀏,Q󰀏SDUWLFXODU󰀢󰀏D󰀏VWUXFWXUH󰀏RU󰀏WRSRORJ\\󰀏FDQ󰀏EH󰀏LPSRVHG󰀏RQ󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏DFRPPXQLFDWRU󰀢󰀏DOORZLQJ󰀏D󰀏PRUH󰀏QDWXUDO󰀏DGGUHVVLQJ󰀏VFKHPH󰀦󰀏:H󰁍OO󰀏GLVFXVV󰀏WRSRORJLHV󰀏LQ󰀏VHFWLRQ󰀏󰀿󰀦󰀭󰀦

󰀔󰀕 :RUNLQJ󰀄ZLWK󰀄*URXSV󰀸󰀄&RQWH[WV󰀸󰀄DQG&RPPXQLFDWRUV

7R󰀏LOOXVWUDWH󰀏WKH󰀏EDVLFV󰀏RI󰀏ZRUNLQJ󰀏ZLWK󰀏FRPPXQLFDWRUV󰀢󰀏OHW󰁍V󰀏FUHDWH󰀏D󰀏FRPPXQLFDWRU󰀏ZKRVHXQGHUO\\LQJ󰀏JURXS󰀏FRQVLVWV󰀏RI󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏ILUVW󰀏URZ󰀏RI󰀏RXU󰀏YLUWXDO󰀏JULG󰀦󰀏6XSSRVH󰀏WKDWMPI_COMM_WORLD󰀏FRQVLVWV󰀏RI󰀏S󰀏SURFHVVHV󰀢󰀏ZKHUH󰀏T󰀂󰀏 󰀏S󰀦󰀏/HW󰁍V󰀏DOVR󰀏VXSSRVH󰀏WKDW󰀏 󰀏󰁂[󰁋T󰀢[󰀏PRG󰀏T󰁃󰀦󰀏6R󰀏WKH󰀏ILUVW󰀏URZ󰀏RI󰀏SURFHVVHV󰀏FRQVLVWV󰀏RI󰀏WKH󰀏SURFHVVHV󰀏ZLWK󰀏UDQNV󰀏󰀶󰀢󰀏󰀩󰀢󰀏󰀦󰀦󰀦󰀦󰀢󰀏T󰀂󰀜󰀏󰀩󰀦󰀏󰁂+HUH󰀢󰀏WKHUDQNV󰀏DUH󰀏LQ󰀏03,B&200B:25/'󰀦󰁃󰀏,Q󰀏RUGHU󰀏WR󰀏FUHDWH󰀏WKH󰀏JURXS󰀏RI󰀏RXU󰀏QHZ󰀏FRPPXQLFDWRU󰀢󰀏ZHFDQ󰀏H[HFXWH󰀏WKH󰀏IROORZLQJ󰀏FRGH󰀦PROGRAM ComCrtINCLUDE 'mpif.h'IMPLICIT NONE

integer, parameter :: MAX_PROCS = 100integer preal p_realinteger qinteger my_rank

integer MPI_GROUP_WORLDinteger first_row_groupinteger first_row_comm

integer process_ranks(0:MAX_PROCS-1)integer procinteger testinteger sum

integer my_rank_in_first_rowinteger ierrCC

test = 0

call MPI_INIT( ierr)

call MPI_COMM_SIZE(MPI_COMM_WORLD, p, ierr )call MPI_COMM_RANK(MPI_COMM_WORLD, my_rank, ierr )C

p_real = pq = sqrt(p_real)C

C Make a list of the processes in the new communicator.do proc = 0, q-1

process_ranks(proc) = procenddoC

C Get the group underlying MPI_COMM_WORLD

call MPI_COMM_GROUP(MPI_COMM_WORLD, MPI_GROUP_WORLD, ierr )C

C Create the new group

call MPI_GROUP_INCL(MPI_GROUP_WORLD, q, process_ranks,+ first_row_group, ierr)

C

C Create the new communicator

call MPI_COMM_CREATE(MPI_COMM_WORLD, first_row_group,+ first_row_comm, ierr)C

7KLV󰀏FRGH󰀏SURFHHGV󰀏LQ󰀏D󰀏IDLUO\\󰀏VWUDLJKWIRUZDUG󰀏IDVKLRQ󰀏WR󰀏EXLOG󰀏WKH󰀏QHZ󰀏FRPPXQLFDWRU󰀦󰀏)LUVW󰀏LWFUHDWHV󰀏D󰀏OLVW󰀏RI󰀏WKH󰀏SURFHVVHV󰀏WR󰀏EH󰀏DVVLJQHG󰀏WR󰀏WKH󰀏QHZ󰀏FRPPXQLFDWRU󰀦󰀏7KHQ󰀏LW󰀏FUHDWHV󰀏D󰀏JURXSFRQVLVWLQJ󰀏RI󰀏WKHVH󰀏SURFHVVHV󰀦󰀏7KLV󰀏UHTXLUHG󰀏WZR󰀏FRPPDQGV󰀺󰀏ILUVW󰀏JHW󰀏WKH󰀏JURXS󰀏DVVRFLDWHG󰀏ZLWKMPI_COMM_WORLD󰀢󰀏VLQFH󰀏WKLV󰀏LV󰀏WKH󰀏JURXS󰀏IURP󰀏ZKLFK󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏QHZ󰀏JURXS󰀏ZLOO󰀏EHWDNHQ󰀼󰀏WKHQ󰀏FUHDWH󰀏WKH󰀏JURXS󰀏ZLWK󰀏MPI_Group_incl󰀦󰀏)LQDOO\\󰀢󰀏WKH󰀏DFWXDO󰀏FRPPXQLFDWRU󰀏LV󰀏FUHDWHGZLWK󰀏D󰀏FDOO󰀏WR󰀏MPI_Comm_create󰀦󰀏7KH󰀏FDOO󰀏WR󰀏MPI_Comm_create󰀏LPSOLFLWO\\󰀏DVVRFLDWHV󰀏D󰀏FRQWH[WZLWK󰀏WKH󰀏QHZ󰀏JURXS󰀦󰀏7KH󰀏UHVXOW󰀏LV󰀏WKH󰀏FRPPXQLFDWRU󰀏first_row_comm󰀦󰀏1RZ󰀏WKH󰀏SURFHVVHV󰀏LQfirst_row_comm󰀏FDQ󰀏SHUIRUP󰀏FROOHFWLYH󰀏FRPPXQLFDWLRQ󰀏RSHUDWLRQV󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏SURFHVV󰀏󰀶󰀏󰁂LQfirst_row_group󰁃󰀏FDQ󰀏EURDGFDVW󰀏$󰀉󰀉󰀏WR󰀏WKH󰀏RWKHU󰀏SURFHVVHV󰀏LQ󰀏first_row_group󰀦integer my_rank_in_first_row

real, allocatable , dimension( :,: ) :: A_00if (my_rank < q) then

call MPI_COMM_RANK(first_row_comm,+ my_rank_in_first_row, ierr)

! Allocate space for A_00, order n_bar.allocate (A_00(n_bar,n_bar))

if (my_rank_in_first_row == 0) then! initialize A_00endif

call MPI_BCAST( A_00, n_bar*n_bar, MPI_INTEGER, 0,+ first_row_comm, ierr)endif

*URXSV󰀏DQG󰀏FRPPXQLFDWRUV󰀏DUH󰀏RSDTXH󰀂REMHFWV󰀦󰀏)URP󰀏D󰀏SUDFWLFDO󰀏VWDQGSRLQW󰀢󰀏WKLV󰀏PHDQV󰀏WKDW󰀏WKH󰀏GHWDLOVRI󰀏WKHLU󰀏LQWHUQDO󰀏UHSUHVHQWDWLRQ󰀏GHSHQG󰀏RQ󰀏WKH󰀏SDUWLFXODU󰀏LPSOHPHQWDWLRQ󰀏RI󰀏03,󰀢󰀏DQG󰀢󰀏DV󰀏DFRQVHTXHQFH󰀢󰀏WKH\\󰀏FDQQRW󰀏EH󰀏GLUHFWO\\󰀏DFFHVVHG󰀏E\\󰀏WKH󰀏XVHU󰀦󰀏5DWKHU󰀏WKH󰀏XVHU󰀏DFFHVVHV󰀏D󰀏KDQGOH󰀏WKDWUHIHUHQFHV󰀏WKH󰀏RSDTXH󰀏REMHFW󰀢󰀏DQG󰀏WKH󰀏RSDTXH󰀏REMHFWV󰀏DUH󰀏PDQLSXODWHG󰀏E\\󰀏VSHFLDO󰀏03,󰀏IXQFWLRQV󰀢󰀏IRUH[DPSOH󰀢󰀏MPI_Comm_create󰀢󰀏MPI_Group_incl󰀢󰀏DQG󰀏MPI_Comm_group󰀦

&RQWH[WV󰀏DUH󰀏QRW󰀏H[SOLFLWO\\󰀏XVHG󰀏LQ󰀏DQ\\󰀏03,󰀏IXQFWLRQV󰀦󰀏5DWKHU󰀏WKH\\󰀏DUH󰀏LPSOLFLWO\\󰀏DVVRFLDWHG󰀏ZLWKJURXSV󰀏ZKHQ󰀏FRPPXQLFDWRUV󰀏DUH󰀏FUHDWHG󰀦󰀏7KH󰀏V\\QWD[󰀏RI󰀏WKH󰀏FRPPDQGV󰀏ZH󰀏XVHG󰀏WR󰀏FUHDWHfirst_row_comm󰀏LV󰀏IDLUO\\󰀏VHOI󰀜H[SODQDWRU\\󰀦󰀏7KH󰀏ILUVW󰀏FRPPDQGMPI_Comm_group(comm, group, ierror)

integer comm, group, ierror

VLPSO\\󰀏UHWXUQV󰀏WKH󰀏JURXS󰀏XQGHUO\\LQJ󰀏WKH󰀏FRPPXQLFDWRU󰀏comm󰀦7KH󰀏VHFRQG󰀏FRPPDQG

MPI_Group_incl(old_group, new_group_size,+ranks_in_old_group, new_group, ierror)integer old_group, new_group_size,

integer ranks_in_old_group(*), new_group, ierror

FUHDWHV󰀏D󰀏QHZ󰀏JURXS󰀏IURP󰀏D󰀏OLVW󰀏RI󰀏SURFHVVHV󰀏LQ󰀏WKH󰀏H[LVWLQJ󰀏JURXS󰀏old_group󰀦󰀏7KH󰀏QXPEHU󰀏RISURFHVVHV󰀏LQ󰀏WKH󰀏QHZ󰀏JURXS󰀏LV󰀏new_group_size󰀢󰀏DQG󰀏WKH󰀏SURFHVVHV󰀏WR󰀏EH󰀏LQFOXGHG󰀏DUH󰀏OLVWHG󰀏LQranks_in_old_group󰀦󰀏3URFHVV󰀏󰀶󰀏LQ󰀏new_group󰀏KDV󰀏UDQN󰀏ranks_in_old_group(0)󰀏LQ󰀏old_group󰀢SURFHVV󰀏󰀩󰀏LQ󰀏new_group󰀏KDV󰀏UDQN󰀏ranks_in_old_group(1)󰀏LQ󰀏old_group󰀢󰀏HWF󰀦7KH󰀏ILQDO󰀏FRPPDQG

MPI_Comm_create(old_comm, new_group, new_comm, ierror)integer old_comm, new_group, new_comm, ierror

DVVRFLDWHV󰀏D󰀏FRQWH[W󰀏ZLWK󰀏WKH󰀏JURXS󰀏new_group󰀏DQG󰀏FUHDWHV󰀏WKH󰀏FRPPXQLFDWRU󰀏new_comm󰀦󰀏$OO󰀏RIWKH󰀏SURFHVVHV󰀏LQ󰀏new_group󰀏EHORQJ󰀏WR󰀏WKH󰀏JURXS󰀏XQGHUO\\LQJ󰀏old_comm󰀦

7KHUH󰀏LV󰀏DQ󰀏H[WUHPHO\\󰀏LPSRUWDQW󰀏GLVWLQFWLRQ󰀏EHWZHHQ󰀏WKH󰀏ILUVW󰀏WZR󰀏IXQFWLRQV󰀏DQG󰀏WKH󰀏WKLUG󰀦MPI_Comm_group󰀏DQG󰀏MPI_Group_incl󰀢󰀏DUH󰀏ERWK󰀏ORFDO󰀏RSHUDWLRQV󰀦󰀏7KDW󰀏LV󰀢󰀏WKHUH󰀏LV󰀏QRFRPPXQLFDWLRQ󰀏DPRQJ󰀏SURFHVVHV󰀏LQYROYHG󰀏LQ󰀏WKHLU󰀏H[HFXWLRQ󰀦󰀏+RZHYHU󰀢󰀏MPI_Comm_create󰀏LV󰀏DFROOHFWLYH󰀏RSHUDWLRQ󰀦󰀏$OO󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏old_comm󰀏PXVW󰀏FDOO󰀏MPI_Comm_create󰀏ZLWK󰀏WKH󰀏VDPHDUJXPHQWV󰀦󰀏7KH󰀏6WDQGDUG󰀏>󰀬@󰀏JLYHV󰀏WKUHH󰀏UHDVRQV󰀏IRU󰀏WKLV󰀺

󰀩󰀦 ,W󰀏DOORZV󰀏WKH󰀏LPSOHPHQWDWLRQ󰀏WR󰀏OD\\HU󰀏MPI_Comm_create󰀏RQ󰀏WRS󰀏RI󰀏UHJXODU󰀏FROOHFWLYHFRPPXQLFDWLRQV󰀦󰀩󰀦 ,W󰀏SURYLGHV󰀏DGGLWLRQDO󰀏VDIHW\\󰀦

󰀩󰀦 ,W󰀏SHUPLWV󰀏LPSOHPHQWDWLRQV󰀏WR󰀏DYRLG󰀏FRPPXQLFDWLRQ󰀏UHODWHG󰀏WR󰀏FRQWH[W󰀏FUHDWLRQ󰀦

1RWH󰀏WKDW󰀏VLQFH󰀏MPI_Comm_create󰀏LV󰀏FROOHFWLYH󰀢󰀏LW󰀏ZLOO󰀏EHKDYH󰀢󰀏LQ󰀏WHUPV󰀏RI󰀏WKH󰀏GDWD󰀏WUDQVPLWWHG󰀢DV󰀏LI󰀏LW󰀏V\\QFKURQL]HV󰀦󰀏,Q󰀏SDUWLFXODU󰀢󰀏LI󰀏VHYHUDO󰀏FRPPXQLFDWRUV󰀏DUH󰀏EHLQJ󰀏FUHDWHG󰀢󰀏WKH\\󰀏PXVW󰀏EH󰀏FUHDWHGLQ󰀏WKH󰀏VDPH󰀏RUGHU󰀏RQ󰀏DOO󰀏WKH󰀏SURFHVVHV󰀦

󰀔󰀕 03,B&RPPBVSOLW

,Q󰀏RXU󰀏PDWUL[󰀏PXOWLSOLFDWLRQ󰀏SURJUDP󰀏ZH󰀏QHHG󰀏WR󰀏FUHDWH󰀏PXOWLSOH󰀏FRPPXQLFDWRUV󰀏󰀜󰀜󰀜

RQH󰀏IRU󰀏HDFK󰀏URZ󰀏RI󰀏SURFHVVHV󰀏DQG󰀏RQH󰀏IRU󰀏HDFK󰀏FROXPQ󰀦󰀏7KLV󰀏ZRXOG󰀏EH󰀏DQ󰀏H[WUHPHO\\WHGLRXV󰀏SURFHVV󰀏LI󰀏S󰀏ZHUH󰀏ODUJH󰀏DQG󰀏ZH󰀏KDG󰀏WR󰀏FUHDWH󰀏HDFK󰀏FRPPXQLFDWRU󰀏XVLQJ󰀏WKHWKUHH󰀏IXQFWLRQV󰀏GLVFXVVHG󰀏LQ󰀏WKH󰀏SUHYLRXV󰀏VHFWLRQ󰀦󰀏)RUWXQDWHO\\󰀢󰀏03,󰀏SURYLGHV󰀏DIXQFWLRQ󰀢󰀏MPI_Comm_split󰀏WKDW󰀏FDQ󰀏FUHDWH󰀏VHYHUDO󰀏FRPPXQLFDWRUV󰀏VLPXOWDQHRXVO\\󰀦$V󰀏DQ󰀏H[DPSOH󰀏RI󰀏LWV󰀏XVH󰀢󰀏ZH󰁍OO󰀏FUHDWH󰀏RQH󰀏FRPPXQLFDWRU󰀏IRU󰀏HDFK󰀏URZ󰀏RI󰀏SURFHVVHV󰀦integer my_row_comm

integer my_row

C my_rank is rank in MPI_COMM_WORLD.C q*q = p

my_row = my_rank/q

call MPI_COMM_SPLIT(MPI_COMM_WORLD, my_row, my_rank,+ my_row_comm, ierr)󰀏

7KH󰀏VLQJOH󰀏FDOO󰀏WR󰀏MPI_Comm_split󰀏FUHDWHV󰀏T󰀏QHZ󰀏FRPPXQLFDWRUV󰀢󰀏DOO󰀏RI󰀏WKHPKDYLQJ󰀏WKH󰀏VDPH󰀏QDPH󰀢󰀏my_row_comm󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏LI󰀏S󰀏 󰀏󰀪󰀢󰀏WKH󰀏JURXSXQGHUO\\LQJ󰀏my_row_comm󰀏ZLOO󰀏FRQVLVW󰀏RI󰀏WKH󰀏SURFHVVHV󰀏󰀶󰀢󰀏󰀩󰀢󰀏DQG󰀏󰀽󰀏RQ󰀏SURFHVVHV󰀏󰀶󰀢󰀩󰀢󰀏DQG󰀏󰀽󰀦󰀏2Q󰀏SURFHVVHV󰀏󰀫󰀢󰀏󰀬󰀢󰀏DQG󰀏󰀭󰀢󰀏WKH󰀏JURXS󰀏XQGHUO\\LQJ󰀏my_row_comm󰀏ZLOOFRQVLVW󰀏RI󰀏WKH󰀏SURFHVVHV󰀏󰀫󰀢󰀏󰀬󰀢󰀏DQG󰀏󰀭󰀢󰀏DQG󰀏RQ󰀏SURFHVVHV󰀏󰀿󰀢󰀏󰀾󰀢󰀏DQG󰀏󰀷󰀏LW󰀏ZLOO󰀏FRQVLVW󰀏RISURFHVVHV󰀏󰀿󰀢󰀏󰀾󰀢󰀏DQG󰀏󰀷󰀦

7KH󰀏V\\QWD[󰀏RI󰀏MPI_Comm_split󰀏LV

MPI_COMM_SPLIT(old_comm, split_key, rank_key,+ new_comm, ierror)

integer old_comm, split_key, rank_key, new_comm,ierror

,W󰀏FUHDWHV󰀏D󰀏QHZ󰀏FRPPXQLFDWRU󰀏IRU󰀏HDFK󰀏YDOXH󰀏RI󰀏split_key󰀦󰀏3URFHVVHV󰀏ZLWK󰀏WKH󰀏VDPHYDOXH󰀏RI󰀏split_key󰀏IRUP󰀏D󰀏QHZ󰀏JURXS󰀦󰀏7KH󰀏UDQN󰀏LQ󰀏WKH󰀏QHZ󰀏JURXS󰀏LV󰀏GHWHUPLQHG󰀏E\\WKH󰀏YDOXH󰀏RI󰀏rank_key󰀦󰀏,I󰀏SURFHVV󰀏$󰀏DQG󰀏SURFHVV󰀏%󰀏FDOO󰀏MPI_Comm_split󰀏ZLWK󰀏WKHVDPH󰀏YDOXH󰀏RI󰀏split_key󰀢󰀏DQG󰀏WKH󰀏rank_key󰀏DUJXPHQW󰀏SDVVHG󰀏E\\󰀏SURFHVV󰀏$󰀏LV󰀏OHVVWKDQ󰀏WKDW󰀏SDVVHG󰀏E\\󰀏SURFHVV󰀏%󰀢󰀏WKHQ󰀏WKH󰀏UDQN󰀏RI󰀏$󰀏LQ󰀏WKH󰀏JURXS󰀏XQGHUO\\LQJnew_comm󰀏ZLOO󰀏EH󰀏OHVV󰀏WKDQ󰀏WKH󰀏UDQN󰀏RI󰀏SURFHVV󰀏%󰀦󰀏,I󰀏WKH\\󰀏FDOO󰀏WKH󰀏IXQFWLRQ󰀏ZLWK󰀏WKHVDPH󰀏YDOXH󰀏RI󰀏rank_key󰀢󰀏WKH󰀏V\\VWHP󰀏ZLOO󰀏DUELWUDULO\\󰀏DVVLJQ󰀏RQH󰀏RI󰀏WKH󰀏SURFHVVHV󰀏DORZHU󰀏UDQN󰀦

MPI_Comm_split󰀏LV󰀏D󰀏FROOHFWLYH󰀏FDOO󰀢󰀏DQG󰀏LW󰀏PXVW󰀏EH󰀏FDOOHG󰀏E\\󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQold_comm󰀦󰀏7KH󰀏IXQFWLRQ󰀏FDQ󰀏EH󰀏XVHG󰀏HYHQ󰀏LI󰀏WKH󰀏XVHU󰀏GRHVQ󰁍W󰀏ZLVK󰀏WR󰀏DVVLJQ󰀏HYHU\\SURFHVV󰀏WR󰀏D󰀏QHZ󰀏FRPPXQLFDWRU󰀦󰀏7KLV󰀏FDQ󰀏EH󰀏DFFRPSOLVKHG󰀏E\\󰀏SDVVLQJ󰀏WKH󰀏SUHGHILQHGFRQVWDQW󰀏MPI_UNDEFINED󰀏DV󰀏WKH󰀏split_key󰀏DUJXPHQW󰀦󰀏3URFHVVHV󰀏GRLQJ󰀏WKLV󰀏ZLOOKDYH󰀏WKH󰀏SUHGHILQHG󰀏YDOXH󰀏MPI_COMM_NULL󰀏UHWXUQHG󰀏LQ󰀏new_comm󰀦

󰀙󰀕 7RSRORJLHV

5HFROOHFW󰀏WKDW󰀏LW󰀏LV󰀏SRVVLEOH󰀏WR󰀏DVVRFLDWH󰀏DGGLWLRQDO󰀏LQIRUPDWLRQ󰀏󰀜󰀜󰀜󰀏LQIRUPDWLRQ󰀏EH\\RQG󰀏WKH󰀏JURXS󰀏DQG

FRQWH[W󰀏󰀜󰀜󰀜󰀏ZLWK󰀏D󰀏FRPPXQLFDWRU󰀦󰀏7KLV󰀏DGGLWLRQDO󰀏LQIRUPDWLRQ󰀏LV󰀏VDLG󰀏WR󰀏EH󰀏FDFKHG󰀏ZLWK󰀏WKHFRPPXQLFDWRU󰀢󰀏DQG󰀏RQH󰀏RI󰀏WKH󰀏PRVW󰀏LPSRUWDQW󰀏SLHFHV󰀏RI󰀏LQIRUPDWLRQ󰀏WKDW󰀏FDQ󰀏EH󰀏FDFKHG󰀏ZLWK󰀏DFRPPXQLFDWRU󰀏LV󰀏D󰀏WRSRORJ\\󰀦󰀏,Q󰀏03,󰀢󰀏D󰀏WRSRORJ\\󰀏LV󰀏MXVW󰀏D󰀏PHFKDQLVP󰀏IRU󰀏DVVRFLDWLQJ󰀏GLIIHUHQWDGGUHVVLQJ󰀏VFKHPHV󰀏ZLWK󰀏WKH󰀏SURFHVVHV󰀏EHORQJLQJ󰀏WR󰀏D󰀏JURXS󰀦󰀏1RWH󰀏WKDW󰀏03,󰀏WRSRORJLHV󰀏DUH󰀏YLUWXDOWRSRORJLHV󰀏󰀜󰀜󰀜󰀏WKHUH󰀏PD\\󰀏EH󰀏QR󰀏VLPSOH󰀏UHODWLRQ󰀏EHWZHHQ󰀏WKH󰀏SURFHVV󰀏VWUXFWXUH󰀏GHILQHG󰀏E\\󰀏D󰀏YLUWXDOWRSRORJ\\󰀢󰀏DQG󰀏WKH󰀏DFWXDO󰀏XQGHUO\\LQJ󰀏SK\\VLFDO󰀏VWUXFWXUH󰀏RI󰀏WKH󰀏SDUDOOHO󰀏PDFKLQH󰀦

7KHUH󰀏DUH󰀏HVVHQWLDOO\\󰀏WZR󰀏W\\SHV󰀏RI󰀏YLUWXDO󰀏WRSRORJLHV󰀏WKDW󰀏FDQ󰀏EH󰀏FUHDWHG󰀏LQ󰀏03,󰀏󰀜󰀜󰀜󰀏D󰀏FDUWHVLDQ󰀏RU󰀏JULGWRSRORJ\\󰀏DQG󰀏D󰀏JUDSK󰀏WRSRORJ\\󰀦󰀏&RQFHSWXDOO\\󰀢󰀏WKH󰀏IRUPHU󰀏LV󰀏VXEVXPHG󰀏E\\󰀏WKH󰀏ODWWHU󰀦󰀏+RZHYHU󰀢EHFDXVH󰀏RI󰀏WKH󰀏LPSRUWDQFH󰀏RI󰀏JULGV󰀏LQ󰀏DSSOLFDWLRQV󰀢󰀏WKHUH󰀏LV󰀏D󰀏VHSDUDWH󰀏FROOHFWLRQ󰀏RI󰀏IXQFWLRQV󰀏LQ󰀏03,ZKRVH󰀏SXUSRVH󰀏LV󰀏WKH󰀏PDQLSXODWLRQ󰀏RI󰀏YLUWXDO󰀏JULGV󰀦

,Q󰀏)R[󰁍V󰀏DOJRULWKP󰀏ZH󰀏ZLVK󰀏WR󰀏LGHQWLI\\󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏MPI_COMM_WORLD󰀏ZLWK󰀏WKHFRRUGLQDWHV󰀏RI󰀏D󰀏VTXDUH󰀏JULG󰀢󰀏DQG󰀏HDFK󰀏URZ󰀏DQG󰀏HDFK󰀏FROXPQ󰀏RI󰀏WKH󰀏JULG󰀏QHHGV󰀏WR󰀏IRUP󰀏LWV󰀏RZQFRPPXQLFDWRU󰀦󰀏/HW󰁍V󰀏ORRN󰀏DW󰀏RQH󰀏PHWKRG󰀏IRU󰀏EXLOGLQJ󰀏WKLV󰀏VWUXFWXUH󰀦

:H󰀏EHJLQ󰀏E\\󰀏DVVRFLDWLQJ󰀏D󰀏VTXDUH󰀏JULG󰀏VWUXFWXUH󰀏ZLWK󰀏MPI_COMM_WORLD󰀦󰀏,Q󰀏RUGHU󰀏WR󰀏GR󰀏WKLVZH󰀏QHHG󰀏WR󰀏VSHFLI\\󰀏WKH󰀏IROORZLQJ󰀏LQIRUPDWLRQ󰀦

󰀩󰀦 7KH󰀏QXPEHU󰀏RI󰀏GLPHQVLRQV󰀏LQ󰀏WKH󰀏JULG󰀦󰀏:H󰀏KDYH󰀏󰀽󰀦

󰀩󰀦 7KH󰀏VL]H󰀏RI󰀏HDFK󰀏GLPHQVLRQ󰀦󰀏,Q󰀏RXU󰀏FDVH󰀢󰀏WKLV󰀏LV󰀏MXVW󰀏WKH󰀏QXPEHU󰀏RI󰀏URZV󰀏DQG󰀏WKH󰀏QXPEHU󰀏RI

FROXPQV󰀦󰀏:H󰀏KDYH󰀏T󰀏URZV󰀏DQG󰀏T󰀏FROXPQV󰀦󰀩󰀦 7KH󰀏SHULRGLFLW\\󰀏RI󰀏HDFK󰀏GLPHQVLRQ󰀦󰀏,Q󰀏RXU󰀏FDVH󰀢󰀏WKLV󰀏LQIRUPDWLRQ󰀏VSHFLILHV󰀏ZKHWKHU󰀏WKH󰀏ILUVW

HQWU\\󰀏LQ󰀏HDFK󰀏URZ󰀏RU󰀏FROXPQ󰀏LV󰀏󰀳DGMDFHQW󰀳󰀏WR󰀏WKH󰀏ODVW󰀏HQWU\\󰀏LQ󰀏WKDW󰀏URZ󰀏RU󰀏FROXPQ󰀢UHVSHFWLYHO\\󰀦󰀏6LQFH󰀏ZH󰀏ZDQW󰀏D󰀏󰀳FLUFXODU󰀳󰁍󰀏VKLIW󰀏RI󰀏WKH󰀏VXEPDWULFHV󰀏LQ󰀏HDFK󰀏FROXPQ󰀢󰀏ZH󰀏ZDQW󰀏WKHVHFRQG󰀏GLPHQVLRQ󰀏WR󰀏EH󰀏SHULRGLF󰀦󰀏,W󰁍V󰀏XQLPSRUWDQW󰀏ZKHWKHU󰀏WKH󰀏ILUVW󰀏GLPHQVLRQ󰀏LV󰀏SHULRGLF󰀦󰀩󰀦 )LQDOO\\󰀢󰀏03,󰀏JLYHV󰀏WKH󰀏XVHU󰀏WKH󰀏RSWLRQ󰀏RI󰀏DOORZLQJ󰀏WKH󰀏V\\VWHP󰀏WR󰀏RSWLPL]H󰀏WKH󰀏PDSSLQJ󰀏RI󰀏WKHJULG󰀏RI󰀏SURFHVVHV󰀏WR󰀏WKH󰀏XQGHUO\\LQJ󰀏SK\\VLFDO󰀏SURFHVVRUV󰀏E\\󰀏SRVVLEO\\󰀏UHRUGHULQJ󰀏WKH󰀏SURFHVVHVLQ󰀏WKH󰀏JURXS󰀏XQGHUO\\LQJ󰀏WKH󰀏FRPPXQLFDWRU󰀦󰀏6LQFH󰀏ZH󰀏GRQ󰁍W󰀏QHHG󰀏WR󰀏SUHVHUYH󰀏WKH󰀏RUGHULQJ󰀏RIWKH󰀏SURFHVVHV󰀏LQ󰀏MPI_COMM_WORLD󰀢󰀏ZH󰀏VKRXOG󰀏DOORZ󰀏WKH󰀏V\\VWHP󰀏WR󰀏UHRUGHU󰀦+DYLQJ󰀏PDGH󰀏DOO󰀏WKHVH󰀏GHFLVLRQV󰀢󰀏ZH󰀏VLPSO\\󰀏H[HFXWH󰀏WKH󰀏IROORZLQJ󰀏FRGH󰀦integer grid_comminteger dim_sizes(0:1)logical wrap_around(0:1)logical reorder = .TRUE.dim_sizes(0) = qdim_sizes(1) = q

wrap_around(0) = .TRUE.wrap_around(1) = .TRUE.

call MPI_CART_CREATE(MPI_COMM_WORLD, 2, dim_sizes,+ wrap_around, reorder, grid_comm, ierr)

$IWHU󰀏H[HFXWLQJ󰀏WKLV󰀏FRGH󰀢󰀏WKH󰀏FRPPXQLFDWRU󰀏grid_comm󰀏ZLOO󰀏FRQWDLQ󰀏DOO󰀏WKH󰀏SURFHVVHV󰀏LQMPI_COMM_WORLD󰀏󰁂SRVVLEO\\󰀏UHRUGHUHG󰁃󰀢󰀏DQG󰀏LW󰀏ZLOO󰀏KDYH󰀏D󰀏WZR󰀜GLPHQVLRQDO󰀏FDUWHVLDQFRRUGLQDWH󰀏V\\VWHP󰀏DVVRFLDWHG󰀦󰀏,Q󰀏RUGHU󰀏IRU󰀏D󰀏SURFHVV󰀏WR󰀏GHWHUPLQH󰀏LWV󰀏FRRUGLQDWHV󰀢󰀏LW󰀏VLPSO\\󰀏FDOOV󰀏WKHIXQFWLRQ󰀏MPI_Cart_coords󰀏󰀺integer coordinates(0:1)integer my_grid_rank

call MPI_COMM_RANK(grid_comm, my_grid_rank, ierr)call MPI_CART_COORDS(grid_comm, my_grid_rank, 2,+ coordinates, ierr)

1RWLFH󰀏WKDW󰀏ZH󰀏QHHGHG󰀏WR󰀏FDOO󰀏MPI_Comm_rank󰀏LQ󰀏RUGHU󰀏WR󰀏JHW󰀏WKH󰀏SURFHVV󰀏UDQN󰀏LQ󰀏grid_comm󰀦7KLV󰀏ZDV󰀏QHFHVVDU\\󰀏EHFDXVH󰀏LQ󰀏RXU󰀏FDOO󰀏WR󰀏MPI_Cart_create󰀏ZH󰀏VHW󰀏WKH󰀏reorder IODJ󰀏WR󰀏.TRUE.󰀏󰀢DQG󰀏KHQFH󰀏WKH󰀏RULJLQDO󰀏SURFHVV󰀏UDQNLQJ󰀏LQ󰀏MPI_COMM_WORLD PD\\󰀏KDYH󰀏EHHQ󰀏FKDQJHG󰀏LQgrid_comm󰀦

7KH󰀏󰀳LQYHUVH󰀳󰁍󰀏WR󰀏MPI_Cart_coords󰀏LV󰀏MPI_Cart_rank󰀦

call MPI_CART_RANK(grid_comm, coordinates, grid_rank,+ ierr)

integer grid_comm, coordinates(*), grid_rank, ierr

*LYHQ󰀏WKH󰀏FRRUGLQDWHV󰀏RI󰀏D󰀏SURFHVV󰀢󰀏MPI_Cart_rank󰀏UHWXUQV󰀏WKH󰀏UDQN󰀏RI󰀏WKH󰀏SURFHVV󰀏LQ󰀏LWV󰀏WKLUGSDUDPHWHU󰀏process_rank󰀦7KH󰀏V\\QWD[󰀏RI󰀏MPI_Cart_create󰀏LV

call MPI_CART_CREATE(old_comm, number_of_dims, dim_sizes,+ periods, reorder, cart_comm, ierror)

integer old_comm, number_of_dims, dim_sizes(*)logical periods(*), reorderinteger cart_comm, ierror

MPI_Cart_create󰀏FUHDWHV󰀏D󰀏QHZ󰀏FRPPXQLFDWRU󰀢󰀏cart_comm󰀏E\\󰀏FDFKLQJ󰀏D󰀏FDUWHVLDQ󰀏WRSRORJ\\󰀏ZLWKold_comm󰀦󰀏,QIRUPDWLRQ󰀏RQ󰀏WKH󰀏VWUXFWXUH󰀏RI󰀏WKH󰀏FDUWHVLDQ󰀏WRSRORJ\\󰀏LV󰀏FRQWDLQHG󰀏LQ󰀏WKH󰀏SDUDPHWHUVnumber_of_dims󰀢󰀏dim_sizes󰀢󰀏DQG󰀏periods󰀦󰀏7KH󰀏ILUVW󰀏RI󰀏WKHVH󰀢󰀏number_of_dims󰀢󰀏FRQWDLQV󰀏WKHQXPEHU󰀏RI󰀏GLPHQVLRQV󰀏LQ󰀏WKH󰀏FDUWHVLDQ󰀏FRRUGLQDWH󰀏V\\VWHP󰀦󰀏7KH󰀏QH[W󰀏WZR󰀢󰀏dim_sizes󰀏DQG󰀏periods󰀢DUH󰀏DUUD\\V󰀏ZLWK󰀏RUGHU󰀏HTXDO󰀏WR󰀏number_of_dims󰀦󰀏7KH󰀏DUUD\\󰀏dim_sizes󰀏VSHFLILHV󰀏WKH󰀏RUGHU󰀏RI󰀏HDFKGLPHQVLRQ󰀢󰀏DQG󰀏periods󰀏VSHFLILHV󰀏ZKHWKHU󰀏HDFK󰀏GLPHQVLRQ󰀏LV󰀏FLUFXODU󰀏RU󰀏OLQHDU󰀦

7KH󰀏SURFHVVHV󰀏LQ󰀏cart_comm󰀏DUH󰀏UDQNHG󰀏LQ󰀏URZ󰀦PDMRU󰀏RUGHU󰀦󰀏7KDW󰀏LV󰀢󰀏WKH󰀏ILUVW󰀏URZ󰀏FRQVLVWV󰀏RISURFHVVHV󰀏󰀶󰀢󰀏󰀩󰀢󰀏󰀦󰀦󰀦󰀦󰀏󰀢󰀏GLPBVL]HV󰁂󰀶󰁃󰀜󰀩󰀢󰀏WKH󰀏VHFRQG󰀏URZ󰀏FRQVLVWV󰀏RI󰀏SURFHVVHV󰀏GLPBVL]HV󰁂󰀶󰁃󰀢GLPBVL]HV󰁂󰀶󰁃󰀣󰀩󰀢󰀏󰀦󰀦󰀦󰀦󰀏󰀢󰀏󰀽󰁎GLPBVL]HV󰁂󰀶󰁃󰀜󰀩󰀢󰀏HWF󰀦󰀏7KXV󰀏LW󰀏PD\\󰀏EH󰀏DGYDQWDJHRXV󰀏WR󰀏FKDQJH󰀏WKH󰀏UHODWLYHUDQNLQJ󰀏RI󰀏WKH󰀏SURFHVVHV󰀏LQ󰀏old_comm󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏VXSSRVH󰀏WKH󰀏SK\\VLFDO󰀏WRSRORJ\\󰀏LV󰀏D󰀏󰀫󰀏[󰀏󰀫󰀏JULG󰀢DQG󰀏WKH󰀏SURFHVVHV󰀏󰁂QXPEHUV󰁃󰀏LQ󰀏old_comm󰀏DUH󰀏DVVLJQHG󰀏WR󰀏WKH󰀏SURFHVVRUV󰀏󰁂JULG󰀏VTXDUHV󰁃󰀏DV󰀏IROORZV󰀦

󰀫󰀶󰀿

󰀬󰀩󰀾

󰀭󰀽󰀷

&OHDUO\\󰀢󰀏WKH󰀏SHUIRUPDQFH󰀏RI󰀏)R[󰁍V󰀏DOJRULWKP󰀏ZRXOG󰀏EH󰀏LPSURYHG󰀏LI󰀏ZH󰀏UH󰀜QXPEHUHG󰀏WKH󰀏SURFHVVHV󰀦+RZHYHU󰀢󰀏VLQFH󰀏WKH󰀏XVHU󰀏GRHVQ󰁍W󰀏NQRZ󰀏ZKDW󰀏WKH󰀏H[DFW󰀏PDSSLQJ󰀏RI󰀏SURFHVVHV󰀏WR󰀏SURFHVVRUV󰀏LV󰀢󰀏ZHPXVW󰀏OHW󰀏WKH󰀏V\\VWHP󰀏GR󰀏LW󰀏E\\󰀏VHWWLQJ󰀏WKH󰀏reorder󰀏SDUDPHWHU󰀏WR󰀏.TRUE. 󰀦

6LQFH󰀏MPI_Cart_create󰀏FRQVWUXFWV󰀏D󰀏QHZ󰀏FRPPXQLFDWRU󰀢󰀏LW󰀏LV󰀏D󰀏FROOHFWLYH󰀏RSHUDWLRQ󰀦7KH󰀏V\\QWD[󰀏RI󰀏WKH󰀏DGGUHVV󰀏LQIRUPDWLRQ󰀏IXQFWLRQV󰀏LV

MPI_Cart_rank(comm, coordinates, rank, ierror)integer comm, coordinates(*), rank, ierror

MPI_Cart_coords(comm, rank, number_of_dims, coordinates,+ ierror)

integer comm, rank, number_of_dims, coordinates(*), ierror

MPI_Cart_rank󰀏UHWXUQV󰀏WKH󰀏UDQN󰀏LQ󰀏WKH󰀏FDUWHVLDQ󰀏FRPPXQLFDWRU󰀏comm󰀏RI󰀏WKH󰀏SURFHVV󰀏ZLWKFDUWHVLDQ󰀏FRRUGLQDWHV󰀏coordinates󰀦󰀏6R󰀏coordinates LV󰀏DQ󰀏DUUD\\󰀏ZLWK󰀏RUGHU󰀏HTXDO󰀏WR󰀏WKH󰀏QXPEHU󰀏RIGLPHQVLRQV󰀏LQ󰀏WKH󰀏FDUWHVLDQ󰀏WRSRORJ\\󰀏DVVRFLDWHG󰀏ZLWK󰀏comm󰀦󰀏MPI_Cart_coords󰀏LV󰀏WKH󰀏LQYHUVH󰀏WRMPI_Cart_rank󰀺󰀏LW󰀏UHWXUQV󰀏WKH󰀏FRRUGLQDWHV󰀏RI󰀏WKH󰀏SURFHVV󰀏ZLWK󰀏UDQN󰀏rank󰀏LQ󰀏WKH󰀏FDUWHVLDQFRPPXQLFDWRU󰀏comm󰀦󰀏1RWH󰀏WKDW󰀏ERWK󰀏RI󰀏WKHVH󰀏IXQFWLRQV󰀏DUH󰀏ORFDO󰀦

󰀔󰀕 03,B&DUWBVXE

:H󰀏FDQ󰀏DOVR󰀏SDUWLWLRQ󰀏D󰀏JULG󰀏LQWR󰀏JULGV󰀏RI󰀏ORZHU󰀏GLPHQVLRQ󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏ZH󰀏FDQ

FUHDWH󰀏D󰀏FRPPXQLFDWRU󰀏IRU󰀏HDFK󰀏URZ󰀏RI󰀏WKH󰀏JULG󰀏DV󰀏IROORZV󰀦logical varying_coords(0:1)integer row_comm

varying_coords(0) = .FALSE.varying_coords(1) = .TRUE.

call MPI_CART_SUB(grid_comm, varying_coords, row_comm,ierr)

7KH󰀏FDOO󰀏WR󰀏MPI_Cart_sub󰀏FUHDWHV󰀏T󰀏QHZ󰀏FRPPXQLFDWRUV󰀦󰀏7KH󰀏varying_coordsDUJXPHQW󰀏LV󰀏DQ󰀏DUUD\\󰀏RI󰀏ERROHDQ󰀦󰀏,W󰀏VSHFLILHV󰀏ZKHWKHU󰀏HDFK󰀏GLPHQVLRQ󰀏󰀳EHORQJV󰀳󰀏WRWKH󰀏QHZ󰀏FRPPXQLFDWRU󰀦󰀏6LQFH󰀏ZH󰁍UH󰀏FUHDWLQJ󰀏FRPPXQLFDWRUV󰀏IRU󰀏WKH󰀏URZV󰀏RI󰀏WKH󰀏JULG󰀢HDFK󰀏QHZ󰀏FRPPXQLFDWRU󰀏FRQVLVWV󰀏RI󰀏WKH󰀏SURFHVVHV󰀏REWDLQHG󰀏E\\󰀏IL[LQJ󰀏WKH󰀏URZFRRUGLQDWH󰀏DQG󰀏OHWWLQJ󰀏WKH󰀏FROXPQ󰀏FRRUGLQDWH󰀏YDU\\󰀦󰀏+HQFH󰀏ZH󰀏DVVLJQHGvarying_coords󰁂󰀶󰁃󰀏WKH󰀏YDOXH󰀏󰀦)$/6(󰀦󰀏󰀜󰀜󰀜󰀏WKH󰀏ILUVW󰀏FRRUGLQDWH󰀏GRHVQ󰁍W󰀏YDU\\󰀏󰀜󰀜󰀜󰀏DQGZH󰀏DVVLJQHG󰀏varying_coords󰁂󰀩󰁃󰀏WKH󰀏YDOXH󰀏󰀦758(󰀦󰀏󰀜󰀜󰀜󰀏WKH󰀏VHFRQG󰀏FRRUGLQDWH󰀏YDULHV󰀦2Q󰀏HDFK󰀏SURFHVV󰀢󰀏WKH󰀏QHZ󰀏FRPPXQLFDWRU󰀏LV󰀏UHWXUQHG󰀏LQ󰀏row_comm󰀦󰀏,Q󰀏RUGHU󰀏WRFUHDWH󰀏WKH󰀏FRPPXQLFDWRUV󰀏IRU󰀏WKH󰀏FROXPQV󰀢󰀏ZH󰀏VLPSO\\󰀏UHYHUVH󰀏WKH󰀏DVVLJQPHQWV󰀏WR󰀏WKHHQWULHV󰀏LQ󰀏varying_coords󰀦integer col_comm

varying_coords(0) = .TRUE.varying_coords(1) = .FALSE.

call MPI_CART_SUB(grid_comm, varying_coords, row_comm,ierr)

1RWH󰀏WKH󰀏VLPLODULW\\󰀏RI󰀏MPI_Cart_sub󰀏WR󰀏MPI_Comm_split󰀦󰀏7KH\\󰀏SHUIRUP󰀏VLPLODUIXQFWLRQV󰀏󰀜󰀜󰀜󰀏WKH\\󰀏ERWK󰀏SDUWLWLRQ󰀏D󰀏FRPPXQLFDWRU󰀏LQWR󰀏D󰀏FROOHFWLRQ󰀏RI󰀏QHZFRPPXQLFDWRUV󰀦󰀏+RZHYHU󰀢󰀏MPI_Cart_sub󰀏FDQ󰀏RQO\\󰀏EH󰀏XVHG󰀏ZLWK󰀏D󰀏FRPPXQLFDWRUWKDW󰀏KDV󰀏DQ󰀏DVVRFLDWHG󰀏FDUWHVLDQ󰀏WRSRORJ\\󰀢󰀏DQG󰀏WKH󰀏QHZ󰀏FRPPXQLFDWRUV󰀏FDQ󰀏RQO\\󰀏EHFUHDWHG󰀏E\\󰀏IL[LQJ󰀏󰁂RU󰀏YDU\\LQJ󰁃󰀏RQH󰀏RU󰀏PRUH󰀏GLPHQVLRQV󰀏RI󰀏WKH󰀏ROG󰀏FRPPXQLFDWRUV󰀦$OVR󰀏QRWH󰀏WKDW󰀏MPI_Cart_sub󰀏LV󰀢󰀏OLNH󰀏MPI_Comm_split󰀢󰀏D󰀏FROOHFWLYH󰀏RSHUDWLRQ󰀦

󰀙󰀕 ,PSOHPHQWDWLRQ󰀄RI󰀄)R[󰀶V󰀄$OJRULWKP

7R󰀏FRPSOHWH󰀏RXU󰀏GLVFXVVLRQ󰀢󰀏OHW󰁍V󰀏ZULWH󰀏WKH󰀏FRGH󰀏WR󰀏LPSOHPHQW󰀏)R[󰁍V󰀏DOJRULWKP󰀦󰀏)LUVW󰀢󰀏ZH󰁍OO󰀏ZULWH󰀏D

IXQFWLRQ󰀏WKDW󰀏FUHDWHV󰀏WKH󰀏YDULRXV󰀏FRPPXQLFDWRUV󰀏DQG󰀏DVVRFLDWHG󰀏LQIRUPDWLRQ󰀦󰀏6LQFH󰀏WKLV󰀏UHTXLUHV󰀏DODUJH󰀏QXPEHU󰀏RI󰀏YDULDEOHV󰀢󰀏DQG󰀏ZH󰁍OO󰀏EH󰀏XVLQJ󰀏WKLV󰀏LQIRUPDWLRQ󰀏LQ󰀏RWKHU󰀏IXQFWLRQV󰀢󰀏ZH󰁍OO󰀏SXW󰀏LW󰀏LQWR󰀏D)RUWUDQ󰀏󰀪󰀶󰀏GHULYHG󰀏W\\SH󰀏WR󰀏IDFLOLWDWH󰀏SDVVLQJ󰀏LW󰀏DPRQJ󰀏WKH󰀏YDULRXV󰀏IXQFWLRQV󰀦

1RWLFH󰀏WKDW󰀏VLQFH󰀏HDFK󰀏RI󰀏RXU󰀏FRPPXQLFDWRUV󰀏KDV󰀏DQ󰀏DVVRFLDWHG󰀏WRSRORJ\\󰀢󰀏ZH󰀏FRQVWUXFWHG󰀏WKHP󰀏XVLQJWKH󰀏WRSRORJ\\󰀏FRQVWUXFWLRQ󰀏IXQFWLRQV󰀏󰀜󰀜󰀜󰀏MPI_Cart_create󰀏DQG󰀏MPI_Cart_sub󰀏󰀜󰀜󰀜󰀏UDWKHU󰀏WKDQ󰀏WKHPRUH󰀏JHQHUDO󰀏FRPPXQLFDWRU󰀏FRQVWUXFWLRQ󰀏IXQFWLRQV󰀏MPI_Comm_create󰀏DQG󰀏MPI_Comm_split󰀦program myfoxinclude 'mpif.h'IMPLICIT NONEtype GRID_INFO_TYPE

integer p ! Total number of processes.

integer comm ! Communicator for the entire grid.integer row_comm ! Communicator for my row.integer col_comm ! Communicator for my col.integer q ! Order of grid.integer my_row ! My row number.integer my_col ! My column number.

integer my_rank ! My rank in the grid communicator.end type GRID_INFO_TYPE

TYPE (GRID_INFO_TYPE) :: grid_infointeger my_rank, ierr

real, allocatable, dimension(:,:) :: A,B,Cinteger n, n_barcall MPI_INIT(ierr)

call Setup_grid(grid_info)

call MPI_Comm_rank(MPI_COMM_WORLD, my_rank, ierr)if (my_rank == 0) then

print *, 'What is the order of the matrices?'read *, nendif

call MPI_BCAST(n,1,MPI_INTEGER, 0, MPI_COMM_WORLD,ierr)n_bar = n/(grid_info%q)

! Allocate local storage for local matrix.allocate( A(n_bar,n_bar) )allocate( B(n_bar,n_bar) )allocate( C(n_bar,n_bar) )A = 1.0B = 2.0

call Fox(n,grid_info,A,B,C,n_bar)print *, Ccontains

subroutine Setup_grid(grid)

TYPE (GRID_INFO_TYPE), intent(inout) :: gridinteger old_rank

integer dimensions(0:1)logical periods(0:1)integer coordinates(0:1)logical varying_coords(0:1)integer ierr

! Set up Global Grid Information.

call MPI_Comm_size(MPI_COMM_WORLD, grid%p, ierr)call MPI_Comm_rank(MPI_COMM_WORLD, old_rank, ierr )grid%q = int(sqrt(dble(grid%p)))dimensions(0) = grid%qdimensions(1) = grid%q

periods(0) = .TRUE.periods(1) = .TRUE.

call MPI_Cart_create(MPI_COMM_WORLD, 2,

+ dimensions, periods, .TRUE. , grid%comm, ierr)call MPI_Comm_rank (grid%comm, grid%my_rank, ierr )call MPI_Cart_coords(grid%comm, grid%my_rank, 2,+ coordinates, ierr )

grid%my_row = coordinates(0)grid%my_col = coordinates(1)

! Set up row and column communicators.varying_coords(0) = .FALSE.varying_coords(1) = .TRUE.

call MPI_Cart_sub(grid%comm,varying_coords,+ grid%row_comm,ierr)varying_coords(0) = .TRUE.varying_coords(1) = .FALSE.

call MPI_Cart_sub(grid%comm,varying_coords,+ grid%col_comm,ierr)end subroutine Setup_grid

subroutine Fox(n,grid,local_A,local_B,loca l_C,n_bar)integer, intent(in) :: n, n_bar

TYPE(GRID_INFO_TYPE), intent(in) :: grid

real, intent(in) , dimension(:,:) :: local_A, local_Breal, intent(out), dimension (:,:) :: local_Creal temp_A( SIZE(A,DIM=1),SIZE(A,DIM=2) )integer step, source, dest, request

integer status(MPI_STATUS_SIZE), bcast_rootlocal_C = 0.0

source = mod( (grid%my_row + 1), grid%q )

dest = mod( (grid%my_row + grid%q -1), (grid%q) )

temp_A = 0.0

do step = 0, grid%q -1

bcast_root = mod( (grid%my_row + step), (grid%q) )if (bcast_root == grid%my_col) then

call MPI_BCAST(local_A,n_bar*n_bar,MPI_REAL,+ bcast_root, grid%row_comm, ierr)

call sgemm('N','N',n_bar,n_bar,n_bar,1.0,

+ local_A,n_bar,local_B,n_bar,1.0,local_C,n_bar)else

call MPI_BCAST(temp_A,n_bar*n_bar,MPI_REAL,+ bcast_root, grid%row_comm, ierr)

call sgemm('N','N',n_bar,n_bar,n_bar,1.0,+ temp_A,n_bar,local_B,n_bar,1.0,local_C,n_bar)endif

call MPI_Send(local_B,n_bar*n_bar,MPI_REAL,dest, 0,+ grid%col_comm, ierr)

call MPI_Recv(local_B,n_bar*n_bar,MPI_REAL,source,0,+ grid%col_comm, status, ierr )enddo

end subroutine Foxend program myfox

󰀔󰀕 :KHUH󰀄7R󰀄*R󰀄)URP󰀄+HUH

󰀔󰀕 :KDW󰀄:H󰀄+DYHQ󰀶W󰀄'LVFXVVHG

03,󰀏LV󰀏D󰀏ODUJH󰀏OLEUDU\\󰀦󰀏7KH󰀏6WDQGDUG󰀏>󰀬@󰀏LV󰀏RYHU󰀏󰀽󰀶󰀶󰀏SDJHV󰀏ORQJ󰀏DQG󰀏LW󰀏GHILQHV󰀏PRUH󰀏WKDQ󰀏󰀩󰀽󰀭

IXQFWLRQV󰀦󰀏$V󰀏D󰀏FRQVHTXHQFH󰀢󰀏WKLV󰀏*XLGH󰀏KDV󰀏FRYHUHG󰀏RQO\\󰀏D󰀏VPDOO󰀏IUDFWLRQ󰀏RI󰀏03,󰀢󰀏DQG󰀏PDQ\\󰀏UHDGHUVZLOO󰀏IDLO󰀏WR󰀏ILQG󰀏D󰀏GLVFXVVLRQ󰀏RI󰀏IXQFWLRQV󰀏WKDW󰀏WKH\\󰀏ZRXOG󰀏ILQG󰀏YHU\\󰀏XVHIXO󰀏LQ󰀏WKHLU󰀏DSSOLFDWLRQV󰀦󰀏6R󰀏ZHEULHIO\\󰀏OLVW󰀏VRPH󰀏RI󰀏WKH󰀏PRUH󰀏LPSRUWDQW󰀏LGHDV󰀏LQ󰀏03,󰀏WKDW󰀏ZH󰀏KDYH󰀏QRW󰀏GLVFXVVHG󰀏KHUH󰀦

󰀩󰀦 &RPPXQLFDWLRQ󰀂0RGHV󰀮󰀏:H󰀏KDYH󰀏XVHG󰀏RQO\\󰀏WKH󰀏VWDQGDUG󰀏FRPPXQLFDWLRQ󰀏PRGH󰀏IRU󰀏send󰀦7KLV󰀏PHDQV󰀏WKDW󰀏LW󰀏LV󰀏XS󰀏WR󰀏WKH󰀏V\\VWHP󰀏WR󰀏GHFLGH󰀏ZKHWKHU󰀏WKH󰀏PHVVDJH󰀏LV󰀏EXIIHUHG󰀦03,󰀏SURYLGHV󰀏WKUHH󰀏RWKHU󰀏FRPPXQLFDWLRQ󰀏PRGHV󰀺󰀏EXIIHUHG󰀢󰀂V\\QFKURQRXV󰀢󰀏DQG󰀏UHDG\\󰀦󰀏,QEXIIHUHG󰀏PRGH󰀢󰀏WKH󰀏XVHU󰀏H[SOLFLWO\\󰀏FRQWUROV󰀏WKH󰀏EXIIHULQJ󰀏RI󰀏RXWJRLQJ󰀏PHVVDJHV󰀦󰀏,QV\\QFKURQRXV󰀏PRGH󰀢󰀏D󰀏VHQG󰀏ZLOO󰀏QRW󰀏FRPSOHWH󰀏XQWLO󰀏D󰀏PDWFKLQJ󰀏UHFHLYH󰀏LV󰀏SRVWHG󰀦󰀏,Q

UHDG\\󰀏PRGH󰀢󰀏D󰀏VHQG󰀏PD\\󰀏EH󰀏VWDUWHG󰀏RQO\\󰀏LI󰀏D󰀏PDWFKLQJ󰀏UHFHLYH󰀏KDV󰀏DOUHDG\\󰀏EHHQSRVWHG󰀦󰀏03,󰀏SURYLGHV󰀏WKUHH󰀏DGGLWLRQDO󰀏VHQG󰀏IXQFWLRQV󰀏IRU󰀏WKHVH󰀏PRGHV󰀦

󰀩󰀦 1RQEORFNLQJ󰀂&RPPXQLFDWLRQ󰀮󰀏:H󰀏KDYH󰀏XVHG󰀏RQO\\󰀏EORFNLQJ󰀏VHQGV󰀏DQG󰀏UHFHLYHV

󰁂MPI_Send DQG󰀏MPI_Recv󰀦󰁃󰀏)RU󰀏WKH󰀏VHQG󰀢󰀏WKLV󰀏PHDQV󰀏WKDW󰀏WKH󰀏FDOO󰀏ZRQ󰁍W󰀏UHWXUQXQWLO󰀏WKH󰀏PHVVDJH󰀏GDWD󰀏DQG󰀏HQYHORSH󰀏KDYH󰀏EHHQ󰀏EXIIHUHG󰀏RU󰀏VHQW󰀏󰀜󰀜󰀜󰀏L󰀦H󰀦󰀢󰀏XQWLO󰀏WKHPHPRU\\󰀏UHIHUHQFHG󰀏LQ󰀏WKH󰀏FDOO󰀏WR󰀏MPI_Send󰀏LV󰀏DYDLODEOH󰀏IRU󰀏UH󰀜XVH󰀦󰀏)RU󰀏WKH󰀏UHFHLYH󰀢WKLV󰀏PHDQV󰀏WKDW󰀏WKH󰀏FDOO󰀏ZRQ󰁍W󰀏UHWXUQ󰀏XQWLO󰀏WKH󰀏GDWD󰀏KDV󰀏EHHQ󰀏UHFHLYHG󰀏LQWR󰀏WKHPHPRU\\󰀏UHIHUHQFHG󰀏LQ󰀏WKH󰀏FDOO󰀏WR󰀏MPI_Recv󰀦󰀏0DQ\\󰀏DSSOLFDWLRQV󰀏FDQ󰀏LPSURYH󰀏WKHLUSHUIRUPDQFH󰀏E\\󰀏XVLQJ󰀏QRQEORFNLQJ󰀏FRPPXQLFDWLRQ󰀦󰀏7KLV󰀏PHDQV󰀏WKDW󰀏WKH󰀏FDOOV󰀏WRVHQG󰁋UHFHLYH󰀏PD\\󰀏UHWXUQ󰀏EHIRUH󰀏WKH󰀏RSHUDWLRQ󰀏FRPSOHWHV󰀦󰀏)RU󰀏H[DPSOH󰀢󰀏LI󰀏WKHPDFKLQH󰀏KDV󰀏D󰀏VHSDUDWH󰀏FRPPXQLFDWLRQ󰀏SURFHVVRU󰀢󰀏D󰀏QRQ󰀜EORFNLQJ󰀏VHQG󰀏FRXOG󰀏VLPSO\\QRWLI\\󰀏WKH󰀏FRPPXQLFDWLRQ󰀏SURFHVVRU󰀏WKDW󰀏LW󰀏VKRXOG󰀏EHJLQ󰀏FRPSRVLQJ󰀏DQG󰀏VHQGLQJ󰀏WKHPHVVDJH󰀦󰀏03,󰀏SURYLGHV󰀏QRQEORFNLQJ󰀏VHQGV󰀏LQ󰀏HDFK󰀏RI󰀏WKH󰀏IRXU󰀏PRGHV󰀏DQG󰀏DQRQEORFNLQJ󰀏UHFHLYH󰀦󰀏,W󰀏DOVR󰀏SURYLGHV󰀏YDULRXV󰀏XWLOLW\\󰀏IXQFWLRQV󰀏IRU󰀏GHWHUPLQLQJ󰀏WKHFRPSOHWLRQ󰀏VWDWXV󰀏RI󰀏D󰀏QRQ󰀜EORFNLQJ󰀏RSHUDWLRQ󰀦󰀩󰀦 ,QWHU󰀦FRPPXQLFDWRUV󰀮󰀏5HFROOHFW󰀏WKDW󰀏03,󰀏SURYLGHV󰀏WZR󰀏W\\SHV󰀏RI󰀏FRPPXQLFDWRUV󰀺󰀏LQWUD󰀜FRPPXQLFDWRUV󰀏DQG󰀏LQWHU󰀜FRPPXQLFDWRUV󰀦󰀏,QWHU󰀜FRPPXQLFDWRUV󰀏FDQ󰀏EH󰀏XVHG󰀏IRUSRLQW󰀜WR󰀜SRLQW󰀏FRPPXQLFDWLRQV󰀏EHWZHHQ󰀏SURFHVVHV󰀏EHORQJLQJ󰀏WR󰀏GLVWLQFW󰀏LQWUD󰀜FRPPXQLFDWRUV󰀦

7KHUH󰀏DUH󰀏PDQ\\󰀏RWKHU󰀏IXQFWLRQV󰀏DYDLODEOH󰀏WR󰀏XVHUV󰀏RI󰀏03,󰀦󰀏,I󰀏ZH󰀏KDYHQ󰁍W󰀏GLVFXVVHG󰀏D󰀏IDFLOLW\\󰀏\\RX󰀏QHHG󰀢SOHDVH󰀏FRQVXOW󰀏WKH󰀏6WDQGDUG󰀏>󰀬@󰀏WR󰀏GHWHUPLQH󰀏ZKHWKHU󰀏LW󰀏LV󰀏SDUW󰀏RI󰀏03,󰀦

󰀔󰀕 ,PSOHPHQWDWLRQV󰀄RI󰀄03,

,I󰀏\\RX󰀏GRQ󰁍W󰀏KDYH󰀏DQ󰀏LPSOHPHQWDWLRQ󰀏RI󰀏03,󰀢󰀏WKHUH󰀏DUH󰀏WKUHH󰀏YHUVLRQV󰀏WKDW󰀏DUH󰀏IUHHO\\󰀏DYDLODEOH󰀏E\\

DQRQ\\PRXV󰀏IWS󰀏IURP󰀏WKH󰀏IROORZLQJ󰀏VLWHV󰀦

• $UJRQQH󰀏1DWLRQDO󰀏/DE󰁋0LVVLVVLSSL󰀏6WDWH󰀏8QLYHUVLW\\󰀦󰀏

info.mcs.anl.gov󰀢󰀏DQG󰀏WKH󰀏GLUHFWRU\\󰀏LV󰀏pub/mpi󰀦

7KH󰀏

DGGUHVV󰀏

LV

• (GLQEXUJK󰀏8QLYHUVLW\\󰀦󰀏7KH󰀏DGGUHVV󰀏LV󰀏ftp.epcc.ed.ac.uk󰀢󰀏DQG󰀏WKH󰀏GLUHFWRU\\󰀏LVpub/chimp/release󰀦• 2KLR󰀏6XSHUFRPSXWHU󰀏&HQWHU󰀦󰀏7KH󰀏DGGUHVV󰀏LV󰀏tbag.osc.edu󰀢󰀏DQG󰀏WKH󰀏GLUHFWRU\\󰀏LV

SXE󰁋ODP󰀦

$OO󰀏RI󰀏WKHVH󰀏UXQ󰀏RQ󰀏QHWZRUNV󰀏RI󰀏81,;󰀏ZRUNVWDWLRQV󰀦󰀏7KH󰀏$UJRQQH󰁋0LVVLVVLSSL󰀏6WDWH󰀏DQG󰀏(GLQEXUJKYHUVLRQV󰀏DOVR󰀏UXQ󰀏RQ󰀏YDULRXV󰀏SDUDOOHO󰀏SURFHVVRUV󰀦󰀏&KHFN󰀏WKH󰀏󰀳5($'0(󰀳󰀏ILOHV󰀏WR󰀏VHH󰀏LI󰀏\\RXUPDFKLQH󰁂V󰁃󰀏DUH󰀏VXSSRUWHG󰀦

󰀔󰀕 0RUH󰀄,QIRUPDWLRQ󰀄RQ󰀄03,

7KHUH󰀏LV󰀏DQ󰀏03,󰀏)$4󰀏DYDLODEOH󰀏E\\󰀏DQRQ\\PRXV󰀏IWS󰀏DW

• 0LVVLVVLSSL󰀏6WDWH󰀏8QLYHUVLW\\󰀦󰀏7KH󰀏DGGUHVV󰀏LV󰀏ftp.erc.msstate.edu󰀢󰀏DQG󰀏WKH󰀏ILOH󰀏LV

pub/mpi/faq󰀦

7KHUH󰀏DUH󰀏DOVR󰀏QXPHURXV󰀏ZHE󰀏SDJHV󰀏GHYRWHG󰀏WR󰀏03,󰀦󰀏$󰀏IHZ󰀏RI󰀏WKHVH󰀏DUH

• http://www.epm.ornl.gov/~walker/mpi󰀦󰀏7KH󰀏2DN󰀏5LGJH󰀏1DWLRQDO󰀏/DE󰀏03,󰀏ZHE

SDJH󰀦

• http://www.erc.msstate.edu/mpi󰀦󰀏7KH󰀏0LVVLVVLSSL󰀏6WDWH󰀏03,󰀏ZHE󰀏SDJH󰀦• http://www.mcs.anl.gov/mpi󰀦󰀏7KH󰀏$UJRQQH󰀏03,󰀏ZHE󰀏SDJH󰀦

(DFK󰀏RI󰀏WKHVH󰀏VLWHV󰀏FRQWDLQV󰀏D󰀏ZHDOWK󰀏RI󰀏LQIRUPDWLRQ󰀏DERXW󰀏03,󰀦󰀏2I󰀏SDUWLFXODU󰀏QRWH󰀢󰀏WKH󰀏0LVVLVVLSSL6WDWH󰀏SDJH󰀏FRQWDLQV󰀏D󰀏ELEOLRJUDSK\\󰀏RI󰀏SDSHUV󰀏RQ󰀏03,󰀢󰀏DQG󰀏WKH󰀏$UJRQQH󰀏SDJH󰀏FRQWDLQV󰀏D󰀏FROOHFWLRQ󰀏RIWHVW󰀏03,󰀏SURJUDPV󰀦

7KH󰀏03,󰀂6WDQGDUG󰀂>󰀬@󰀏LV󰀏FXUUHQWO\\󰀏DYDLODEOH󰀏IURP󰀏HDFK󰀏RI󰀏WKH󰀏VLWHV󰀏DERYH󰀦󰀏7KLV󰀏LV󰀢󰀏RI󰀏FRXUVH󰀢󰀏WKHGHILQLWLYH󰀏VWDWHPHQW󰀏RI󰀏ZKDW󰀏03,󰀏LV󰀦󰀏6R󰀏LI󰀏\\RX󰁍UH󰀏QRW󰀏FOHDU󰀏RQ󰀏VRPHWKLQJ󰀢󰀏WKLV󰀏LV󰀏WKH󰀏ILQDO󰀏DUELWHU󰀦󰀏,WDOVR󰀏FRQWDLQV󰀏D󰀏ODUJH󰀏QXPEHU󰀏RI󰀏QLFH󰀏H[DPSOHV󰀏RI󰀏XVHV󰀏RI󰀏WKH󰀏YDULRXV󰀏03,󰀏IXQFWLRQV󰀦󰀏6R󰀏LW󰀏LVFRQVLGHUDEO\\󰀏PRUH󰀏WKDQ󰀏MXVW󰀏D󰀏UHIHUHQFH󰀦󰀏&XUUHQWO\\󰀢󰀏VHYHUDO󰀏PHPEHUV󰀏RI󰀏WKH󰀏03,󰀏)RUXP󰀏DUH󰀏ZRUNLQJRQ󰀏DQ󰀏DQQRWDWHG󰀏YHUVLRQ󰀏RI󰀏WKH󰀏03,󰀏VWDQGDUG󰀏>󰀭@󰀦

7KH󰀏ERRN󰀏>󰀽@󰀏LV󰀏D󰀏WXWRULDO󰀏LQWURGXFWLRQ󰀏WR󰀏03,󰀦󰀏,W󰀏SURYLGHV󰀏QXPHURXV󰀏FRPSOHWH󰀏H[DPSOHV󰀏RI󰀏03,SURJUDPV󰀦

7KH󰀏ERRN󰀏>󰀿@󰀏FRQWDLQV󰀏D󰀏WXWRULDO󰀏LQWURGXFWLRQ󰀏WR󰀏03,󰀏󰁂RQ󰀏ZKLFK󰀏WKLV󰀏JXLGH󰀏LV󰀏EDVHG󰁃󰀦󰀏,W󰀏DOVR󰀏FRQWDLQVD󰀏PRUH󰀏JHQHUDO󰀏LQWURGXFWLRQ󰀏WR󰀏SDUDOOHO󰀏SURFHVVLQJ󰀏DQG󰀏WKH󰀏SURJUDPPLQJ󰀏RI󰀏PHVVDJH󰀜SDVVLQJPDFKLQHV󰀦

7KH󰀏8VHQHW󰀏QHZVJURXS󰀢󰀏comp.parallel.mpi󰀢󰀏SURYLGHV󰀏LQIRUPDWLRQ󰀏RQ󰀏XSGDWHV󰀏WR󰀏DOO󰀏RI󰀏WKHVHGRFXPHQWV󰀏DQG󰀏VRIWZDUH󰀦

󰀔󰀕 7KH󰀄)XWXUH󰀄RI󰀄03,

$V󰀏LW󰀏LV󰀏FXUUHQWO\\󰀏GHILQHG󰀢󰀏03,󰀏IDLOV󰀏WR󰀏VSHFLI\\󰀏WZR󰀏FULWLFDO󰀏FRQFHSWV󰀺󰀏,󰁋2󰀏DQG󰀏WKH

FUHDWLRQ󰁋GHVWUXFWLRQ󰀏RI󰀏SURFHVVHV󰀦󰀏:RUN󰀏KDV󰀏DOUHDG\\󰀏EHHQ󰀏VWDUWHG󰀏RQ󰀏WKH󰀏GHYHORSPHQW󰀏RI󰀏ERWK󰀏,󰁋2IDFLOLWLHV󰀏DQG󰀏G\\QDPLF󰀏SURFHVV󰀏FUHDWLRQ󰀦󰀏,QIRUPDWLRQ󰀏RQ󰀏WKH󰀏IRUPHU󰀏FDQ󰀏EH󰀏REWDLQHG󰀏IURPhttp://lovelace.nas.nasa.gov/MPI-IO/mpi-io.html󰀢󰀏DQG󰀏LQIRUPDWLRQ󰀏RQ󰀏WKH󰀏ODWWHU󰀏FDQ󰀏EH󰀏IRXQGRQ󰀏WKH󰀏$UJRQQH󰀏03,󰀏ZHE󰀏SDJH󰀦󰀏6LJQLILFDQW󰀏GHYHORSPHQWV󰀏DUH󰀏LQYDULDEO\\󰀏SRVWHG󰀏WRcomp.parallel.mpi󰀦󰀏

󰀔󰀕 &RPSLOLQJ󰀄DQG󰀄5XQQLQJ󰀄03,3URJUDPV

7KLV󰀏VHFWLRQ󰀏LV󰀏LQWHQGHG󰀏WR󰀏JLYH󰀏WKH󰀏RXWOLQH󰀏RI󰀏KRZ󰀏WR󰀏FRPSLOH󰀏DQG󰀏UXQ󰀏D󰀏SURJUDP󰀏LQ󰀏WKH󰀏,%063󰀽󰀦

03,󰀏SURJUDP󰀏ZULWWHQ󰀏LQ󰀏)RUWUDQ󰀏RU󰀏)RUWUDQ󰀏󰀪󰀶󰀏FDQ󰀏EH󰀏FRPSLOHG󰀏XVLQJ󰀏WKH󰀏IROORZLQJFRPPDQG󰀏󰀺

mpif77 program.f

%\\󰀏GHIDXOW󰀢󰀏WKH󰀏SURJUDP󰀏ZLOO󰀏EH󰀏UXQQLQJ󰀏RQ󰀏󰀬󰀏SURFHVVRUV󰀏RI󰀏WKH󰀏63󰀽󰀦󰀏7KH󰀏SURJUDP󰀏FDQ󰀏EHLQYRNHG󰀏E\\󰀏WKH󰀏QDPH󰀏RI󰀏H[HFXWDEOHa.out

7KH󰀏QXPEHU󰀏RI󰀏SURFHVVHV󰀏LV󰀏FRQWUROHG󰀏E\\󰀏WKH󰀏HQYLURQPHQW󰀏YDULDEOH󰀏03B352&6󰀦󰀏7KH󰀏ZHESDJH󰀏KWWS󰀺󰁋󰁋ZZZ󰀦KNX󰀦KN󰁋FF󰁋VS󰀽󰁋WHFKQLFDO󰁋VHWHQY󰀦KWPO󰀏KDV󰀏PDQXDOV󰀏IRU󰀏VHWWLQJ󰀏WKHHQYLURQPHQW󰀏YDULDEOH󰀦

󰀙󰀕 5HIHUHQFH

>󰀩@󰀏*HRIIUH\\󰀏)R[󰀢󰀏HW󰀏DO󰀦󰀢󰀏6ROYLQJ󰀂3UREOHPV󰀂RQ󰀂&RQFXUUHQW󰀂3URFHVVRUV󰀢󰀏(QJOHZRRG󰀏&OLIIV󰀢󰀏1-󰀢󰀏3UHQWLFH󰀜󰀜+DOO󰀢󰀩󰀪󰀷󰀷󰀦

>󰀽@󰀏:LOOLDP󰀏*URSS󰀢󰀏(ZLQJ󰀏/XVN󰀢󰀏DQG󰀏$QWKRQ\\󰀏6NMHOOXP󰀢󰀏8VLQJ󰀂03,󰀰󰀂3RUWDEOH󰀂3DUDOOHO󰀂3URJUDPPLQJ󰀂ZLWKWKH󰀂0HVVDJH󰀦3DVVLQJ󰀂,QWHUIDFH󰀢󰀏&DPEULGJH󰀢󰀏0$󰀢󰀏0,7󰀏3UHVV󰀢󰀏󰀩󰀪󰀪󰀬󰀦

>󰀫@󰀏%ULDQ󰀏:󰀦󰀏.HUQLJKDQ󰀏DQG󰀏'HQQLV󰀏0󰀦󰀏5LWFKLH󰀢󰀏7KH󰀂&󰀂3URJUDPPLQJ󰀂/DQJXDJH󰀢󰀏󰀽QG󰀏HG󰀦󰀢󰀏(QJOHZRRG&OLIIV󰀢󰀏1-󰀢󰀏3UHQWLFH󰀜󰀜+DOO󰀢󰀏󰀩󰀪󰀷󰀷󰀦

>󰀬@󰀏0HVVDJH󰀏3DVVLQJ󰀏,QWHUIDFH󰀏)RUXP󰀢󰀏03,󰀰󰀂$󰀂0HVVDJH󰀦3DVVLQJ󰀂,QWHUIDFH󰀂6WDQGDUG󰀢󰀏,QWHUQDWLRQDO󰀏-RXUQDORI󰀏6XSHUFRPSXWHU󰀏$SSOLFDWLRQV󰀢󰀏YRO󰀏󰀷󰀢󰀏QRV󰀏󰀫󰁋󰀬󰀢󰀏󰀩󰀪󰀪󰀬󰀦󰀏$OVR󰀏DYDLODEOH󰀏DV󰀏7HFKQLFDO󰀏5HSRUW󰀏&6󰀜󰀪󰀬󰀜󰀽󰀫󰀶󰀢&RPSXWHU󰀏6FLHQFH󰀏'HSW󰀦󰀢󰀏8QLYHUVLW\\󰀏RI󰀏7HQQHVVHH󰀢󰀏.QR[YLOOH󰀢󰀏71󰀢󰀏󰀩󰀪󰀪󰀬󰀦

>󰀭@󰀏6WHYH󰀏2WWR󰀢󰀏HW󰀏DO󰀦󰀢󰀏03,󰀂$QQRWDWHG󰀂5HIHUHQFH󰀂0DQXDO󰀢󰀏&DPEULGJH󰀢󰀏0$󰀢󰀏0,7󰀏3UHVV󰀢󰀏WR󰀏DSSHDU󰀦>󰀿@󰀏3HWHU󰀏6󰀦󰀏3DFKHFR󰀢󰀏3URJUDPPLQJ󰀂3DUDOOHO󰀂ZLWK󰀂03,󰀢󰀏6DQ󰀏)UDQFLVFR󰀢󰀏&$󰀢󰀏0RUJDQ󰀏.DXIPDQQ󰀦󰀏󰀩󰀪󰀪󰀾󰀦>󰀾@󰀏3HWHU󰀏6󰀦󰀏3DFKHFR󰀢󰀏$󰀂8VHU·V󰀂*XLGH󰀂WR󰀂03,󰀢󰀏󰀩󰀪󰀪󰀭󰀦