· KLDP.org · KLDP.net · KLDP Wiki · KLDP BBS ·
Docbook Sgml/HPC-KLDP

HPC(High Performance Computer) Linux Cluster HowTo

HPC(High Performance Computer) Linux Cluster HowTo

ÀÌÁøÈ£

System Engineer
System Programer
DBA

           
        

v0.8, 2003/05/12

ÀÌ ¹®¼­´Â µ¿±âÁ¾(homogeneous system) °£ÀÇ Å¬·¯½ºÅ͸¦ ÅëÇÏ¿© º´·Äó¸® ¹æ½ÄÀÇ ½´ÆÛÄÄÇ»ÆÃÀ» Á¦ÀÛÇÏ´Â ¹æ½ÄÀ» ±â¼úÇÏ¿´½À´Ï´Ù. ±âº»ÀûÀ¸·Î beowolf ¹æ½Ä À» µû¶óÇÏ°í ÀÖ°í, ÀÀ¿ëÀ» ÇÏ¿´À¸³ª, Çѱ۷ΠµÈ °ü·Ã¹®¼­°¡ ¸¹ÀÌ ¾ø´Ù´Âµ¥ Âø¾È, ¹®¼­È­ ÇÏ°Ô µÇ¾ú½À´Ï´Ù.

고친 과정
고침 0.92003-11-05고친이 ljn
MPICH ¼³Á¤ÆÄÀÏ ¼öÁ¤
고침 0.82003-05-12고친이 ljn
¿ÀÅ»ÀÚ ±³Á¤
고침 0.72003-05-12고친이 ljn
ÃÖÃÊ ÀÛ¼º


1장. ¼­·Ð

ÀÌ ¹®¼­´Â GPL(GNU PUBLIC LICENSE) ¿¡ ÀÇÇØ ÀÚÀ¯·ÎÀÌ ¹èÆ÷µÉ¼ö ÀÖ½À´Ï´Ù. ¶óÀ̼¾½º¿¡ °üÇÑ ¼³¸íÀº http://www.fsf.org/copyleft/gpl.html¿¡¼­ Âü°íÇÒ¼ö ÀÖ½À´Ï´Ù.

Copyright (C) 2003 ÀÌÁøÈ£


1.1. µé¾î°¡±â Àü¿¡

ÀÌ ¹®¼­´Â Ŭ·¯½ºÅÍ ¿¡ »ç¿ëµÉ ¸ðµç Machine ÀÌ µ¿ÀÏÇÑ ±âÁ¾ ÇÏ¿¡¼­ ±¸µ¿ÇÑ´Ù°í °¡Á¤ÇÕ´Ï´Ù. °¢ÀÚÀÇ ½Ã½ºÅÛ µéÀº µ¶¸³µÈ boot ¿µ¿ªÀ¸·Î ±¸µ¿µÇ¸ç °¢°¢ÀÇ Åë½ÅÀº FastEthernet À¸·Î (10/100 or 1000 Mbps) ¿¬°áµÇ¾î ÀÖÀ½À» °¡Á¤ÇÕ´Ï´Ù. ±×¸®°í ´ë±Ô¸ð Ŭ·¯½ºÅÍ(32 Node ÀÌ»ó) ±¸Ãà½Ã º»¹®¼­ÀÇ ³»¿ë´ë·Î Àû¿ëÀÌ ¾ÈµÉ¼öµµ ÀÖ½À´Ï´Ù. ÀÌ°ÍÀº ´ëºÎºÐ À¯´Ð½º NFS(Network File System) ÀÇ I/O ¹®Á¦¿Í ¹ÐÁ¢ÇÑ °ü·ÃÀÌ ÀÖ½À´Ï´Ù. Beowolf ¿Í °ü·ÃµÈ ¸¹Àº ¹®¼­µéÀÌ Àֱ⠶§¹®¿¡ ¿©±â¼­´Â ¸¹Àº ³»¿ëÀÇ ÁÖ¼®À̳ª ¼³¸íÀ» ÇÏÁö ¾Ê°í ´çÀå ½Ç¹«¿¡¼­ »ç¿ëÇÒ¼ö ÀÖ´Â ¼öÁØÀÇ ¹®¼­¸¦ °£°áÇÏ°í °¡´ÉÇÑÇÑ ¸íÈ®ÇÏ°Ô ¸¸µå´Â °ÍÀÌ ÁÖ ¸ñÀû ÀÔ´Ï´Ù.


1.2. °¨»çÀÇ ±Û

ÀÌ ¹®¼­¸¦ ÀÛ¼ºÇϴµ¥ µµ¿òÀ» ÁֽźÐ, ƯÈ÷ Çѱ¹ ¿Ü±¹¾î´ë ÄÄÇ»ÅÍ °øÇаú RTDCS Lab ÀÇ ±è°æ¹Î ´Ô °ú ÇÑÈñÁ¤ ´Ô ¿¡°Ô °¨»çÀÇ ¸»À» ÀüÇÕ´Ï´Ù. Ŭ·¯½ºÅÍ °ü·Ã ÀÚ·á, ȤÀº ¹®¼­¸¦ ã´Âµ¥ ¸¹Àº½Ã°£À» Àý¾àÇÏ°Ô ÇØÁÖ¾ú½À´Ï´Ù.


1.3. Çǵå¹é

ÀÌ ¹®¼­¿¡ ´ëÇÑ ¹ßÀüÀûÀÎ Á¦¾ÈÀ̳ª ¼öÁ¤»çÇ×, ¹®Á¦Á¡ µî¿¡ ´ëÇÑ Çǵå¹éÀº ¾ðÁ¦µçÁö ȯ¿µÇÕ´Ï´Ù. ·Î ¸ÞÀÏÀ» º¸³» ÁֽʽÿÀ. (´Ü Ŭ·¯½ºÅÍ È¯°æÀÇ ¼³Ä¡,½ÇÇàÀÇ ¿À·ù¿¡ °üÇÑ ¹®ÀÇ»çÇ×Àº Á¤ÁßÈ÷ »ç¾çÇÕ´Ï´Ù)


2장. °³¿ä

2.1. Çϵå¿þ¾î ÀÇ ¼±ÅÃ

Ŭ·¯½ºÅÍ ¸¦ ±¸ÃàÇϱâ À§Çؼ­´Â ¾î¶°ÇÑ Çϵå¿þ¾î¸¦ »ç¿ëÇÏ¿© ¾î´ÀÁ¤µµ ±Ô¸ðÀÇ Å¬·¯½ºÅ͸¦ ±¸ÃàÇÒ °Í ÀÎÁö H/W ÀûÀÎ ¼±ÅÃÀ» ÇØ¾ß ÇÑ´Ù. ´ÙÀ½Àº H/W ¼±Åÿ¡ ´ëÇÑ Alpha-11 ÇÁ·ÎÁ§Æ®ÆÀ(http://www.alpha11.com) ÀÇ ¸î°¡Áö Á¶¾ðÀÌ´Ù.


2.1.1. ÇÁ·Î¼¼¼­(CPU)

Ŭ·¯½ºÅ͸¦ ±¸ÃàÇÏ¿© ¿øÇÏ´Â ÄÄÇ»Æà ÆÄ¿ö¸¦ ¾ò±âÀ§ÇÑ µÎ°¡Áö °í·Á»çÇ×ÀÌ ÀÖ´Ù. Àú¼º´ÉÀÇ ÄÄÇ»Å͸¦ ´Ù¼ö ¿¬°áÇÒ°ÍÀΰ¡ °í¼º´ÉÀÇ ÄÄÇ»Å͸¦ ¼Ò¼ö ¿¬°áÇÒ °ÍÀΰ¡¸¦ ³õ°í °í·ÁÇØ¾ß ÇÑ´Ù. ÀϹÝÀûÀ¸·Î ³ëµåÀÇ ¼ö°¡ Áõ°¡ÇÒ¼ö·Ï ÇÁ·Î¼¼¼­°£ÀÇ Åë½ÅÀÌ Áõ°¡ÇϹǷΠ³×Æ®¿öÅ© ºñ¿ëÀÌ Áõ°¡ÇÏ°Ô µÇ°í µ¿±âÈ­¿¡ ÇÊ¿äÇÑ ½Ã°£ÀÌ Áõ°¡ÇϹǷΠº´·ÄÈ­ È¿À² ¿ª½Ã ÁÙ¾îµé°Ô µÈ´Ù. µû¶ó¼­ °í¼º´É ÄÄÇ»Å͸¦ ¼Ò¼ö ¿¬°áÇÏ´Â °ÍÀÌ È¿À²¸é¿¡¼­´Â ÁÁ´Ù. ±×·¯³ª °í¼º´É ÄÄÇ»ÅÍÀÇ °¡°ÝÀº ±× ¼º´É¿¡ ºñÇØ ºñ½Î±â ¶§¹®¿¡ ŸÇùÀ» ÇØ¾ß ÇÑ´Ù. Compaq ÀÇ ALPHA ÇÁ·Î¼¼¼­´Â Floting °è»ê´É·Â¿¡ À־ µ¿ÀÏ clock ÀÇ intel Pentium ¸Ó½Å¿¡ ºñÇØ ¾à 2.5¹è Á¤µµ ºü¸£¸ç ´ëºÎºÐÀÇ ¼öÄ¡¸ðµ¨µéÀÌ floating ¿¬»êÀÌ ÁÖ¸¦ ÀÌ·ç´Â °ÍÀ» °¨¾ÈÇÏ¸é ¼öÄ¡¸ðµ¨À» À§ÇÑ Å¬·¯½ºÅÍ ±¸Ãà¿¡ ÀûÇÕÇÏ´Ù. ¶ÇÇÑ 64bit ¾ÆÅ°ÅØó À̹ǷΠ±×¿¡ µû¸¥ IO °ü·Ã ¼Óµµ Çâ»ó¿¡ µµ¿òÀ» ÁÙ¼ö ÀÖ´Ù. AMD ¿Í INTEL ÀÇ ix86 À¸·Î ´ëº¯µÇ´Â ÇÁ·Î¼¼¼­µéÀº Àú°¡¿¡ Ŭ·¯½ºÅ͸¦ ±¸¼ºÇÒ¼ö ÀÖÀ¸¸ç ´Ù¾çÇÑ H/W ¿Í S/W ÀÇ Áö¿øÀ» ¹ÞÀ»¼ö ÀÖ´Â ÀåÁ¡ÀÌ ÀÖ´Ù.


2.1.2. ³×Æ®¿öÅ© ÀåÄ¡(NIC), ½ºÀ§Äª Çãºê

°¡Àå ÀϹÝÀûÀÎ °æ¿ì´Â 100Mbps ½ºÀ§Äª Çãºê¿Í ·£Ä«µå¸¦ »ç¿ëÇÏ¿© ±¸¼ºµÈ´Ù. 100Mbps ³×Æ®¿öÅ© ÀåÄ¡¸¦ »ç¿ëÇÏ¿© Á¦ÀÛÇÑ Å¬·¯½ºÅÍ´Â ¸¹Àº »ç¿ëÀڵ鿡 ÀÇÇØ ÃæºÐÈ÷ »ç¿ëÇÒ ¸¸ÇÑ Àåºñ¶ó´Â °ÍÀÌ ÀÔÁõµÇ¾ú´Ù. ¸¸¾à ´õ ÁÁÀº ³×Æ®¿öÅ© ÀåÄ¡¸¦ »ç¿ëÇϱ⸦ ¿øÇÑ´Ù¸é ¹Ì¸®³Ý(MyriNet) °ú ±â°¡ºñÆ®ÀÌ´õ³Ý(Gigabit) À» °í·ÁÇÒ¼ö ÀÖÀ¸³ª À̵é Àåºñ´Â °í°¡ÀÌ´Ù. ƯÈ÷ ¹Ì¸®³ÝÀº Àü¿ë ÇÁ·ÎÅäÄÝÀ» »ç¿ëÇÒ°æ¿ì ÆÐŶÀü¼Û½Ã Áö¿¬½Ã°£(Latency)ÀÌ ¸Å¿ì ÀÛÀº Ŭ·¯½ºÅÍ Àü¿ë ³×Æ®¿÷ ÀåÄ¡ ÀÌÁö¸¸ TCP °¡ Áö¿øµÇÁö ¾Ê´Â ¹®Á¦Á¡Àº ÀÖ´Ù. ÃÖ±Ù TCP ¸¦ ¿¡¹Ä·¹ÀÌ¼Ç ÇÒ¼ö ÀÖ´Â ¹æ¹ýÀÌ ³ª¿Í ÀÖÀ¸³ª ¾ÆÁ÷ ¾ÈÁ¤È­ µÇ¾î ÀÖÁö ¾ÊÀº °É·Î ¾Ë°íÀÖ´Ù.


2.1.3. ±âŸ ÁÖº¯ÀåÄ¡

¸Þ¸ð¸®´Â 1 bit ¿¡·¯ º¸Á¤´É·ÂÀÌ ÀÖ´Â ECC ·¥°ú ÀϹݷ¥ÀÌ °í·ÁµÉ¼ö ÀÖ´Ù. Á»´õ ½Å·Ú¼º ÀÖ´Â ½Ã½ºÅÛÀ» ±¸ÃàÇϱâ À§Çؼ­´Â ECC ·¥À» »ç¿ëÇÏ´Â °ÍÀÌ ÁÁÀ¸³ª °¡°ÝÀº Á»´õ ºñ½Î´Ù. ¸ÞÀκ¸µåÀÇ ¼±Åÿ¡ À־´Â ¼º´Éº¸´Ù´Â ¾ÈÁ¤¼º¿¡ ÁßÁ¡À» µÎ¾î¼­ ¼±ÅÃÇÏ´Â °ÍÀÌ ¹Ù¶÷Á÷ÇÏ´Ù. Ŭ·¯½ºÅÍ´Â ´Ù¼öÀÇ ÄÄÇ»ÅÍ·Î ±¸¼ºµÇ¹Ç·Î Çϵå¿þ¾î°¡ ¹®Á¦¸¦ ÀÏÀ¸Å³ È®·üÀÌ ±×¸¸Å­ Áõ°¡ÇÏ°Ô µÇ±â ¶§¹®ÀÌ´Ù. Storage(ÀúÀåÀåÄ¡) ÀÇ °æ¿ì ÀϹÝÀûÀ¸·Î IDE ¹æ½Ä°ú SCSI ¹æ½ÄÀÇ HDD °¡ °í·ÁµÉ¼ö ÀÖ´Ù. ÀÌ´Â ÀüÀûÀ¸·Î ¾î¶² ÇÁ·Î±×·¥À» ¼öÇàÇÒ °ÍÀÎÁö¿¡ µû¶ó °áÁ¤µÉ¼ö ÀÖ´Ù. I/O °¡ ºó¹øÀÌ ¹ß»ýÇÏ´Â ÇÁ·Î±×·¥À» ¼öÇàÇѴٰųª µ¿½Ã¿¡ ´Ù¼öÀÇ ÇÁ·Î¼¼½º¿¡ ÀÇÇؼ­ I/O °¡ ÀÌ·ç¾î Áø´Ù¸é SCSI ¸¦ °í·ÁÇØ¾ß ÇÒ°ÍÀÌ´Ù. ±×¸®°í ´õ ³ªÀº ¾ÈÁ¤¼º°ú ¼Óµµ¸¦ ¿øÇÑ´Ù¸é °í°¡ÀÇ RAID ¿ª½Ã °í·ÁµÉ¼ö ÀÖ´Ù. ÇÑÆí ´Ù¼öÀÇ Çϵåµð½ºÅ©¸¦ ÀÌ¿ëÇÏ¿© ¼ÒÇÁÆ®¿þ¾îÀûÀ¸·Î RAID ¸¦ ±¸¼ºÇÏ´Â ¹æ¹ýµµ ÀÖ´Ù.


2.2. ¼ÒÇÁÆ®¿þ¾î ÀÇ ¼±ÅÃ

Ŭ·¯½ºÅ͸¦ À§ÇÑ ¼ÒÇÁÆ®¿þ¾î¸¦ Å©°Ô 3°¡Áö·Î ³ª´«´Ù¸é Ŭ·¯½ºÅÍ¿¡ »ç¿ëµÉ OS, Ŭ·¯½ºÅ͸¦ ±¸ÃàÇϴµ¥ ÇÊ¿ä ÇÑ Middleware, ¸¶Áö¸·À¸·Î Ŭ·¯½ºÅ͸¦ ±¸¼ºÇÑÈÄ »ç¿ëÇÒ ÇÁ·Î±×·¥À¸·Î ³ª´­¼ö ÀÖ´Ù. ´ÙÀ½Àº À̵鿡 ´ëÇÑ °£´ÜÇÑ Á¶¾ðÀÌ´Ù.


2.2.1. OS ¿¡ ´ëÇÑ °í·Á»çÇ×

°¡Àå ³Î¸® »ç¿ëµÇ´Â OS ·Î´Â ¿ª½Ã ¸®´ª½º¸¦ µé¼ö ÀÖ´Ù. ÇöÀç ¸®´ª½º¸¦ ÀÌ¿ëÇÏ¿© Ŭ·¯½ºÅ͸¦ ±¸ÃàÇϱâ À§ÇÑ ¸¹Àº ±â¼ú¹®¼­¿Í ¶óÀ̺귯¸®, ¸ð´ÏÅ͸µ Åø, Å¥À׽ýºÅÛ µîÀÌ °ø°³µÇ¾î ÀÖÀ¸¸ç ½ÉÁö¾î´Â Ŭ·¯½ºÅ͸µ À» À§ÇÑ Ä¿³ÎÀ̹ÌÁö, ¹èÆ÷ÆÇ µîµµ ³ª¿Í ÀÖ´Ù. ¹°·Ð ¸®´ª½º»Ó ¾Æ´Ï¶ó UNIX ¿Í MS ÀÇ Window ·Îµµ Ŭ·¯½ºÅ͸¦ ±¸ÃàÇÒ¼ö ÀÖ´Ù.


2.2.2. MiddleWare ¿¡ ´ëÇÑ °í·Á»çÇ×

¿©±â¼­ ¸»ÇÏ´Â ¹Ìµé¿þ¾î¶õ º´·Ä ¶óÀ̺귯¸®, ¸ð´ÏÅ͸µ Åø, Å¥À׽ýºÅÛ µî Ŭ·¯½ºÅÍ °ü·Ã ¼ÒÇÁÆ®¿þ¾î¸¦ ÁöĪÇÑ´Ù. ¸ÕÀú º´·Ä ¶óÀ̺귯¸®´Â ¹«¼öÈ÷ ¸¹ÀÌ ÀÖÀ¸¸ç À̵éÁß ¿øÇÏ´Â ¶óÀ̺귯¸®¸¦ Çϳª ¶Ç´Â ¿©·¯ °³ ¼±ÅÃÇؼ­ ¼³Ä¡ÇÒ¼ö ÀÖ´Ù. ¸ð´ÏÅ͸µ Åø°ú Å¥À׽ýºÅÛ ¿ª½Ã ´Ù¼ö°¡ °ø°³µÇ¾î ÀÖÀ¸¸ç »ó¿ëÅøµéµµ ³ª¿Í ÀÖ´Ù. À̵鿡 ´ëÇÑ ÀÚ¼¼ÇÑ Á¤º¸´Â Reference Sites ¿¡ ³ª¿Í ÀÖ´Â °¢ ¼ÒÇÁÆ®¿þ¾îÀÇ È¨ÆäÀÌÁö¸¦ Âü°íÇϱ⠹ٶõ´Ù.


2.2.3. ÇÁ·Î±×·¥ º´·ÄÈ­ ¿¡ ´ëÇÑ °í·Á»çÇ×

±âÁ¸ÀÇ ¿öÅ©½ºÅ×ÀÌ¼Ç ¿¡¼­ µ¹¾Æ°¡°í ÀÖ´Â ÇÁ·Î±×·¥À» Ŭ·¯½ºÅÍ ¿¡¼­ ½ÇÇàÇÑ´Ù°í Çؼ­ ¼º´ÉÀÌ ÁÁ¾ÆÁö´Â °ÍÀº ¾Æ´Ï´Ù. Ŭ·¯½ºÅ͸¦ ºñ·ÔÇÑ º´·Äó¸® ÄÄÇ»ÅÍ´Â º´·ÄÈ­ µÈ Äڵ带 ¿ä±¸ÇÑ´Ù. Áï ½Ã¸®¾ó ÇÁ·Î±×·¥À» º´·ÄÈ­ ÇÑÈÄ À̸¦ Ŭ·¯½ºÅÍ¿¡¼­ ½ÇÇàÇØ¾ß ÇÑ´Ù. C, C++, Fortran À¸·Î ¸¸µç ÇÁ·Î±×·¥Àº ÀüÅëÀûÀ¸·Î º´·ÄÈ­°¡ °¡´ÉÇÏ´Ù.


2.3. ¿ä±¸ »çÇ×

¿©±â¼­´Â 7´ëÀÇ ÄÄÇ»Å͸¦ Ŭ·¯½ºÅ͸µ ÇÒ°ÍÀ̸ç ÀÌÈÄ ÇÊ¿äÇÑ Àåºñ´Â ´ÙÀ½°ú °°´Ù. ÄÄÇ»ÅÍ 7´ë, °¢ ÄÄÇ»ÅÍ´Â CPU, RAM, BOARD, HDD µîÀÇ ±âº»ÀûÀÎ ÀåÄ¡¸¦ °¡Áö°í ÀÖ´Ù. ÄÄÇ»ÅÍ°£ °¢°¢ Lan Card 2°³ ¾¿(ÇÊÀÚ´Â NFS ¿Í Gateway ¿ë Åë½Å¿ëÀ¸·Î Çϳª, ³ª¸ÓÁö 1°³ÀÇ ·£Ä«µå´Â ³ëµå°£ÀÇ ½ÇÁúÀûÀÎ °è»êÀÛ¾÷ ¿ë Åë½ÅÀ¸·Î »ç¿ëÇϱâ À§ÇØ ³ëµå´ç 2°³ÀÇ ·£Ä«µå¸¦ ¼³Ä¡ Çß´Ù. ¹°·Ð 1°³ÀÇ ·£Ä«µå ¸¸À¸·Îµµ ¼­ºñ½º Çϴµ¥ ¹®Á¦µÉ °ÍÀº ¾ø´Ù.) ½ºÀ§Ä¡ Çãºê 1°³, ¸ð´ÏÅÍ, Å°º¸µå, ¸¶¿ì½º µîÀÌ ÇÊ¿äÇÏ´Ù.

Ŭ·¯½ºÅ͸¦ ¸¸µé°í ¸ð´ÏÅ͸µ Çϴµ¥´Â ±âº»ÀûÀÎ ¸®´ª½º ±â¼úÀÌ ¿ä±¸µÈ´Ù. ÀÌ·¯ÇÑ ±â¼úµéÀº ÀÌ ¹®¼­ÀÇ ¹üÀ§¸¦ ³Ñ¾î¼­±â ¶§¹®¿¡ ´Ù·çÁö ¾Ê´Â´Ù. ÀÌ·¯ÇÑ ³»¿ëµéÀº http://www.kldp.org ³ª http://www.tldp.org ¿¡¼­ ¸¹Àº ÀڷḦ ¾òÀ»¼ö ÀÖ´Ù.


3장. ¼³Ä¡Çϱâ

3.1. ±âº» »çÇ×

ÀϹÝÀûÀ¸·Î Ŭ·¯½ºÅ͸µ ½Ã½ºÅÛ¿¡´Â 1 ´ëÀÇ °ü¸®³ëµå¿Í n ´ëÀÇ ÄÄÇ»Æà ³ëµå·Î ±¸¼ºµÇ¾î ÀÖ´Ù. °ü¸®³ëµå´Â ¸»±×´ë·Î Ŭ·¯½ºÅÍ ½Ã½ºÅÛÀ» °ü¸®ÇÏ´Â ³ëµå ÀÌ°í ÄÄÇ»Æà ³ëµå´Â ÁÖ·Î ¿¬»êÀ» ´ã´çÇÏ°Ô µÇ´Â ³ëµå ÀÌ´Ù. °ü¸®³ëµå¿Í ÄÄÇ»Æà ³ëµå´Â ¹°¸®ÀûÀ¸·Î ºÐ¸®½ÃÅ°´Â °ÍÀÌ ÁÁÀ¸³ª ¿©±â¼± ƯÁ¤ ³ëµå¸¦ °ü¸®³ëµå¿Í ÄÄÇ»Æà ³ëµåµµ ÇÔ²² ¼¼Æà ÇÒ°ÍÀÌ´Ù. Ŭ·¯½ºÅÍ ½Ã½ºÅÛÀÌ ±¸ÃàµÈ ÈÄ¿¡ »ç¿ëÀÚ ¹× °ü¸®ÀÚ´Â ´Â °ü¸®³ëµå¿¡ Á¢¼ÓÇÏ¿© Àüü Ŭ·¯½ºÅÍ ½Ã½ºÅÛÀ» »ç¿ë, °ü¸®ÇÏ°Ô µÉ°ÍÀÌ´Ù. Çϵåµð½ºÅ©´Â ¸ðµç Ŭ·¯½ºÅÍ ³ëµå°¡ °¢°¢ µ¶¸³ÀûÀÎ µð½ºÅ©¸¦ º¸À¯ÇÏ°í ÀÖ°í °¢°¢ÀÇ ¹°¸®ÀûÀÎ ºÎÆ® ½Ã½ºÅÛÀ» °®Ãß°í ÀÖ´Ù.


3.2. OS Install

¿©·¯Á¾·ùÀÇ ¸®´ª½º ¹öÀü, Á¦Ç°µéÀÌ ÀÖÀ¸³ª ¿©±â¼­ ¼³Ä¡ÇÒ Á¦Ç°Àº Redhat Linux 7.3 (Valhalla) ¸¦ ¼³Ä¡ Çϱâ·Î ÇÑ´Ù. º» HowTo ¿¡¼­ Á¦°øÇϴ Ŭ·¯½ºÅÍ Á¦ÀÛ ±â¹ýÀº ´ë±Ô¸ð Ŭ·¯½ºÅÍ (32 Node ÀÌ»ó) ¿¡¼­´Â ±×´ë·Î Àû¿ëÀÌ ¾ÈµÉ¼öµµ ÀÖ´Ù. ÀÌÀ¯´Â ±âº»ÀûÀ¸·Î Beowolf Family ¿¡¼­ Ŭ·¯½ºÅÍ ³¢¸®ÀÇ File °øÀ¯´ÂNFS (Network File System) ¸¦ »ç¿ëÇÏ°í Àִµ¥, ´ë·®ÀÇ Å¬·¯½ºÅÍ ¸¦ NFS ·Î ¹­À»°æ¿ì ½É°¢ÇÑ I/O º´¸ñÇö»óÀÌ ÀϾ°Ô µÈ´Ù. ÆÄÀϽýºÅÛ°ú °ü·ÃÇÏ¿© ÇöÀç ¸¹Àº ¿¬±¸°¡ ÀÌ·ïÁö°í ÀÖÀ¸¸ç ÁÁÀº °á°ú¹° µéÀÌ ¸¹ÀÌ ³ª¿À°í ÀÖ´Ù. ±×·¯³ª ¿©±â¼­´Â ÆÄÀϽýºÅÛÀÇ ¼¼ºÎ °ü·Ã³»¿ëÀ» ´Ù·çÁö ¾ÊÀ»°ÍÀÌ´Ù. Áغñ°¡ µÇ¾úÀ¸¸é °¢ Node ¿¡ Redhat ¸®´ª½º¸¦ ¼³Ä¡ÇÑ´Ù. °ü¸®³ëµå°¡ µÉ ÄÄÇ»ÅÍ´Â ¼³Ä¡È­¸é ¿¡¼­ NFS Server ÆÐÅ°Áö¸¦ ¹Ýµå½Ã ¼±Åà Çϵµ·Ï ÇÑ´Ù. ¼³Ä¡¿¡ ´ëÇØ ±Ã±ÝÇÑ°ÍÀÌ ÀÖ´Ù¸é °ü·Ã¹®¼­¸¦ Âü°íÇϱ⠹ٶõ´Ù. Àß ¸ð¸£°ÚÀ¸¸é Full Install ÈÄ ¾µ¸ð¾ø´Â ÆÐÅ°Áö¸¦ Çϳª¾¿ Á¦°Å Çϴ°ÍÀ» ±ÇÀåÇÑ´Ù.


3.2.1. Bug patching, Software Upgrade

ºÒÇàÇÏ°Ôµµ ¾î¶² OS µçÁö Ãë¾àÁ¡À̳ª °áÇÔÀÌ ¾ø´Â °ÍÀº ¾ø´Ù. Redhat Linux µµ ¿¹¿Ü´Â ¾Æ´Ï¸ç, °ü·Ã ¹ö±×¿Í ¾÷µ¥ÀÌÆ® ¿¡ ´ëÇؼ­ http://www.redhat.com/apps/support/errata/ ¿¡ errata ¹®¼­·Î »ó¼¼È÷ ±â¼úÇØ ³õ¾Ò´Ù. ÇÊÀÚ°¡ ¹Ýµå½Ã ±ÇÀåÇÏ´Â °ÍÀº gcc °ü·Ã ÆÐÅ°Áö¸¦ change ÇÏ´Â °ÍÀÌ´Ù. Redhat 7.x ´ë¿¡ µé¾î°¡¼­ gcc °ü·Ã ¹ö±× ¸®Æ÷ÆÃÀº ÀÌ·ç Çì¾Æ¸±¼ö ¾øÀÌ ¸¹ÀÌ ³ª¿À°í ÀÖ´Ù. Redhat 7.3 ÀÇ ±âº» gcc ÆÐÅ°Áö´Â ³öµÎ°í anonymous °èÁ¤À¸·Î ´ÙÀ½ÀÇ url ¿¡¼­ gcc 2.95 ¹öÀüÀ» ´Ù¿î ¹Þ´Â´Ù. ftp:// mirror.oops.org/pub/Linux/Redhat/RPMS/7.3/gcc/ ÇØ´ç µð·ºÅ丮ÀÇ ÆÄÀϵéÀ» ÀüºÎ ´Ù¿î ¹Þ¾Æ¼­ ¼³Ä¡ ÇØ¾ß ÇÑ´Ù. ¶Ç´Â http://gcc.gnu.org ¿¡¼­ gcc 2.95 ¹öÀüÀÇ RPM ÆÄÀϵéÀ» ´Ù¿î¹Þ¾Æ¼­ ¼³Ä¡Çصµ ÁÁ´Ù. ¼³Ä¡ÇÏ·Á´Â gcc ¹öÀüÀÌ ÇöÀç°Í º¸´Ù ³·±â ¶§¹®¿¡ ¼³Ä¡½Ã ÀÇÁ¸¼º ¿¡·¯³ª ¹öÀü ¿¡·¯°¡ ³¯¼ö ÀÖ´Ù. ¹«½ÃÇÏ°í ¼³Ä¡Çϵµ·Ï ÇÏÀÚ.

[root@master root]# rpm -Uvh --force --nodeps gcc-2.95.4-3.ix86.rpm
¸í·ÉÀ¸·Î ¼³Ä¡ÇÑ´Ù.


3.2.2. Tuning

ÇÊÀÚÀÇ °æÇè»ó ¼­¹ö OS ÀÇ Ä¿³ÎÆ©´× À̳ª ÃÖÀûÈ­ ¼¼ÆÃÀº ±âº»¼¼Æà µÇ¾îÁø ¼­¹ö ÀÇ ¼º´Éº¸´Ù Ä¿´Ù¶õ ¼º´ÉÇâ»óÀ» °¡Á®¿ÀÁö ¸øÇÑ´Ù. (ÃÖ°íÀÇ ¼º´ÉÂ÷À̸¦ º¸ÀÎ °ÍÀÌ ¾à 15% Á¤µµ¿´´Ù ±×°Íµµ ƯÁ¤±â´É ¿¡¼­¸¸..) ¹°·Ð ¾îÇø®ÄÉÀ̼ÇÀÇ À߸øµÈ ¼³°è³ª DB ¼­¹öÀÇ ºÒ¾ÈÁ¤ÇÑ ¸ðµ¨¸µ µîÀº ½É°¢ÇÑ ¼º´ÉÀå¾Ö¸¦ ÃÊ·¡Çϱ⵵ Çϴµ¥ ÀÌ·±°ÍµéÀº ¾îµð±îÁö³ª À߸øµÈ °ÍÀ» ¹Ù·ÎÀâ´Â °ÍÀÌÁö, Æ©´×ÀÌ ¾Æ´Ï´Ù. ±×°É ¹Ù·ÎÀâ´Â °ÍÀº ¿©·¯ºÐ ¸òÀÌ´Ù. ¸®´ª½º Ä¿³Î Æ©´×À̳ª Device Æ©´×¿¡ ´ëÇؼ­´Â ³»¿ëÀÌ ³Ê¹« ¹æ´ëÇÏ°í ¶Ç Çϵå¿þ¾î¿¡ ƯȭµÈ ³»¿ëµéÀÌ ÀÖÀ» ¼ö¹Û¿¡ ¾ø±â ¶§¹®¿¡ ¿©±â¼­´Â ´Ù·çÁö ¾ÊÀ»°ÍÀÌ´Ù. º°µµÀÇ ÁÁÀº ¹®¼­µéÀ̳ª Ã¥µéÀ» Âü°íÇϱ⠹ٶõ´Ù.


3.3. Networking

3.3.1. »ç¼³ ³×Æ®¿öÅ© ±¸Ãà

»ç¼³ ³×Æ®¿öÅ©¸¦ ±¸ÃàÇÏ´Â ÀÌÀ¯´Â ´ÙÀ½°ú °°´Ù.

  • Àüü Ŭ·¯½ºÅÍ ½Ã½ºÅÛ °ü¸®ÀÇ Æí¸®¼º (¸ðµç »ç¿ëÀÚ°¡ °ü¸®³ëµå ¿¡¼­ ÀÛ¾÷À» ÇÒ°Í À̱⠶§¹®¿¡ Ŭ·¯½ºÅÍ °ü¸®°¡ ÆíÀÌ)

  • º¸¾ÈÀÇ Æí¸®¼º (°ü¸®³ëµå ¸¸ º¸¾È¿¡ ½Å°æ¾²¸é µÈ´Ù ³ª¸ÓÁö´Â »ç¼³¸ÁÀ¸·Î ¹­¿© ÀÖ¾î ¿ÜºÎ ÆÐŶµî¿¡ ÀüÇô ¿µÇâÀ» ¹ÞÁö ¾ÊÀ½)


3.3.2. °ü¸®³ëµå ¼Â¾÷

OS ¼³Ä¡°¡ ³¡³µ´Ù¸é ¸ðµç ³ëµå¿¡¼­ ÀÛ¾÷ÇÒ¼ö ÀÖ´Â ÀÏ¹Ý À¯Àú °èÁ¤À» Çϳª ¸¸µéÀÚ. ½ÇÁ¦ °èÁ¤À̸§Àº ¾î¶°ÇÑ °ÍÀÌµç »ó°ü¾ø´Ù.

[root@master root]# useradd -d /home/share micro
[root@master root]# passwd micro

Ŭ·¯½ºÅÍ ³ëµå°£ ³×Æ®¿öÅ© ¼¼ÆÃÀ» ÇÑ´Ù. ÇÊÀÚÀÇ ¿¬±¸½Ç ³»ºÎ ³×Æ®¿÷Àº »ç¼³¸ÁÀ¸·Î ÀÌ·ç¾îÁ® ÀÖ´Ù. µû¶ó¼­ °ü¸®³ëµå ÀÇ IP ÁÖ¼Ò´Â Á¢¼Ó°¡´ÉÇÑ (³»ºÎ ³×Æ®¿÷¿¡¼­ °ü¸®³ëµå·Î) ip ÁÖ¼Ò¿©¾ß ÇÑ´Ù. ¿©±â¼­ ÁÖ¾îÁö´Â ip ÁÖ¼Ò´Â ÇÊÀÚÀÇ È¯°æ¿¡ ¸Â°Ô ¼¼ÆÃµÈ °Í ÀÌ¸ç ½ÇÁ¦·Î´Â ¾î¶°ÇÑ °ªÀÌ ÁÖ¾îÁ®µµ »ó°ü¾ø´Ù.

[root@master root]# /sbin/ifconfig 
eth0      Link encap:Ethernet  HWaddr 00:E0:81:20:07:84
inet addr:10.1.1.1  Bcast:10.1.1.255  Mask:255.255.255.0
eth0:0    Link encap:Ethernet  HWaddr 00:E0:81:20:07:85
inet addr:192.168.0.64 Bcast:192.168.0.255 Mask:255.255.255.0
eth1      Link encap:Ethernet  HWaddr 00:E0:81:20:07:85
inet addr:10.1.2.1  Bcast:10.1.2.255  Mask:255.255.255.0
	   

eth0 ÀÇ 10.1.1.0~254 C class »ç¼³ IP ´ë¿ªÀº ³ëµå°£ NFS ¸¦ À§ÇÑ ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º ÀÌ´Ù. ¸ðµç ³ëµå´Â eh0 ¸¦ ÅëÇØ NFS ¼­ºñ½º¸¦ ÁÖ°í ¹ÞÀ» °ÍÀÌ´Ù. eth0:0 Àº ÇÊÀÚ°¡ ÀÓÀÇ·Î ¸¶½ºÄ¿·¹À̵ùÀ» ÅëÇØ °¡»ó IP ¸¦ ÁØ°ÍÀÌ¸ç ³»ºÎ ȤÀº ¿ÜºÎ »ç¿ëÀÚ ³×Æ®¿÷ ȯ°æ¿¡¼­ Á¢¼ÓÇÒ IP ÁÖ¼Ò¸¦ ¼¼ÆÃÇÑ´Ù. ¸ðµç »ç¿ëÀÚ ¹× °ü¸®ÀÚ´Â °ü¸®³ëµåÀÇ eth0:0 ÀÇ ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º ·Î Á¢¼Ó ÇÒ°ÍÀÌ´Ù. eth1 ÀÇ 10.1.2.0 ~ 254 C class »ç¼³ IP ´ë¿ªÀº °è»êÀÛ¾÷ ¿ë Åë½ÅÀ» À§ÇÑ ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º ÀÌ´Ù. °ü¸®¿ë, NFS ¿ë Åë½Å¸Á°ú ³í¸®ÀûÀ¸·Î ºÐ¸®¸¦ ½ÃÄÑ µ¶¸³ÀûÀÎ ÀÎÅÍÆäÀ̽º¸¦ »ç¿ëÇϵµ·Ï ÇÑ´Ù. ÇÑÆí ¶ó¿ìÆÃÀº ¾Æ·¡¿Í °°ÀÌ ¼³Á¤ÇØ ÁÖ¾î¾ß ÇÑ´Ù. netconfg ¸í·ÉÀ̳ª xwindow »ó¿¡¼­ netcfg ¸í·ÉÀ¸·Î ¼³Á¤ÇÒ¼ö ÀÖ´Ù.

[root@master root]# /sbin/route 
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *              	255.255.255.0   U     0      0        0 eth0
10.1.1.0        *               255.255.255.0   U     0      0        0 eth0
10.1.2.0        *               255.255.255.0   U     0      0        0 eth1
127.0.0.0       *               255.0.0.0       U     0      0        0 lo
default         your.gateway   	0.0.0.0         UG    0      0        0 eth0
	   

/etc/hosts ÆÄÀÏÀ» ÆíÁý ÇÑ´Ù. Ŭ·¯½ºÅ͸¦ ÀÌ·ç°í ÀÖ´Â °¢³ëµåÀÇ ÁÖ¼Ò¿Í À̸§À» Á¤ ÀÇ ÇØÁØ´Ù. ÀÌ°ÍÀº °ü¸®³ëµå¿Í ¸ðµç ÄÄÇ»Æà ³ëµå¿¡ µ¿ÀÏÇÏ°Ô ¼³Á¤ÇØ ÁØ´Ù.

[root@master root]# vi /etc/hosts
# °¢ node ¿ë ip ´ë¿ª
10.1.2.1        node00          master
10.1.2.2        node01
10.1.2.3        node02
10.1.2.4        node03
10.1.2.5        node04
10.1.2.6        node05
10.1.2.7        node06
127.0.0.1       localhost
# NFS ¿ë IP ´ë¿ª
10.1.1.1	nfsserver
10.1.1.2	nfs01
10.1.1.3	nfs02
10.1.1.4	nfs03
10.1.1.5	nfs04
10.1.1.6	nfs05
10.1.1.7	nfs06
	   

°øÀ¯µð·ºÅ丮 ¸¦ ¼³Á¤ ÇÑ´Ù. ¿©±â¼± /home/share ¸¦ °øÀ¯µð·ºÅ丮·Î ¼³Á¤ÇÑ´Ù. ´ÙÀ½°ú °°ÀÌ /etc/exports ÆÄÀÏÀ» ÆíÁýÇÏÀÚ.

[root@master root]# cat /etc/exports
/home/share     *(rw)
	   

±×¸®°í ´ÙÀ½ÀÇ ¸í·ÉÀ» »ç¿ëÇÏ¿© nfs ¼­¹ö¸¦ ´Ù½Ã ½ÃÀÛÇØÁÖÀÚ.

[root@master root]# /etc/rc.d/init.d/nfs restart
	   

¸¶½ºÅÍ ³ëµå¿¡ nfs µ¥¸óÀÌ Á¦´ë·Î ¶°ÀÖ´ÂÁö È®ÀÎÇØ º¸ÀÚ. ¾Æ·¡¿Í ºñ½ÁÇÏ°Ô ³ª¿Í¾ß ÇÑ´Ù.

[root@master root]# ps -auex | grep nfs
root      4633  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4634  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4635  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4636  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4637  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4638  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4639  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
root      4640  0.0  0.0     0    0 ?        SW   May01   0:00 [nfsd]
	   

nfs µ¥¸óÀÌ ¶°ÀÖÁö ¾Ê´Ù¸é nfs ´Â RPC(Remote Procedure Call) À» »ç¿ëÇϱ⠶§¹®¿¡ Æ÷Æ®¸ÅÆÛ (Port mapper) ¶ó´Â µ¥¸óÀÌ ¸ÕÀú ¶°ÀÖ¾î¾ß ÇÑ´Ù. À̸¦ È®ÀÎÇØ º¸ÀÚ.

[root@master root]# rpcinfo -p
program vers proto   port
100000    2   tcp    111  portmapper
100000    2   udp    111  portmapper
¡¦¡¦¡¦
	   

ÀÌ µ¥¸óÀÌ ¶° ÀÖÁö ¾Ê´Ù¸é ´ÙÀ½°ú °°Àº ¸í·ÉÀ» »ç¿ëÇÏ¿© portmapper ¸¦ ¸ÕÀú ¶ç¿ì°í nfs ¸¦ Àç½ÃÀÛ Çغ¸ÀÚ.

[root@master root]# /etc/rc.d/init.d/portmap start
	   

À§ µÎ°¡Áö ¸ðµÎ Á¤»óÀûÀ¸·Î ÀÛµ¿ÇÑ´Ù¸é nfs °¡ Á¦´ë·Î µ¿ÀÛ ÇÒ°ÍÀÌ´Ù. ±×·¯³ª ±×·¸ Áö ¾Ê´Ù¸é nfs °¡ Ä¿³Î¿¡¼­ Áö¿øµÇÁö ¾Ê´Â °æ¿ìÀϼö ÀÖ´Ù. À̸¦ À§Çؼ­´Â Ä¿³Î ÄÄ ÆÄÀÏÀ» ÇØÁÖ¾î¾ß ÇÑ´Ù. Ä¿³Î ÄÄÆÄÀϽà nfs °ü·Ã ¿É¼ÇµéÀ» ¸ðµÎ Ä¿³Î¿¡ Æ÷ÇÔ ½ÃÄÑ ÁÖ¾î¾ß ÇÑ´Ù. ÀÌ°ÍÀº Ä¿³Î ÄÄÆÄÀÏ¿¡ Àͼ÷ÇÑ »ç¿ëÀÚ ¶ó¸é º° ¾î·Á¿òÀÌ ¾ø°ÚÁö¸¸ ±× ·¸Áö ¾ÊÀº »ç¿ëÀÚ¶ó¸é KLDP homepage¿¡¼­ "kernel compile" À̶ó´Â °Ë»ö¾î·Î °Ë»öÇÏ¸é ¸¹Àº ¹®¼­¸¦ ãÀ»¼ö ÀÖÀ»°ÍÀÌ´Ù. À̸¦ Âü°íÇÏÀÚ. Ä¿³Î ÄÄÆÄÀÏ ¿É¼Ç¿¡¼­ FileSystem -> Network File Systems a NFS °ü·ÃµÈ ¿É¼ÇÀº ¸ðµÎ ¼±ÅÃÀ» ÇÏ°í ÄÄÆÄÀÏ À» ÇÑ´Ù. Ä¿³Î ÄÄÆÄÀÏÀ» ¸¶ÃÆÀ¸¸é Ä¿³Î°ú ¸ðµâµéÀ» ÀûÀçÇÏ°í ÀçºÎÆà ÇÑÈÄ nfs ¸¦ ´Ù½Ã ½ÃÀÛ ÇØ º¸ÀÚ.

ÀÌ»óÀÌ ¾øÀ¸¸é rsh À» Çã°¡ÇÑ´Ù. Ŭ·¯½ºÅÍ ³»ÀÇ ¸ðµç ³ëµåµé°£¿¡ rsh ÀÌ °¡´ÉÇϵµ ·Ï ¼¼ÆÃÇØ¾ß ÇÑ´Ù. ÀÌ°ÍÀº º´·Ä ¶óÀ̺귯¸® ÀÇ ÇϳªÀÎ MPI (Message Passing Interface) °¡ Á¤»óÀÛµ¿Çϱâ À§ÇØ ÇÊ¿äÇÏ´Ù. rsh °ü·Ã ¼¼ÆÃÀº °ü¸® ³ëµå¿Í ÄÄÇ»Æà ³ëµå¿¡ µ¿ÀÏÇÏ°Ô ¼³Á¤ÇØ Áà¾ß¸¸ ÇÑ´Ù. /etc/xinetd.d µð·ºÅ丮 ¿¡¼­ rsh , rlogin, rexec µîÀÇ ÆÄÀÏÀ» ÆíÁýÇÑ´Ù.

[root@master root]# vi /etc/xinetd.d/rsh
service shell
	{
        disable = no
	socket_type = stream
	...
	  

À§¿Í °°ÀÌ disable ¶õÀ» no ·Î ¹Ù²ãÁÖÀÚ. ÇØ´ç ÆÄÀÏÀ» ¸ðµÎ ¼öÁ¤ÇßÀ¸¸é xinetd µ¥ ¸óÀ» Àç½ÃÀÛ Çϸé Àû¿ëÀÌ µÈ´Ù.

[root@master root]# /etc/rc.d/init.d/xinetd restart
	  

ÀÌÁ¦ ¾î¶² ÄÄÇ»ÅÍ µéÀÌ Àڽſ¡°Ô rsh ÀÌ °¡´ÉÇϵµ·Ï Çã°¡ÇÒÁö¸¦ Á¤ÀÇÇØ Áà¾ß ÇÑ´Ù. °ü·Ã ÆÄÀÏÀº .rhosts ÆÄÀÏ¿¡ Àû¿ëÇÏ´Â ¹ý°ú /etc/hosts.equiv ÆÄÀÏ¿¡ Àû¿ëÇϴ¹ý ÀÌ Àִµ¥ ¿©±â¼­´Â /etc/hosts.equiv ¿¡ Àû¿ëÇϵµ·Ï ÇÏ°Ú´Ù.

[root@master root]# cat /etc/hosts.equiv
master node00
node01
node02
node03
node04
node05
node06
	  

/etc/hosts.equiv ÆÄÀÏÀÌ ¾øÀ¸¸é ¸¸µé°í À§¿Í °°ÀÌ È£½ºÆ® ³×ÀÓÀ» Àû¾îÁÖÀÚ. ´ÙÀ½Àº root À¯Àú°¡ rsh À» ÀÌ¿ëÇÏ¿© ·Î±×ÀÎ ÇÒ¼ö ÀÖµµ·Ï /etc/securetty ÆÄÀÏÀ» ¼öÁ¤ÇØ ÁÖµµ·Ï ÇÑ´Ù.

[root@master root]# vi /etc/securetty
vc/1
vc/2
vc/3
¡¦¡¦
tty1
tty2
¡¦¡¦
rsh
rexec
rlogin
	  

À§¿Í °°ÀÌ ¸¶Áö¸· ¶óÀο¡ rsh °ü·Ã ¸í·ÉÀ» Ãß°¡ ½ÃŲ´Ù. ´ÙÀ½À¸·Î ³ëµå°£ rsh , rlogin ½Ã Æнº¿öµå ÁúÀÇ ÀýÂ÷¸¦ »ý·«Çϱâ À§Çؼ­ /etc/pam.d µð·ºÅ丮ÀÇ rsh, rlogin, rexec ÆÄÀÏÀ» ¼öÁ¤ÇØÁÖÀÚ.

	  [root@master root]# vi /etc/pam.d/rsh
#%PAM-1.0
auth       sufficient   /lib/security/pam_nologin.so
auth       sufficient   /lib/security/pam_securetty.so
auth       sufficient     /lib/security/pam_env.so
auth       sufficient     /lib/security/pam_rhosts_auth.so
account    sufficient   /lib/security/pam_stack.so service=system-auth
session    sufficient   /lib/security/pam_stack.so service=system-auth
[root@master root]# vi /etc/pam.d/rlogin
#%PAM-1.0
# For root login to succeed here with pam_securetty, "rlogin" must be
# listed in /etc/securetty.
auth       sufficient   /lib/security/pam_nologin.so
auth       sufficient   /lib/security/pam_securetty.so
auth       sufficient     /lib/security/pam_env.so
auth       sufficient   /lib/security/pam_rhosts_auth.so
auth       sufficient   /lib/security/pam_stack.so service=system-auth
account    sufficient   /lib/security/pam_stack.so service=system-auth
password   sufficient   /lib/security/pam_stack.so service=system-auth
session    sufficient   /lib/security/pam_stack.so service=system-auth
[root@master root]# vi /etc/pam.d/rexec 
#%PAM-1.0
# For root login to succeed here with pam_securetty, "rexec" must be
# listed in /etc/securetty.
auth       sufficient   /lib/security/pam_nologin.so
auth       sufficient   /lib/security/pam_securetty.so
auth       sufficient     /lib/security/pam_env.so
auth       sufficient   /lib/security/pam_stack.so service=system-auth
account    sufficient   /lib/security/pam_stack.so service=system-auth
session    sufficient   /lib/security/pam_stack.so service=system-auth
	  

¼öÁ¤À» ³¡¸¶ÃÆÀ¸¸é xinetd µ¥¸óÀ» Àç½ÃÀÛ ÇÑ´Ù.

	  [root@master root]# /etc/rc.d/init.d/xinetd restart
	  

3.3.3. ÄÄÇ»Æóëµå ¼Â¾÷

°ü¸®³ëµå¿Í ¸¶Âù°¡Áö·Î ¸®´ª½º¸¦ ¼³Ä¡Çϵµ·Ï ÇÏÀÚ ±×¸®°í °¢ »ç¿ëÀÚ´Â ¸ðµç ³ëµå¿¡ µ¿ÀÏÇÑ °èÁ¤À» °¡Áö°í ÀÖ¾î¾ß ÇÑ´Ù. ±ÇÀåÇÏ´Â ¹æ¹ýÀº °ü¸®³ëµåÀÇ /etc/passwd, /etc/passwd- ÆÄÀÏ°ú /etc/shadow, /etc/shadow- ÆÄÀϵé, /etc/group , /etc/group- ÆÄÀϵéÀ» ¸ðµç ³ëµå¿¡ Ä«ÇÇ ÇÏ´Â °ÍÀÌ´Ù. ÀÌ·Î½á °ü¸®³ëµå¿¡¼­ Ãß°¡ÇÑ À¯Àú°èÁ¤ µéÀ» ¸ðµç ³ëµå¿¡¼­ »ç¿ëÇÒ¼ö ÀÖ°Ô µÇ¾ú´Ù. ´ÙÀ½Àº ³×Æ®¿öÅ© ¼³Á¤¿¡ µé¾î°¡º¸ÀÚ. ÀüüÀûÀÎ ¼³Á¤Àº °ü¸®³ëµå¿Í Å©°Ô ´Ù¸¦¹Ù°¡ ¾ø´Ù.

           [root@node01 root]# /sbin/ifconfig 
eth0      Link encap:Ethernet  HWaddr 00:E0:81:20:07:84
inet addr:10.1.1.2 Bcast:10.1.1.255  Mask:255.255.255.0
eth1      Link encap:Ethernet  HWaddr 00:E0:81:20:07:85
inet addr:10.1.2.2  Bcast:10.1.2.255  Mask:255.255.255.0
           

eth0 ÀÇ 10.1.1.0~254 C class »ç¼³ IP ´ë¿ªÀº ³ëµå°£ NFS ¸¦ À§ÇÑ ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º ÀÌ´Ù. ¸ðµç ³ëµå´Â eh0 ¸¦ ÅëÇØ NFS ¼­ºñ½º¸¦ ÁÖ°í ¹ÞÀ» °ÍÀÌ´Ù. eth1 ÀÇ 10.1.2.0 ~ 254 C class »ç¼³ IP ´ë¿ªÀº °è»êÀÛ¾÷ ¿ë Åë½ÅÀ» À§ÇÑ ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º ÀÌ´Ù. °ü¸®¿ë °ú NFS ¿ë Åë½Å¸Á°ú ³í¸®ÀûÀ¸·Î ºÐ¸®¸¦ ½ÃÄÑ µ¶¸³ÀûÀÎ ÀÎÅÍÆäÀ̽º¸¦ »ç¿ëÇÑ´Ù. ÇÑÆí ¶ó¿ìÆÃÀº ¾Æ·¡¿Í °°ÀÌ ¼³Á¤ÇØ ÁÖ¾î¾ß ÇÑ´Ù. ±âº» °ÔÀÌÆ®¿þÀÌ ´Â °ü¸® ³ëµå°¡ µÈ´Ù. netconfg ¸í·ÉÀ̳ª xwindow »ó¿¡¼­ netcfg ¸í·ÉÀ¸·Î ¼³Á¤ÇÒ¼ö ÀÖ´Ù.

	   [micro@node01 share]$ /sbin/route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.1.1.0        *               255.255.255.0   U     0      0        0 eth0
10.1.2.0        *               255.255.255.0   U     0      0        0 eth1
127.0.0.0       *               255.0.0.0       U     0      0        0 lo
default         node00          0.0.0.0         UG    0      0        0 eth1
	   

etc/hosts ÆÄÀÏÀ» ÆíÁý ÇÑ´Ù. Ŭ·¯½ºÅ͸¦ ÀÌ·ç°í ÀÖ´Â °¢³ëµåÀÇ ÁÖ¼Ò¿Í À̸§À» Á¤ÀÇ ÇØÁØ´Ù. ÀÌ°ÍÀº °ü¸®³ëµå¿Í ¸ðµç ÄÄÇ»Æà ³ëµå¿¡ µ¿ÀÏÇÏ°Ô ¼³Á¤ÇØ ÁØ´Ù.

[root@master root]# vi /etc/hosts
# °¢ node ¿ë ip ´ë¿ª
10.1.2.1        node00          master
10.1.2.2        node01
10.1.2.3        node02
10.1.2.4        node03
10.1.2.5        node04
10.1.2.6        node05
10.1.2.7        node06
127.0.0.1       localhost
# NFS ¿ë IP ´ë¿ª
10.1.1.1		nfsserver
10.1.1.2		nfs01
10.1.1.3		nfs02
10.1.1.4		nfs03
10.1.1.5		nfs04
10.1.1.6		nfs05
10.1.1.7		nfs06
	   

°øÀ¯µð·ºÅ丮 ¸¦ ¼³Á¤ ÇÑ´Ù. ¿©±â¼± /home/share ¸¦ °øÀ¯µð·ºÅ丮·Î ¼³Á¤ÇÑ´Ù. ÀÌ¹Ì °ü¸®³ëµå ¿¡¼­´Â ¸ðµç node °¡ °ü¸®³ëµå ÀÚ½ÅÀÇ /home/share ¸¦ ¸¶¿îÆ® Çϴ°ÍÀ» Çã¶ôÇß´Ù. µû¶ó¼­ ÄÄÇ»Æà ³ëµå ¿¡¼­´Â °ü¸®³ëµåÀÇ /home/share °ø°£À» /etc/fstab ¿¡ Ãß°¡ÇØ ÁÖµµ·Ï ÇÏÀÚ.

	   [root@node01 /]# vi /etc/fstab
¡¦¡¦¡¦¡¦
node00:/home/share      /home/share             nfs     bg,intr,noac    0 0
	   

°ü¸®³ëµå node00 ÀÇ /home/share ¸¦ ÀÚ½ÅÀÇ /home/share ·Î ¸¶¿îÆ® ÇÏ ´Â ¸í·ÉÀÌ´Ù. ÆÄÀϽýºÅÛÀº nfs ÀÌ°í nfs ¸¶¿îÆà ¿É¼ÇÀº bg, intr, noac À¸ ·Î ¼¼Æà ÇØÁØ´Ù. bg ´Â ¸¸¾à ù¹ø° NFS ¸¶¿îÆ® ½Ãµµ°¡ ŸÀӾƿô¿¡ °É¸®¸é ¹é±×¶ó¿îµå ¿¡¼­ ½ÇÇàÀ» °è¼ÓÇÑ´Ù. noac ¿É¼ÇÀº ¹Ýµå½Ã ¼³Á¤ÇØÁà¾ß ÇÑ´Ù. MPI ¶óÀ̺귯¸® (ƯÈ÷ MPICH) ¿¡¼­ NFS V3 ¿Í I/O °ü·Ã Ãæµ¹ ¹®Á¦°¡ ¸®Æ÷Æà µÇ¾ú´Âµ¥, °ü·Ã ¹ö±×¸¦ ¸·±â À§ÇØ noac ¿É¼ÇÀÌ ÇÊ¿äÇÏ´Ù. ÀÌ°ÍÀº NFS ÀÇ ¼Ó¼º ij½¬¸¦ ÇØÁ¦ÇÏ´Â °ÍÀ¸·Î ¼­¹ö È¿À²À» ¶³¾î¶ß¸®±â´Â ÇÏÁö¸¸ µÎ°³ÀÇ ´Ù¸¥ NFS Ŭ¶óÀ̾ðÆ®·Î ÇÏ¿©±Ý ¼­¹ö»óÀÇ °øÅë ÆÄÀÏ ½Ã½ºÅÛ¿¡ ¾²±â ÀÛ¾÷À» ÇÒ¶§ ÁÁÀº È¿À²À» ¾òÀ»¼ö ÀÖ°Ô ÇØÁØ´Ù. ÀÌÁ¦ /home/share °øÀ¯µð·ºÅ丮 ¸¦ ¸¶¿îÆà Çغ¸ÀÚ.

	   [root@node01 /]# mount -t nfs node00:/home/share /home/share
	   

ÀÌ»ó¾øÀÌ ¸¶¿îÆÃÀÌ µÇ¾úÀ¸¸é rsh ¼Â¾÷¿¡ µé¾î°¡µµ·Ï ÇÑ´Ù. rsh ¼¼ÆÃÀº ±â º»ÀûÀ¸·Î °ü¸®³ëµå¿Í ´Ù¸£Áö ¾Ê´Ù. rsh °ü·Ã ¼Â¾÷ÀÌ ¿Ï·á µÇ¾úÀ¸¸é ÀÌÁ¦ °¢ È£½ºÆ® °£¿¡´Â rsh ¸í·ÉÀÌ °¡´ÉÇØ Áú °ÍÀÌ´Ù. ´ÙÀ½Àº rsh ÀÌ Á¤»óÀûÀ¸·Î µ¿ÀÛ ÇÏ´ÂÁö È®ÀÎÇØ º¸µµ·Ï ÇÑ´Ù. °ü¸® ³ëµå¿¡¼­ ÄÄÇ»Æà ³ëµå·Î rsh ÀÌ µÇ´ÂÁö È®ÀÎÇØ º¸ÀÚ.

	   [micro@master micro]$ rsh node01
Last login: ¡¦¡¦¡¦
[micro@node01 micro]$
	   

ÄÄÇ»Æà ³ëµå¿¡¼­ °ü¸®³ëµå·Î rsh ÀÌ µÇ´ÂÁö È®ÀÎÇØ º¸ÀÚ.

	   [micro@node01 micro]$ rsh master
Last login: ¡¦¡¦¡¦
[micro@master micro]$
	   

3.3.4. ½Ã°£ µ¿±âÈ­

Ŭ·¯½ºÅÍ °£ ½Ã°£À» µ¿±âÈ­ ÇÑ´Ù´Â°Ç ¸Å¿ì Áß¿äÇÑ ÀÏÀÌ´Ù. ¾î¶°ÇÑ ÀÌÀ¯°¡ ÀÖ´õ¶óµµ Ŭ·¯½ºÅÍ°£¿¡ ½Ã°£À» ¸ÂÃçÁà¾ß ÇÑ´Ù. ¸ðµç ½Ã½ºÅÛÀÇ ½Ã°¢À» ÇöÀç½Ã°¢À¸·Î ¸ÂÃá´Ù. ±× ´ÙÀ½ ±× ½Ã°¢À» CMOS ½Ç½Ã°£ ½Ã°£À¸·Î "hwclock -w" À» ÀÌ¿ë º¯°æÇÑ´Ù. ½Ã°£µ¿±âÈ­ÀÇ ÁÁÀº ÆÐÅ°ÁöÁß Çϳª´Â ¼öÇÐÀÚ Ãâ½ÅÀÎ D.J.Bernstein ÀÌ ¸¸µç clockspeed °¡ ÀÖ´Ù. ÀÚ¼¼ÇÑ ³»¿ëÀº ´ÙÀ½¿¡¼­ È®ÀÎÇÒ¼ö ÀÖ´Ù. http://cr.yp.to/clockspeed.html ȤÀº http://kldp.org/~eunjea/clockspeed.php À§ÀÇ url ¿¡¼­ clockspeed ¼Ò½º¸¦ ´Ù¿î·Îµå ÇÑ´Ù.

[root@master opt]# wget http://cr.yp.to/clockspeed/clockspeed-0.62.tar.gz
¡¦¡¦¡¦¡¦¡¦¡¦
[root@master opt]# tar xzf clockspeed-0.62.tar.gz
[root@master opt]# cd clockspeed-0.62
[root@master opt]# make
[root@master opt]# make setup check
	   

ÆÄÀϵéÀº /usr/local/clockspeed ¿¡ ¼³Ä¡µÈ´Ù. ½ÇÇàÆÄÀϵéÀº /usr/local/clockspeed/bin ¿¡ ¼³Ä¡µÇ¹Ç·Î »ç¿ëÀÚÀÇ $PATH ¿¡ Ãß°¡ÇØ ÁÖµµ·Ï ÇÑ´Ù. ´ÙÀ½Àº ½ÇÁ¦ µ¿±âÈ­ ÇÏ´Â °úÁ¤ÀÌ´Ù. ¸ÕÀú ntp ¼­¹ö¿Í Ŭ·¯½ºÅͼ­¹ö ¿ÍÀÇ ½Ã°£ µ¿±âÈ­¸¦ ÇÑ´Ù. ¿©±â¼­´Â ntp ¼­¹ö·Î ntp.nasa.gov ¸¦ ÀÌ¿ëÇß´Ù.

[root@master /]# nslookup 
¡¦¡¦¡¦¡¦¡¦¡¦¡¦
> ntp.nasa.gov
¡¦¡¦¡¦¡¦¡¦¡¦¡¦
Non-authoritative answer:
ntp.nasa.gov    canonical name = ntp-nasa.arc.nasa.gov.
Name:   ntp-nasa.arc.nasa.gov
Address: 198.123.30.132   ©¬----- ntp ¼­¹ö ÁÖ¼Ò
[root@master /]# sntpclock 198.123.30.132 | clockadd 
	   

ÀÌÁ¦ ntp ¼­¹ö¿Í Ŭ·¯½ºÅÍ ¿ÍÀÇ ½Ã°£À» µ¿±âÈ­ Çß´Ù. ÀÌÁ¦´Â Á»Àü¿¡ ntp ¼­¹ö¿Í µ¿±âÈ­¸¦ ¸ÂÃá master ¼­¹ö¿Í ³ª¸ÓÁö Ŭ·¯½ºÅÍµé °£ÀÇ µ¿±âÈ­ ÀÛ¾÷À» Çϵµ·Ï ÇÏÀÚ. µ¿±âÈ­ÀÇ ±âÁØÀÌ µÇ´Â ¼­¹ö´Â master ¼­¹ö ÀÌ´Ù. ±âº»ÀûÀÎ ½ÇÇàÀº master ¼­¹ö¿¡¼­ taiclockd ¼­¹ö µ¥¸óÀ» ½ÇÇà½ÃÄÑ ÁÖ°í ³ª¸ÓÁö ¼­¹ö¿¡¼­ taiclock Ŭ¶óÀ̾ðÆ® ¸¦ »ç¿ëÇÏ¿© µ¿±âÈ­ ¸¦ ½ÃÄÑÁÖ´Â ¹æ½ÄÀÌ´Ù.

[micro@master /]$ taiclockd &   ©¬--- master ¼­¹ö (root ±ÇÇÑÀ¸·Î ½ÇÇàÇÏÁö ¾Ê¾Æµµ µÊ)
[micro@node01 /]$ taiclock 10.1.2.1 > /tmp/adjust   ©¬--- Ŭ¶óÀ̾ðÆ® ¼­¹ö
[micro@node01 /]$ clockview < /tmp/adjust  ©¬-- ½Ã°£Â÷ È®ÀÎ
before: 2003-05-06 10:50:05.824066000000000000
after:  2003-05-06 10:50:32.290600000000000000
	   

¾à 23ÃÊ Á¤µµ Â÷ÀÌ°¡ ³ª°í ÀÖ´Ù. ½Ã°£À» ¸ÂÃçÁÖÀÚ.

[micro@node01 /]$ clockadd < /tmp/adjust 
[root@node01 /]# /sbin/hwclock -w 
	   

À§ÀÇ °úÁ¤À» Ŭ·¯½ºÅÍ ½Ã½ºÅÛ ¿¡¼­ Á¤±âÀûÀ¸·Î ½ÇÇàÇÏ¸é µÈ´Ù. ÇÊÀÚ´Â master ¼­¹ö ¿¡ ´ÙÀ½°ú °°ÀÌ shell script ¸¦ ¸¸µé¾î cron À» »ç¿ëÇÏ¿© Á¤±âÀûÀ¸·Î Ŭ·¯½ºÅÍ°£ ½Ã°£À» µ¿±âÈ­ ÇÏ°í ÀÖ´Ù.

[root@master bin]# vi /root/bin/sync_nodes_time.sh
#!/bin/sh

#
# First, correct time of this machine
#
/usr/local/clockspeed/bin/sntpclock 198.123.30.132 | \ 
/usr/local/clockspeed/bin/clockadd 
/sbin/hwclock -w 
#
# Next, sync time of each nodes
#

echo Start taiclockd ...
/usr/local/clockspeed/bin/taiclockd &

echo Correct time of node01
rsh node01 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
echo Correct time of node02
rsh node02 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
echo Correct time of node03
rsh node03 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
echo Correct time of node04
rsh node04 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
echo Correct time of node05
rsh node05 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
echo Correct time of node06
rsh node06 '/usr/local/clockspeed/bin/taiclock 10.1.3.1 > /tmp/adjust ; \
                /usr/local/clockspeed/bin/clockadd < /tmp/adjust; \
                /sbin/hwclock -w'
               
echo Stop taiclockd ...
killall taiclockd

echo done.
# end of file
	   

3.4. Parallel Library

3.4.1. ¼³Ä¡Çϱ⿡ ¾Õ¼­

º´·Ä ¶óÀ̺귯¸® ¿¡ »ç¿ëµÇ´Â Middle ware ´Â Ŭ·¯½ºÅÍ ½Ã½ºÅÛÀÏ °æ¿ì Å©°Ô PVM(Parallel Virtual Machine) °ú MPI (Message Passing Interface) ·Î ³ª´·¼ö ÀÖ´Ù. ¾çÂÊ ¸ðµÎ °ü·Ã¹®¼­°¡ »ó´çÈ÷ ¸¹ÀÌ ÀÖÀ¸¸ç ÇÑ°¡Áö Â÷ÀÌÁ¡À» ²ÅÀ¸¸é PVM Àº À̱âÁ¾°£ÀÇ ½Ã½ºÅÛ¿¡¼­ µ¹¾Æ°£´Ù´Â Á¡°ú MPI ´Â µ¿±âÁ¾°£ÀÇ Å¬·¯½ºÅÍ ¿¡¼­ °í·ÁµÈ ¶óÀ̺귯¸® ¶ó´Â Á¡ÀÌ´Ù. PVM VS MPI ¿¡ °ü·ÃµÈ ¸¹Àº ³íÀï°Å¸®°¡ ²Ï ¿À·¡ÀüºÎÅÍ ÁøÇàµÇ¾î ¿Ô°í ¾ÆÁ÷µµ ³íÀïÀÌ ¹ú¾îÁö´Â °÷µµ ÀÖ´Ù. ¿©±â¼­´Â µ¿±âÁ¾ °£ÀÇ Å¬·¯½ºÅÍ ½Ã½ºÅÛÀ¸·Î ²Ù¹Ð°Í À̹ǷΠMPI ¸¦ ¼³Ä¡ ÇÒ°ÍÀÌ´Ù. MPI ´Â º´·Ä ¶óÀ̺귯¸®ÀÇ »ç½Ç»óÀÇ Ç¥ÁØÀ¸·Î ÀÚ¸®Àâ¾Ò´Ù. ´ëÇ¥ÀûÀÎ MPI ¶óÀ̺귯¸® ¿¡´Â LAM-MPI ¿Í MPICH °¡ ¸¹ÀÌ »ç¿ëµÈ´Ù. ±âº»ÀûÀ¸·Î MPI ÀÇ Åë½ÅÀº rsh À» ÀÌ¿ëÇÏ¸ç º´·Ä ÇÁ·Î¼¼¼­°£ Á¡´ëÁ¡ (Point-to-Point) Åë½ÅÀ» ÇÑ´Ù. MPI ¸¦ ÀÌ¿ëÇÏ¿© ½ÇÇàÇÒ º´·Ä ÇÁ·Î±×·¥Àº ³í¸®Àû, ȤÀº ¹°¸®ÀûÀ¸·Î Ŭ·¯½ºÅÍ °¢ ³ëµå°£ µ¿ÀÏÇÑ µð·ºÅ丮¿¡ Á¸ÀçÇØ¾ß ÇÑ´Ù.


3.4.2. LAM-MPI

LAM-MPI ¸¦ ¼³Ä¡ÇØ º¸ÀÚ. http://www.lam-mpi.org ¿¡¼­ lam ÃֽŹöÀüÀ» ¹Þ¾Æ¿ÀÀÚ. À̱ÛÀ» ¾²´Â ½ÃÁ¡¿¡¼­´Â 6.5.9 °¡ ÃֽŹöÀü ÀÌ´Ù. Âü°í·Î »õ·Î¿î ¹öÀüÀÌ ³ª¿Ã¶§ ¼³Ä¡ ¹æ¹ýÀÌ º¯°æµÉ¼ö ÀÖÀ¸¹Ç·Î ´Ù¿î¹ÞÀº ÆÄÀÏ¿¡ ÀÖ´Â ¼³Ä¡¹®¼­¸¦ Âü°íÇϱ⠹ٶõ´Ù.

[root@master root]# wget http://www.lam-mpi.org/download/files/lam-6.5.9.tar.gz
--15:07:35--  http://www.lam-mpi.org/download/files/lam-6.5.9.tar.gz
           => `lam-6.5.9.tar.gz'
Resolving www.lam-mpi.org... done.
Connecting to www.lam-mpi.org[129.79.245.239]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2,184,991 [application/x-gzip]

100%[==========================================>] 2,184,991    197.43K/s    ETA 00:00
15:07:52 (197.43 KB/s) - `lam-6.5.9.tar.gz' saved [2184991/2184991]
[root@master root]# tar xzf lam-6.5.9.tar.gz
[root@master root]# cd lam-6.5.9
[root@master root]# ./configure --prefix=/usr/local/lam
¡¦¡¦¡¦¡¦¡¦¡¦
[root@master root]# make
¡¦¡¦¡¦¡¦¡¦¡¦
[root@master root]# make install
¡¦¡¦¡¦¡¦¡¦¡¦
	   

ÀÌÁ¦ lam-mpi ¼³Ä¡°¡ ³¡³µ´Ù. ±×·¯³ª ½ÇÇàÀ» À§Çؼ­ ¸î°¡Áö ¼³Á¤ÀÌ ³²¾Æ ÀÖ´Ù. »ç¿ëÀÚ°¡ LamMPI ¸¦ »ç¿ëÇϱâ À§Çؼ­´Â ¸ÕÀú ¸î°¡Áö ȯ°æº¯¼ö¸¦ ¼³ Á¤ÇØ Áà¾ß ÇÑ´Ù. ¸ðµç »ç¿ëÀÚ°¡ µ¿ÀÏÇÏ°Ô Àû¿ë ¹Þµµ·Ï /etc/bashrc ³ª /etc/profile ¿¡ ´ÙÀ½À» Ãß°¡ÇØ ÁÖÀÚ.

[root@master root]# vi /etc/bashrc
¡¦¡¦¡¦¡¦¡¦¡¦
LAMHOME=/usr/local/lam
PATH=$PATH:/usr/local/lam/bin
export LAMHOME PATH
	   

ÀÌÁ¦ ¸ðµç »ç¿ëÀÚ°¡ ·Î±×ÀÎ ÇÒ¶§ LAMHOME °ú PATH °¡ Àû´çÇÏ°Ô ¼³Á¤ µÉ °ÍÀÌ´Ù. ÀÌÁ¦ Lam À» ½ÇÇàÇØ º¸µµ·Ï ÇÏÀÚ. Lam team ¿¡¼­ ±ÇÀåÇÏ´Â ¹æ¹ýÀº ¸Ó½Å³×ÀÓ ÀÌ ±â¼úµÇ¾î ÀÖ´Â ÆÄÀÏÀ» ¸¸µé¾î¼­ lamd µ¥¸ó°ú µ¿½Ã¿¡ ºÎÆà ½ÃÅ°´Â °ÍÀÌ´Ù.

[root@master root]# vi /etc/lamhosts 
node00
node01
¡¦¡¦¡¦
node06
	   

À§¿Í °°ÀÌ lamhosts ÆÄÀÏÀ» ¸¸µé¾î¼­ lamd ÀÇ ½Ãµ¿ ÇÁ·Î±×·¥ÀÎ lamboot ¿¡ ÆĶó¸ÞÅ͸¦ ³Ñ°ÜÁÖµµ·Ï ÇÏÀÚ. Lamd µ¥¸óÀÇ ½ÇÇà¹æ¹ýÀº ´ÙÀ½°ú °°´Ù. Root °èÁ¤ÀÌ ¾Æ´Ñ ÀÏ¹Ý °èÁ¤À¸·Î ½ÇÇàÇØ¾ß ÇÑ´Ù.

[micro@master micro]$ lamboot -v -b /etc/lamhosts 
LAM 6.5.9/MPI 2 C++/ROMIO - Indiana University

Executing hboot on n0 (node00 - 1 CPU)...
Executing hboot on n1 (node01 - 1 CPU)...
Executing hboot on n2 (node02 - 1 CPU)...
Executing hboot on n3 (node03 - 1 CPU)...
Executing hboot on n4 (node04 - 1 CPU)...
Executing hboot on n5 (node05 - 1 CPU)...
Executing hboot on n6 (node06 - 1 CPU)...
topology done
	  

À§¿Í °°Àº ¸Þ½ÃÁö°¡ ³ª¿Í¾ß ÇÑ´Ù. ±×¸®°í ½ÇÇàµÇ¾î ÀÖ´Â lamd ÀÇ »óŸ¦ º¸·Á¸é lamnodes ·Î È®ÀÎÇØ º¸¸é µÈ´Ù.

[micro@master micro]$ lamnodes
n0      node00:1
n1      node01:1
n2      node02:1
n3      node03:1
n4      node04:1
n5      node05:1
n6      node06:1
	  

lamd µ¥¸óÀ» ½ÇÇàÇØ ºÃÀ¸´Ï shutdown µµ Çغ¸µµ·Ï ÇÏÀÚ. Lamd µ¥¸óÀÇ ÁßÁö´Â wipe ÀÇ ½ÇÇàÀ¸·Î °¡´ÉÇÏ´Ù.

[micro@master micro]$ wipe -v -b /etc/lamhosts 
LAM 6.5.9/MPI 2 C++/ROMIO - Indiana University 

Executing tkill on n0 (node00)...
Executing tkill on n1 (node01)...
Executing tkill on n2 (node02)...
Executing tkill on n3 (node03)...
Executing tkill on n4 (node04)...
Executing tkill on n5 (node05)...
Executing tkill on n6 (node06)...
	  

Lam-mpi ¸¦ ÀÌ¿ëÇÏ¿© º´·Ä ÇÁ·Î±×·¥À» ½ÇÇà½ÃÄÑ º¸ÀÚ. Lamd µ¥¸óÀ» ¶ç¿î´Ù. ÇÊÀÚ´Â °£´ÜÇÑ shell script ¸¦ ¸¸µé¾î¼­ /etc/rc.d/init.d/ µð·ºÅä ¿¡ À§Ä¡½ÃÅ°°í runlevel À̳ª command »ó¿¡¼­ lamd µ¥¸óÀ» start, stop ÇÒ¼ö ÀÖµµ·Ï Çß´Ù. ´ÙÀ½Àº ÇØ´ç script ÆÄÀÏÀÌ´Ù. ÁÖÀÇÇÒ Á¡Àº ¿©·¯ºÐÀÌ ½Ç ÇàÇϴ Ŭ·¯½ºÅÍ È¯°æ°ú ÇÊÀÚ°¡ ±¸ÇöÇÑ È¯°æÀÌ ¼­·Î ´Ù¸¦¼ö ÀÖÀ¸´Ï »ç¿ëÀÚ È¯°æ¿¡ ¸Â°Ô ¾à°£ÀÇ ¼öÁ¤À» °¡ÇØ¾ß ÇÒ°ÍÀÌ´Ù.

[micro@master micro]$ vi /etc/rc.d/init.d/lamd 
#! /bin/sh
#

# PATH your lam-mpi
#export PATH=$PATH:/usr/local/lam/bin

# Source function library.
. /etc/init.d/functions

# See how we were called.

case "$1" in
  start)
        echo -n "Starting lamd"
        if [ "$UID" == "500" ];then  		# lam À» ½ÇÇàÇÒ ÀÏ¹Ý À¯Àú°èÁ¤ÀÇ UID
                lamboot -v -b /etc/lamhosts
        else
                su - micro -c 'lamboot -v -b /etc/lamhosts' 
        fi
        echo
        ;;
  stop)
        echo -n "Stopping lamd"
        if [ "$UID" == "500" ];then
                wipe -v -b /etc/lamhosts
        else
                su - micro -c 'wipe -v -b /etc/lamhosts'
        fi
        echo
        ;;
  status)
        echo -n "shows status nodes..."
        if [ "$UID" == "500" ];then
                lamnodes
        else
                su - micro -c 'lamnodes'
        fi
        echo
        ;;
  restart)
        $0 stop
        $0 start
        ;;
  *)
        echo $"Usage: $0 {start|stop|status|restart}"
        exit 0
esac

exit $?
[micro@master micro]$ /etc/rc.d/init.d/lamd start 
Starting lamd
LAM 6.5.9/MPI 2 C++/ROMIO - Indiana University

Executing hboot on n0 (node00 - 1 CPU)...
Executing hboot on n1 (node01 - 1 CPU)...
Executing hboot on n2 (node02 - 1 CPU)...
Executing hboot on n3 (node03 - 1 CPU)...
¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦
	
	  

´ÙÀ½Àº Ŭ·¯½ºÅÍ ÇÁ·Î¼¼¼­ °£¿¡ °£´ÜÇÑ ¸Þ½ÃÁö Àü´ÞÀ» ÁÖ°í¹Þ´Â C ·Î ¸¸µç MPI ÇÁ·Î±×·¥ ÀÌ´Ù

/*
 * Laboratory for Scientific Computing
 * http://www.lam-mpi.org/tutorials/
 * University of Notre Dame
 *
 * MPI Tutorial
 * The cannonical ring program
 *
 * Mail questions regarding tutorial material to mpi@lam-mpi.org
 */

#include <stdio.h>
#include "mpi.h"    /* MPI Header ¸¦ ÀÎŬ·çµå ÇؾßÇÔ */


int main(int argc, char *argv[])
{
  MPI_Status status;
  int num, rank, size, tag, next, from;

  /* Start up MPI */

  MPI_Init ( ,  );
  MPI_Comm_rank(MPI_COMM_WORLD, );
  MPI_Comm_size(MPI_COMM_WORLD, );
 
  /* Arbitrarily choose 201 to be our tag.  Calculate the */
  /* rank of the next process in the ring.  Use the modulus */
  /* operator so that the last process "wraps around" to rank */
  /* zero. */

  tag = 201;
  next = (rank + 1) % size;
  from = (rank + size - 1) % size;

  /* If we are the "console" process, get a integer from the */
  /* user to specify how many times we want to go around the */
  /* ring */

  if (rank == 0) {
    printf("Enter the number of times around the ring: ");
    scanf("%d", #) ;

    printf("Process %d sending %d to %d\n", rank, num, next);
    MPI_Send(#, 1, MPI_INT, next, tag, MPI_COMM_WORLD);
  }

  /* Pass the message around the ring.  The exit mechanism works */
  /* as follows: the message (a positive integer) is passed */
  /* around the ring.  Each time is passes rank 0, it is decremented. */
  /* When each processes receives the 0 message, it passes it on */
  /* to the next process and then quits.  By passing the 0 first, */
  /* every process gets the 0 message and can quit normally. */

  while (1) {

    MPI_Recv(#, 1, MPI_INT, from, tag, MPI_COMM_WORLD, );
    printf("Process %d received %d\n", rank, num);

    if (rank == 0) {
      num--;
      printf("Process 0 decremented num\n");
    }

    printf("Process %d sending %d to %d\n", rank, num, next);
    MPI_Send(#, 1, MPI_INT, next, tag, MPI_COMM_WORLD);

    if (num == 0) {
      printf("Process %d exiting\n", rank);
      break;
    }
  }

  /* The last process does one extra send to process 0, which needs */
  /* to be received before the program can exit */

  if (rank == 0)
    MPI_Recv(#, 1, MPI_INT, from, tag, MPI_COMM_WORLD, );

  /* Quit */

  MPI_Finalize();
  return 0;
}
	  

¾Õ¼­µµ ¾ê±âÇßµíÀÌ º´·Ä·Î ½ÇÇàÇÒ ÇÁ·Î±×·¥Àº ³í¸®Àû ÀÌ´ø ¹°¸®ÀûÀÌ´ø ÇÁ·Î±×·¥À» ½ÇÇàÇÒ °¢°¢ Ŭ·¯½ºÅÍ °£¿¡ µ¿ÀÏÇÑ À§Ä¡(Path) ¿¡ Á¸ÀçÇØ¾ß ÇÑ´Ù. Ŭ·¯½ºÅÍ°£¿¡ NFS, ȤÀº ´Ù¸¥ °¡»ó ÆÄÀϽýºÅÛ À¸·Î °øÀ¯ÁßÀÎ À§Ä¡¿¡ ÇÁ·Î±×·¥À» À§Ä¡½ÃÅ°°Å³ª ȤÀº ƯÁ¤ µð·ºÅ丮¿¡¼­ ½ÇÇàÇÒ°æ¿ì ¸ðµç ³ëµå°£¿¡ µ¿ÀÏÇÑ À§Ä¡¿¡ Ä«ÇǸ¦ ÇÑÈÄ ½ÇÇàÀ» ÇϸéµÈ´Ù. ÀÌÁ¦ ÄÄÆÄÀÏÀ» Çغ¸ÀÚ. gcc ³ª egcs ´ë½Å lam-mpi ÀÇ mpicc ¸¦ ÀÌ¿ëÇØ¾ß ÇÑ´Ù.

[micro@master micro]$ mpicc -o ring MPI_C_SAMPLE.c
	  

ÀÌÁ¦ ÄÄÆÄÀÏÇÑ º´·ÄÇÁ·Î±×·¥À» µ¹·Áº¸µµ·Ï ÇÏÀÚ. ÇÁ·Î±×·¥ ½ÇÇàÀº mpirun ¸í·ÉÀ¸·Î ½ÇÇàÇϸç mpirun ¿¡´Â ¿©·¯°¡Áö ¿É¼ÇÀÌ ÀÖ´Ù. °ü·Ã ³»¿ëÀº man mpirun À̳ª mpirun --help ¸í·ÉÀ¸·Î ¾Ë¼ö°¡ ÀÖ´Ù.

[micro@master micro]$ mpirun -O -np 2 ./ring 
Enter the number of times around the ring: 150 
¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦
Process 1 received 3
Process 1 sending 3 to 0
Process 0 received 3
Process 0 decremented num
Process 0 sending 2 to 1
Process 1 received 2
Process 1 sending 2 to 0
Process 0 received 2
Process 0 decremented num
Process 0 sending 1 to 1
Process 1 received 1
Process 1 sending 1 to 0
Process 0 received 1
Process 0 decremented num
Process 0 sending 0 to 1
Process 0 exiting
Process 1 received 0
Process 1 sending 0 to 0
Process 1 exiting
	   

lam-mpi ÀÇ mpirun ÀÇ -O ¿É¼ÇÀº ÇÁ·Î±×·¥À» ½ÇÇàÇÒ Å¬·¯½ºÅ͸¦ ±âº»ÀûÀ¸·Î homogeneous system (µ¿±âÁ¾) À¸·Î °¡Á¤ÇÑ´Ù. µû¶ó¼­ ¸Þ½ÃÁö ÆнÌÇϴµ¥ À־ º°µµÀÇ transaction ÀýÂ÷¸¦ °ÅÄ¡Áö ¾Ê´Â´Ù. Low type À¸·Î Á÷Á¢ Åë½ÅÇÏ´Â °ÍÀÌ´Ù. -np 2 ´Â ÇÁ·Î¼¼¼­ÀÇ ¼ö¸¦ 2°³·Î ½ÇÇàÇÑ´Ù´Â °ÍÀÌ´Ù. ÇØ´çÇÁ·Î±×·¥À» 2 node ½Ã½ºÅÛ(CPU °¡ 1°³ ÀÏ°æ¿ì) ¿¡¼­ ½ÇÇàÇÑ´Ù´Â °ÍÀ» ¶æÇÑ´Ù. ÀÌ»ó¾øÀÌ ÇÁ·Î±×·¥ÀÌ µ¹¾Æ°¬´Ù¸é ±âº»ÀûÀΠŬ·¯½ºÅÍ ½Ã½ºÅÛÀÌ ±¸¼ºµÈ°ÍÀÌ´Ù. °°Àº ÇÁ·Î±×·¥À» ¿©·¯ºÐÀÇ Å¬·¯½ºÅÍ ³ëµå °³¼ö¿¡ ¸Â°Ôµµ µ¹·Áº¸µµ·Ï ÇÏÀÚ.

¸¶Áö¸·À¸·Î lam-mpi ÀÇ °ü·ÃµÈ ¿©·¯Á¤º¸´Â man À» ÀÌ¿ë mpirun , lamclean , lamboot µîÀÇ ÇÊ¿äÇÑ Á¤º¸¸¦ ¾òÀ»¼ö ÀÖÀ¸¸ç ¶ÇÇÑ http://www.lam-mpi.org ¿¡¼­ ¹«¼öÈ÷ ¸¹Àº Á¤º¸¸¦ ¾òÀ»¼ö ÀÖ´Ù. (FAQ ³ª Tutorial µîÀº Çѹø¾¿ Àо±â¸¦ ±ÇÀåÇÑ´Ù)


3.4.3. MPICH

MPICH (MPI Chameleon) À» ¼³Ä¡ÇØ º¸ÀÚ. ¾Õ¼­ ¾ê±âÇßµíÀÌ °¡Àå ¸¹ÀÌ »ç¿ëµÇ°í ÀÖ´Â MPI ¶óÀ̺귯¸® Áß Çϳª°¡ ¹Ù·Î MPICH ÀÌ´Ù. ´Ù¾çÇÑ Çϵå¿þ¾î¸¦ Áö¿øÇϸç, °¢Á¾ ½Ã½ºÅÛ ÆÛÆ÷¸Õ½º º¥Ä¡¸¶Å© ÇÁ·Î±×·¥ÀÇ Ã´µµ·Î µµ ¸¹ÀÌ »ç¿ëµÇ°í ÀÖ´Ù. ¸¹Àº ºÐ·®ÀÇ À¯Àú Guide ¿Í ¸Å´º¾óÀÌ ÀÖÀ¸¸ç MPICH ¸¦ »ç¿ëÇÏ°Ô µÈ´Ù¸é ÃÖ¼ÒÇÑ Àû¾îµµ CH_P4MPD(http://www.mcs.anl.gov/mpi/mpich/docs/mpichman-chp4mpd/mpichman-chp4mpd.htm) Device ¿¡ °ü·ÃµÈ Manual (ÀÌ ¹®¼­¿¡¼­´Â ch_p4mpd device ¸¦ ¼±ÅÃÇÏ¿© ¼³Ä¡ ÇÒ°ÍÀÌ´Ù.) °ú FAQ (http://www-unix.mcs.anl.gov/mpi/mpich/docs/faq.htm) ¸¦ Àо±â¸¦ ±ÇÀåÇÑ´Ù. Áغñ µÇ¾úÀ¸¸é ¼Ò½ºÆÄÀÏÀ» ´Ù¿î·Îµå ÇÏÀÚ.

[micro@master share]$ wget ftp://ftp.mcs.anl.gov/pub/mpi/mpich.tar.gz
	   

¾ÕÃàÀ» Ç®°í ¼³Ä¡¸¦ ÇÑ´Ù. ¿©±â¼­´Â /usr/local/mpich-1.2.5 ¿¡ ¼³Ä¡¸¦ ÇÒ°ÍÀÌ´Ù.

[micro@master share]$ tar xzf mpich.tar.gz 
[micro@master share]$ cd mpich-1.2.5 
[micro@master mpich-1.2.5]$ ./configure --with-device=ch_p4mpd --prefix=/usr/local/mpich-1.2.5 
[micro@master mpich-1.2.5]$ make; make install
	   

ÀÌ»ó¾øÀÌ ¼³Ä¡°¡ µÇ¾ú´Ù¸é /usr/local/mpich-1.2.5/share/machines.LINUX ¿¡ »ç¿ëÇÒ È£½ºÆ®³×ÀÓ À» Àû¾îÁØ´Ù.

[micro@master mpich-1.2.5]$ vi /usr/local/mpich-1.2.5/share/machines.LINUX
node00
node01
node03
¡¦¡¦¡¦
node06
	  

³ª¸ÓÁö Ŭ·¯½ºÅÍ ¿¡¼­µµ °°Àº ¿É¼ÇÀ¸·Î ¼³Ä¡¸¦ ÇÑ´Ù. ¸ðµç Ŭ·¯½ºÅÍ¿¡ ¼³Ä¡°¡ µÇ¾ú´Ù¸é °£´ÜÇÑ sample ÇÁ·Î±×·¥À» µ¹·Áº¸µµ·Ï ÇÏÀÚ. Mpich ¸¦ ÀÌ¿ëÇÏ¿© º´·ÄÇÁ·Î±×·¥À» ½ÇÇàÇÏ·Á¸é LAM-MPI ¿Í ¸¶Âù°¡Áö·Î MPICH Àü¿ë µ¥¸óÀÎ MPD °¡ ¸ÕÀú ¶°ÀÖ¾î¾ß ÇÑ´Ù. MPD ¸¦ ½ÇÇà½ÃÅ°ÀÚ. ÀϹݰèÁ¤À¸·Î ½ÇÇà½ÃÄѵµ µÈ´Ù.<¼öÁ¤> MPICH ¸Þ´º¾ó¿¡ ³ª¿ÍÀÖµíÀÌ MPD ¸¦ ½ÇÇà½Ãų À¯Àú Ȩ µð·ºÅ丮¿¡ .mpd.conf ÆÄÀÏÀ» »ý¼ºÇØÁØ´Ù. password=[your password] -> ÃÖ¼ÒÇÑÀÇ º¸¾ÈÁ¤Ã¥À» ÁؼöÇϱâÀ§ÇØ »ý¼ºÇÏ´Â °ÍÀÌ´Ï Æнº¿öµå´Â ¾Æ¹«·¡µµ »ó°üÀÌ ¾ø´Ù.

[micro@master share]$ echo 'password=dkdkdk' > .mpd.conf
[micro@master share]$ chmod 600 .mpd.conf

[micro@master share]$ /usr/local/mpich-1.2.5/bin/mpd &
	  

¸¶½ºÅÍ ¼­¹ö¿¡¼­ mpd ¸¦ ½ÇÇà½ÃÅ°°í ³ª¸ÓÁö ¼­¹ö¿¡¼­´Â ½ÇÇà¹æ¹ýÀÌ ¾à°£ ´Ù¸£´Ù. ¸ÕÀú mpd µ¥¸óÀÇ ¼ÒÄÏ ¹øÈ£¸¦ ¾Ë¾Æ¾ß ÇÑ´Ù. ´ÙÀ½ÀÇ ¸í·ÉÀ¸·Î ¾Ë ¼ö°¡ ÀÖ´Ù.

[micro@master share]$ /usr/local/mpich-1.2.5/bin/mpdtrace 
mpdtrace: master_33253: ihs=node00_33253  rhs=node00_33253  rhs2=node00_33253 gen=2
	  

À§¿¡¼­ º¸¸é mpd µ¥¸óÀÌ 33253 ¼ÒÄÏÀ¸·Î ¹ÙÀεù µÇ¾ú´Ù´Â °ÍÀ» ¾Ë¼öÀÖ´Ù. ³ª¸ÓÁö Ŭ·¯½ºÅÍ ¿¡¼­µµ mpd µ¥¸óÀ» ½ÇÇà½ÃÄÑ ÁØ´Ù. Mpd µ¥¸ó³¢¸®ÀÇ ¼Ò ÄÏ Åë½ÅÀ» À§ÇØ ¸¶½ºÅÍ ¼­¹ö¿Í ¿¬µ¿ÇÑ´Ù.

[micro@master share]$ rsh node01
[micro@node01 share]$ /usr/local/mpich/bin/mpd -h master -p 33253 &
	  

-h ¿É¼ÇÀº Ŭ·¯½ºÅÍ ¸¶½ºÅÍ ¼­¹öÀÇ hostname À» Àû¾îÁÖ¸é µÇ¸ç, -p ¿É¼Ç À¸·Î ¼ÒÄϹøÈ£¸¦ Àû¾îÁÖ¸é µÈ´Ù. Node01 »Ó ¾Æ´Ï¶ó ³ª¸ÓÁö Ŭ·¯½ºÅÍ¿¡¼­ µµ mpd ¸¦ ½ÇÇà½ÃŲÈÄ ¿¹Á¦·ÎµÈ º´·ÄÇÁ·Î±×·¥À» ½ÇÇàÇغ¸µµ·Ï ÇÏÀÚ. ¿¹Á¦ ÇÁ·Î±×·¥Àº mpich examples ¿¡ ÀÖ´Â ¿øÁÖÀ²À» ±¸ÇÏ´Â °£´ÜÇÑ ÇÁ·Î±×·¥ ÀÌ´Ù.

[micro@master share]$ cd mpich-1.2.5/examples/basic 
[micro@master basic]$ make cpi 
[[micro@master basic]$ /usr/local/mpich-1.2.5/bin/mpirun -np 4 ./cpi 
Process 0 of 4 on node00
Process 1 of 4 on node06
Process 2 of 4 on node05
Process 3 of 4 on node04
pi is approximately 3.1415926544231239, Error is 0.0000000008333307
wall clock time = 0.002665
	 

À§¿Í ºñ½ÁÇÑ °á°ú°¡ ³ª¿Í¾ß ÇÑ´Ù. ¸¸ÀÏ ¿Ã¹Ù¸¥ °á°ú°¡ ³ª¿ÀÁö ¾Ê°Å³ª ¿¡·¯ °¡ ¹ß»ýÇß´Ù¸é, mpich ¸Å´º¾óÀ» Âü°í¸¦ Çϰųª, Case of Trouble (http://www-unix.mcs.anl.gov/mpi/mpich/docs/mpichman-chp4mpd/node59.htm#Node59) À» Âü°íÇϵµ·Ï ÇÏÀÚ.


4장. Benchmark

4.1. ¼Ò°³

ÇöÀç ¸¸µé¾îÁø ½Ã½ºÅÛ »ó¿¡¼­ °¢Á¾ ±âÁØÀÌ µÉ¼öÀÖ´Â ÆÛÆ÷¸Õ½º¸¦ ÃøÁ¤ÇÏ´Â °ÍÀº ¸Å¿ì Áß¿äÇÑ ÀÏÀÌ´Ù. Benchmark ´Â ½Ã½ºÅÛÀÇ ¼º´ÉÀ» üũÇغ»´Ù´Â Á߿伺µµ ÀÖÁö¸¸ ´õ ³ª¾Æ°¡ ÇâÈÄ System À» Upgrade ÇØ ³ª°¡´Â °úÁ¤Áß¿¡ ¼º´ÉÀÌ ½ÇÁ¦·Î Çâ»óµÇ¾ú´À³Ä´Â °üÁ¡¿¡¼­ Áß¿äÇÑ ÁöħÀÌ µÇ´Â°ÍÀÌ´Ù. ¹°·Ð ÀÚ½ÅÀÌ »ç¿ëÇÏ°í ÀÖ´Â º´·Ä Application À» µ¹·Áº¸´Â ÀÏÀº ¿ì¼±ÀûÀÌ´Ù. ±×·¯³ª ±×¹Û¿¡ ÀϹÝÀûÀÎ °ø°³/Ç¥ÁØ Benchmark Program À» µ¹·Áº»´Ù¸é ´Ù¸¥ Cluster ½Ã½ºÅÛ°ú »óÈ£ ¼º´Éºñ±³¸¦ ÇÒ ¼ö ÀÖ´Â Áß¿ä ÀڷḦ È®º¸ Çϴ°ÍÀÌ´Ù. ÈçÈ÷ ½±°Ô±¸ÇÒ¼ö ÀÖ´Â ´ëÇ¥ÀûÀÎ º´·Ä Benchmark Program µéÀº ´ÙÀ½°ú °°´Ù.


4.2. NetPIPE (Network Protocol Independent Performance Evaluator)

³×Æ®¿÷ ÆÛÆ÷¸Õ½º ÃøÁ¤ º¥Ä¡¸¶Å© ÇÁ·Î±×·¥. Ethernet Ä«µå, ȤÀº ³×Æ®¿÷ ÀÎÅÍÆäÀ̽º °£¿¡ °£´ÜÇÑ Ping-Pong Å×½ºÆ®¸¦ ÇÏ¿© ¸Þ½ÃÁö Size ¸¦ º¯°æÇØ°¡¸é¼­ ´Ù¾çÇÑ ÃøÁ¤À» ÇÑ´Ù. ¿©·¯Á¾·ùÀÇ Çϵå¿þ¾î¸¦ Áö¿øÇϸç, PVM, MPI , MPI-2 µî º´·Ä ¼ÒÇÁÆ®¿þ¾î¿Í ³×Æ®¿÷ ÆÛÆ÷¸Õ½º ÀÇ º´ÇÕÃøÁ¤µµ °¡´ÉÇÏ´Ù.


4.3. NPB (Nas Parallel Benchmark)

CFD(Computational Fluid Dynamics) ¾îÇø®ÄÉÀÌ¼Ç ¿¡ °ü·ÃÇÑ 8°³ÀÇ ´Ù¾çÇÑ Benchmark Test. ³×Æ®¿÷ ¼º´É»Ó ¾Æ´Ï¶ó Computational ¼º´É µî Cluster ÀÇ ´Ù¾çÇÑ ¼º´ÉµéÀ» Á¾ÇÕÀûÀ¸·Î °¡´ÆÇØ º¼¼ö ÀÖ´Â ÀǹÌÀÖ´Â Benchmark Program.


4.4. SCALAPACK (Scalable LAPACK)

SCALAPACK Àº ¿©·¯ ¼±Çü´ë¼ö Math Library µé·Î ÀÌ·ç¾îÁø º¥Ä¡¸¶Å© ÇÁ·Î±×·¥ÀÌ´Ù. ƯÈ÷ ÀÌ°ÍÀº Àü¼¼°è ½´ÆÛÄÄÇ»ÅÍ ·©Å·(http://www.top500.org) À» ÃøÁ¤ÇÒ ¶§ ¾²¿©Áö´Â º¥Ä¡¸¶Å© ·Î½á Network ¼º´Éº¸´Ù´Â Computing ¼º´ÉÀÌ ÁÖ·Î ¹Ý¿µµÈ´Ù.


4.5. Install & Run

ÀÌÁ¦ ¿©·¯ºÐÀÇ Å¬·¯½ºÅÍ ½Ã½ºÅÛ¿¡ º¥Ä¡¸¶Å© ÇÁ·Î±×·¥À» ¼³Ä¡Çغ¸°í Á÷Á¢ ¼º´ÉÀ» üũ Çغ¸ÀÚ. ÀÌ ¹®¼­¿¡¼­´Â NetPIPE ¿Í SCALAPACK À» ¼³Ä¡, Å×½ºÆ® Çغ¼°ÍÀÌ´Ù.


4.5.1. Å×½ºÆ® »ç¾ç

  • CPU : AMD MP 1900+ 1.5Ghz 256KB Cash (SINGLE)

  • RAM : 2Giga PC 2100 ECC Memory

  • HDD : 10000 RPM SCSI

  • NIC : 3Com Gigabit Card, 3Com 10/100 Lancard


4.5.2. NetPIPE

NetPIPE ´Â ¸Ó½Å°£¿¡ ȤÀº ¿¡´õ³Ý Ä«µå°£¿¡ ³×Æ®¿÷ ¼º´ÉÀ» ÃøÁ¤ÇÒ¼ö ÀÖ´Â º¥Ä¡¸¶Å© ÇÁ·Î±×·¥ ÀÌ´Ù. PVM °ú MPI µÑ´Ù Áö¿øÇÏ°í ÀÖÀ¸¸ç ¿©·¯ ±âÁ¾ÀÇ Çϵå¿þ¾î µµ Áö¿øÇÏ°í ÀÖ´Ù. ÀνºÅçÀ» Çغ¸ÀÚ. ´ÙÀ½ÀÇ url ¿¡¼­ ¼Ò½º¸¦ ´Ù¿î¹Þ´Â´Ù. À̱ÛÀ» ¾²´Â ÇöÀç NetPIPE ÀÇ ÃֽŹöÀüÀº 3.3 ÀÌ´Ù. http://www.scl.ameslab.gov/netpipe/ Ŭ·¯½ºÅÍ°£ °øÀ¯µð·ºÅ丮 (ÇÊÀÚÀÇ È¯°æ¿¡¼­´Â /home/share ) ¿¡¼­ ÀÛ¾÷À» ÇÏ´Â °ÍÀÌ Á»´õ Æí¸®ÇÏ´Ù.

[micro@master share]$ tar xzf NetPIPE_3.3.tar.gz 
[micro@master share]$ cd NetPIPE_3.3
	  

Makefile À» ÆíÁýÇϵµ·Ï ÇÏÀÚ. ´ëºÎºÐ ¼Õ´îºÎºÐÀº °ÅÀÇ ¾ø°í MPI ÀÇ ¼³Á¤ ºÎºÐ¸¸ ÆíÁýÇØÁÖ¸é µÈ´Ù. MPICC ÄÄÆÄÀÏ·¯ ¸¦ ÇØ´ç »ç¿ëÀÚÀÇ ½Ã½ºÅÛ ¿¡ ¸Â ´Â ÄÄÆÄÀÏ·¯¸¦ ÁöÁ¤ÇØÁÖ¸é µÈ´Ù. Áï LAM-MPI ·Î NetPIPE ¸¦ º¥Ä¡¸¶Å· ÇÒ°æ¿ì´Â LAM ÀÇ mpicc ¸¦ ÁöÁ¤ÇØÁÖ¸é µÇ°í MPICH ÀÇ ¼º´ÉÀ» Å×½ºÆ® Çغ¸·Á¸é MPICH ÀÇ mpicc ¸¦ ÁöÁ¤ÇØÁÖ¸é µÈ´Ù.

[micro@master NetPIPE_3.3]$ vi makefile 
# For MPI, mpicc will set up the proper include and library paths
MPICC       = /usr/local/mpich/bin/mpicc    	# ©¬ À̺κÐ
¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦.
MPI2CC   = /usr/local/mpich/bin/mpicc		# ©¬ À̺κÐ
	  

¼öÁ¤À» ÇßÀ¸¸é ÄÄÆÄÀÏÀ» Çغ¸µµ·Ï ÇÑ´Ù. ´Ü¼øÈ÷ tcp ÀÇ ¼º´ÉÀ» Å×½ºÆà ÇØ º¸±â À§Çؼ± make tcp ·Î ÄÄÆÄÀÏ ÇÏ¸é µÈ´Ù.

[micro@master NetPIPE_3.3]$ make tcp
	  

±âº»ÀûÀ¸·Î NetPIPE ÀÇ ÆÛÆ÷¸Õ½º ÃøÁ¤Àº ¾ç¹æÇâ ping-pong Å×½ºÆ® ÀÌ´Ù. Tcp ¼º´É º¥Ä¡¸¶Å· À» Çϴ°æ¿ì ÇÑÂÊ¿¡¼± receiver °¡ µÇ°í ´Ù¸¥ ÇÑÂÊ¿¡ ¼± sender ·Î ½ÇÇà½ÃÄÑ ÁÖ¸é µÈ´Ù.

[micro@master NetPIPE_3.3]$ ./NPtcp -r &
[micro@master NetPIPE_3.3]$ rsh node01
[micro@node01 NetPIPE_3.3]$ ./NPtcp -t -h master 
Send and receive buffers are 512000 and 512000 bytes
(A bug in Linux doubles the requested buffer sizes)
Now starting the main loop
  0:       1 bytes    500 times -->      0.10 Mbps in      78.87 usec
  1:       2 bytes   1267 times -->      0.19 Mbps in      78.89 usec
  2:       3 bytes   1267 times -->      0.29 Mbps in      78.84 usec
  3:       4 bytes    845 times -->      0.39 Mbps in      79.14 usec
4:       6 bytes    947 times -->      0.58 Mbps in      79.00 usec
¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦
123: 8388611 bytes      3 times -->    505.81 Mbps in  126529.34 usec
	 

ÇÊÀÚÀÇ È¯°æ¿¡¼­´Â PEAK °á°ú°¡ ´ÙÀ½°ú °°ÀÌ ³ª¿Ô´Ù.

표 4-1. NetPIPE º¥Ä¡¸¶Å© °á°ú (tcp)

Message SizeBandwidthLatency (Usec)
8.3 Mbytes505 Mbps126529

ÀÌÁ¦ MPI ¸¦ ÀÌ¿ëÇÑ ³×Æ®¿÷ ¼º´ÉÀ» ÃøÁ¤ÇØ º¸µµ·Ï ÇÏÀÚ. Âü°í·Î ÀÌ ¹®¼­¿¡¼­´Â lam-mpi ÀÇ mpi ¼º´ÉÀ» ÃøÁ¤ÇØ º¼°ÍÀÌ´Ù. ¾Õ¼­ makefile ¿¡ mpicc ¸¦ ¼öÁ¤ÇØ ÁáÀ¸¸é ÄÄÆÄÀÏÀ» Çϵµ·Ï ÇÑ´Ù.

[micro@master NetPIPE_3.3]$ make mpi 
	 

Npmpi ÇÁ·Î±×·¥ÀÌ ÄÄÆÄÀÏ µÇ¾úÀ» °ÍÀÌ´Ù. Lam-mpi ÀÇ mpirun À¸·Î ½ÇÇà ÇØ º¸µµ·Ï ÇÏÀÚ.

[micro@master NetPIPE_3.3]$ mpirun -O -np 2 ./Npmpi
0: master
1: node01
0:       1 bytes    500 times -->	0.09 Mbps in      81.29 usec
1:       2 bytes   1230 times -->      0.19 Mbps in      81.40 usec
2:       3 bytes   1228 times -->      0.28 Mbps in      81.38 usec
3:       4 bytes    819 times -->      0.38 Mbps in      81.16 usec
¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦¡¦
123: 8388611 bytes      3 times -->    504.27 Mbps in  126917.33 usec
	 

lam-mpi ÀÇ mpi ³×Æ®¿÷ ÃÖ´ë ¼º´ÉÀº ´ÙÀ½°ú °°ÀÌ ³ª¿Ô´Ù.

표 4-2. NetPIPE º¥Ä¡¸¶Å© °á°ú (mpi)

Message SizeBandwidthLatency (Usec)
8.3 Mbytes508 Mbps94347

¸¶Âù°¡Áö·Î mpich ÀÇ mpi ³×Æ®¿÷ ÆÛÆ÷¸Õ½ºµµ Çѹø Å×½ºÆ® Çغ¸±â ¹Ù¶õ´Ù.


4.5.3. SCALAPACK (LINPACK Benchmark)

SCALAPACK Àº ¼±Çü´ë¼ö ÀÇ Çظ¦ ±¸ÇÏ´Â ÆÐÅ°Áö·Î SCALAPACK ¿¡ ±âº»ÀûÀ¸·Î Æ÷ÇԵǾî ÀÖÀ¸¸ç ¸¹Àº ºÎºÐÀÌ ºÎµ¿¼Ò¼öÁ¡ ¿¬»êÀ¸·Î ±¸¼ºµÇ¾î ÀÖ´Ù. LINPACK º¥Ä¡¸¶Å© ¿¡¼­ ÁßÁ¡ÀûÀ¸·Î »ç¿ëµÇ´Â ·çƾµéÀº Gauss ¼Ò°Å¹ýÀ» ÀÌ¿ëÇÑ N °³ÀÇ ¼±Çü¹æÁ¤½Ä ÀÇ Çظ¦ ±¸ÇÏ´Â °ÍÀ¸·Î BLAS (Basic Linear Algebra Subprograms) ¿¡ Æ÷ÇԵǾî ÀÖ´Ù. BLAS ´Â LINPACK º¥Ä¡¸¶Å© ¿¡¼­ °¡Àå ±âº»ÀÌ µÇ´Â ¶óÀ̺귯¸® ·Î½á ±âº»ÀûÀÎ ¼±Çü´ë¼ö ¿¬»êÇÔ¼ö µéÀ» ±¸ÇöÇسõÀº ÁýÇÕÀÌ´Ù. ÀÌ°ÍÀº Fortran À¸·Î Â¥¿©Á® ÀÖÀ¸¸ç BLAS ¶óÀ̺귯¸® ³»ÀÇ °¢ ÇÔ¼öµéÀº ¿¬»êÀÚ¿Í ¿¬»ê°á°ú°¡ Vector ³Ä Matrix ³Ä ¿¡ µû¶ó °è»ê ·¹º§ÀÌ ³ª´µ¾î Áø´Ù. ÀÌ BLAS ¸¦ ÀÌ¿ëÇØ º¥Ä¡¸¶Å·À» ÇÒ¼öµµ ÀÖÁö¸¸ ATLAS (Automatically Tuned Linear Algebra Software) ¸¦ ÀÌ¿ëÇÏ¿© ÇØ´ç Ç÷§Æû¿¡ ÃÖÀûÈ­µÈ ·çƾ ¶óÀ̺귯¸® ¸¦ »ý¼ºÇÒ¼öµµ ÀÖ´Ù. ÀÌ ¹®¼­¿¡¼­´Â BLAS, BLACS, ATLAS, SCALAPACK À» ÀÌ¿ëÇÏ¿© º¥Ä¡¸¶Å·À» ÇÒ°ÍÀÌ´Ù. ¿©±â¼­´Â ¼³Ä¡ ¹× ½ÇÇàÀÇ °£°áÇÑ Guide ¸¸À» Á¦½ÃÇÒ ¿¹Á¤À̸ç ÀÌ ¹®¼­ ¿ÜÀÇ ±âŸ ÀÚ¼¼ÇÑ ³»¿ëÀº SCALAPACK ÀÇ È¨ÆäÀÌÁö http://www.netlib.org/scalapack/ À̳ª Çѱ¹Å¬·¯½ºÅÍ ±â¼ú¼¾ÅÍ(http://www.cluster.or.kr/board/read.php?table=benchmark) ÀÇ º¥Ä¡¸¶Å© Guide ¸¦ Âü°íÇϱ⠹ٶõ´Ù.


4.5.3.1. LINPACK

http://www.netlib.org/blas/blas.tgz ¿¡¼­ BLAS ¸¦ ´Ù¿î ¹Þ¾Æ¼­ ¼³Ä¡ÇÑ´Ù.

[micro@master share]$ mkdir BLAS
[micro@master share]$ cd BLAS
[micro@master share]$ tar xzf blas.tgz 
	    

ÄÄÆÄÀÏ ÇÑ´Ù. Âü°í·Î ÇØ´ç ÇÁ·Î¼¼¼­¿¡ ÃÖÀûÈ­µÈ ÄÄÆÄÀÏ·¯ ¸¦ »ç¿ëÇÏ¸é ¼º´ÉÇâ»óÀ» º¼¼ö ÀÖ´Ù (Intel ÀÇ pgcc ³ª Compaq ÀÇ ccc µîµî..)

[micro@master share]$ f77 -c *.f 
	    

»ý¼ºµÈ ¿ÀºêÁ§Æ® ÆÄÀϵé (È®ÀåÀÚ°¡ *.o) À» ¶óÀ̺귯¸® ·Î ¸¸µç´Ù.

[micro@master share]$ ar cr blas_LINUX.a *.o 
	    

BLACS (Basic Linear Algebra Communication Subprograms) ¼³Ä¡. BLACS ´Â ´Ù¾çÇÑ ºÐ»ê¸Þ¸ð¸® ȯ°æ¿¡¼­ ÇÁ·Î¼¼¼­°£ ¸Þ½ÃÁö Åë½ÅÀ» À§ ÇÑ ¼±Çü´ë¼ö ¶óÀ̺귯¸® ÀÌ´Ù. PVM °ú MPI ¿ëÀÌ µû·Î ÀÖÀ¸¹Ç·Î ÇÊ¿ä ÇÑ ÆÄÀÏÀ» ´Ù¿î¹Þ´Â´Ù. ¿©±â¼­´Â MPI ¸¦ »ç¿ëÇϹǷΠhttp://www.netlib.org/blacs/ ¿¡¼­ mpiblacs.tgz ¿Í blacstester.tgz ¸¦ ´Ù¿î ¹Þ´Â´Ù. ´ÙÀ½ mpiblacs.tgz ¸¦ ¾ÐÃàÀ» Ç®¸é BLACS µð·ºÅ丮°¡ »ý±ä´Ù.

[micro@master share]$ tar xzf mpiblacs.tgz 
[micro@master share]$ tar xzf blacstester.tgz BLACS/TESTING/*
[micro@master share]$ cd BLACS 
	    

BMAKES µð·ºÅ丮¿¡¼­ ¸Ó½Å¿¡ ¸Â´Â Bmake ÆÄÀÏÀ» BLACS µð·ºÅ丮 ·Î º¹»çÇÑ´Ù.

[micro@master BLACS]$ cp BMAKES/Bmake.MPI-LINUX ./Bmake.inc 
	    

Bmake.inc ÆÄÀÏÀ» ÆíÁýÇÑ´Ù. Bmake.inc ÆÄÀÏÀº 3°¡Áö Section À¸·Î Á¤ÀǵǾî ÀÖÀ¸¸ç °¢ ¼½¼ÇÀº ÄÄÆÄÀÏ °úÁ¤¿¡¼­ ÇÊ¿äÇÑ ¿©·¯ ¸ÅÅ©·Î¸¦ Á¤ ÀÇ ÇÑ´Ù. ¼½¼Ç 1 ¿¡¼­´Â ¶óÀ̺귯¸®¿Í ½ÇÇàÆÄÀÏÀÇ À§Ä¡¸¦ ÁöÁ¤ÇÏ°í make °á°ú·Î »ý¼ºµÇ´Â ÆÄÀÏÀÇ À̸§À» ÁöÁ¤ÇÒ ¶§ ÀÌ¿ëÇÏ´Â ¸ÅÅ©·Î¸¦ Á¤ÀÇÇÑ´Ù. Section 2 ¿¡¼­´Â BLACS ¿¡¼­ ÀÌ¿ëÇÏ´Â C Preprocessor °ª À» Á¤ÀÇÇÑ´Ù. Section 3 ¿¡¼± ÄÄÆÄÀÏ·¯¿Í ¸µÄ¿/·Î´õ ¸¦ ¼³Á¤ÇÏ´Â ¸ÅÅ©·Î ¸¦ Á¤ÀÇÇÑ´Ù.

[micro@master BLACS]$ vi Bmake.inc
#============ SECTION 1: PATHS AND LIBRARIES =======================
SHELL = /bin/sh			<- »ç¿ëÇÒ ½©ÀÇ Á¾·ù

BTOPdir = $(HOME)/BLACS  	<- BLACS ÀÇ Top Level µð·ºÅ丮

COMMLIB = MPI			<- »ç¿ëÇÒ communication ¶óÀ̺귯¸® CMMD, 					MPI, PVM, MPL, NX Áß Çϳª

PLAT = LINUX               	<- Ç÷¿Æû

BLACSdir    = $(BTOPdir)/LIB	<- BLACS ¶óÀ̺귯¸®ÀÇ À§Ä¡
BLACSDBGLVL = 1			<- µð¹ö±ë ·¹º§ (0 = NO, 1 = YES)
BLACSFINIT  = $(BLACSdir)/blacsF77init_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL).a
BLACSCINIT  = $(BLACSdir)/blacsCinit_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL).a
BLACSLIB    = $(BLACSdir)/blacs_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL).a
			<- ¶óÀ̺귯¸® À̸§µé 

MPIdir = /usr/local/mpich		<- MPICH ÀÇ À§Ä¡
MPIdev = ch_p4mpd			<- MPICH Device ÀÇ Á¾·ù
MPIplat = LINUX			
MPILIBdir = $(MPIdir)/$(MPIdev)/lib	<- MPICH ¶óÀ̺귯¸® À§Ä¡
MPIINCdir = $(MPIdir)/$(MPIdev)/include	<- MPICH Çì´õ ÆÄÀÏ À§Ä¡
MPILIB = $(MPILIBdir)/libmpich.a	<- MPICH ¶óÀ̺귯¸® ÆÄÀÏ

BTLIBS = $(BLACSFINIT) $(BLACSLIB) $(BLACSFINIT) $(MPILIB)
			<- Å×½ºÆÿ¡ ÇÊ¿äÇÑ ¶óÀ̺귯¸® µé.
INSTdir = $(BTOPdir)/INSTALL/EXE
TESTdir = $(BTOPdir)/TESTING/EXE
FTESTexe = $(TESTdir)/xFbtest_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL)
CTESTexe = $(TESTdir)/xCbtest_$(COMMLIB)-$(PLAT)-$(BLACSDBGLVL)
#================= End SECTION 1===============================
#============== SECTION 2: BLACS INTERNALS ========================
SYSINC = -I$(MPIINCdir)
INTFACE = -Df77IsF2C	<- Fortran77 ¿¡¼­ C ·Î ÀÎÅÍÆäÀ̽º ¹æ¹ý Add_, NoChange, UpCase, ¶Ç´Â f77IsF2C È®½ÇÇÏÁö ¾ÊÀ» °æ¿ì INSTALL/EXE/xintface ÇÁ·Î±×·¥À» ¼öÇàÇØ º¼°Í
SENDIS =           <- -DSndIsLocBlk ·Î Á¤ÀÇÇϸé MPI_Send °¡
                   locally-blocking ·çƾÀ¸·Î ¼öÇàµÈ¾î ´õ È¿À²ÀûÀÌ´Ù.
		   ºñ¿öµÑ °æ¿ì globally-blockingÀ¸·Î °¡Á¤ÇÑ´Ù.
BUF = 
TRANSCOMM = -DuseMpich <- ¿©±â¿¡ Á¤ÀÇ	µÈ ÆĶó¹ÌÅÍ´Â Ç÷§Æû ¸¶´Ù Ʋ¸®´Ù. ÁÖ¼®¿¡¼­´Â BLACS/INSTALL/xtc_CsameF77 °ú BLACS/INSTALL/xtc_UseMpich ¸¦ ½ÇÇàÇϵµ·Ï Áö½ÃÇÏ°í ÀÖ´Ù. xtc_CsameF77 °ú xtc_UseMpich ¸¦ »ý¼ºÇÏ´Â ¹æ¹ýÀº ´ÙÀ½°ú °°´Ù.
	$ BLACS/INSTALL/make xtc_CsameF77
$ BLACS/INSTALL/make xtc_UseMpich
ÀÌ ÆÄÀÏÀ» ½ÇÇà½ÃÅ°¸é ¼³Á¤ÇÒ °ªÀÌ Ãâ·ÂµÈ´Ù.
	$ BLACS/INSTALL/EXE/mpirun -np 2 xtc_CsameF77
.............
Set TRANSCOMM = -DUseMpich
$ BLACS/INSTALL/EXE/xtc_UseMpich
Set TRANSCOMM = -DuseMpich
WHATMPI =
SYSERRORS =
DEBUGLVL = -DBlacsDebugLvl=$(BLACSDBGLVL)
DEFS1 = -DSYSINC $(SYSINC) $(INTFACE) $(DEFBSTOP) $(DEFCOMBTOP) $(DEBUGLVL)
BLACSDEFS = $(DEFS1) $(SENDIS) $(BUFF) $(TRANSCOMM) $(WHATMPI) $(SYSERRORS)
#================= End SECTION 2===============================
#================= SECTION 3: COMPILERS ============================
F77            = f77            <- fortran ÄÄÆÄÀÏ·¯
#F77NO_OPTFLAGS = -Nx400
F77FLAGS       = $(F77NO_OPTFLAGS) -O
F77LOADER      = $(F77)
F77LOADFLAGS   =
CC             = gcc            <- C ÄÄÆÄÀÏ·¯
CCFLAGS        = -O4
CCLOADER       = $(CC)
CCLOADFLAGS    =

ARCH      = ar
ARCHFLAGS = r
RANLIB    = ranlib
#================= End SECTION 3 ===============================
	   

ÄÄÆÄÀÏ ÇÑ´Ù

[micro@master BLACS]$ make mpi 
	   

LIB/blacs_MPI-LINUX-1.a È­ÀÏÀÌ »ý¼ºµÇ¾î¾ß ÇÑ´Ù. SRC/ µð·ºÅ丮¿¡´Â »ç¿ëÀÚ°¡ È£ÃâÇÒ ¼ö ÀÖ´Â ·çƾµéÀÌ µé¾î ÀÖ°í, ¸ðµÎ C¿Í Fortran77 ÀÎÅÍÆäÀ̽º¸¦ °¡Áö°í ÀÖ´Ù. ¸ðµç non-communication ·çƾµéÀº blacs_ ¶ó´Â Á¢µÎ¾î·Î ½ÃÀ۵ȴÙ. BLACS ³»ºÎ ·çƾ°ú Àü¿ª º¯¼ö(global variables)µéÀº ¸ðµÎ BI_ ¶ó´Â Á¢µÎ¾î¸¦ °¡Áö°í ÀÖ´Ù.

ATLAS (Automatically Tuned Linear Algebra Software) ¼³Ä¡. ATLAS Ȩ (http://math-atlas.sourceforge.net) ¿¡¼­ °ü·Ã ÆÄÀÏÀ» ´Ù¿î ¹Þ¾Æ ¼³Ä¡¸¦ ÇÑ´Ù. À̱ÛÀ» ¾²´Â ½ÃÁ¡¿¡¼­ÀÇ ATLAS ÃֽŹöÀüÀº 3.5.2 ÀÌ´Ù.

[micro@master share]$ tar xzf atlas3.5.2.tar.gz
[micro@master share]$ cd ATLAS
[micro@master ATLAS]$ make config CC=gcc  #CC ¸¦ ÁöÁ¤ÇÏÁö ¾ÊÀ¸¸é gcc °¡ »ç¿ëµÈ´Ù.
[micro@master ATLAS]$ make config
gcc -o xconfig config.c
./xconfig
ATLAS configure started.
160
159
¡¦¡¦
001
Enter number at top left of screen [0]: 160 # È­¸é¿¡ º¸ÀÌ´Â °¡Àå Å« ¼ö¸¦ Àû´Â´Ù.
====================================================================
IMPORTANT
====================================================================
Before going any further, check
http://math-atlas.sourceforge.net/errata.html.
This is the ATLAS errata file, which keeps a running count of all known
ATLAS bugs and system problems, with associated workarounds or fixes.
IF YOU DO NOT CHECK THIS FILE, YOU MAY BE COMPILING A LIBRARY WITH KNOWN BUGS.

Have you scoped the errata file? [y]: y 	# errata ¹®¼­¸¦ Àоµµ·Ï ÇÑ´Ù.
Configure will ask a series of questions, in one of two forms. The first form of question is a menu of choices. One option in almost all menus is
'Other/UNKNOWN'. If you are unsure of the answer, always choose this option.
¡¦¡¦¡¦¡¦..(»ý·«)¡¦¡¦¡¦.
Are you ready to continue? [y]: y 
I need to know if you are using a cross-compiler (i.e., you are compiling on a different architecture than you want the library built for).

Are you using a cross-compiler? [n]: n
Probing to make operating system determination:
Operating system configured as Linux # ¸Â´ÂÁö È®ÀÎ

Probing for architecture:
Architecture is set to ATHLON # ¸Â´ÂÁö È®ÀÎ

Probing for supported ISA extensions:
make[2]: *** [atlas_run] Error 132
make[1]: *** [IRun_SSE1] Error 2
SSE2: NO.
SSE1: DETECTED!
Number of CPUs: 1
Required cache flush detected as : 524288 bytes
Looking for compilers:

F77 = /usr/bin/g77 -funroll-all-loops -O3
CC = /usr/bin/gcc -fomit-frame-pointer -O3 -funroll-all-loops
MCC = /usr/bin/gcc -fomit-frame-pointer -O

Looking for BLAS (this may take a while):
Unable to find usable BLAS, BLASlib left blank.
FINDING tar, gzip, AND gunzip
tar : /bin/tar
gzip : /bin/gzip
gunzip : /bin/gunzip


ATLAS has default parameters for OS='Linux' and system='ATHLON'.
If you want to just trust these default values, you can use express setup,
drastically reducing the amount of questions you are required to answer

use express setup? [y]: y
¡¦¡¦¡¦¡¦¡¦
Enter Architecture name (ARCH) [Linux_ATHLONSSE1]: Enter 

[micro@master ATLAS]$ make install arch=< arch>
	   

arch ´Â ¾ÆÅ°ÅØÃÄ À̸§ÀÌ´Ù. config °úÁ¤¿¡¼­ ¸¶Áö¸·¿¡ Ãâ·ÂµÈ´Ù. À§ÀÇ config °úÁ¤¿¡¼­ Ãâ·ÂµÈ ´ë·Î make install arch=Linux_ATHLONSSE1 ¸¦ ÀÔ·ÂÇÑ´Ù.

[micro@master ATLAS]$ make install arch=Linux_ATHLONSSE1
.........
(»ý·« 1½Ã°£ ÀÌ»ó ¼Ò¿ä)
ATLAS install complete. Examine
ATLAS/bin//INSTALL_LOG/SUMMARY.LOG for details.
	   

´ÙÀ½Àº SCALAPACK À» ¼³Ä¡ÇÑ´Ù. MPICH, BLAS, BLACS °¡ ¼³Ä¡µÇ¾î ÀÖ¾î¾ß ÇÑ´Ù. http://www.netlib.org/scalapack/ ¿¡¼­ ÃֽŹöÀüÀÇ scalapack À» ´Ù¿î¹ÞÀºÈÄ ¾ÐÃàÀ» Ç®¸é SCALAPACK µð·ºÅ丮°¡ »ý±ä´Ù.

[micro@master share]$ tar xzf scalapack.tgz 
	   

Slmake.inc ÆÄÀÏÀ» ÆíÁýÇÑ´Ù. ÀÌ ÆÄÀÏÀº ¸ðµç Makefile ¿¡¼­ include µÇ¾î »ç¿ë µÇ¸ç ¼³Ä¡¿¡ ÇÊ¿äÇÑ ¸ÅÅ©·Î¸¦ Á¤ÀÇÇÑ´Ù. INSTALL µð·ºÅ丮¸¦ ÂüÁ¶ ÇÏ¿© Àڽſ¡°Ô ¸Â´Â Slmake.inc ÆÄÀÏÀ» º¹»çÇÏ¿© ÆíÁýÇÑ´Ù.

[micro@master share]$ cd SCALAPACK
[micro@master SCALAPACK]$ cp INSTALL/SLmake.LINUX ./SLmake.inc
[micro@master SCALAPACK]$ vi SLmake.inc 
	   

´ëºÎºÐÀÇ °ªµéÀº ±âº»°ªÀ» ±×´ë·Î ÀÌ¿ëÇÏ°í, ¾Õ¼­ ATLAS ¸¦ ÀÌ¿ë Ç÷§Æû ¿¡ ÃÖÀûÈ­µÈ BLAS ¸¦ »ý¼ºÇßÀ¸´Ï °ü·Ã¼³Á¤À» ¸ÂÃß¾î ÁÖµµ·Ï ÇÑ´Ù.

############################################################################
#
#  Program:         ScaLAPACK
#
#  Module:          SLmake.inc
#
#  Purpose:         Top-level Definitions
#
#  Creation date:   February 15, 2000
#
#  Modified:
#
#  Send bug reports, comments or suggestions to scalapack@cs.utk.edu
#
############################################################################
#
SHELL         = /bin/sh
#
#  The complete path to the top level of ScaLAPACK directory, usually
#  $(HOME)/SCALAPACK
#
home          = $(HOME)/SCALAPACK
#
#  The platform identifier to suffix to the end of library names
#
PLAT          = LINUX
#
#  BLACS setup.  All version need the debug level (0 or 1),
#  and the directory where the BLACS libraries are
#
BLACSDBGLVL   = 1
BLACSdir      = $(HOME)/BLACS/LIB
#
#  MPI setup; tailor to your system if using MPIBLACS
#  Will need to comment out these 6 lines if using PVM
#
USEMPI        = -DUsingMpiBlacs
#SMPLIB        = /usr/lib/mpi/build/LINUX/ch_p4/lib/libmpich.a
SMPLIB        = /usr/local/mpich/lib/libmpich.a
BLACSFINIT    = $(BLACSdir)/blacsF77init_MPI-$(PLAT)-$(BLACSDBGLVL).a
BLACSCINIT    = $(BLACSdir)/blacsCinit_MPI-$(PLAT)-$(BLACSDBGLVL).a
BLACSLIB      = $(BLACSdir)/blacs_MPI-$(PLAT)-$(BLACSDBGLVL).a
TESTINGdir    = $(home)/TESTING

#
#  PVMBLACS setup, uncomment next 6 lines if using PVM
#
#USEMPI        =
#SMPLIB        = $(PVM_ROOT)/lib/$(PLAT)/libpvm3.a
#BLACSFINIT    =
#BLACSCINIT    =
#BLACSLIB      = $(BLACSdir)/blacs_PVM-$(PLAT)-$(BLACSDBGLVL).a
#TESTINGdir    = $(HOME)/pvm3/bin/$(PLAT)

CBLACSLIB     = $(BLACSCINIT) $(BLACSLIB) $(BLACSCINIT)
FBLACSLIB     = $(BLACSFINIT) $(BLACSLIB) $(BLACSFINIT)

#
#  The directories to find the various pieces of ScaLapack
#
PBLASdir      = $(home)/PBLAS
SRCdir        = $(home)/SRC
TESTdir       = $(home)/TESTING
PBLASTSTdir   = $(TESTINGdir)
TOOLSdir      = $(home)/TOOLS
REDISTdir     = $(home)/REDIST
REDISTTSTdir  = $(TESTINGdir)
#
#  The fortran and C compilers, loaders, and their flags
#
F77           = /usr/local/mpich/bin/mpif77
CC            = /usr/local/mpich/bin/mpicc
NOOPT        = 
F77FLAGS     =  -funroll-all-loops -O3 $(NOOPT)
DRVOPTS      = $(F77FLAGS)
CCFLAGS      = -O4
SRCFLAG       =
#F77LOADER     = $(F77)
F77LOADER     = $(F77)
CCLOADER      = $(CC)
F77LOADFLAGS  =
CCLOADFLAGS   =
#
#  C preprocessor defs for compilation 
#  (-DNoChange, -DAdd_, -DUpCase, or -Df77IsF2C)
#
CDEFS         = -Df77IsF2C -DNO_IEEE $(USEMPI)
#
#  The archiver and the flag(s) to use when building archive (library)
#  Also the ranlib routine.  If your system has no ranlib, set RANLIB = echo
#
ARCH          = ar
ARCHFLAGS     = cr
RANLIB        = ranlib
#
#  The name of the libraries to be created/linked to
#
SCALAPACKLIB  = $(home)/libscalapack.a
#BLASLIB       = $(HOME)/BLAS/blas_LINUX.a
# ATLAS ÀÇ BLAS ¶óÀ̺귯¸®¸¦ ÁöÁ¤ÇØ ÁØ´Ù.
BLASLIB       = -L$(HOME)/ATLAS/lib/Linux_ATHLONSSE1 -lf77blas -latlas
#
PBLIBS        = $(SCALAPACKLIB) $(FBLACSLIB) $(BLASLIB) $(SMPLIB)
PRLIBS        = $(SCALAPACKLIB) $(CBLACSLIB) $(SMPLIB)
RLIBS         = $(SCALAPACKLIB) $(FBLACSLIB) $(CBLACSLIB) $(BLASLIB) $(SMPLIB)
LIBS          = $(PBLIBS)
############################################################################
	   

ÄÄÆÄÀÏ ÇÑ´Ù. ÄÄÆÄÀÏ °úÁ¤¿¡¼­ ¿¡·¯°¡ »ý±â¸é SLmake.inc ÆÄÀÏÀ» ¼öÁ¤ÇÏ °í ´Ù½Ã ÄÄÆÄÀÏ ÇÑ´Ù.

[micro@master SCALAPACK]$ make lib 
	   

SCALAPACK µð·ºÅ丮 ¾Æ·¡¿¡ libscalapack.a ÆÄÀÏÀÌ ¸¸µé¾î Áø´Ù. ¿©±â ±îÁö ÀÌ»óÀÌ ¾ø´Ù¸é °£´ÜÇÑ Å×½ºÆ® ÇÁ·Î±×·¥À» µ¹·Áº¸ÀÚ.

[micro@master SCALAPACK]$ cd TESTING
[micro@master TESTING]$ cd LIN
[micro@master LIN]$ make double
[micro@master LIN]$ cd ..
[micro@master TESTING]$ /usr/local/mpich/bin/mpirun -np [ÇÁ·Î¼¼¼­ °³¼ö] ./xdlu 
ScaLAPACK Ax=b by LU factorization.
'MPI Machine'

Tests of the parallel real double precision LU factorization and solve.
The following scaled residual checks will be computed:
 Solve residual         = ||Ax - b|| / (||x|| * ||A|| * eps * N)
 Factorization residual = ||A - LU|| / (||A|| * eps * N)
The matrix A is randomly generated for each test.

An explanation of the input/output parameters follows:
TIME    : Indicates whether WALL or CPU time was used.
M       : The number of rows in the matrix A.
N       : The number of columns in the matrix A.
NB      : The size of the square blocks the matrix A is split into.
NRHS    : The total number of RHS to solve for.
NBRHS   : The number of RHS to be put on a column of processes before going
          on to the next column of processes.
P       : The number of process rows.
Q       : The number of process columns.
THRESH  : If a residual value is less than THRESH, CHECK is flagged as PASSED
LU time : Time in seconds to factor the matrix
Sol Time: Time in seconds to solve the system.
MFLOPS  : Rate of execution for factor and solve.

The following parameter values will be used:
  M       :         10000
  N       :         10000
  NB      :            36
  NRHS    :             3
  NBRHS   :             3
  P       :             1
  Q       :             7

Relative machine precision (eps) is taken to be       0.111022E-15
Routines pass computational tests if scaled residual is less than   1.0000

TIME     M     N  NB NRHS NBRHS    P    Q  LU Time Sol Time  MFLOPS  CHECK
---- ----- ----- --- ---- ----- ---- ---- -------- -------- -------- ------
WALL 10000 10000  36     3    3    1    7   100.52     0.46  6607.53 PASSED
Finished      1 tests, with the following results:
    1 tests completed and passed residual checks.
    0 tests completed and failed residual checks.
    0 tests skipped because of illegal input values.

END OF TESTS.
	   

À§¿Í ºñ½ÁÇÑ °á°ú°¡ ³ª¿Í¾ß ÇÑ´Ù. ´ÙÀ½Àº SCALAPACKÀÌ ¼³Ä¡µÇ¾ú´Ù°í °¡Á¤ÇÏ°í, ´ÜÀÏ ³ëµå¿¡¼­ LINPACK Benchmark¸¦ ¼öÇàÇÏ´Â ¹æ¹ýÀ» ¼³¸íÇÑ´Ù. ATLAS ÃÖÀûÈ­ ·çƾÀ» ÀÌ¿ëÇÏ·Á¸é SLmake.inc¿¡¼­ ATLAS¿¡¼­ Á¦°øÇÏ´Â BLAS ·çƾÀ» ÀÌ¿ëÇϵµ·Ï ¼³Á¤ÇÑ´Ù.

[micro@master SCALAPACK]$ vi SLmake.inc 
......(»ý·«)
BLASLIB       = -L$(HOME)/ATLAS/lib/Linux_ATHLONSSE1 -lf77blas -latlas
......(»ý·«)
	   

SCALAPACK/TESTING µð·ºÅ丮¿¡´Â ±âº»ÀûÀ¸·Î 13°³ÀÇ .dat È­ÀÏ°ú LIN, EIG µð·ºÅ丮°¡ ¼³Ä¡µÈ´Ù. LIN, EIG µð·ºÅ丮¿¡´Â º¥Ä¡¸¶Å©¸¦ ¼öÇàÇÏ´Â FortranÀ¸·Î ÀÛ¼ºµÈ ¼Ò½ºÄÚµå¿Í MakefileÀÌ µé¾î ÀÖ´Ù. LIN µð·ºÅ丮´Â Linear Equations Testing À» À§ÇÑ ÇÁ·Î±×·¥µéÀÌ µé¾î ÀÖ´Ù. EIG µð·ºÅ丮¿¡´Â Eigenroutine TestingÀ» À§ÇÑ ÇÁ·Î±×·¥ÀÌ µé¾î ÀÖ´Ù. TESTING µð·ºÅ丮¿¡ Æ÷ÇÔµÈ .dat È­ÀÏ°ú ¿ëµµ´Â ´ÙÀ½°ú °°´Ù.

BLLT.dat 'ScaLAPACK, Version 1.2, banded linear systems input file'
BLU.dat  'ScaLAPACK, Version 1.2, banded linear systems input file'
BRD.dat  'ScaLAPACK BRD input file'
HRD.dat  'ScaLAPACK HRD input file'
INV.dat  'ScaLAPACK, Version 1.0, Matrix Inversion Testing input file'
LLT.dat  'ScaLAPACK, LLt factorization input file'
LS.dat   'ScaLAPACK LS solve input file'
LU.dat   'SCALAPACK, LU factorization input file'
NEP.dat  'SCALAPACK NEP (Nonsymmetric Eigenvalue Problem) input file'
QR.dat   'ScaLAPACK, Orthogonal factorizations input file'
SEP.dat  'ScaLAPACK Symmetric Eigensolver Test File'
SVD.dat  'ScaLAPACK Singular Value Decomposition  input file'
TRD.dat  'ScaLAPACK TRD computation input file'
	   

.dat ÆÄÀÏ¿¡´Â °¢°¢ÀÇ º¥Ä¡¸¶Å©¿¡ ÇÊ¿äÇÑ º¯¼öµéÀÌ ÀúÀåµÈ´Ù. Å×½ºÆ® ¿¡ ÇÊ¿äÇÑ LU.dat ÆÄÀÏÀ» »ìÆ캸ÀÚ.

[micro@master TESTING]$ vi LU.dat 
-- LU.dat --
'SCALAPACK, LU factorization input file'
'MPI Machine'
'LU.out'                output file name (if any)
6                       device out
4                       number of problems sizes
4 10 17 13 23 31 57     values of M
4 12 13 13 23 31 50     values of N
3                       number of NB's
2 3 4 5                 values of NB
3                       number of NRHS's
1 3 9 28                values of NRHS
3                       Number of NBRHS's
1 3 5 7                 values of NBRHS
4                       number of process grids (ordered pairs of P & Q)
1 2 1 4 2 3 8           values of P
1 2 4 1 3 2 1           values of Q
1.0                     threshold
T                       (T or F) Test Cond. Est. and Iter. Ref. Routines
-- LU.dat --
	  

´ë·«ÀûÀÎ Çü½ÄÀº °¢ º¥Ä¡¸¶Å©¿¡ ÇÊ¿äÇÑ °ªµé°ú ÇØ´ç °ªµéÀÇ ¼ýÀÚ·Î Á¤ÀÇ µÇ¾î ÀÖ´Ù. ¿¹¸¦ µé¾î number of problems sizes °¡ 4 À̹ǷΠM °ú N ÀÇ º¥Ä¡¸¶Å© Å×½ºÆà ¿¡ »ç¿ëµÉ °ªÀº M = 4,10,17,13 ÀÌ°í N = 4,12,13,23 ÀÌ´Ù. ±×¸®°í values of P,Q ´Â ÇÁ·Î¼¼¼­ÀÇ Grid ¸¦ ¶æÇÑ´Ù. P ´Â ÇÁ·Î¼¼¼­ÀÇ row ¸¦ ¶æÇϸç Q ´Â column À» ¶æÇÑ´Ù. À§ÀÇ °æ¿ì P = 1 , Q = 1 À̸é 1 X 1 = 1 ÀÌ´Ï 1 °³ÀÇ ÇÁ·Î¼¼¼­ ¿¡¼­ ½ÇÇàµÈ´Ù´Â °ÍÀ» ¶æÇÑ´Ù. µÎ¹ø° ÀÇ °æ¿ìÀÎ P = 2 , Q = 2 ÀÇ °æ¿ì 2 X 2 = 4 ÀÌ´Ï 4 °³ÀÇ ÇÁ·Î¼¼¼­(Node) ¿¡¼­ ½ÇÇàµÈ´Ù´Â °ÍÀ» ¶æÇÑ´Ù. ÇØ´ç º¥Ä¡¸¶Å· ÇÁ·Î±×·¥À» »ý¼ºÇÏ·Á¸é LIN À̳ª EIG µð·ºÅ丮 ¿¡¼­ make [type] À» ½ÇÇàÇÏ¸é µÈ´Ù. ÇØ´ç type Àº single, double, complex, complex16 4 °¡Áö °¡ ÀÖ´Ù. ¿¹¸¦ µé¾î make single Àº single precision floating point ¸¦ À§ÇÑ º¥Ä¡¸¶Å· ÇÁ·Î±×·¥ÀÌ »ý¼ºµÈ´Ù. make all Àº 4°¡Áö Çü¿¡ ´ëÇÑ ¸ðµç º¥Ä¡¸¶Å· ÇÁ·Î±×·¥À» ÇѲ¨¹ø¿¡ ÄÄÆÄÀÏ ÇÑ´Ù. ÇØ´ç type ¿¡ ´ëÇÑ ÆÄÀϸí°ú °³¼ö´Â ´ÙÀ½°ú °°´Ù.

[micro@master LIN]$ make single 
[micro@master LIN]$ ls ../xs* 
xsdblu*  xsdtlu*  xsgblu*  xsinv*  xsllt*  xsls*  xslu*  xspbllt*  xsptllt*  xsqr*
[micro@master LIN]$ make double
[micro@master LIN]$ ls ../xd*
xddblu*  xddtlu*  xdgblu*  xdinv*  xdllt*  xdls*  xdlu*  xdpbllt*  xdptllt*  xdqr*
[micro@master LIN]$ make complex 
[micro@master LIN]$ ls ../xc*
xcdblu*  xcdtlu*  xcgblu*  xcinv*  xcllt*  xcls*  xclu*  xcpbllt*  xcptllt*  xcqr*
[micro@master LIN]$ make complex16 
[micro@master LIN]$ ls ../xz* 
xzdblu*  xzdtlu*  xzgblu*  xzinv*  xzllt*  xzls*  xzlu*  xzpbllt*  xzptllt*  xzqr*
	  

ÀÌ¿Í °°ÀÌ ÃÑ 40°³ÀÇ ½ÇÇàÆÄÀÏ µéÀÌ »ý¼ºµÇ´Âµ¥ ¸¶Âù°¡Áö·Î EIG µð·ºÅ丮 ¿¡¼­µµ °°Àº Çü½ÄÀ¸·Î make ¸¦ Çϸé ÇÁ·Î±×·¥ÀÌ »ý¼ºµÈ´Ù. °¢°¢ÀÇ ÇÁ·Î±× ·¥À» ½ÇÇàÇÏ¿© º¸ÀÚ. MPI ¸¦ ÀÌ¿ëÇÏ¿© ½ÇÇàÇÏ·Á¸é mpirun -np N program À» ½ÇÇàÇÏ¸é µÈ´Ù. LINPACK ÀÇ º¥Ä¡¸¶Å© ¿¡¼­´Â LU.dat ÆÄÀÏÀÇ ÆĶó¸ÞÅÍ ¼³Á¤À» ÀÌ¿ëÇÏ¿© º¥Ä¡¸¶Å© ÇÒ¼ö ÀÖ´Ù. ±âº»ÀûÀ¸·Î ScaLAPACKÀº ºí·° ´ÜÀ§·Î ¿¬»êÀ» ¼öÇàÇϸç, Ŭ·¯½ºÅÍ¿Í °°Àº º´·Ä ÄÄÇ»ÅÍ¿¡¼­ ÃÖ´ë ¼º´ÉÀ» ¾ò±â À§Çؼ­´Â ÁÖ¾îÁø ÄÄÇ»ÅÍ¿¡ ÀûÀýÇÑ ºí·ÏÀÇ Å©±â¸¦ ±¸ÇÏ¿©¾ß ÇÑ´Ù. ÀÌ´Â °è»êÀ¸·Î ´ë·«ÀûÀÎ °ªÀ» ±¸ÇÑ ÈÄ¿¡, ¸¹Àº ½ÇÇàÀ» °ÅÃÄ °æÇèÀûÀ¸·Î ¾òÀ» ¼ö ÀÖ´Ù. ±×¸®°í ÄÄÇ»ÅÍ°¡ ¼öÇàÇÒ ¼ö ÀÖ´Â ÃÖ´ë ¹®Á¦ Å©±â(Nmax)¸¦ ¾ò±â À§Çؼ­´Â, ÇϳªÀÇ ÇÁ·Î¼¼¼­¿¡¼­ ¹®Á¦ Å©±â¸¦ Á¡Â÷·Î ´Ã¸®¸é¼­ ÁÖ¾îÁø ¸Þ¸ð¸®¿¡ ³ëµå°¡ Æ÷¿ëÇÒ ¼ö ÀÖ´Â ÃÖ´ë Å©±â¸¦ ¾Ë¾Æ³»¾ß ÇÑ´Ù. ¸¶Áö¸·À¸·Î À̸¦ ¹ÙÅÁÀ¸·Î ¸¹Àº ³ëµå¸¦ °¡Áø º´·Ä ÄÄÇ»ÅÍ¿¡¼­ ¼öÇàÇÒ ¼ö ÀÖ´Â ÃÖ´ëÀÇ Å©±â·Î LU ÀμöºÐÇØ ·çÆÃÀ» ¼öÇà½ÃÄѼ­ ÃÖ´ëÀÇ ¼º´É(Rmax)À» ¾òÀ» ¼ö ÀÖ´Ù.

Fortran À¸·Î Â¥¿©Áø LIN/pdludriver.f ÆÄÀÏ¿¡¼­ TOTMEM ÀÇ °ªÀ» º¯È­½ÃÅ°¸é¼­ Segmentation fault °¡ ¹ß»ýÇÏ´Â ¹üÀ§¸¦ Á¶»çÇØ º¸ÀÚ. ÀÌ°ÍÀº ÁÖ ¸Þ¸ð¸® »Ó¸¸ ¾Æ´Ï¶ó ½º¿Ò¿µ¿ªÀÇ Å©±â¿¡ ¿µÇâÀ» ¹Þ´Â´Ù. ½º¿Ò ¿µ¿ªÀÇ Å©±âº¸´Ù TOTMEM °ªÀ» Å©°Ô Çϸé Segmentation fault ¸¦ ÀÏÀ¸Å³ °ÍÀÌ´Ù. ¹°·Ð ¼Ò½º¸¦ ¼öÁ¤ÇÏ°í ³ª¸é ÄÄÆÄÀÏÀ» ´Ù½Ã ÇØ¾ß ÇÑ´Ù. 2GB ¸Þ¸ð¸®, 500MB ½º¿Ò¿µ¿ªÀÇ ÁÖ¾îÁø Á¶°Ç¿¡¼­ TOTMEM À» 500000000 ·Î Á¤ÇÏ¿´´Ù. TESTING µð·ºÅ丮ÀÇ LU.dat À» ´ÙÀ½°ú °°ÀÌ ¼öÁ¤ÇÏ°í xdlu ¸¦ ½ÇÇà½ÃÄÑ º¸ÀÚ. °è»ê¿¡ ÇÊ¿äÇÑ ¸Þ¸ð¸®¾çÀÌ ¸ÞÀÎ ¸Þ¸ð¸®ÀÇ Å©±âº¸´Ù Å©¸é ½º¿Ò¿µ¿ªÀÇ ¾ï¼¼½º¸¦ À§Çؼ­ Çϵåµð½ºÅ©°¡ µ¿ÀÛÇÏ´Â °ÍÀ» º¼¼ö ÀÖÀ»°ÍÀÌ´Ù.

-- LU.dat --
'SCALAPACK, LU factorization input file'
'MPI Machine'
'LU.out'                output file name (if any)
6                       device out
6                       number of problems sizes
1000 1200 1400 1600 1800 2000 values of M
1000 1200 1400 1600 1800 2000 values of N
1                       number of NB's
60		        values of NB
1                       number of NRHS's
1 	               values of NRHS
1                       Number of NBRHS's
1	                 values of NBRHS
1                       number of process grids (ordered pairs of P & Q)
1		       values of P
1		       values of Q
1.0                     threshold
T                       (T or F) Test Cond. Est. and Iter. Ref. Routines
-- LU.dat --
	  

ÀÌ°ÍÀ» 2¹ø ½ÇÇàÇÑ °á°ú´Â ´ÙÀ½°ú °°´Ù. ù¹ø° ½ÇÇàÇÑ °á°ú

TIME     M     N  NB NRHS NBRHS    P    Q  LU Time Sol Time  MFLOPS  CHECK
---- ----- ----- --- ---- ----- ---- ---- -------- -------- -------- ------
WALL  1000  1000  60     1    1    1    1     0.64     0.01  1026.42 PASSED
WALL  1200  1200  60     1    1    1    1     1.05     0.02  1078.62 PASSED
WALL  1400  1400  60     1    1    1    1     1.67     0.02  1083.34 PASSED
WALL  1600  1600  60     1    1    1    1     2.29     0.03  1177.45 PASSED
WALL  1800  1800  60     1    1    1    1     3.13     0.04  1227.84 PASSED
WALL  2000  2000  60     1    1    1    1     4.37     0.05  1207.76 PASSED
	  

µÎ¹ø° ½ÇÇàÇÑ °á°ú

TIME     M     N  NB NRHS NBRHS    P    Q  LU Time Sol Time  MFLOPS  CHECK
---- ----- ----- --- ---- ----- ---- ---- -------- -------- -------- ------

WALL  1000  1000  60     1    1    1    1     0.63     0.01  1032.06 PASSED
WALL  1200  1200  60     1    1    1    1     1.05     0.02  1079.69 PASSED
WALL  1400  1400  60     1    1    1    1     1.59     0.02  1134.49 PASSED
WALL  1600  1600  60     1    1    1    1     2.28     0.03  1184.27 PASSED
WALL  1800  1800  60     1    1    1    1     3.12     0.04  1231.93 PASSED
WALL  2000  2000  60     1    1    1    1     4.37     0.05  1207.30 PASSED
	  

¹®Á¦ÀÇ Å©±â°¡ Ä¿Áú¼ö·Ï MFLOPS °¡ Áõ°¡ÇÏ´Ù°¡ SWAP À» »ç¿ëÇÒ Á¤µµ°¡ µÇ¸é ¼º´ÉÀÌ ¶³¾îÁö´Â °ÍÀ» ¾Ë¼ö ÀÖ´Ù. ´ÙÀ½Àº NB¸¦ ¹Ù²Ù¸é¼­ ¼öÇàÀ» Çغ¸ÀÚ. M °ú N À» °¢ÀÚÀÇ ½Ã½ºÅÛ¿¡ ¸Â°Ô ¼öÁ¤Çϸ鼭 NB ¸¦ 28 ¿¡¼­ 60 ±îÁö º¯È­ ½ÃŲ´Ù.

TIME     M     N  NB NRHS NBRHS    P    Q  LU Time Sol Time  MFLOPS  CHECK
---- ----- ----- --- ---- ----- ---- ---- -------- -------- -------- ------

WALL  5000  5000  28     1    1    1    7    19.85     0.14  4170.73 PASSED
WALL  5000  5000  30     1    1    1    7    14.85     0.14  5562.10 PASSED
WALL  5000  5000  32     1    1    1    7    15.40     0.13  5367.77 PASSED
WALL  5000  5000  34     1    1    1    7    15.89     0.15  5198.10 PASSED
WALL  7000  7000  28     1    1    1    7    49.39     0.24  4608.81 PASSED
WALL  7000  7000  30     1    1    1    7    37.77     0.27  6013.41 PASSED
WALL  7000  7000  32     1    1    1    7    38.96     0.25  5833.21 PASSED
WALL  7000  7000  34     1    1    1    7    39.07     0.26  5816.04 PASSED
WALL 10000 10000  28     1    1    1    7   133.66     0.41  4973.60 PASSED
WALL 10000 10000  30     1    1    1    7    99.69     0.45  6659.18 PASSED
WALL 10000 10000  32     1    1    1    7   102.15     0.43  6500.25 PASSED
WALL 10000 10000  34     1    1    1    7   101.73     0.40  6529.03 PASSED
	 

À§ÀÇ ½ÇÇè¿¡¼­ ÃøÁ¤µÈ ÃÖ°í ¼º´ÉÀº M=N=10000 NB=30 À϶§ 6659.18 MFLOPS ÀÌ´Ù. M=N °ªÀ» ÅëÀÏÇÏ°í NB ÀÇ ÃÖÀûÈ­ °ªÀ» ã¾Æº¸µµ·Ï ÇÏÀÚ.

TIME     M     N  NB NRHS NBRHS    P    Q  LU Time Sol Time  MFLOPS  CHECK
---- ----- ----- --- ---- ----- ---- ---- -------- -------- -------- ------

WALL  1000  1000  28     1    1    1    7     1.09     0.03   595.45 PASSED
WALL  1000  1000  30     1    1    1    7     0.33     0.02  1939.09 PASSED
WALL  1000  1000  32     1    1    1    7     0.33     0.02  1911.82 PASSED
WALL  1000  1000  34     1    1    1    7     0.37     0.02  1699.22 PASSED
WALL  1000  1000  36     1    1    1    7     0.38     0.02  1695.32 PASSED
WALL  1000  1000  38     1    1    1    7     0.40     0.02  1606.20 PASSED
	 

M=N=1000 À¸·Î ÅëÀϽÃÅ°°í ÃøÁ¤ÇßÀ»°æ¿ì NB °¡ 30 ÀÏ°æ¿ì °¡Àå ÁÁÀº ¼º´ÉÀ» º¸ÀÓ.


4.5.3.2. HPL (High-Performance Linpack Benchmark)

´ÙÀ½Àº ´ë¿ë·® ¸Þ¸ð¸® ½Ã½ºÅÛ À» º¥Ä¡¸¶Å© Çϴµ¥ ¾²ÀÌ´Â (Àü¼¼°è ½´ÆÛÄÄÇ»Æà ¼øÀ§¸¦ ¸Å±â´Â TOP 500 Site ¿¡¼­ »ç¿ëÇÏ´Â ÇÁ·Î±×·¥) HPL À» ÀÌ¿ëÇÏ¿© º¥Ä¡¸¶Å· À» Çغ¸ÀÚ. HPL À» ¼³Ä¡Çϱâ Àü¿¡ BLAS , MPICH CBLAS µîÀÌ ¼³Ä¡µÇ¾î ÀÖ¾î¾ß ÇÑ´Ù. ¿©±â¼­´Â ATLAS ÀÇ BLAS ·çƾÀ» ÀÌ¿ëÇÒ °Í À̱⠶§¹®¿¡ ATLAS µµ ¼³Ä¡µÇ¾î ÀÖ¾î¾ß ÇÑ´Ù. CBLAS ¼³Ä¡´Â Çѱ¹ Ŭ·¯½ºÅÍ ±â¼ú¼¾ÅÍ http://www.cluster.or.kr/board/read.php?table=benchmark=3 ¸¦ Âü°íÇϰųª ¿©±â¸¦ Âü°íÇϵµ·Ï ÇÑ´Ù. http://www.netlib.org/blas/ Hpl À» ´Ù¿î¹Þ¾Æ¼­ ¾ÐÃàÀ» Ǭ´Ù. http://www.netlib.org/benchmark/hpl/

[micro@master share]$ tar xzf hpl.tgz
	    

hpl µð·ºÅ丮 ¾ÈÀÇ setup µð·ºÅ丮 ¿¡¼­ ÇØ´çÇ÷§Æû¿¡ ¸Â´Â make ÆÄÀÏÀ» hpl Top µð·ºÅ丮 ¾ÈÀ¸·Î º¹»çÇÑ´Ù. ¿©±â¼­´Â Linux ÀÇ Athlon Ĩ, ±×¸® °í BLAS ÀÇ C ÀÎÅÍÆäÀ̽ºÀÎ CBLAS ¸¦ »ç¿ëÇÒ °Í À̹ǷΠÆÄÀϸíÀº ´ÙÀ½ °ú °°´Ù.

[micro@master share]$ cd hpl 
[micro@master hpl]$ cp setup/Make.Linux_ATHLON_CBLAS . 
	    

ÇØ´çÆÄÀÏÀ» ¼öÁ¤Çϵµ·Ï ÇÑ´Ù.

[micro@master hpl]$ vi Make.Linux_ATHLON_CBLAS
------ Make.Linux_ATHLON_CBLAS -------
SHELL        = /bin/sh
#
CD           = cd
CP           = cp
LN_S         = ln -s
MKDIR        = mkdir
RM           = /bin/rm -f
TOUCH        = touch

ARCH         = Linux_ATHLON_CBLAS

TOPdir       = $(HOME)/hpl
INCdir       = $(TOPdir)/include
BINdir       = $(TOPdir)/bin/$(ARCH)
LIBdir       = $(TOPdir)/lib/$(ARCH)
#
HPLlib       = $(LIBdir)/libhpl.a

#CC           = gcc
CC           = /usr/local/mpich/bin/mpicc  <- MPICH ÀÇ C ÄÄÆÄÀÏ·¯
NOOPT        =
#CCFLAGS      = -fomit-frame-pointer -O3 -funroll-loops -W -Wall
CCFLAGS      = -fomit-frame-pointer -O3 -funroll-loops
#
#LINKER       = gcc
LINKER       = /usr/local/mpich/bin/mpicc
LINKFLAGS    = $(CCFLAGS)
#
ARCHIVER     = ar
ARFLAGS      = r
RANLIB       = echo

MPdir        = /usr/local/mpich
MPinc        = -I$(MPdir)/include
MPlib        = $(MPdir)/lib/libmpich.a

F2CDEFS      =
NOOPT        =
F77          = /usr/local/mpich/bin/mpif77
F77LOADER    = /usr/local/mpich/bin/mpif77
F77FLAGS     = -O $(NOOPT)

LAdir        = $(HOME)/ATLAS/lib/Linux_ATHLONSSE1
LAinc        = $(HOME)/ATLAS/include/Linux_ATHLONSSE1
LAlib        = $(LAdir)/libcblas.a $(LAdir)/libatlas.a

#HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc)
		<- À­ºÎºÐÀ» ¾Æ·¡¿Í °°ÀÌ ¼öÁ¤ÇÑ´Ù.
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) -I$(LAinc) $(MPinc)
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib)

HPL_OPTS     = -DHPL_CALL_CBLAS
HPL_DEFS     = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
------ Make.Linux_ATHLON_CBLAS -------
	    

´ÙÀ½ Make.top ÆÄÀÏ°ú Makefile ÀÇ arch ºÎºÐÀ» ¼öÁ¤ÇØ ÁØ´Ù.

#
arch             = Linux_ATHLON_CBLAS
#
	    

ÄÄÆÄÀÏ ÇÑ´Ù. Make arch=[ÇØ´ç½Ã½ºÅÛ] À» ÀÔ·ÂÇÏÀÚ. ±×·³ bin µð·ºÅ丮 ¾Æ ·¡¿¡ [ÇØ´ç½Ã½ºÅÛ] µð·ºÅ丮°¡ »ý¼ºµÆÀ»°ÍÀÌ´Ù.

[micro@master hpl]$ make arch=Linux_ATHLON_CBLAS 
[micro@master hpl]$ cd bin/Linux_ATHLON_CBLAS 
	    

bin/Linux_ATHLON_CBLAS µð·ºÅ丮¿¡ °¡º¸¸é HPL.dat ÆÄÀÏ°ú xhpl ÆÄ ÀÏÀÌ º¸ÀÏ°ÍÀÌ´Ù. HPL.dat ÆÄÀÏÀº ¾Õ¼­ LINPACK º¥Ä¡¸¶Å· ¿¡ ȯ°æ¼³Á¤ ÆÄÀÏó·³ ¿©·¯°¡Áö º¥Ä¡¸¶Å·¿¡ ÇÊ¿äÇÑ ÆĶó¹ÌÅÍ µéÀ» ¼³Á¤ÇÏ´Â °÷ÀÌ°í, xhpl ½ÇÇàÆÄÀÏÀº ½ÇÁúÀûÀ¸·Î º¥Ä¡¸¶Å·¿¡ µ¹¸®´Â ÇÁ·Î±×·¥ÀÌ´Ù. ±×·³ HPL.dat ÆÄÀÏÀÇ Æ÷¸ËÀ» »ìÆ캸ÀÚ.

[micro@master hpl]$ vi HPL.dat
----- HPL.dat -----
HPLinpack benchmark input file
Innovative Computing Laboratory, University of Tennessee
HPL.out      output file name (if any)
6            device out (6=stdout,7=stderr,file)
1            # of problems sizes (N)
10000
1           # of NBs
85           NBs
1            # of process grids (P x Q)
1            Ps
7            Qs
16.0         threshold
1            # of panel fact
1            PFACTs (0=left, 1=Crout, 2=Right)
1            # of recursive stopping criterium
4          NBMINs (>= 1)
1            # of panels in recursion
2            NDIVs
1            # of recursive panel fact.
2        RFACTs (0=left, 1=Crout, 2=Right)
1            # of broadcast
1            BCASTs (0=1rg,1=1rM,2=2rg,3=2rM,4=Lng,5=LnM)
1            # of lookahead depth
1            DEPTHs (>=0)
2            SWAP (0=bin-exch,1=long,2=mix)
64          swapping threshold
0            L1 in (0=transposed,1=no-transposed) form
0            U  in (0=transposed,1=no-transposed) form
1            Equilibration (0=no,1=yes)
8            memory alignment in double (> 0)
----- HPL.dat -----
	    

±âº»ÀûÀ¸·Î LINPACK º¥Ä¡¸¶Å©ÀÇ LU.dat °ú Å©°Ô ´Ù¸£Áö ¾Ê´Ù´Â °ÍÀ» ¾Ë¼ö°¡ ÀÖ´Ù. ¸î°¡Áö Â÷ÀÌÁ¡Àº Problem size °¡ 1Â÷¿ø À¸·Î ¹Ù²ï°Í°ú Swapping threshold ¸¦ ÁöÁ¤ÇÒ¼ö ÀÖ´Ù´Â °Í µîÀε¥ ÀÚ¼¼ÇÑ ³»¿ëÀº ÇØ´ç Æ©´× ÆäÀÌÁö http://www.netlib.org/benchmark/hpl/tuning.html ¸¦ Âü°íÇϵµ·Ï ÇÏÀÚ. xhpl À» ½ÇÇàÇØ º¸µµ·Ï ÇÑ´Ù.

[micro@master Linux_ATHLON_CBLAS]$ mpirun -np 7 xhpl
====================================================================
HPLinpack 1.0  --  High-Performance Linpack benchmark  --  September 27, 2000
Written by A. Petitet and R. Clint Whaley,  Innovative Computing Labs.,  UTK
====================================================================

An explanation of the input/output parameters follows:
T/V    : Wall time / encoded variant.
N      : The order of the coefficient matrix A.
NB     : The partitioning blocking factor.
P      : The number of process rows.
Q      : The number of process columns.
Time   : Time in seconds to solve the linear system.
Gflops : Rate of execution for solving the linear system.

The following parameter values will be used:

N      :   10000
NB     :    85
P      :       1
Q      :       7
PFACT  :   Crout
NBMIN  :       4
NDIV   :       2
RFACT  :   Right
BCAST  :  1ringM
DEPTH  :       1
SWAP   : Mix (threshold = 64)
L1     : transposed form
U      : transposed form
EQUIL  : yes
ALIGN  : 8 double precision words
============================================================================
T/V                N    NB     P     Q               Time             Gflops
----------------------------------------------------------------------------
W11R2C4        10000    85     1     7              70.49          9.460e+00
----------------------------------------------------------------------------
||Ax-b||_oo / ( eps * ||A||_1  * N        ) =        0.0646673 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_1  * ||x||_1  ) =        0.0153022 ...... PASSED
||Ax-b||_oo / ( eps * ||A||_oo * ||x||_oo ) =        0.0034203 ...... PASSED
============================================================================
	    

À§ÀÇ °á°ú´Â N = 10000 , NB = 85 À϶§ 9.46Gflops °¡ ³ª¿Ô´Ù. LINPACK °ú ¸¶Âù°¡Áö·Î ¿©·¯ºÐÀÇ ½Ã½ºÅÛ È¯°æ¿¡ ¸Â°Ô problem size ¿Í NB ¸¦ ÀûÀýÈ÷ ¼öÁ¤ÇØ °¡¸é¼­ ½Ã½ºÅÛÀÌ ¼öÇàÇÒ¼ö ÀÖ´Â ÃÖ°í¼º´ÉÀ» À̲ø¾î ³»º¸ÀÚ. HPL.dat ÆÄÀÏÀ» ¼öÁ¤ÇÑ´ÙÀ½ ÄÄÆÄÀÏÀ» ´Ù½Ã ÇÑ´Ù.

[micro@master Linux_ATHLON_CBLAS]$ rm -f ./xhpl
[micro@master Linux_ATHLON_CBLAS]$ cd ../../
[micro@master Linux_ATHLON_CBLAS]$ make clean
[micro@master Linux_ATHLON_CBLAS]$ make all
	    

5장. Reference sites

  • Internet Performance Tuning http://www-iepm.slac.stanford.edu/

  • System Tuning for Linux http://people.redhat.com/alikins/system_tuning.html

  • Linux Performance tuning http://linuxperf.nl.linux.org/

  • MPICH http://www-unix.mcs.anl.gov/mpi/mpich/

  • Linux Kernel Guide http://kldp.org/KoreanDoc/html/Kernel-KLDP/

  • Linux Kernel Hacker's Guide http://tldp.org/LDP/khg/HyperNews/get/khg.html

  • NetPIPE Benchmark http://www.scl.ameslab.gov/

  • Rocks Cluster http://www.rocksclusters.org/Rocks/

  • Top500 SuperComputer http://www.top500.org

  • LAM-MPI http://www.lam-mpi.org

  • Beowolf Project http://www.beowulf.org

  • POSDATA http://cluster.posdata.co.kr

  • Samsung Alpha-11 Cluster http://www.alpha11.com

  • Cluster References http://www.cluster.or.kr

  • SCALAPACK http://www.netlib.org/scalapack




sponsored by andamiro
sponsored by cdnetworks
sponsored by HP

Valid XHTML 1.0! Valid CSS! powered by MoniWiki
last modified 2005-05-16 17:50:37
Processing time 0.0023 sec