Скачиваем исходники :
| linux-2.6.26.1.tar.bz2 - само ядро |
| patch-2.6.26.1.bz2 - патч |
| отсюда можно скачать mosix, но качаем именно для нашей версии ядра |
Исходные данные :
| 172.16.30.10 - centos1 |
| 172.16.30.20 - centos2 |
| 172.16.30.30 - centos3 |
Распоковываем архивы и раскидываем их по каталогам
[root@centos5k1 ~]# tar -xjf linux-2.6.26.1.tar.bz2 [root@centos5k1 ~]# tar -xjf MOSIX-2.24.2.2.tbz [root@centos5k1 ~]# bzip2 -d patch-2.6.26.1.bz2 [root@centos5k1 ~]# cp -r linux-2.6.26.1 /usr/src/kernels/ [root@centos5k1 ~]# cp patch-2.6.26.1 /usr/src/kernels/linux-2.6.26.1/
Раскидываем файлы mosix, так как указано ниже
[root@centos5k1 ~]# cd mosix-2.24.2.2/ File Where chmod ------------------------------------------------------------------------------ bestnode /bin u+s defaults/features /etc/mosix/features defaults/tunes /etc/mosix/tunes migrate /bin mon /bin mosconf* /sbin mospipe /bin mos_checkconf /sbin mosctl /bin mosd /sbin mosenv /bin mosixd /sbin moskillall /bin mosps /bin u+s mosq /bin u+s mosqueue /bin mosrun /bin u+s native /bin postald /sbin qmd /sbin remoted /sbin setcl /sbin setpe /sbin testload /bin tune /bin tune_mosix /bin mosix /etc/init.d/ man/* /usr/share/man/
Накладываем патчи на ядро, сначало для системы, потом от mosix
патч для системы patch-2.6.26.1
[root@centos5k1 linux-2.6.26.1]# patch -p1 -R -i patch-2.6.26.1 patching file Documentation/networking/udplite.txt patching file Documentation/video4linux/cx18.txt patching file Makefile patching file arch/ia64/kvm/kvm-ia64.c patching file arch/powerpc/kvm/powerpc.c patching file arch/powerpc/platforms/cell/cbe_cpufreq_pmi.c patching file arch/s390/kvm/kvm-s390.c patching file arch/sparc64/kernel/irq.c patching file arch/sparc64/kernel/ldc.c patching file arch/sparc64/kernel/time.c patching file arch/um/include/init.h patching file arch/x86/Kconfig.cpu patching file arch/x86/kernel/acpi/sleep.c patching file arch/x86/kernel/i387.c patching file arch/x86/kernel/ptrace.c patching file arch/x86/kernel/reboot.c patching file arch/x86/kvm/mmu.c patching file arch/x86/kvm/svm.c patching file arch/x86/kvm/vmx.c patching file arch/x86/kvm/x86.c patching file arch/x86/kvm/x86_emulate.c patching file arch/x86/mm/init_64.c patching file drivers/acpi/processor_perflib.c patching file drivers/cpufreq/cpufreq.c patching file drivers/dma/iop-adma.c patching file drivers/ide/ide-cd.c patching file drivers/media/dvb/dvb-usb/dib0700_devices.c patching file drivers/media/dvb/dvb-usb/dvb-usb-ids.h patching file drivers/media/video/Kconfig patching file drivers/media/video/cx18/cx18-firmware.c patching file drivers/media/video/cx23885/cx23885-core.c patching file drivers/media/video/saa7134/saa7134-cards.c patching file drivers/media/video/saa7134/saa7134-core.c patching file drivers/media/video/uvc/Kconfig patching file drivers/media/video/uvc/uvc_driver.c patching file drivers/media/video/uvc/uvc_status.c patching file drivers/media/video/uvc/uvc_video.c patching file drivers/media/video/uvc/uvcvideo.h patching file drivers/mmc/host/pxamci.c patching file drivers/net/hamradio/hdlcdrv.c patching file drivers/net/ixgbe/ixgbe_main.c patching file drivers/net/myri10ge/myri10ge.c patching file drivers/net/wireless/ath5k/base.c patching file drivers/net/wireless/b43legacy/main.c patching file drivers/rtc/rtc-at91rm9200.c patching file drivers/spi/mpc52xx_psc_spi.c patching file drivers/usb/serial/usb-serial.c patching file fs/dquot.c patching file fs/ecryptfs/crypto.c patching file fs/isofs/rock.c patching file fs/libfs.c patching file fs/proc/task_mmu.c patching file include/asm-arm/bitops.h patching file include/asm-generic/vmlinux.lds.h patching file include/asm-sparc64/io.h patching file include/asm-x86/signal.h patching file include/linux/cpufreq.h patching file include/linux/ipv6.h patching file include/linux/kvm_host.h patching file kernel/cpuset.c patching file kernel/marker.c patching file kernel/rcupreempt.c patching file kernel/sys_ni.c patching file mm/filemap.c patching file mm/shmem.c patching file net/ipv4/tcp_input.c patching file net/ipv4/udp.c patching file net/ipv6/ip6_fib.c patching file scripts/Makefile.modpost patching file scripts/mod/modpost.c patching file sound/pci/trident/trident_main.c patching file virt/kvm/kvm_main.c
копируем патч для mosix и накладываем его на ядро
[root@centos5k1 linux-2.6.26.1]# cp /root/mosix-2.24.2.2/other/patch-2.6.26.1 /usr/src/kernels/linux-2.6.26.1/ [root@centos5k1 linux-2.6.26.1]# patch -p1 -i patch-2.6.26.1 patching file Makefile patching file arch/x86/configs/i386_defconfig patching file arch/x86/configs/x86_64_defconfig patching file arch/x86/ia32/sys_ia32.c patching file arch/x86/kernel/process_32.c patching file arch/x86/kernel/process_64.c patching file arch/x86/kernel/ptrace.c patching file arch/x86/kernel/setup_64.c patching file arch/x86/kernel/signal_32.c patching file arch/x86/kernel/signal_64.c patching file arch/x86/kernel/vsyscall_64.c patching file arch/x86/mm/init_64.c patching file arch/x86/vdso/vdso32-setup.c patching file arch/x86/vdso/vma.c patching file fs/binfmt_elf.c patching file fs/exec.c patching file fs/pipe.c patching file fs/proc/base.c patching file fs/proc/internal.h patching file include/asm-x86/vgtod.h patching file include/asm-x86/vsyscall.h patching file include/linux/connect.h patching file include/linux/init_task.h patching file include/linux/mosix.h patching file include/linux/pipe_fs_i.h patching file include/linux/prctl.h patching file include/linux/ptrace.h patching file include/linux/sched.h patching file init/Kconfig patching file init/main.c patching file kernel/exit.c patching file kernel/fork.c patching file kernel/ptrace.c patching file kernel/sched.c patching file kernel/signal.c patching file kernel/sys.c patching file kernel/timer.c patching file mm/memory.c patching file mm/oom_kill.c patching file mm/swapfile.c patching file mosix/Makefile patching file mosix/config.c patching file mosix/connect.c patching file mosix/load.c patching file mosix/mosadmin.c patching file mosix/mosproc.c
Изменим в файл Makefile, один параметр, теперь наше ядро будет называться так linux-2.6.26-mosix
[root@centos5k1 ~]# cd /usr/src/kernels/linux-2.6.26.1/ [root@centos5k1 linux-2.6.26.1]# cat Makefile | grep "EXTRAVERSION =" EXTRAVERSION = -mosix
Собираем наше ядро
Чистка
[root@centos5k1 linux-2.6.26.1]# make mrproper
Создаем фаил .config - его можно создать, например с помощью команды "make oldconfig".
[root@virtfedora linux-2.6.26.1]# make oldconfig HOSTCC scripts/basic/fixdep HOSTCC scripts/basic/docproc HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/kxgettext.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/lex.zconf.c SHIPPED scripts/kconfig/zconf.hash.c ......................................................................................... CRC-CCITT functions (CRC_CCITT) [M/y/?] m CRC16 functions (CRC16) [M/y/?] m CRC ITU-T V.41 functions (CRC_ITU_T) [M/y/?] m CRC32 functions (CRC32) [Y/?] y CRC7 functions (CRC7) [N/m/y/?] n CRC32c (Castagnoli, et al) Cyclic Redundancy-Check (LIBCRC32C) [M/y/?] m # # configuration written to .config #
Обратите на это внимание:
NOTE: if you modify the kernel configuration, do not touch the following: a. DO NOT CONFIGURE the "CONFIG_SECURITY" option: 'Security Options' / 'enable different security models' b. DO NOT CONFIGURE the "CONFIG_HEADERS_CHECK" option: 'Kernel hacking' / 'Run 'make headers_check' when building vmlinux'
Проверяем включение опций mosix
[root@centos5k1 linux-2.6.26.1]# cat .config | grep MOSIX CONFIG_MOSIX=y [root@centos5k1 linux-2.6.26.1]# cat .config | grep CONFIG_BINFMT_ELF CONFIG_BINFMT_ELF=y [root@centos5k1 linux-2.6.26.1]# cat .config | grep CONFIG_PROC_FS CONFIG_PROC_FS=y
Соберем и установим модули(make)
[root@centos5k1 linux-2.6.26.1]# make scripts/kconfig/conf -s arch/x86/Kconfig CHK include/linux/version.h UPD include/linux/version.h CHK include/linux/utsrelease.h UPD include/linux/utsrelease.h SYMLINK include/asm -> include/asm-x86 CC kernel/bounds.s GEN include/linux/bounds.h CC arch/x86/kernel/asm-offsets.s GEN include/asm-x86/asm-offsets.h CALL scripts/checksyscalls.sh CC scripts/mod/empty.o HOSTCC scripts/mod/mk_elfconfig MKELF scripts/mod/elfconfig.h HOSTCC scripts/mod/file2alias.o HOSTCC scripts/mod/modpost.o HOSTCC scripts/mod/sumversion.o HOSTLD scripts/mod/modpost HOSTCC scripts/kallsyms ....................................................
Устанавливаем ядро (make modules_install install)
[root@centos5k1 linux-2.6.26.1]# make modules_install install INSTALL sound/pci/snd-azt3328.ko INSTALL sound/pci/snd-bt87x.ko INSTALL sound/pci/snd-cmipci.ko INSTALL sound/pci/snd-cs4281.ko INSTALL sound/pci/snd-cs5530.ko INSTALL sound/pci/snd-ens1370.ko INSTALL sound/pci/snd-ens1371.ko INSTALL sound/pci/snd-es1938.ko INSTALL sound/pci/snd-es1968.ko INSTALL sound/pci/snd-fm801.ko INSTALL sound/pci/snd-intel8x0.ko INSTALL sound/pci/snd-intel8x0m.ko INSTALL sound/pci/snd-maestro3.ko INSTALL sound/pci/snd-rme32.ko INSTALL sound/pci/snd-rme96.ko INSTALL sound/pci/snd-sis7019.ko .................................................................. DEPMOD 2.6.26-mosix sh /usr/src/kernels/linux-2.6.26.1/arch/x86/boot/install.sh 2.6.26-mosix arch/x86/boot/bzImage System.map "/boot"
Проверяем создание файлов и загрузчик(grub) и перегружаемся
[root@centos5k1 linux-2.6.26.1]# ls /boot/ | grep 2.6.26 initrd-2.6.26-mosix.img System.map-2.6.26-mosix vmlinuz-2.6.26-mosix [root@centos5k1 linux-2.6.26.1]# ls /lib/modules/ | grep 2.6.26 2.6.26-mosix
title Centos (2.6.26-mosix)
root (hd0,0)
kernel /vmlinuz-2.6.26-mosix ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.26-mosix.img
Настройка mosix
Запускаем mosconf_cluster, добавляем 172.16.30.10
[root@centos5k1 ~]# mosconf_cluster There are no nodes in your cluster yet: ======================================= To add a new set of nodes to your cluster, type 'n'. To turn on advanced options, type '+'. For help, type 'h'. To save and exit, type 'q'. (to abandon all changes and exit, type 'Q') Option :- n Adding new node(s) to the cluster: First host-name or IP address :- 172.16.30.10 Number of nodes :- 1 Nodes in your cluster: ====================== 1. 192.168.50.14 To add a new set of nodes to your cluster, type 'n'. To modify an entry, type its number. To delete an entry, type 'd' followed by that entry-number (eg. d1). To turn on advanced options, type '+'. For help, type 'h'. To save and exit, type 'q'. (to abandon all changes and exit, type 'Q') Option :- q Cluster configuration was saved. Failed creating temporary file: No such file or directory OK to also update the logical node numbers [Y/n]? y Suggesting to assign '172.16.30.10' as the central queue manager for the cluster (but be cautious if you mix 32-bit and 64-bit nodes in the same cluster) OK to update it now [Y/n]? y
Запускаем mosconf, выбираем 7 пункт (Become part of a multi-cluster organizational Grid)
[root@centos5k1 mosix]# mosconf MOSIX CONFIGURATION =================== If this is your cluster's file-server and you want to configure MOSIX for a set of nodes with a common root, please type their common root directory. Otherwise, if you want to configure the node that you are running on, just press:- What would you like to configure? ================================= 1. Which nodes are in this cluster 2. Authentication (ESSENTIAL) 3. Logical node numbering 4. Queueing policies 5. Freezing policies 6. Miscellaneous policies 7. Become part of a multi-cluster organizational Grid Configure what :- 7 There are no partners in your multi-cluster Grid yet: ===================================================== Please Type the name of a new partner to configure (or press to exit) :- What would you like to configure next? ====================================== 1. Which nodes are in this cluster 2. Partner clusters in your multi-cluster organizational Grid 3. Authentication (ESSENTIAL) 4. Logical node numbering 5. Queueing policies 6. Freezing policies 7. Miscellaneous policies q. Exit Configure what :- q [root@centos5k1 mosix]#
Что получилось в результате:
[root@centos5k1 mosix]# pwd /etc/mosix [root@centos5k1 mosix]# ls features freeze.conf mosix.map partners queue.conf tunes userview.map var [root@centos5k1 mosix]# cat userview.map 1 172.16.30.10 1 [root@centos5k1 mosix]# cat mosix.map 172.16.30.10 p [root@centos5k1 mosix]#
Добавляем остальные машины в кластер, я прописал их в ручную в фаил, кстати они уже должны быть настроены точно также, сам centos стоит у меня на виртуальной машине, поэтому просто копировал ее в другую папку, запустил и сменил ip
[root@centos5k1 mosix]# cat mosix.map 172.16.30.10 1 p 172.16.30.20 1 p 172.16.30.30 1 p [root@centos5k1 mosix]# cat userview.map 1 172.16.30.10 1 2 172.16.30.20 1 3 172.16.30.30 1 [root@centos5k1 mosix]#
Запускаем mosix
[root@centos5k1 mosix]# service mosix start Starting MOSIX... MOSD - MOSIX Version 2.24.2.2 [ OK ] [root@centos5k1 mosix]#
Тестируем работу кластера
Запустим монитор нагрузки "mosmon", нанем будет отражено три наших кластера(1 2 3)
Давай например запустим программу(john, как ее ставить рассказывать не буду) расшифровки паролей(допустим самбовских ). Запустим с первого кластера, направим на третьи, а наблюдать будем со второго через mon. Будем использовать утилиту mosrun
[root@centos5k1 ~]# mosrun -e -j3 /root/john-1.7.3.1/run/john /root/smbpasswd Loaded 229 password hashes with no different salts (LM DES [32/32 BS])
Как видно из рисунка почти вся нагрузка перешла на третьи кластер(процесс remoted), при этом нагрузка на первом кластере была равна 1-3% , либо ее вообще небыло.
В заключение скажу , что этот кластер подойдет для вычисления паралельных программ, например MPI