NatSemi Geode / bproc nodes Crude howto by David W. Hendricks ( cro_marmot_at_comcast.net ) Sept 24, 2003 This is intended to help the other three people out there wanting to make a Beowulf cluster out of those neat little Advantech PCM-5823 "biscuit SBC" boards. With some modification, perhaps it can also help to get a node running with a VGA console or other projects involving LinuxBIOS. This HOWTO will probably be updated a lot. This is not a final draft. Stuff is out of order, I used an outdated kernel and bproc version, and there are probably many spelling, grammar, verb tense, and otherwise stupid errors throughout this document. Note: I tried to be consistent, but if not just be aware at the terms "frontend node" and "master node" may be used interchangably. Also, a lot of commands were done using bpsh . The syntax is identical minus the bpsh if you do not have BProc. Here is the order things needs to happen: Download BProc ( bproc.sf.net ), a Linux kernel ( kernel.org ), freebios ( freebios.sf.net ), and etherboot ( etherboot.sf.net ). I suggest downloading bproc before the kernel since you'll need to know which kernel your version of BProc will patch. Building a phase 2 image This is the image that will be downloaded by the slave nodes from the master node. 1. Build 2.4.19 kernel for frontend and slave nodes. Compiled vanilla 2.4.20 source and applied BProc 3.2.5 patch (Also compiled from source) My config file can be found at: http://www.home.comcast.net/~zen_weasel/linuxbios_stuff/pcm-5823/Geode_Beowulf_Cluster_HOWTO Used a different source tree for the laptop's kernel and Geode kernel (Wouldn't have to worry about screwing up the two different config files as much). For the Geode kernel: Fixed IRQ routing table (arch/i386/kernel/pci-irq.c) by commenting out ^1 in pirq_cyrix_get and pirq_cyrix_set functions: static int pirq_cyrix_get(struct pci_dev *router, struct pci_dev *dev, int pirq) { return read_config_nybble(router, 0x5C, (pirq-1)/*^1*/); } static int pirq_cyrix_set(struct pci_dev *router, struct pci_dev *dev, int pirq, int irq) { write_config_nybble(router, 0x5C, (pirq-1)/*^1*/, irq); return 1; } Also, I disabled module version checking in frontend node's config... modversions was just getting confused in this case. Lastly, after your new kernel is built for the frontend node and booted, make sure the bproc and vmadump modules are loaded. Modprobe command should pick them up without any warnings. 2. Edit /etc/dhcpd.conf file. The IP addresses, netmask, MAC addresses, etc. can be changed to go in any network environment. The values set below are just examples: ddns-update-style ad-hoc; shared-network net { subnet 10.128.0.0 netmask 255.255.0.0 { # range 10.0.4.10 10.0.4.42; } subnet 10.0.4.0 netmask 255.255.255.0 { # range 10.0.4.10 10.0.4.42; option routers 10.0.4.1; } } host n1 { hardware ethernet 0:d0:c9:56:03:29; #MAC address fixed-address 10.0.4.10; #First IP in range set earlier filename "/tftpboot/geode"; } Make sure TFTPD is set up correctly before banging your head on the wall trying to toubleshoot an otherwise trivial issue. I suggest using the tftp client and trying to manuall pull the image down to make sure you have the path set correctly in your dhcpd.conf file. To add more nodes, copy and paste the lines starting at host n1 { to the closing brace of host n1, then change the MAC address and the fixed address to whatever you want the next node to look like. 3. Edit /etc/beowulf/config file The only things that really need to be changed are the MAC address assignments, IP address range, and the number of nodes. Check the example (beowulf.config) for correct syntax in case the included file is not good enough. Note: If you make changes to dhcpd.conf (Add another node, change MAC / IP addresses, change boot images, etc), you will have to restart the dhcpd deamon for the changes to take effect. To do this, you can usually either run service dhcpd restart (Redhat and similar distributions) or go into /etc/init.d and run ./dhcpd restart. Likewise, if you make changes to the /etc/beowulf/config file, you must restart the beowulf deamon. Any nodes affected by those changes should be restarted for the changes to take effect. 4. Make bootable phase 2 Geode image A. Make sure your Geode's kernel has console on serial support enabled (under character devices) and VGA disabled (under framebuffer support). VGA support may come later, but as of now it's not supported for this board. Also, make sure your Geode's kernel is patched with bproc if you're setting up a cluster: /usr/src/bproc/make LINUX=/usr/src/linux-2.4.xx-geode_src_tree /usr/src/bproc/make LINUX=/usr/src/linux-2.4.xx-geode_src_tree modules /usr/src/bproc/make LINUX=/usr/src/linux-2.4.xx-geode_src_tree modules_install /usr/src/bproc/make LINUX=/usr/src/linux-2.4.xx-geode_src_tree install B. Use the beoboot utliity to build a phase 2 (-2) stand-alone image (-i): beoboot -i -2 -k /geode/source/tree/arch/i386/boot/bzImage -o geode-image C. Assuming you use an ELF image to boot (This is set in the config file used for NLBconfig), use the mkelfImage utility (From the etherboot source tree) to make a bootable ELF image: mkelfImage --ramdisk=geode.initrd --command-line="root=/dev/ram0 panic=30 apm=off console=ttyS0,115200" geode-image elfImage The image made by beoboot is used, not the bzImage from the kernel D. mv elfImage /tftpboot/tftpboot/geode My image was made with beoboot cm.1.4 and mkelfImage version 1.16. Done! Other useful stuff tcpdump -i host ------ Setting up LinuxBIOS on the Geode 1. Since this step can thoroughly thrash your BIOS and render the machine inoperable, let's first make a backup. In the freebios/util directory, there is a utility called dumpdevmem. bpsh 0 dumpdevemem 0xfff8 0x8000 > /tmp/backup 2. Build a kernel with working IRQ tables (Make appropriate changes to pci_irq.c as described in step 1 of setting up a phase 2 image). 3. Find your payload Temporarily located at http://www.flagen.com/~sc/eepro100.ebi This will be used later on. 4. Make appropriate config files A. Download the freebios source tree (http://freebios.sf.net). B. Go to the freebios/util/config directory and use the NLBConfig.py program to make the config files for you: python NLBCongig.py /path/to/freebios/src/mainboard/advantech/pcm-5823/config.example /path/to/freebios The syntax goes: python NLBConfig.py You may wish to modify (Or relocate) the config.example. To get it working, I had to add option CONFIG_COMPRESS=0 in my config file since this board does not support shadowed BIOS. Once NLBConfig has successfully been run, it should have created a directory called pcm-5823 in whichever directory you're presently working in. cd to it. 5. Build the BIOS For this, you'll need the flash_rom program in the freebios/util/flash_and_burn directory. We're going to use it a few times for the rest of the procedure (And a lot more if something goes wrong), so it would be wise to link or copy it to the /usr/sbin directory or somewhere else where it's universally accessable. Now you should be in the directory that was created by NLBConfig. make clean and make. You should now have a file called romimage. In my case, I had to concatenate the romimage on top of itself since it was exactly half the size (256K) it should have been (512K): cat romimage romimage > 512k-image If you're already running bproc, you can Copy this image to the slave node like this: bpcp 512k-image 0:/ (For node 0) The make sure /dev/mem exists. If not do this: bpsh 0 mknod /dev/mem c 1 1 And finally, flash it: bpsh 0 flash_rom -w 512k-image If you aren't running bproc, copy the image and flash_rom through rcp, scp, floppy, cd, etc. and just do everything else without the 'bpsh in front. You should now have LinuxBIOS working on your node.