Welcome
to the
KludgeKollection
| Article |
Synopsis |
| KlusTriX |
We have just released our own
customized distribution of ClusterKNOPPIX. It is basically
ClusterKNOPPIX with a number of patches, fixes, optimizations (and the
entire KludgeKollection) installed and ready-to-go! Details and
explanations are available at our new website, courteously hosted by
the Ghana Unix Users Group. |
| Hey,
man! Don't mount that network file system off root! |
Dave Hitz of Network Appliance explains why
mounting network filesystems off root (/) is really baaaaaaad
juju. Read the article, then understand. The article discusses NFS, but does not appear to be specific to NFS. In fact, the principles detailed would seem to apply to any network file system. Maybe it works okay for you, but my experience has been that buring the oMFS (or NFS) mounts a few levels down (per the article's recommendations) has resulted in much less oMFS related trouble, especially when a node crashes and burns. Enjoy! P. S. Errors in the edited article provided are my fault, not that of Dave Hitz. |
| INSTALL SCRIPT now available for
the KludgeKollection. |
Included with the
KludgeKollection "KludgeKlump" tarball or available as a separate
download, the KludgeKollection Installer Script is available now!
It fixes some minor bugs (such as the CPUJOB and IOJOB not being able
to
find mosrun) . It also includes an option to install Wim
Vandermissen's new Kernel Sources for use with Debian-based
distributions (particularly ClusterKNOPPIX 3.3
(09/24). (Thanks, Wim!!!!) Due to space constraints imposed
by my ISP, unfortunately, you must download the .deb separately.
(See instructions below.) UPDATED 10/21/2003: New, completely re-written! Fully modularized! Lots of new bugs! (Just kidding. Hope I squashed 'em all!). Now works with associated files 'debian_package_list' and 'rpm_package_list' as well as 'do_ssh_hosts' to allow the user to specify/add/remove .debs (or RPMS) targeted for installation. NEW FEATURE: Automatically detects already installed packages. It doesn't attempt to reinstall them! Installs a few more .debian packages, offering the user full control over each. Also, offers a patched version of /etc/init.d/openmosix which checks to make sure the user running it is root and will also automagically locate setpe, mosctl and omdiscd on its own. (Originally, the openmosix startup script would not return an error upon failing to start due to permissions issues. It would, in fact, appear to run properly! Further, since the locate of mosctl, setpe, and omdiscd seem to be changed routinely without notice, it's about time the openmosix startup script was smart enough to find them on its own. ;-P ) |
| New: SNAPBACK SNAPTROL INTRUD_ID SNOO-P SSSH NEWSCRIPT MCTL PATCHED for openMosix: cpujob iojob nomig runhome nodecay slowdecay fastdecay ADDED for OPENMOSIX: runon |
|
Item |
Synopsis |
|---|---|
| NEWLY UPGRADED! KludgeKollection Installer Script do_ssh_hosts (Edit with list of your servers before running the installer script if you would like "Super SSH Capabilities.") debian_package_list rpm_package_list kludgelist Sorry, folks! Mr. Brain-Dead, here, (Me) forgot to include this rather critical file! This little number is necessary for the installer to install all these lovely KludgeKollection scripts! Wim Vandermissen's ClusterKNOPPIX 3.3 kernel source |
WHAT
IT IS: This is the first "official
beta" of the KludgeKollection install script. WHAT IT DOES: It installs the utilities listed below as well as adding a number of debian (or RPM) packages such as Sun's Java, openoffice.org, some utilities for Window Maker, aterm, and such like. Most of these are optional, and some items are installed for Debian but not for RPM-based distros. REQUIRES: do_ssh_hosts, debian_package_list, kludgelist rpm_package_list, the contents of the KludgeKollection (easily downloaded as the "KludgeKlump" tarball, below). The installer script and the associated files should all be run from the same directory, preferably /root/bin. This may mean copying do_ssh_hosts from /usr/local/bin/ to /root/bin. (Normally, on my cluster, the KludgeKlump is extracted in /root/bin/ and the install program is run from /root/bin/. Running it from other locations will produce unpredictable results). HABITAT: /root/bin TYPE: Bash script PENDING IMPROVEMENTS: Will routinely be updated as new utilities are added to the Kollection and old/deprecated ones removed. NOTE: This Install script was written primarily for use with ClusterKNOPPIX 3.3 (Sept 24 release), but should work with most Debian or RPM-based distros. NEW! This script repairs the problems with CPUJOB and IOJOB wherein they cannot find mosrun because mosrun is normally in /bin and not /usr/local/bin. The script repairs some other minor bugs/issues with hard-disk installed ClusterKNOPPIX 3.3 file locations and the like. It may work with equivalent bugs on upgraded distros that use the same files, i.e. CPUJOB and IOJOB. The script can also install the .deb of Wim Vandermissen's latest (2.4.22-1SMP) kernel sources, provided you download the .deb package at http://bofh.be/clusterknoppix/download.htm and then place the .deb in /root/bin. (The script will install it from there and then provide a brief instruction as to your next step.) The script has been rewritten for speed and better user control. It also now installs a patched version of the openMosix startup script (see below for details.). The script now autodetects already installed debian packages, and will skip those, prompting only to install those in debian_package_list that are NOT already installed. The script now installs sssh and optionally creates links based upon an updated do_ssh_hosts file such that it becomes possible to ssh to a host simply by typing that host's name at the command-line, e.g. "liberty" would result in opening an SSH session to the host called liberty. NOTE: The links and sssh program are placed in /usr/local/bin. |
| NEWLY revised
error-codes and a check to ensure that Snapback is running on the right
host! SnapBack snapback_hostkey New feature. Contains one word, namely the name of the host that collects snapback backups. Snapback simply uses this as a basic check to make sure it is executing on the right machine. (I guess I filled up the hard disk on my laptop one too many times doing a snapback backup when I thought I was SSH'd into Liberty where the big RAID array is....) The file here contains the word "liberty", the name of my backup server. You need to change this to the name of your backup server, or Snapback will give you error 20 and abort. backup_exclude I had to add a bogus line, "bogus_nonsense" to the top of this file so that it would download properly. Seems having a "/" as the first character throws most browsers for a loop. Go figure. If you want, you can remove the bogus line from the file so it looks like the following: /tmp /dev /proc /mfs /mnt (Without leading/trailing CR.) Or you can leave it. Snapback doesn't care and rsync (which is actually what uses the file) doesn't seem to be bothered by it (though, if you have a file called "bogus_nonsense", it will be excluded from backup.) NOTE: /root/bin/backup_exclude is required for snapback to operate properly (Not to try to back up inappropriate filesystems like /proc or /dev or /mfs) The file is supplied with appropriate defaults for most systems (and all machines on my network and cluster.) |
WHAT
IT IS: SNAPBACK is a modified version of Mike Rubel's RSYNC
Snapshot Rotation utility. Mike's site is here. WHAT IT DOES: Like Mike's original work, this program allows the user to make multiple "snapshots" of the target filesystem. It's got a few added features (Bugs?) when compared to Mike Rubel's original. In particular: 1. The user may specify any number of snapshots (backup sets) up to the amount of available storage space. 2. Snapback has an interactive capability: If no command-line parameters are specified, it will prompt the user for the appropriate information. 3. Snapback can back up Windows shares through SAMBA. These must be mounted in /mnt/servername/sharename with the smbclient software provided with SAMBA. In addition, Snapback must be instructed that the backup is a SAMBA backup by providing the appropriate prompt ("SAMBA") on the command line or by selecting SAMBA interactively when prompted for the Backup Type. 4. Snapback uses RSYNC directly in ALL operations. For backing up UNIX machines, it uses SSH for security. Snapback does not directly support RSH. 5. Snapback offers an option to back up a UNIX filesystem and only that filesystem, exclusive of any other filesystem mounted under the target filesystem. 6. Snapback offers an option to back up the target filesystem AND any filesystems mounted thereunder. 7. Snapback only runs under the ROOT account. It will not run if the UID is not ROOT. 8. Snapback automatically and intelligently detects and locates required ancillary programs. It will abort if it does not find all required ancillary programs. 9. Snapback includes fairly thorough, if somewhat terse, built-in help. 10. Snapback is completely modularized into functions for easy maintenance and upgrade. 11. Newly revised: Error codes are sorted by function, e.g. GETARGS() returns errors from 30 to 40, CHKCMD() returns errors 1 to 9,R_U_ROOT() returns 10 to 19, and CHECK_HOST() returns 20-29, so now you can tell which function is actually passing errors if something goes wrong. 12. New Feature: Now checks to ensure that it is running on the appropriate host (the machine you have chosen to receive the backup files when Snapback runs), so you avoid doing what I did a few too many times, namely forgetting to ssh to the backup server before running snapback, and thereby filling up the hard disk on the wrong machine with a spurious backup. REQUIRES: ask, ssh, SAMBA, backup_exclude, snapback_hostkey, id, seq, echo, mount, smbmount, rm, mv, cp, touch, rsync, hostname. All of these, except ask are part of most regular *NIX distros, while ask is included as part of the KludgeKollection. HABITAT: /usr/local/bin and/or /root/bin (Intended only for use only by ROOT, though.) Default executable supplied with the KludgeKollection is flagged 744. In addition, SNAPBACK checks the UID of the user running it to make sure that it is ROOT. Finally, the "backup_exclude and snapback_hostkey" files are expected to be in /root/bin (even though a copy is also placed in /usr/local/bin/. This is a safety feature. The program will not run if this file is missing!) TYPE: Bash script. NOTES: No guarantees. This is a potentially lethal program, so use it with care. I use it extensively, and it works for me--I've pretty much phased out the use of dump and tar in favor of snapback. I think it's that good, but, of course, YMMV. Anyway, use it in good health and thanks to Mike Rubel for the original! NOTE: I use a directory called /usr2/backup as the "root" of my backup system. Thus, for example, the machine called "marinduque" is backed up into directories off /usr2/backup/marinduque. There is a variable called BACKUP_ROOT, which is defined near the top of the MAIN function (It's on Line 429 in the current version.) You can change this to reflect whatever backup root you feel is appropriate, e.g. /headquarters_net/archives or /scully/mulder or whatever. Current version is 11132k3-a . PLANNED IMPROVEMENTS: This thing is actually pretty good. May add a "second level" capability like Mike's where one program can keep a daily rotation and another (or another part of this one) keeps a weekly or monthly rotation. May also make it capable of parsing snapback_hostkey for multiple authorized hosts. We'll see.... I'm open to (constructive) suggestions. |
| Snaptrol windows_boxen linux_boxen snapback_hostkey gen_backup_tree |
WHAT
IT IS: A set of scripts that automates running Snapback
(above)
on multiple machines on a network. WHAT IT DOES: Though the scripts are fairly complex, its goal in life is simple. It reads two files, windows_boxen and linux_boxen and uses these to run Snapback with appropriate command-line switches. It defaults to keeping two weeks (14 days) worth of backups through Snapback for both Windows and Linux hosts on the same network, though it will accept other settings. It will back up only the servers and hosts listed in the aforementioned files. Snaptrol can be used as a cron-job, so that backups become automatic and require little, if any, user intervention. REQUIRES: ssh, windows_boxen, linux_boxen, snapback, ask a functioning SAMBA installation (for backing up Windows boxes) and rsync. Like Snapback, Snaptrol was created using the "newscript" mastering utility (Available below), so it includes a self-check to ensure that it is running on the appropriate host, as root, and that all the necessary programs are installed. NICE TO HAVE: gen_backup_tree will create the file-structure required for snaptrol and snapback to work. Snapback can create the required file for each backup as it runs, but it does not create the mount-points for externally mounted filesystems (such as Windows filesystems mounted via SAMBA (read-only) for backup). gen_backup_tree neatly automates this potentially tedious task. It uses the same "linux_boxen" and "windows_boxen" scripts that the other scripts use. Please use with care: It's very early beta and still somewhat buggy! HABITAT: /usr/local/bin (for the executable, snaptrol) /root/bin for the control files (windows_boxen and linux_boxen) TYPE: Bash script, control text files. NOTES: Text files provided (windows_boxen and linux_boxen) are provided. They contain sample hostnames. You will need to change these files to reflect the names of the hosts on your network that YOU want backed up. You will need to edit /root/bin/snapback_hostkey to contain the name(s) of the machines authorized to run snaptrol and snapback. On my network, only Liberty has this permission. NOTE: Snaptrol uses this file to identify the authorized backup host and perform a special backup of this machine. Obviously, if this file is missing, or contains the wrong name(s), the machines in question will be backed up incorrectly or not at all. They may even try to back up the backup directories, which can lead to an endless loop and fill up the hard disks of the backup host real fast, which would be A Very Bad Thing. PLANNED IMPROVEMENTS: Modify the part of the program that handles the host so that it will parse a file with multiple hostnames in it, rather than dealing with just one. I'm not entirely certain that this is such a great idea, though. It is probably best to have one host listed in the host key for each fileserver running snapback and snaptrol. You probably don't want one snaptrol server backing up another.... |
| NEW sssh-SuperSSH |
WHAT
IT IS: A small script from
Linux Server Hacks by Rib Flickenger (O'Reilly) that enables
establishing SSH connections to hosts by simply specifying the host's
name on the command-line. WHAT IT DOES: Through establishing symbolic links to /usr/local/bin/sssh, it becomes possible to ssh to a host (such as Liberty) by simply typing "liberty" at the command-prompt. It is possible to pass commands along, e.g. "liberty df -kl" as well. Clearly, in combination with execution of do_ssh, this utility would make SSH a dream! REQUIRES: ssh, do_ssh_hosts, optionally, the KludgeKollection installer script. HABITAT: /usr/local/bin TYPE: Bash script, symlinks (made from do_ssh_hosts) NOTES: Leave the sssh program itself flagged 555 for safety's sake. (Otherwise, it seems that it may get overwritten.) IMPORTANT: The KludgeKollection install script creates the symlinks for you, provided you have updated /usr/local/bin/do_ssh_hosts with the appropriate host-names first). Otherwise, you can skip the KludgeKollection installer and create the symlinks on your own. sssh does NOT create any symlinks on its own. |
|
WHAT IT IS: A
poor-man's DSH.
REQUIRES: do_ssh_hosts, ask HABITAT: /usr/local/bin TYPE: Bash
script. UPGRADE: Now will execute simple programs (that don't require command-line switches) on remote hosts. In such a case, the command-line is do_massdo
/usr/sbin/up2date (for executing up2date on each host in
the
do_ssh_hosts file). The original command-line, do_massdo remains
effective for interactive use, e.g. |
|
| ponys (Po' Man's
NIS-like thingie) Okay. I think this thing's finally fixed.... |
WHAT
IT IS: Ponys is a user/group/samba user account duplicator. WHAT IT DOES: Ponys will read the ponys-users file in its current directory (usually /root/bin/ or /usr/local/bin) which contains a list of the user-accounts to process. Then, for each account, it will go to a master server which has the account already set up and rsync the contents of the home directory for that account to the home directory of the host from which it is being run. Then, ponys will duplicate the passwd, group, shadow, and gshadow files from the master server to the current host. Finally (and optionally) ponys will copy /etc/samba/smbpasswd from the master server to the same location on the current host. REQUIRES: ask, ponys-users file Must be run from the same directory as contains ponys-users HABITAT: /root/bin/ (for security reasons. This is a dangerous tool: While it will run from most anywhere, you don't want the average user getting their paws on it!) TYPE: Bash script. NOTES: ponys-users is NOT included. As this is a simple text list of the account-names you are interested in creating, and that ALREADY EXIST on the master server, I figured it would be easier for you to just 'roll your own'. Mine contains this: vmorgo afreed blinnane mshea mmiller (and so on) Ponys does not creat anything on its own--it expects to DUPLICATE accounts that have already been created on a master server. Ponys has some pretty nice error-handling and a neat little bug-check feature, if I say so myself. It also is completely modularized into functions, so should serve as an example of fairly good (if not terribly succinct) BASH scripting practice. It's probably not a bad idea to run this thing every now and again as a cron-job on each server EXCEPT the "golden master". PLANNED IMPROVEMENTS: Implement a version that pushes updates out to nodes rather than requiring to be run FROM nodes. Thus, only one instance of the program is required, running on the master server. (It should be a simple matter of reversing the direction of the rsync commands in the functions....) |
| do_ssh_hosts |
WHAT IT IS: do_ssh_hosts
is the equivalent of /etc/hosts. It is a
simple, text-formatted listing of hostnames which is used as an input
for other scripts in the KludgeKollection. I have provided a
sample do_ssh_hosts for your edification. Of course, you must
make
your own file with your own server names. WHAT IT DOES:
do_ssh_hosts is required by several programs in the KludgeKollection. REQUIRES: (None) HABITAT: /usr/local/bin TYPE: Bash script. |
|
ask |
WHAT IT IS: Ask
is a simple script that allows the programmer to ask the
user yes/no questions. Choice is a C program that prompts for a numeric choice to be entered at the keyboard. It returns an errorcode corresponding to the number entered. This should make the program useful in interactive scripts requiring a choice be made from more than two options. WHAT IT DOES:
Ask
returns an error code of 0 (True) for
"yes" and 1 (False) for "no". REQUIRES: (None) HABITAT: /usr/local/bin TYPE: NOTES: ask "Continue? [Y/n]" in a script will cause the user to see Coninue? [Y/n] on the screen when the script reaches the invocation of ask. PLANNED IMPROVEMENTS: |
|
WHAT IT IS: Just what
we've all been waiting for! A new way to corrupt kernel
builds! No, but seriously, folks, I use this VERY EARLY BETA
script to hand-roll my own kernels because I find that I often forget
one or another of the required steps, and I got frickkin' sick of
having to look it up every time. This thing will even pre-patch
your kernel for you (Provided the patch is in the appropriate place and
is provided in BZIP2 format.
Oh, yeah: This
version includes an option (which it will prompt you for if you
don't specify it on the command-line) to take advantage of multiple
processors in SMP machines. (It probably won't work on a cluster
because most of the compile operations are very short and finish before
triggering the openMosix migration algorith.) In case you are running, my 2.4.25-based FrankenKernel
(available at http://klustrix.ghuug.org) was made with this program. REQUIRES: bunzip2,
usual collection of tools used to make kernels, e.g. GCC, etc.
Also requires a properly installed kernel source tree (obviously). HABITAT: /usr/local/bin TYPE: Bash script PLANNED IMPROVEMENTS:
Needs better error-checking.
Under some conditions, it will return "Success" even when it has
actually failed. This is because the echo command in line 2
ALWAYS
works and we erroniously check the error-level for that, rather than,
as
we should, the "ssh $j hostname" part of line 2. By the time you
read this, aquaint will probably have been updated to correct this bug. |
|
|
WHAT IT IS: do_rsh is
a a script that enables passwordless rsh for all
users on a host. It uses do_ssh_hosts as an input file for
enabling passwordless access to all the hosts listed therein.
Optionally, it will even enable passwordless rsh for the root account
(not a good idea security-wise). NOTE: do_rsh requires a
linux distro that uses xinetd, rather than the older inetd
metadaemon. Thus, for example, while it works on RedHat 7.3 and
Mandrake 9.1 (and should work on later versions of both), the portion
of
it that modifies the settings for the rsh and rlogin services as well
as
the root-login enable portion will not work on Debian Woody
(ClusterKNOPPIX 3.2/3.3). You would have to modify
initd on your own and take other steps to enable root access through
RSH. REQUIRES: rsh, rlogin, xinetd, chkconfig, ask, do_ssh_hosts. HABITAT: /root/bin (recommended); should work in /usr/local/bin. For safety's sake, this program should be run only by ROOT. TYPE: Bash script. PLANNED IMPROVEMENTS:
Modify the code so that it will
refuse to run unless the logged-in user is root. NOTE: The
openSSH version included with ClusterKNOPPIX 3.3 is bloody
fast. I only used RSH because SSH seemed so slow and CPU
intensive.
At this point, the faster openSSH will probably soon obviate the need
for this program and it will be deprecated. Complete documentation of what do_rsh does is included in the "rsh_rlogin_enable-LINUX.HOWTO" text file. |
|
|
do_ssh
|
WHAT IT IS: do_ssh automates
the process of passwordless SSH authentication. WHAT IT DOES: It enables passwordless ssh for ONE user on a host. It is designed to be run by any user on a host, and it enables passwordless access to all the hosts listed in do_ssh_host for only the user running the program. This makes it somewhat more secure than the more common option of enabling global passwordless ssh on a host, since, ostensibly, a system administrator could control who has access to, and can run, do_ssh, thereby avoiding the problem of conferring passwordless ssh access to individuals who should not have it. do_ssh may be run by root in order to obtain passwordless ssh for the root account on each host listed in do_ssh_hosts. REQUIRES: ssh, do_ssh_hosts. Optionally, the root password is required if the system administrator is willing to allow the user to restart the sshd daemon on each host after updating the known_hosts files. Otherwise, the user's changes do not take effect until sshd is restarted on each host. HABITAT: /usr/local/bin TYPE: Bash script PLANNED IMPROVEMENTS: 1. Modify the code to add an option to enable global passwordless access to a host by modifying /etc/ssh/ssh_known_hosts and /etc/ssh/sshd_config. (This may actually come in the form of a separate script that requires root access to run.) 2. Recode as a stand-alone C program. |
| do_ssh_ck "Works for me beta" Try it, you may like it. Or, it may eat your children, dogs, disks, etc..... |
WHAT
IT IS: do_ssh_ck is a slightly modified
version of do_ssh intended for use with ClusterKNOPPIX running from
CD. It should work with any distro, since the only major mod was
to change the search location for do_ssh_hosts from /usr/local/bin to
the
root directory (/). The program, do_ssh_ck itself should run from
anywhere on the system. WARNING: It still may have bugs
(particularly the code that is supposed to delete an existing
masterkeys
file preparatory to replacing it with a new one. Feel free to
modify the code (but post me the fixed code so I can put it on this
website for all to enjoy.) WHAT IT DOES: See do_ssh, above. REQUIRES: ssh, do_ssh_hosts. See do_ssh, above. do_ssh_hosts must be in / (the root directory) rather than in /usr/local/bin as is the case in previous versions. HABITAT: Anywhere. You can put it in your home directory, in /root, or wherever. It should work. TYPE: Bash script. PLANNED IMPROVEMENTS: DEBUG IT! I'm pretty sure there's still a bug or two in there somewhere.... This version will probably supercede the original do_ssh because it no longer requires ASK and because it "Works For Me" on ClusterKNOPPIX when ClusterKNOPPIX is running from the CD-ROM. This version should work on any system in so far as I know. May recode as stand-alone C program. IMPORTANT NOTES: 1. do_ssh_ck will need to be re-run each time ClusterKNOPPIX is restarted from the CD. You may wish to rename the program to "do_ssh" before running it to keep things simple. 2. do_ssh_ck will stop and restart the sshd daemon on ClusterKNOPPIX (or any other distro). I have seen this cause ssh to lock up (only on ClusterKNOPPIX), so beware. I DO know that this is not a problem of do_ssh, but rather a problem with the version of ssh included with ClusterKNOPPIX since manual restarts at the console sometimes have thee same problem.) What does the "_ck" ending on do_ssh_ck mean? ClusterKNOPPIX, of course! |
| nuke |
WHAT IT IS:
This is a script that allows you to kill a process by
specifying all or part of its name, rather than having to look up a
PID. The *NIX command KILLALL purports to do the same thing, as
does PKILL, but
my impression is that NUKE just seems to work better.... REQUIRES: kill, grep HABITAT: /usr/local/bin TYPE: Bash script. PLANNED IMPROVEMENTS: Error handling and reporting. Maybe. |
| pararip |
WHAT IT IS:
This is the infamous Parallel Ripper. It's one's right out of the
openMosix
howto. Pararip will read a music CD and convert the .CDDA music
files thereupon into .WAV files in the CURRENT DIRECTORY. Then,
again, working in the CURRENT DIRECTORY, pararip will use your
openMosix
cluster to convert the WAV files into either ogg (oggenc) or mp3
(bladeenc) files. Where it used to take the better part of an hour or so to encode .WAV to MP3 files with bladeenc on a PIII 500, I now routinely see encoding speeds on the order of 2 to 5 minutes for an entire CD's worth of MP3 files using a cluster with 8 CPUs distributed over 5 nodes! REQUIRES: oggenc, bladeenc, ask, cdparanoia, a properly configured openMosix cluster. HABITAT: pararip lives in /usr/local/bin, but the program MUST be executed from the directory in which you want the converted .ogg or .mp3 files to reside. TYPE: Bash script. PLANNED IMPROVEMENTS: Code is a bit messy and needs to be cleaned/tightened up. Error-checking is not that great, but it runs pretty well. COMMENTS: It is worth noting that bladeenc seems to "parallelize" far better onto a cluster than does oggenc. NOTES: Latest version is able to clean up after itself and includes some other nice features like the ability to just make the WAV files, convert existing WAV files to OGG or MP3 and so on. And, hey, the code's been cleaned up (a bit)! |
| multisync parasync |
WHAT IT IS:
This pair of indespensible copy utilities do
essentially the same thing: They copy the specified directories
(or files) to all machines listed on the command-line. Multisync
does so sequentially, one host after the other, and requires ssh.
Parasync, on the other hand, parallelizes the copies so that they all
run at the same time. Additionally, parasync may work with rsh as
well as ssh. Parasync and multisync both depend upon rsync, and so are susceptable to the same benefits and limitations (and dangers!) as that program. REQUIRES: rsync, ssh, rsh (parasync only) HABITAT: /usr/local/bin TYPE: Bash script PLANNED IMPROVEMENTS:
Add a "checkprogs" function that makes sure required programs (rsh and
ssh) are present before the script runs (similar to the one in the
soon-to-be-released SNAPBACK program. NOTE: Though it
is not readily apparent, both programs can copy a
single file--just specify the complete path and filename on both the
"source" and "destination" prompts for each program. |
| wmaker-terminal |
WHAT IT IS: For
reasons that remain a mystery to me, I've seen too many
examples of Window Maker installed that have lousy terminal
support. That is, the little "Terminal" icon on the desktop
usually points directly to xterm or some other terminal program in such
a way that that program can only be launched once before the icon
deactivates. -g gnome-terminal -a aterm -e Eterm -x xterm -r rxvt
REQUIRES: At least one of the supported terminal programs (Most distros at least come with xterm and rxvt.) This program does NOT actually require Window Maker, though it was originally written for use with same. HABITAT: /usr/local/bin TYPE: Bash script |
| setistart |
WHAT IT IS: Setistart
is one of the oldest kludges in the Kludge
Kollection. It is used to control setiathome clients.
REQUIRES:
./setimaster off the current directory
containing the setiathome program. Recommended: operational
openMosix cluster. HABITAT: Your
home directory or /usr/local/bin (default) TYPE: Bash script PLANNED IMPROVEMENTS:
This one needs a re-write. The
code leaves a bit to be desired, and is rather messy. But, it
works pretty well, at least for me. Take a look at the code to
see
how it works (and where it needs improvements.) |
| cpujob iojob runon (New--openMosix doesn't come with this anymore, so I thought I'd make one... It's handy to have....) nomig runhome nodecay slowdecay fastdecay |
WHAT
IT IS: Patched versions of the the openMosix userland
scripts
for openMosix userland tools 0.3.4.1 as supplied with ClusterKNOPPIX
3.3 (Sept 24, 2003 release) and presumably as supplied in general. WHAT IT DOES: Identical to original versions, except that these have a very minor edit that instructs them to use /bin/mosrun instead of /usr/local/binmosrun, since mosrun is in /bin and not /usr/local/bin. REQUIRES: OpenMosix and OpenMosix userland tools installations (Userland tools should be 0.3.4.1 or version current as of 09/24/2003.) HABITAT: /bin TYPE: Bash script. PLANNED IMPROVEMENTS: It is hoped that the openMosix userland tools will be edited/modified so that these patches (and the openMosix one below) are no longer required. Yes, folks, we're shooting for deprecation, here! NOTES: These patches may be installed optionally through the KludgeKollection Installer Script (KKIS). |
| openmosix |
WHAT
IT IS: A patched version of the startup script for current
versions of openMosix (As provided with ClusterKNOPPIX 3.3 (September
24, 2003 release, and presumably all other releases of openMosix). WHAT IT DOES: The stock version of /etc/init.d/openmosix does NOT check for root privileges before attempting to start. In addition, it is hard-wired to expect to find SETPE and OMDISCD in /sbin and MOSCTL in /bin. Unfortunately, the trend seems to be to move these files to other locations, so the script fails to find them. This new version not only insists upon being run by ROOT (It returns an appropriate error-message otherwise), but also will AUTOMAGICALLY locate and "memorize" the locations of SETPE, MOSCTL and OMDISCD. Thus, no matter where these files are, as long as they are in the search-path, the openMosix startup script will find them. (It will also helpfully tell you where they are!) There... Isn't that much nicer? REQUIRES: the "which" command; A complete installation of openMosix. The program is nothing without openMosix! HABITAT: /etc/init.d/ or wherever openMosix lives on your machine. (By default, the KludgeKollection installer will place the file in /etc/init.d/openmosix.) TYPE: Bash startup script. NOTES: This patch may be installed optionally through the KludgeKollection Installer Script (KKIS). |
| Serverturbo Newly rewritten. Modularized, newscript(tm) script. |
WHAT
IT IS: Serverturbo performs some optimizations
to improve the performance of your server, specifically by making
changes to settings in /proc and hdparm It is a very simple program and
has
the ability to restore the changed settings to the (RedHat 7.3)
defaults. WHAT IT DOES: The program modifies settings for hard-disk performance through hdparm and max_readahead and min_readahead (essentially modifying the disk buffers (disk-cache)). It also optionally modifies some settings for the TCP/IP stack and for the virtual memory manager. To have the settings made permanent and if you don't want to use/don't have the sysctl utility, you may want to put a call to serverturbo at the end of rcd.local, e.g. /usr/local/binserverturbo -o to enable all performance boost parameters. REQUIRES: ask. HABITAT: /usr/local/bin TYPE: Bash script UPGRADED: Executing "serverturbo" by itself on the command-line will automatically set all optimizations to the defaults that come with Debian Woody (A safe, conservative set). It will not, however, touch the hard disk optimization settings. NEW: Single switch for either disabling or enabling ALL optimizations in one shot. NEW: Support for hard disk performance tuning. NEW: Now supports Debian(Woody) and ClusterKNOPPIX 3.3 NEW: Completely re-written and modularized in new-script formant for easy maintenance, modification and troubleshooting. NEW: Now produces a lovely aroma faintly reminiscent of lilacs! NOTES: You can, and perhaps should, modify the code in this script to suit the needs of your particular distribution. Serverturbo has been tested with Debian (ClusterKNOPPIX 3.3), RedHat 7.3 and Mandrake 9.1, and should work with these, or any similar distro. I left the Bugcheck in that comes up just before the program runs its optimizations. This makes it kind of useless in automated scripts, but handy for checking parameters before starting the actual optimizations. If you don't like this, simply comment out the call to the "BUGPAUSE" function in "MAIN". It's nearly at the end of the script. PLANNED IMPROVEMENTS: The built-in help needs work. I'm considering making the parameters for the virtual memory manager individually tuneable, rather than the hardwired defaults they presently are. I really need to do this. The fifth parameter (normally 500) is apparently related to how often the VMM flushes/updates/does something with the disk. When serverturbo -o is run, this gets set to 150. On most machines, that's fine, but on my laptop, the ticking is [)@^^|\| annoying, so I usually go and set that one back to 500.... |
| mctl |
WHAT
IT IS: Allows the execution of multiple mosctl commands
on one command-line. With minimal modification, should be able to
execute a large number of ANY command on one command-line. Note
that it operates serially, rather than spawning all the commands at one
time. (If you want a "spawner", take a look at either Pararip or
Parasync on this site and go ahead and trick out the code. Feel
free to post me on the changed version. If it's totally cool,
I'll pust it here.) WHAT IT DOES: Moreno Baricevic has rewritten the original version of this utility so now any number of mosctl commands may be executed on one command-line. Any multipart commands (such as "setspeed 25000" have to be wrapped in quotes so that the two parts are not interpreted separately. In the present incarnation, the program will assemble the final command-line and ask you for confirmation before executing it. Try it. You will like it. Moreno has done a magnificent job of commenting his source-code (Ignore his comments about his English being bad--he writes better than most Americans I know!), so there's a lot to learn just from reading through the listing. In particular, he uses arrays in shell-scripting--a trick I never even knew about! REQUIRES: mosctl, a working openMosix installation. HABITAT: /usr/local/bin TYPE: Bash script. |
| newscript newscript_template |
WHAT
IT IS: This is a script-generator. It uses the
newscript-template to generate the basic shell-script, which the user
then may modify to suit a particular purpose. WHAT IT DOES: Executing the command: newscript snoo-p will generate a skeleton shell-script called "snoo-p", which is already flagged executable by all and ready to edit by root. This new script is placed in /root/bin. Use of the newscript utility dramatically reduces development-time when writing scripts, because it eliminates the need to duplicate code that is common to all scripts. It also standardizes the script format so that debugging is vastly facilitated. Finally (and hopefully), it encourages good programming form through the use of simple, easily maintained functions and modules. REQUIRES: Root access, newscript-template, ask, date. COMMENTS: newscript_template is fairly feature-rich. It includes a number of functions and a shell "main" for the user to complete. The functions provided in newscript_template include, but are not limited to:
/usr/local/bin (newscript) /root/bin/ (newscript_template) TYPE: Bash script. NOTES: By default, newscript is configured to be used by /root. However, consideration is made for the possibility of any use using this handy utility. To that end, the KludgeKollection Installer Script places a copy of newscript_template in /usr/local/bin. If you want all users to be able to use newscript and newscript_template, you will need to comment out the call to "R_U_ROOT" in the MAIN function of the program. You also will need to change references in the newscript program so that the program looks for /usr/local/binnewscript_template instead of /root/bin/newscript_template. It is very easy to modify newscript_template without touching newscript, itself. This gives the program total flexibility in the type and nature of the scripts it generates, such flexibility being completely independent of newscript, itself. UPDATED: Automatically date-stamps new programs with a version number based upon the current date. UPDATED: Newscript also stamps the generated shell-scripts with the version number of Newscript_template used to generate them. By the way: Newscript, itself, was the first shell script program generated using Newscript template! |
| Snoo-p intruder_list List of the IP addresses of intruders--hosts to be looked up through snoo-p. The sample list provided here is a listing of actual IPs detected as sources for intrusion attempts against my firewall! intrud_id Simple script that extracts the IP addresses of intrusion attempts from firewall logs. |
WHAT
IT IS: Snoo-p (Pronounced as "Snoopy", Charlie Brown's
dog.) is a program that will attempt to find out the relevant
information about one or more given IP's or hostnames. It
automatically generates a report for each hostname or IP tested,
suitable for submitting as a complaint-form to that person's ISP's
"abuse@" address. This one is so simple to operate that there's actually more info here about the intrud_id auxiliary utility for use with snoo-p than there is about snoo-p, itself! WHAT IT DOES: Snoo-p, by default, will use intruder_list as its input file. It will parse this file, performing a lookup on each IP address or hostname in the list. For each entry in the list, it will report pertinent information including who owns the address, various registration data, and so on. In addition, it will construct a ready-to-e-mail complaint report for that entry, which it will save in a file called <entry>_intruder.txt. For example, if testing 69.71.82.221, it will save the pre-made complaint report in 69.71.82.221_intruder.txt in the current directory. The program normally imposes a delay of 6 seconds between each look-up so as to avoid deluging the whois servers with an overload of incoming requests. (Whois actually generates a warning about not deluging whois servers with too many automated requests on pain of grevious bodily harm) If you wish to override this setting (at your own risk!), you may do so by passing the program two arguments: The first is the name of the file it should use for input (even if it is intruder_list) and the second argument (not documented in the program's on-line help 'cause you ain't supposed to use it!) is a number representing the number of seconds between requests to the whois servers. intrud_id Most firewall log files report intrusion attempts using certain keywords such as "Source", "Inbound", or "Originating IP" to identify the addresses of hosts that are originating intrusion attempts or scans. Often, these logs are long and filled with irrelevant information. The small script intrud_id has been provided as a convenience for processing these logfiles: Its syntax is: intrud_id [input filename] [keyword] (optional: [-c/-u]) (optional: >[output filename]) Example: intrud_id firewall.log Source >intruder_list The program will read the firewall.log file, searching for all lines containing the word "Source". For each of those, it would a couple of "cut" operations, looking for the appropriate IP address. It will deposit its output in the file called intruder_list in the current directory. If this parameter is omitted, the output goes to stderr (the screen). The copy of intrud_id provided here is customized to work with my particular firewall, and it works flawlessly. Unless your firewall generates log files in the same format as mine does, you are going to have to play with the field delimiters and field specifiers (-d and -f) in intrud_id's "cut" commands to get it to cut the right field(s) for you. You may have to add (or remove) cut commands, or maybe even resort to awk. But once that's done, intrud_id should generate a nice, numerically ascending sorted list of IPs extracted from your firewall log--with all duplicates removed, of course! The list will be in the proper format for use with snoo-p. This is very handy for those of us with better things to do than parse source IPs out of 3000 lines of firewall log every day! The optional -c and -u arguments may be passed to intrud_id to make it return the number of entries it finds in the source-file containing the keyword and IP address (-c), or the number of unique lines containg the keyword and an IP address (-u). On the sample log I tried it on, I had 129 IPs total, of which 110 were unique. REQUIRES: cut, ask, intruder_list, whois (current version as of Sept 2003). intrud_id will also require an input-file and a search keyword. It is not an interactive utility--it will sit there doing nothing if typed without any command-line arguments. (Sorry, kids. No error-checking on this one--it's a quick one-off, not a proper newscript-based shell program.) HABITAT: /usr/local/bin NOTE: intruder_list must be in the CURRENT DIRECTORY, regardless of the location of snoo-p. The output files will be created in the CURRENT DIRECTORY. NOTES: If the intruder_list file is missing from the current directory, the program will exit with an error message. It is recommended that this program be run by root or an administrator who knows what he or she is doing. While the program is not particularly dangerous, it is really intended for use by system administrators and/or personnel in charge of security and intrusion detection/reporting. Each line of the intruder_list file must contain one IP address or hostname. An actual sample, with addresses of hosts that have attempted to bypass my firewall is provided to the left. You may add to this file or replace it as you see fit for your particular situation. |
| set_services_for_openmosix |
WHAT
IT IS: This utility compares the
current settings for services on your host with those taken from a
"golden" host, e.g. one for which all the services have been set to the
desired or 'ideal' settings. It then creates a 'diff' file from
the two files. The user is offered the opportunity to let the program change the service settings on their host to match those of the ideal host using the generated diff-file as input. REQUIRES: xinetd, chkconfig, RedHat 7.3 or later, Mandrake 9.0 or later (tested), should work with any system that uses xinetd and chkconfig. Probably will NOT work with Debian Woody or other inetd-based distros. HABITAT: /root/bin (Do NOT put this in /usr/local/bin. It would probably work, but the last thing you want is this program falling into the wrong hands. It turns off potentially critical services, after all! TYPE: Bash script. NOTE: In the fervent belief of economy and stability, this program only turns OFF services. It does NOT turn any on. Ever. This is a safety/stability feature. PLANNED IMPROVEMENTS: Probably convert this to Perl or C stand-alone. Maybe. Then, again, maybe not. It works pretty well as a script and is well commented/documented in the code. |
| openMosix LoadCpuLimit HowTo |
WHAT
IT IS: At Cristiano De Michele's
suggestion, I have revised the openMosix LoadCPU Limiter Howto to
provide additional detail and instruction, particularly on how to use
the various settings offered by the new loadcpulimit feature included
with recent versions of openMosix. As I am not yet familiar with the load limiter, please review the file for correctness and please let me know where I have erred! Thanks! REQUIRES: openMosix. HABITAT: None TYPE: Text-file howto. PLANNED IMPROVEMENTS: Should be revised to correct any errors. Pending peer review.... (It's up to you gang. Let me know where I goofed....) |
| KludgeKlump:
The Tarball |
Don't feel like
downloading each
utility? Get 'em all right here, including index.html (This
page.) Includes the three RPMs listed below. |
| Some
Useful Packages acl-2.2.3-3.i386.rpm bladeenc-0.94.2-4.i686.rpm star-1.4.1-1.i686.rpm openmosix_0.3.4-1_i386.deb BEnc-0942-LinuxSid-i386.deb NOTE: The BEnc-0942-LinuxSid-i386.deb file is actually made from an RPM using alien. (It does work--I've used it without problems.) |
Here are some useful RPMS and .debs for use with
current
and upcoming Kludge Kollection utilities. Bladeenc is required if
you wish to use pararip to make MP3 files, while star is a backup
utility similar to TAR but faster and more feature-rich. Star
requires the acl rpm. All three rpms will be available through
the
upcoming KludgeKollection installer. NOTE: There may be a dependency issue with acl-2.2.3-3.i386.rpm, so you may not wish to try to install star-1.4.1-1.i686.rpm (unless you want to try resolving the dependencies yourself.) |
| Coming Soon... |
A utility to set /proc/self/lock to 0 and possibly to edit
/etc/profile to contain the "echo 0 > /proc/self/lock". |
| Comments? Complaints? Submissions |
Comments? Complaints? Suggestions? Want to
submit your own program to the Kludge Kollection? Just click the
link to the left.... |
| |
|