SWAMP -- the Script Workflow Analysis for MultiProcessing system
First version, 2006
Author: Daniel L. Wang, wangd@uci.edu
--------------------------------------------------

System requirements:
-- NCO, the netCDF Operators http://nco.sourceforge.net  
   SWAMP has been tested with NCO 3.
  

Install steps:
-- Configure OPeNDAP to use SWAMP as its netCDF (.nc) handler.
Edit dap-server.rc, located in the same directory as OPeNDAP's
nph-dods.  Probably at /var/www/cgi-bin/dap-server.rc 
  Look for existing "handler ... dap_nc_handler line"
  Note the existing path to dap_nc_handler
  Replace with path to dap_nc_swamp

-- Configure SWAMP in swamp_handler.conf --> should be placed in the
   same place as dap_nc_swamp 
  
Setting up the ramdisk:
Using a ramdisk is highly recommended.  Two aspects of SWAMP
performance are highly affected by the availability of a ramdisk:
SQLite db access and temporary file storage.  Linux kernels began
including a tmpfs ramdisk at /dev/shm by default early in the 2.4
kernel series.  /dev/shm is typically sized as half the available
physical RAM.  SWAMP writes its SQLite db file in /dev/shm in its
default configuration.  Therefore, runtime SWAMP state (running
scripts, task progress) does not survive reboots by default.

Using a ramdisk as temporary (intermediate) file storage greatly
reduces disk contention in parallel jobs.  You can point SWAMP at
/dev/shm as its scratch space if you don't mind cluttering up that
directory.  Otherwise, you can:
a) make a subdirectory under /dev/shm for scratchspace
b) make a mount point for another tmpfs filesystem, and add a line in
/etc/fstab to mount a bigger (or smaller) ramdisk.

Example /etc/fstab line:
none    /var/www/swamptmp    tmpfs    mode=1777,size=15G   0   2

"none" is a device identifier (e.g. /dev/hda1 for disk mounts), which
is ignored by the tmpfs module

"/var/www/swamptmp" is the mount point for the filesystem

"tmpfs" specifies that we are using the tmpfs filesystem (and not
"ext3" or "xfs" as in the case of disk-based filesystems)

"mode=1777,size=15G"  means to mount with mode 1777 and a maximum size
of 15G (always < physical RAM, our system has 16G physical, so we
limit the size to help ensure that the kernel and running NCO subtasks
have enough memory to operate without swapping to disk.  Currently,
SWAMP does not limit its scratch file usage, aside from deleting
intermediate files once they have been fully consumed.

0 -- do not dump
2 -- mount on pass 2.  We should mount it after its mount point is
mounted (probably the / filesystem)


-------
Common Problems:
Is SWAMP finding the config file properly?
Is SWAMP allowed to write to the scratch path and the output path?
Is SWAMP allowed to write to the database?
Is the database consistent?  You can refresh the database with the db tool.
