From kaiyew_lum at nus.edu.sg Wed Feb 5 11:51:01 2003 From: kaiyew_lum at nus.edu.sg (Lum Kai Yew) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Scheduling under Scyld Beowulf Message-ID: <88F7031A498BD84185E72A4763109ABA55D5AC@MBXSRV06.stf.nus.edu.sg> Hi. I've just ordered the basic CD from Linux Central for evaluation. My lab currently has a mini cluster of 4 nodes (dual CPU), but we are seriously think about expanding it to at least 64 nodes. My question is: does Scyld-Beowulf have a job scheduling feature, or can I run OpenPBS under Scyld-Beowulf -- and how? Thanks a lot! Kai-Yew Lum Temasek Laboratories Singapore From becker at scyld.com Wed Feb 5 15:51:01 2003 From: becker at scyld.com (Donald Becker) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Scheduling under Scyld Beowulf In-Reply-To: <88F7031A498BD84185E72A4763109ABA55D5AC@MBXSRV06.stf.nus.edu.sg> Message-ID: On Wed, 5 Feb 2003, Lum Kai Yew wrote: > Hi. I've just ordered the basic CD from Linux Central for evaluation. > My lab currently has a mini cluster of 4 nodes (dual CPU), but we are > seriously think about expanding it to at least 64 nodes. > > My question is: does Scyld-Beowulf have a job scheduling feature, or > can I run OpenPBS under Scyld-Beowulf -- and how? Yes, and yes. The Basic Edition releases do not include a scheduler. The professional editions includes several queuing and job assignment tools. You can use most schedulers with Scyld Beowulf. There is a version of PBS Pro available, and installations have used OpenPBS, LSF and SGE. In most cases it's best to treat the cluster as a single machine, and have the scheduling daemon run only on the master. The scheduling/queuing tools included in the professional edition are oriented to simplicity of use on a single cluster that is dedicated to a small number of users and applications. Most commercial schedulers use sophisticated rules of who and what type of programs can use multiple systems across a site, necessarily leading to much more complexity in configuration and use. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From becker at scyld.com Fri Feb 14 11:59:01 2003 From: becker at scyld.com (Donald Becker) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] RE: Creating user accounts.... In-Reply-To: <000c01c2d3b3$c56fcb40$f8b25541@nh.mpisofttech.com> Message-ID: On Thu, 13 Feb 2003, Srihari Angaluri wrote: > Is there any serious performance/scalability issue to using NIS, Yes. A difference between a clusters and a collection of workstations is that a cluster-wide job will cause all nodes to be active at once. Rather than getting scattered name service requests, all requests will arrive at once. More than about 25 clients will cause a NIS server to drop requests, and even in the best case you end up with a serialization point and slow-downs. The current practice is changing the name system, both implementation and administrative model, to match each installion. - People use NIS when there is a rapidly changing user base - explicitly copy files when there is a small user and node set, or - synchronize, large mostly-static files with 'rsync'. The file approach is typically implemented with ad hoc scripts, which work well for the people that wrote the scripts but make it difficult to trace an error to the change that caused the error. We developed BeoNSS, a cluster nameservice, to address the administration and scaling problems. The advantage of our BeoNSS system is that it - Scales very well - Uses an unchanged administrative model - Allows a single point of administration (add users on just the master), - May be extended to support multiple administrative domains (each master sharing a cluster might have their own user list). > as > opposed to copying the individual files to each and every node on the > cluster? Is this even a desirable option for large clusters, for > example? What if I need to add more accounts? You missed a big one: what about machines that are down or non-responsive (!) when you update? If you are willing to deal with more complexity and administration, copying files to cluster nodes with full installs will result in good scaling, but... then you have administration issues. We designed our architecture on the following principle: compute nodes exist to perform computations for a master. Using BeoNSS, compute nodes only need know about the users that are actively running jobs. They don't keep any persistant configuration that might be outdated. Only the master needs the full user list. Look at this from end-goal perspective: there shouldn't be an administrative change or a compute node performance difference between the master having 10 users, having 10,000 users, or supporting arbitrary job submission from an even larger user base. We still have to try to keep UIDs less that 32767 but that's a compatibility issue with old-style 16 bit UIDs, not a design limit. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From linuxgro at yahoo.com.mx Sat Feb 15 17:14:00 2003 From: linuxgro at yahoo.com.mx (linuxgro@yahoo.com.mx) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Demo cd Message-ID: <20030215205645.GA1240@dapricom.creacionesdeangeles.com> Hi! Im a Mexican computer science student and I'm taking my course of distributed systems, I told my teacher about Beowulf clusters and we are very interested about them, the only problem is that we have not any software, and I know Scyld is the best and easier choice for a beowulf... So I want to know how can I get a Scyld Demo CD, We can buy one from you, but It would be easier for us if we could download an iso image. Well, thanks for your answers and have a nice weekend. David Flores. From paliwal at jhu.edu Tue Feb 18 09:17:16 2003 From: paliwal at jhu.edu (Amit Paliwal) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] fortran compilers Message-ID: <3E516DEC.4F8B7533@jhu.edu> Hi All, I am a beowulf newbie . We have a 16 node cluster running on intel P4s . Past 15 days i have been looking for some advice as to which commercial fortran 90 compiler would work best on our cluster(which is compatible with scyld). we purchased PGI CDk just to find that it is not compatible with scyld . Do i need license for just front end node to compile ( i.e master node )or for all the nodes. please advise... thanks, Amit. From becker at scyld.com Wed Feb 19 11:34:01 2003 From: becker at scyld.com (Donald Becker) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] RE: Creating user accounts.... In-Reply-To: Message-ID: On Wed, 19 Feb 2003, RANGI, JAI wrote: > Thanks for all of you for giving me the ideas for setting up a script for > user accounts managements... > Here is what I did and at this point of time it is working just fine ( Might > not be very fancy for some of you) for our cluster...... > > passwd $1 > matchusers.sh ... > rcp /etc/group geonode1:/etc/group > rcp /etc/passwd geonode1:/etc/passwd While this will likely function with careful use, it is not robust. Consider Simultaneous updates to Down nodes With simultaneous 'rcp' you run the slim but non-zero chance that you will clear or corrupt /etc/passwd. The problem with the many script-based cluster "management" software systems is that they will almost always work for the people that wrote them. Without realizing it, they avoid exercising the fragile parts of the system. But pass it on to someone else, and the first thing that happens is "everyone update your password now". > I did some reading on LDAP that also looks a nice options for accounts > managements. Seems like none have done worked on it, I am Still hunting for > some time to work on LDAP, I will try to post some more info on that soon... LDAP is "just another directory service". That's good, because the concepts from other directory services transfer. It's throughly out of favor with Unix shops, which is bad because you will need help at some point. LDP is cleaner and more efficient in some areas, but the semantics are not what you would expect for a system that should have had the benefit of earlier experience. -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993 From JAI_RANGI at SDSTATE.EDU Wed Feb 19 12:38:23 2003 From: JAI_RANGI at SDSTATE.EDU (RANGI, JAI) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] RE: Creating user accounts.... Message-ID: Thanks for all of you for giving me the ideas for setting up a script for user accounts managements... Here is what I did and at this point of time it is working just fine ( Might not be very fancy for some of you) for our cluster...... I created a script for password change like this passwd $1 matchusers.sh This will prompt for the new password and will update all the nodes.... matchusers.sh script looks like this rcp /etc/group geonode1:/etc/group rcp /etc/passwd geonode1:/etc/passwd rcp /etc/shadow geonode1:/etc/shadow rcp /etc/group geonode2:/etc/group rcp /etc/passwd geonode2:/etc/passwd rcp /etc/shadow geonode2:/etc/shadow rcp /etc/group geonode3:/etc/group rcp /etc/passwd geonode3:/etc/passwd rcp /etc/shadow geonode3:/etc/shadow . . . for all the nodes in cluster, Now if the password is changed for any user it being automatically updated on all the nodes. I did some reading on LDAP that also looks a nice options for accounts managements. Seems like none have done worked on it, I am Still hunting for some time to work on LDAP, I will try to post some more info on that soon... Jai Rangi ------------------------------------------------------- In the world with no fences, why would you need Gates ? - Linux ------------------------------------------------------- From jeffrey.b.layton at lmco.com Wed Feb 19 18:10:01 2003 From: jeffrey.b.layton at lmco.com (Jeff Layton) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Re: Creating user accounts.... References: Message-ID: <3E53D1E6.C28D9A06@lmco.com> > > I did some reading on LDAP that also looks a nice options for accounts > > managements. Seems like none have done worked on it, I am Still hunting for > > some time to work on LDAP, I will try to post some more info on that soon... > > LDAP is "just another directory service". That's good, because the > concepts from other directory services transfer. It's throughly out of > favor with Unix shops, which is bad because you will need help at some > point. > LDP is cleaner and more efficient in some areas, but the semantics are not > what you would expect for a system that should have had the benefit of > earlier experience. What are the pluses and minuses of LDAP vs. NIS for directory services? You can even through LDP into that mix if you like. What I'm looking for is some comparison of NIS vs. LDAP vs. files for manageability, scalability, and impact on network performance. Thanks! Jeff -- Jeff Layton Senior Engineer Lockheed-Martin Aeronautical Company - Marietta Aerodynamics & CFD "Is it possible to overclock a cattle prod?" - Irv Mullins This email may contain confidential information. If you have received this email in error, please delete it immediately, and inform me of the mistake by return email. Any form of reproduction, or further dissemination of this email is strictly prohibited. Also, please note that opinions expressed in this email are those of the author, and are not necessarily those of the Lockheed-Martin Corporation. From Jack.Cheng at motorola.com Thu Feb 20 12:46:01 2003 From: Jack.Cheng at motorola.com (Cheng Jack-QA3581) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Directing output files to the slave nodes Message-ID: <49B18DE31018D5118FBA009027B0FE9907772DB8@IL27EXM07.cig.mot.com> Hi, I inherited a Scyld Beowulf cluster recently and have a question on how to direct output files to specific slave nodes when executing a job from the master node. For example, for the 20 slave nodes in my Beowulf cluster, say that there are 20 users. User 1 wants to run an application and store the output files on slave node #1, user 2 would like to do the same but store the files on slave node #2, and so on. Each user would like to run his/her application program from the master and have the process run on each node in the cluster (i.e. use the command "bprun -NX"). My understanding is that "bprun -NX" will process the job on each node and then place the output on the master node. What would be the best way for the users to direct output files to specific slave nodes? Thanks for the help. - Jack From timo.lassmann at cgb.ki.se Fri Feb 28 05:00:00 2003 From: timo.lassmann at cgb.ki.se (Timo Lassmann) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Database searches on Scyld Message-ID: <1046426354.1082.15.camel@caga> Hi everyone, I am working in a bioinformatics group in Stockholm. We had a beowulf cluster for a long time and are now thinking of switching to the new scyld beowulf version. We mainly use the cluster for database searches and in our old setup we had to have local database copies installed on all nodes for extra speed. Since in beowulf 2 everything is only installed on the master node, would our parallel searches run as fast? And if it will be as fast how difficult is it to install pvm based applications over to the new setup? Thank you very much in advance, Timo Lassmann From becker at scyld.com Fri Feb 28 06:47:01 2003 From: becker at scyld.com (Donald Becker) Date: Tue Nov 9 01:14:27 2010 Subject: [scyld-users] Database searches on Scyld In-Reply-To: <1046426354.1082.15.camel@caga> Message-ID: On 28 Feb 2003, Timo Lassmann wrote: > I am working in a bioinformatics group in Stockholm. We had a beowulf > cluster for a long time and are now thinking of switching to the new > scyld beowulf version. We mainly use the cluster for database searches > and in our old setup we had to have local database copies installed on > all nodes for extra speed. Since in beowulf 2 everything is only > installed on the master node, would our parallel searches run as fast? An advantage of the Scyld Beowulf system is that there is a broad range of file system configurations. The demo and turn-key CDs are set up at one end of the range, with the master operating only from the CD-ROM and compute nodes not mounting any network or local disk file systems. The initial installation of the full system has the master using a file system on the local disk, with the compute nodes booting without a filesystem. After initial installation almost all sites configure the compute nodes to use local disks for swap space and NFS mount /home. Most sites configure local disk partitions for data file storage. Some sites configure the root partition on local disk, and NFS mount or copy over all of the standard binaries so that the system looks like a traditional Beowulf FS. Other sites configure local storage as a parallel file system with PVFS or GFS. With all of these configurations there is the advantage of the "diskless administration" model. The computes nodes always boot into the safe no-file-system mode. They are configured by a master machine, using only configuration files on that master. When local file systems are enabled on the compute node, the master configuration has the option to Always make a fresh file system with 'mkfs' Make a fresh file system if the existing one fails 'fsck' Leave the node operational but in 'Error' state if any 'fsck' fails. > And if it will be as fast how difficult is it to install pvm based > applications over to the new setup? Easier than a traditional PVM installation: the cluster system knows the list of available nodes the job initiation semantics are cleaner and faster there is not the usual 'rsh'/'ssh' permission problem -- Donald Becker becker@scyld.com Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Scyld Beowulf cluster system Annapolis MD 21403 410-990-9993