[bproc]bpsh and pipes

[Jag]

--0rSojgWGcpz+ezC3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, 24 Apr 2001, Dan Smith wrote:

>=20
> I have two questions.  What does bpsh -n really do?  And how do you pipe
> output from one program to another program while using bpsh? For instance,
>=20
> bpsh 1 foo | bar
>=20
> where "foo" is a program that produces large amounts of data and writes t=
hem
> to stdout and "bar" is a program that processes that data and receives it
> from stdin.  Using the above example as is seems to pump everything to the
> master node and then back again creating lots of network traffic.  What is
> the correct way to do this?  Thanks.

Your problem is that the pipe is being processed on the master node.
Your shell is saying I need to run 'bpsh 1 foo' (which will send its
stdout back over the network), then pipe the output to the locally
running program 'bar'

I know of no 'good' way to do what you want on a Scyld cluster.  My
first recomendation would be to make it all one program.   My second
would me to make sure foo and bar can be execusted on the remote node
(ie, binaries exported over nfs), then create a file that simply reads
'foo | bar' then do 'bpsh 1 bash file'  Bash will start, be moved to
node 1, there it will read the file you created, then execute foo and
bar, doing the piping the way you want.


Jag

--0rSojgWGcpz+ezC3
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE66Na7+pq97aGGtXARAqH3AKCZ2KQuO7FHom0acHubw20Te4EFBwCbBcpc
MxmRixjxi1wFmt8EylvG3I0=
=hEc9
-----END PGP SIGNATURE-----

--0rSojgWGcpz+ezC3--