Network Your Shell Scripts with Netpipes
Pages: 1, 2

A Single-Paired Transaction

The simplest interaction is a single-paired transaction. These two invocations are run separately in two difference consoles.

The first one to invoke is this one, which sets up a server waiting for a connection. The first one is:

faucet 2000 --in --out sh -c "cat <&0>from_client.txt;ps aux"

Two commands run on the server side. The first one, cat<&0>from_client.txt, takes the output from the contacting client connection as standard input and directs it into a text file called from_client.txt. The second command, ps aux, directs its output to the client stream. This output is sent only after the standard input that was sent by the client side has completed -- after the client has closed the socket connection.

This is the second process. It starts on the client side:

hose localhost 2000 --out --in sh -c "(ps aux;sockdown 1 1); \
	grep postgres > from_server.txt"

This command is similar to the server invocation in that there are two sets of commands. The first set is ps aux; sockdown 1 1. Since the --out switch was the first switch used, the ps aux sends its standard output over the TCP connection. The sockdown 1 1 closes the socket connection. You can run several commands with different output, as in cmd1; cmd2; cmd3...; sockdown 1 1, but the sockdown command must always come last.

The second client-side command, grep postgres > from_server.txt, takes the output that the server sent from its ps aux, filtering out all mentions of any process that has the word postgres in them, and saves those results to the file from_server.txt.

This example (see the screenshot) works because the client side has performed a shutdown system call on the client's standard output file descriptor, making the socket read-only. Adding another paired transaction of input and output to the commands will fail.

A Multi-Paired Transaction

The single-paired transaction of the last example succeeds in making one paired transaction by closing the write socket on one end; this unfortunately means that you can't have more than one pair of back-and-forth communications. netpipes has another function that permits multi-paired datastreams to create a running dialog between the two socket connections of client and server. This two-way communication uses the Session Control Protocol (SCP). SCP is a standard proposed by Sun Systems a few years back that enables several transactions to run concurrently. netpipes implements SCP by spawning a separate process for each transaction pair. It then uses a utility called encapsulate that attaches them to specific file descriptors.

This screenshot shows a simple question-and-answer session between the server and client:

Figure 1. A client/server Q&A session.

Here are the server and client scripts:

#!/bin/bash
# This is the server side script, server.sh
# define the question array
Q=("Hello, who are you?"\
"What is your last name?"\
"What city do you live in?")

# carry out the interrogation
for i in 0 1 2
do
encapsulate --fd 3 -so5i4 sh -c \
	"echo ${Q[i]}>&5;exec 5>&-;cat<&4"
done

#!/bin/bash
# This is the client side script, client.sh
#define the answer array
A=("My name is Robert"\
"My last name is Bernier"\
"I live in Ottawa")

# give the answers
for i in 0 1 2
do
encapsulate --fd 3 -si4o5 sh -c "cat<&4;echo ${A[i]}>&5"
done

The two scripts use three file descriptors; file descriptor 4 serves as input, file descriptor 5 is the output, and file descriptor 3 multiplexes the sub-process channels of file descriptors 4 and 5. The multiplexor, file descriptor 3, actually carries the other two file descriptors.

The command exec 5>&- closes file descriptor 5, thus making it possible for the socket to receive standard input from file descriptor 4. By the way, the client and server descriptors don't have to use the same numbers, but it is less confusing that way.

Notice how the switches -si4o5 and -so5i4 have a reversed order between client and server. You must follow this convention or the connection will fail!

Streaming Multimedia Files

Here's an interesting way to stream MP3s, MOVs, and AVIs. All you need to do is to set up the server as such:

faucet 2000 --out sh -c "cat War\ -\ Low\ Rider.mp3" -daemon

and run your client this way:

hose localhost 2000 --in sh -c "cat<&0 | mplayer - -cache 8192"

What's really cool is that you can run as many clients as you want at the same time -- each stream is independent! With a little scripting, you can make even make an interactive session using SCP where the client can choose the multimedia file to play.

Caveats

Running netpipes multiplexed connections is not the most responsive mechanism when it comes to socket connections.

You'll need to understand redirection, file descriptors, and how they work in bash (or your preferred shell) in order to take advantage of the full power of netpipes. bash may not have the most intuitive scripting language, but it certainly is among the most powerful.

Debugging a netpipes script can be an adventure into the unknown, because there are no error messages explaining why a connection fails. Take your time and start by writing small scripts.

Pick one way of doing things. Be consistent with file descriptor naming conventions between the client and server. Use strace and ps to see what the utilities are doing. Reading the netpipes man pages lends meaning to the saying that there's more than one way to skin a cat.

Of course, watch out when reading the man pages. They are confusing and could stand a bit of cleanup.

Want a little more security? The original netpipes package included a utility called ssl-auth. As far as I can tell, this utility no longer exists, but you can no doubt replicate it using something like stunnel.

You don't need to use bash. Perl or any other language that uses file descriptors will work fine, too.

Ideas

What can you do with this utility? One activity that comes to mind is to run a menu-driven utility: one side sends over a text file with choices, and the other side carries out actions based on the choices taken. Wrap this in stunnel to make for a very efficient and secure data exchange system -- perfect for those non-root accounts that need to set up socket connections on the fly and don't have the traditional services available for them. This is a great mechanism for lightweight utilities. You could even incorporate them into emergency boot distributions.

References

• The man pages are indispensable: netpipes, faucet, hose, sockdown, encapsulate.

• bash scripting and redirection

• SCP, the Session Control Protocol

Robert Bernier is the PostgreSQL business intelligence analyst for SRA America, a subsidiary of Software Research America (SRA).

---

Return to ONLamp.com.