Palantir - a video/audio/data interactive streaming server
palantir [ -d DEV ] [ -c CH ] [ -s NxM ] [ -j QUALITY ] [ -g ] [ -f[FILE] ] [ -m ] [ -x ] [ -C FILE] [ -q ] [ -p PORT ] [ -S DEV ] [ -t TIME ] [ -T TIME ] [ -F[FILE] ] [ -P TIME ]
palantir is a streaming server featuring live video, audio and data channels.
The video channel encodes the input of a video4linux device in a sequence of JPEG frames and transmits it as a multipart/x-mixed-replace HTTP stream, which some browsers are able to display natively.
The audio channel is compressed according to the bandwidth-sparing GSM 06.10 standard. Audio communication is bidirectional (clients can talk to the server); only one audio connection at a time is allowed to the server.
Another bidirectional channel is available for user-defined data, which can be used to remotely control devices connected to the server (for example some pan/tilt/zoom hardware) as well as to transmit environmental information to the clients. Again, only one client at a time can send data to the server, but unlike audio, data from the server are available to all connected clients simultaneously. (For a detailed discussion of data input/output see palantir.conf(5).)
Control (i.e. ownership of the bidirectional audio channel and the data transmission channel) is granted on a FIFO basis to clients requesting it; the control period, after which a client is preempted if other requests are pending, has a configurable time length.
Fields of application for palantir range from surveillance to entertainment, industrial process control and site monitoring.
palantir comes with two client applications - a Java applet and a stand-alone Windows client (see CLIENTS below). Both can display the live video stream and interact with the server through the data channel. The Windows client can also handle audio communication.
The most basic interface to the video stream is a web browser supporting the multipart/x-mixed-replace HTTP format. The list of such browsers currently includes Netscape, Mozilla and its derivatives, and (on the Macintosh platform only) Internet Explorer. The live stream can be embedded in a HTML page just like any other static image, with something along the line of <IMG SRC=``http://hostname:3000''>. (Be sure to include the TCP port in the URL - see option -p above, or the TCPPort configuration file directive.)
Keep in mind, however, that this arrangement will likely be prone to latency problems on slow clients: since there is no way for the browser to acknowledge received frames, the server just keeps pushing frames as soon as they are available at the capture device; if the client (due to hardware or connectivity limitations) cannot sustain the server's rate, undelivered frames start accumulating in the server's TCP transmit queue until it gets saturated. From that point on the server is forced to slow its pace, but the queued frames account for an unrecoverable lag between server and client whose extent can be estimated as {TCP transmit queue size in bytes (usually 64K)} / {average frame size in bytes}.
If a connection to palantir is made from a web browser unable to correctly display the stream as an embedded image (e.g. Internet Explorer on the Windows platform), a single frame is delivered and the connection is immediately closed. The same result can be achieved on all browsers by appending ``?mode=single'' to the server URL, as in <IMG SRC=``http://hostname:3000/?mode=single''>.
The two client applications supplied with palantir acknowledge every frame received from the server, which in turn waits for this confirmation before sending the next frame, in order to keep latency to a minimum. Connections involving these clients are tagged as ``closed loop'' in the server log.
The Windows client requires no installation; users should be able to download it from the site running palantir and run it at once (possibly after unzipping). It can exploit all the cool features of palantir, including bidirectional audio. The Windows client also allows the video and audio channels to be separately turned on and off, in order to make efficient use of low-bandwith links.
The Java applet client is somewhat more limited, as it has no audio capabilities, but since it can effectively run on a wide range of Java-enabled browsers on different platforms it has a much larger potential user base.
The preferred way of deploying these clients to the final user is to have them served by the same host running palantir. (Due to a well-known limitation of Java applets, this is also the only way of having the Java client work.)
The status of the devices connected to the palantir server can be accessed through a web browser by appending the special string ``?mode=devices'' to the server URL, as in ``http://hostname:3000/?mode=devices''.
Using a reasonable frame size and quality, a decent frame rate (>1 f/s) can be obtained even on slow analog modem connections, without sacrificing the audio stream.
On a LAN, the maximum achievable frame rate essentially depends on the ability of both server and clients to efficiently compress or uncompress the JPEG frames, and ultimately on the capture hardware speed.
If palantir appears unreachable from outside your LAN, please double-check your firewall configuration. palantir usually listens to ports 3000 and 3001; these can be changed with the -p option or the TCPPort configuration file directive..
David Santinoli <david at santinoli.com>.