general

news
downloads

projects

streamline
beltway buffers
pipesfs
model-t

documentation

introduction
papers
slides
videos
examples
dev. manual
browse code

mailing list

join / leave
archive

development

svn download
svn log
browse svn
people

vrije universiteit amsterdam logo
Streamline is a Vrije Universiteit Amsterdam research project.

opensource logo
All code is made available under a mixed GNU Lesser General Public (LGPL) and Simplified BSD license.

:: examples

Below are a handful of toy example applications that work with Streamline 1.6. Download the demo application to give these a try.

With version 1.8 the language changed. Use the expressions listed in the grey box, instead.

Tcpdump 1

Insert the following request (by issuing insert <request>) to make Streamline parse a tcpdump tracefile, run it through BPF to only select port 80 traffic and then print out some details on screen. we use a tracefile instead of live data because the demo application doesn't allow in-kernel processing. Full Streamline also gives you the (ethernet) function for live input.

(pcapin,expression="tracefile") > (bpf,expression="src port 80") > (inspect)
pcapin tracefile | bpf "src port 80" | inspect

Don't know what a tcpdump tracefile is? It is a file containing a network stream as captured by the application tcpdump using the option -w (or more generally any application based on libpcap, really).

Tcpdump 2

The same can be achieved without going through BPF as follows: again parse the file, but use Streamline's classification support to filter out port 80 traffic. Each function generates a 16bit classifier. Dependent functions can selectively run based on this classifier.

(pcapin,expression="tracefile") > (tcp) > (sourceport) -80-> (inspect)
pcapin tracefile | tcp | sourceport |80 inspect

Layer 7 filtering

Traditionally, application level ('layer 7') filtering has been difficult due to the strict socket(..) API that sits below the application. The following Streamline example shows how a webserver can be secured against sophisticated attacks. We preprocess the IP packets as before, but now reconstruct TCP streams. These are filtered through an AhoCorasick matching engine to through out known attacks. Thereafter we run a protocol-specific filter (prospector) and then generate per-flow output. Live output is work in progress, for the time being data is sent to diskfiles, one file per flow.

(pcapin,expression="tracefile") > (tcp) > (destport) -80-> (tcpreassemble) > (ac_file,"patternfile") > (ips_prospector) > (outfiles,expression="safehttp")
pcapin tracefile | tcp | destport |80 untcp | ac_file patternfile | filesout "safehttp"
Or try this shorter version to extract HTTP traffic:
pcapin tracefile | tcp | http | filesout "safehttp"

NB: the prospector filter has been removed in 1.7

Transcoding

We now look at a higher level example. Streamline can also take input from a socket and similarly output over a socket. In the following snippet Streamline connects to a running TCP server at port 5051 and opens up a server itself at port 5050. Input received on port 5050 is transcoded by the rot13 function and then sent out to the server at port 5051. To run this test start a simple server (I suggest `netcat -l -p 5051`), insert the request, and then open a simple client (e.g., `netcat localhost 5050`) on the commandline. Anything you type in the client will be shown in the serverwindow encoded.

(ipv4_receive,expression="t 127.0.0.1 5050") > (rot13) > (ipv4_transmit,expression="t 127.0.0.1 5051")
tcpsock 5050 | rot13 | tcpsock "127.0.0.1:5051"

For this example you could do the same with just netcat and UNIX pipes. Indeed, Streamline's syntax is reminiscent of pipes. It extends these with full boolean support, however, and allows crossing over into kernelspace and further away.

Playing Around

You can further experiment with the application by connecting your own graphs of requests. Accepted operands are '>' for dependencies, '|' for parallel processing (a OR b), '&' is similar (a AND b) and '!>' will only forward rejects. Use help for an overview of actions. Most interesting are probably the shown insert and list instances to see the status of the active function instances.

Are you a student interested in systems research? Streamline is a Vrije Universiteit Amsterdam research project. We're always looking for exceptional candidates for our Master's program.