Saturday, September 23, 2006

Throughput Testing Through a Bridge

In my earlier posts I've discussed throughput testing. Now I'm going to introduce an inline system as a bridge. You could imagine that this system might be a firewall, or run Snort in inline mode. For cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 purposes of this post, however, we're just going to see what effect cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge has on throughput between a client and server.

This is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 new system. It's called cel600, and it's running cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 same GENERIC.POLLING kernel mentioned earlier.

FreeBSD 6.1-RELEASE-p6 #0: Sun Sep 17 17:09:24 EDT 2006
root@kbld.taosecurity.com:/usr/obj/usr/src/sys/GENERIC.POLLING
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel Celeron (598.19-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x686 Stepping = 6
Features=0x383f9ff<>
T,PSE36,MMX,FXSR,SSE>
real memory = 401260544 (382 MB)
avail memory = 383201280 (365 MB)

This system has two dual NICs in it. em0 and em1 are Gigabit fiber, and em2 and em3 are Gigabit copper.

cel600:/root# ifconfig em0
em0: flags=8943 mtu 1500
options=48
inet6 fe80::204:23ff:feb1:7f22%em0 prefixlen 64 scopeid 0x1
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r 00:04:23:b1:7f:22
media: Ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rnet autoselect (1000baseSX )
status: active
cel600:/root# ifconfig em1
em1: flags=8943 mtu 1500
options=48
inet6 fe80::204:23ff:feb1:7f23%em1 prefixlen 64 scopeid 0x2
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r 00:04:23:b1:7f:23
media: Ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rnet autoselect (1000baseSX )
status: active
cel600:/root# ifconfig em2
em2: flags=8943 mtu 1500
options=48
inet6 fe80::204:23ff:fec5:4e80%em2 prefixlen 64 scopeid 0x3
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r 00:04:23:c5:4e:80
media: Ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rnet autoselect (1000baseTX )
status: active
cel600:/root# ifconfig em3
em3: flags=8943 mtu 1500
options=48
inet6 fe80::204:23ff:fec5:4e81%em3 prefixlen 64 scopeid 0x4
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r 00:04:23:c5:4e:81
media: Ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rnet autoselect (1000baseTX )
status: active

I configure cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365m in /etc/rc.conf this way:

ifconfig_em0="polling up"
ifconfig_em1="polling up"
ifconfig_em2="polling up"
ifconfig_em3="polling up"
cloned_interfaces="bridge0 bridge1"
ifconfig_bridge0="addm em0 addm em1 monitor up"
ifconfig_bridge1="addm em2 addm em3 monitor up"

The end result is two bridge interfaces.

bridge0: flags=48043 mtu 1500
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r ac:de:48:e5:e7:69
priority 32768 hellotime 2 fwddelay 15 maxage 20
member: em1 flags=3
member: em0 flags=3
cel600:/root# ifconfig bridge1
bridge1: flags=48043>UP,BROADCAST,RUNNING,MULTICAST,MONITOR> mtu 1500
ecá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r ac:de:48:0c:26:66
priority 32768 hellotime 2 fwddelay 15 maxage 20
member: em3 flags=3
member: em2 flags=3

Notice cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365se two pseudo-interfaces are both in MONITOR mode. That was set automatically.

With cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge in place, I can conduct throughput tests.

Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 client's view.

asa633:/root# iperf -c 172.16.6.2 -t 60 -i 5
------------------------------------------------------------
Client connecting to 172.16.6.2, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.6.1 port 57355 connected with 172.16.6.2 port 5001
[ 3] 0.0- 5.0 sec 55.9 MBytes 93.9 Mbits/sec
[ 3] 5.0-10.0 sec 51.6 MBytes 86.6 Mbits/sec
[ 3] 10.0-15.0 sec 72.3 MBytes 121 Mbits/sec
[ 3] 15.0-20.0 sec 54.6 MBytes 91.6 Mbits/sec
[ 3] 20.0-25.0 sec 61.4 MBytes 103 Mbits/sec
[ 3] 25.0-30.0 sec 75.4 MBytes 127 Mbits/sec
[ 3] 30.0-35.0 sec 60.2 MBytes 101 Mbits/sec
[ 3] 35.0-40.0 sec 47.8 MBytes 80.2 Mbits/sec
[ 3] 40.0-45.0 sec 74.7 MBytes 125 Mbits/sec
[ 3] 45.0-50.0 sec 59.0 MBytes 99.0 Mbits/sec
[ 3] 50.0-55.0 sec 54.0 MBytes 90.6 Mbits/sec
[ 3] 55.0-60.0 sec 76.8 MBytes 129 Mbits/sec
[ 3] 0.0-60.0 sec 744 MBytes 104 Mbits/sec

Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 server's view.

poweredge:/root# iperf -s -B 172.16.6.2
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 172.16.6.2
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 172.16.6.2 port 5001 connected with 172.16.6.1 port 57355
[ 4] 0.0-60.0 sec 744 MBytes 104 Mbits/sec

Compared to cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 straight-through tests, you can see cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 effect on throughput caused by cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge.

[ 4] 0.0-60.0 sec 1.19 GBytes 170 Mbits/sec

Of interest during cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 test is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 interrupt count on cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge.

last pid: 728; load averages: 0.00, 0.09, 0.06 up 0+00:06:36 17:58:40
22 processes: 1 running, 21 sleeping
CPU states: 0.4% user, 0.0% nice, 0.4% system, 17.1% interrupt, 82.1% idle
Mem: 7572K Active, 4776K Inact, 16M Wired, 8912K Buf, 339M Free
Swap: 768M Total, 768M Free

Let's try cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 UDP test. First, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 client view.

asa633:/root# iperf -c 172.16.6.2 -u -t 60 -i 5 -b 500M
------------------------------------------------------------
Client connecting to 172.16.6.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.6.1 port 51356 connected with 172.16.6.2 port 5001
[ 3] 0.0- 5.0 sec 169 MBytes 284 Mbits/sec
[ 3] 5.0-10.0 sec 169 MBytes 284 Mbits/sec
[ 3] 10.0-15.0 sec 171 MBytes 287 Mbits/sec
[ 3] 15.0-20.0 sec 171 MBytes 287 Mbits/sec
[ 3] 20.0-25.0 sec 171 MBytes 287 Mbits/sec
[ 3] 25.0-30.0 sec 171 MBytes 287 Mbits/sec
[ 3] 30.0-35.0 sec 171 MBytes 287 Mbits/sec
[ 3] 35.0-40.0 sec 172 MBytes 288 Mbits/sec
[ 3] 40.0-45.0 sec 172 MBytes 288 Mbits/sec
[ 3] 45.0-50.0 sec 172 MBytes 288 Mbits/sec
[ 3] 50.0-55.0 sec 172 MBytes 288 Mbits/sec
[ 3] 0.0-60.0 sec 2.00 GBytes 287 Mbits/sec
[ 3] Sent 1463703 datagrams
[ 3] Server Report:
[ 3] 0.0-60.0 sec 1.93 GBytes 276 Mbits/sec 0.014 ms 53386/1463702 (3.6%)
[ 3] 0.0-60.0 sec 1 datagrams received out-of-order

Now cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 server view.

poweredge:/root# iperf -s -u -B 172.16.6.2
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 172.16.6.2
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.6.2 port 5001 connected with 172.16.6.1 port 51356
[ 3] 0.0-60.0 sec 1.93 GBytes 276 Mbits/sec 0.014 ms 53386/1463702 (3.6%)
[ 3] 0.0-60.0 sec 1 datagrams received out-of-order

Here's cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 result from cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 straight-through test.

[ 3] 0.0-60.0 sec 1.94 GBytes 277 Mbits/sec 0.056 ms 62312/1478219 (4.2%)

The results are almost identical.

Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge's interrupt count as shown in a top excerpt.

last pid: 751; load averages: 0.00, 0.03, 0.04 up 0+00:10:20 18:02:24
22 processes: 1 running, 21 sleeping
CPU states: 0.0% user, 0.0% nice, 0.4% system, 19.8% interrupt, 79.8% idle
Mem: 7564K Active, 4788K Inact, 16M Wired, 8928K Buf, 339M Free
Swap: 768M Total, 768M Free

With cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 Gigabit fiber tests done, let's look at Gigabit copper.

First, a TCP test as seen by cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 client.

asa633:/root# iperf -c 172.16.7.2 -t 60 -i 5
------------------------------------------------------------
Client connecting to 172.16.7.2, TCP port 5001
TCP window size: 32.5 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.7.1 port 58824 connected with 172.16.7.2 port 5001
[ 3] 0.0- 5.0 sec 76.3 MBytes 128 Mbits/sec
[ 3] 5.0-10.0 sec 76.3 MBytes 128 Mbits/sec
[ 3] 10.0-15.0 sec 76.8 MBytes 129 Mbits/sec
[ 3] 15.0-20.0 sec 76.6 MBytes 129 Mbits/sec
[ 3] 20.0-25.0 sec 76.8 MBytes 129 Mbits/sec
[ 3] 25.0-30.0 sec 75.4 MBytes 127 Mbits/sec
[ 3] 30.0-35.0 sec 76.3 MBytes 128 Mbits/sec
[ 3] 35.0-40.0 sec 76.1 MBytes 128 Mbits/sec
[ 3] 40.0-45.0 sec 76.5 MBytes 128 Mbits/sec
[ 3] 45.0-50.0 sec 75.4 MBytes 126 Mbits/sec
[ 3] 50.0-55.0 sec 76.7 MBytes 129 Mbits/sec
[ 3] 55.0-60.0 sec 76.4 MBytes 128 Mbits/sec
[ 3] 0.0-60.0 sec 916 MBytes 128 Mbits/sec

Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 server's view.

poweredge:/root# iperf -s -B 172.16.7.2
------------------------------------------------------------
Server listening on TCP port 5001
Binding to local address 172.16.7.2
TCP window size: 64.0 KByte (default)
------------------------------------------------------------
[ 4] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 58824
[ 4] 0.0-60.0 sec 916 MBytes 128 Mbits/sec

That is better than cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 result for fiber from above.

[ 4] 0.0-60.0 sec 744 MBytes 104 Mbits/sec

It's not as good as cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 result for straight-through copper.

[ 4] 0.0-60.0 sec 1.16 GBytes 166 Mbits/sec

It seemed as though cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge interrupt count was lower than cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 fiber TCP tests.

last pid: 754; load averages: 0.00, 0.01, 0.02 up 0+00:13:48 18:05:52
22 processes: 1 running, 21 sleeping
CPU states: 0.0% user, 0.0% nice, 0.4% system, 16.7% interrupt, 82.9% idle
Mem: 7560K Active, 4792K Inact, 16M Wired, 8928K Buf, 339M Free
Swap: 768M Total, 768M Free

Finally, UDP copper tests. Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 client view.

asa633:/root# iperf -c 172.16.7.2 -u -t 60 -i 5 -b 500M
------------------------------------------------------------
Client connecting to 172.16.7.2, UDP port 5001
Sending 1470 byte datagrams
UDP buffer size: 9.00 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.7.1 port 62131 connected with 172.16.7.2 port 5001
[ 3] 0.0- 5.0 sec 129 MBytes 217 Mbits/sec
[ 3] 5.0-10.0 sec 129 MBytes 217 Mbits/sec
[ 3] 10.0-15.0 sec 129 MBytes 217 Mbits/sec
[ 3] 15.0-20.0 sec 129 MBytes 217 Mbits/sec
[ 3] 20.0-25.0 sec 129 MBytes 217 Mbits/sec
[ 3] 25.0-30.0 sec 129 MBytes 216 Mbits/sec
[ 3] 30.0-35.0 sec 129 MBytes 216 Mbits/sec
[ 3] 35.0-40.0 sec 129 MBytes 216 Mbits/sec
[ 3] 40.0-45.0 sec 129 MBytes 216 Mbits/sec
[ 3] 45.0-50.0 sec 129 MBytes 216 Mbits/sec
[ 3] 50.0-55.0 sec 129 MBytes 216 Mbits/sec
[ 3] 0.0-60.0 sec 1.51 GBytes 216 Mbits/sec
[ 3] Sent 1103828 datagrams
[ 3] Server Report:
[ 3] 0.0-60.0 sec 1.46 GBytes 209 Mbits/sec 0.047 ms 35057/1103827 (3.2%)
[ 3] 0.0-60.0 sec 1 datagrams received out-of-order

Here is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 server view.

poweredge:/root# iperf -s -u -B 172.16.7.2
------------------------------------------------------------
Server listening on UDP port 5001
Binding to local address 172.16.7.2
Receiving 1470 byte datagrams
UDP buffer size: 41.1 KByte (default)
------------------------------------------------------------
[ 3] local 172.16.7.2 port 5001 connected with 172.16.7.1 port 62131
[ 3] 0.0-60.0 sec 1.46 GBytes 209 Mbits/sec 0.047 ms 35057/1103827 (3.2%)
[ 3] 0.0-60.0 sec 1 datagrams received out-of-order

Let's compare that to cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 fiber UDP test from above.

[ 3] 0.0-60.0 sec 1.93 GBytes 276 Mbits/sec 0.014 ms 53386/1463702 (3.6%)

This time, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 results are much worse than cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 UDP over fiber results.

When I tested UDP over crossover copper, this was cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 result.

[ 3] 0.0-60.0 sec 1.86 GBytes 267 Mbits/sec 0.024 ms 40962/1401730 (2.9%)

The top excerpt is about cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 same as cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 fiber UDP test.

last pid: 754; load averages: 0.01, 0.01, 0.01 up 0+00:16:21 18:08:25
22 processes: 1 running, 21 sleeping
CPU states: 0.0% user, 0.0% nice, 0.0% system, 17.1% interrupt, 82.9% idle
Mem: 7564K Active, 4788K Inact, 16M Wired, 8928K Buf, 339M Free
Swap: 768M Total, 768M Free

It's not really feasible to make any solid assumptions based on cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365se tests. They're basically get to get a ballpark feel for cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 capabilities of a given architecture, but you need to repeat cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365m multiple times to get some confidence in cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 results.

If you want built-in repeatability and confidence testing, try Netperf.

With cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365se results, however, I have some idea of what I can expect from this particular hardware setup, namely a bridge between a client sending data to a server.

  • TCP over fiber: about 104 Mbps

  • UDP over fiber: about 276 Mbps

  • TCP over copper: about 128 Mbps

  • UDP over copper: about 209 Mbps


Rounding down, and acting conservatively, I would feel this setup could handle somewhere around 100 Mbps (aggregated) over fiber and around 125 Mbps over copper. Note this says nothing about any software running on cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 bridge and its ability to do whatever function it is designed to perform. This is just a throughput estimate.

In my next related posts I'll introduce bypass switches and see how cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365y influence this process.

I'll also rework cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 configuration into straight-through, bridged, and switched modes to test latency using ping.

3 comments:

Shirkdog said...

I tried to send traffic as fast as possible with straight through GIG copper from an OpenBSD box to a Linux box, cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 fastest I could send was 350Mb.

Just like you have mentioned, your hardware is cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 main bottle neck, and this was a horribly configured install on a Dual Xeon box. I am going
to try to setup a FreeBSD box and see if I can do cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 configuration (as was told to me), with a ramdrive and get gig speed :-)

Anonymous said...

I Richard.- I'm reading your book "The TAO of Network Security Monitoring", it's an excellent book, it open my mind about monitoring. Now I'm in chapter 9, I'll give you my comments when I finish all cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 book.
I have 2 question about this blog...
First, I use FreeBSD firewalls with IPFW(not in bridge). You recommend cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 use of "polling" on all cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 interfaces (inside, outside, DMZ)?
Second, it's good idea to add anocá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365r NIC to this firewall and use it to capture all data like you say in your book? I also run squid to cache content and some rules.

REGARDS

Richard Bejtlich said...

Hi Carlos,

I recommend using polling if you need cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 extra performance it provides. Ocá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365rwise cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365re's no need.

I don't recommend capturing data on a firewall. Use a separate sensor for monitoring duties. In an emergency, you might collect on cá cược thể thao bet365_cách nạp tiền vào bet365_ đăng ký bet365 firewall.