neuhalfen.name

A random collection of posts

Solaris Network Trouble in Xen: Throughput Degrades to Nill

Permalink

Update: The answer is here

Topology

My internal network is laid out with three bridges: brred, brorange and brgreen with nets 172.20.2/24, 172.20.1/24 and 172.20.0/24 respectively.

  • brred has two members: The ‘red’ interface of my endian[2] firewall and the ISP facing physical nic. The endian-domU connects to the ISP via PPPoE (with the PPP-packets forwarded over brred to the ISP).
  • brgreen / brorange are bridges for the internal net and the DMZ.
    Routing is done via the endian domU.

My OpenSolaris domU is attached to brgreen.

Tests

I tested the following combinations:

Downloading from a server on brorange and downloading from the Internet. My Solaris domU was tested attached to brgreen and attached to brorange.

  • brgreen downloading from brorange : good
  • brorange downloading from brorange : good
  • brgreen downloading from internet : bad
  • brorange downloading from internet : bad

Communicating with the Internet clearly is the problem here, though the question is why.

Sniffing

Some sniffing on the bridge where the nic is attached results in example [3] when throughput is good. Snipet [4] is a capture showing the degrading throughput.

Answers?

Here

References

  • 1 uname -a -> SunOS sol200811 5.11 snv_101b i86pc i386 i86xpv Solaris
    IP: 172.20.0.24
    Installed from the RC-2 CD.
  • 2 http://www.endian.com/en/community/
  • 3 good troughput (tcpdump, downloading from dlcl-vip1.netcologne.de)
    12:49:10.632597 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 786485:787897(1412) ack 124 win 46 12:49:10.633710 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 787897:789309(1412) ack 124 win 46 12:49:10.634297 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 789309:790721(1412) ack 124 win 46 12:49:10.635062 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 790721:792133(1412) ack 124 win 46 12:49:10.636011 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 792133:793545(1412) ack 124 win 46 12:49:10.637043 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 793545:794957(1412) ack 124 win 46 12:49:10.637990 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 794957:796369(1412) ack 124 win 46 12:49:10.638738 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 796369:797781(1412) ack 124 win 46 12:49:10.638825 IP 172.20.0.24.42223 > dlcl-vip1.netcologne.de.http: . ack 797781 win 48008 12:49:10.639511 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 797781:799193(1412) ack 124 win 46 12:49:10.640456 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 799193:800605(1412) ack 124 win 46 12:49:10.641208 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 800605:802017(1412) ack 124 win 46 12:49:10.642215 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 802017:803429(1412) ack 124 win 46 12:49:10.643401 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 803429:804841(1412) ack 124 win 46 12:49:10.643918 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 804841:806253(1412) ack 124 win 46 12:49:10.644883 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 806253:807665(1412) ack 124 win 46 12:49:10.645633 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.42223: . 807665:809077(1412) ack 124 win 46 12:49:10.645726 IP 172.20.0.24.42223 > dlcl-vip1.netcologne.de.http: . ack 809077 win 50832 ...
  • 4 Throughput degrading and finally dropping down to nill
    18:55:21.351112 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2233121:2234481(1360) ack 124 win 46 18:55:21.351142 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: P 2234481:2235841(1360) ack 124 win 46 18:55:21.351144 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2235841:2237201(1360) ack 124 win 46 18:55:21.351176 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2237201:2238561(1360) ack 124 win 46 18:55:21.351177 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2238561:2239921(1360) ack 124 win 46 18:55:21.351208 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2239921:2241281(1360) ack 124 win 46 18:55:21.351209 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2241281:2242641(1360) ack 124 win 46 18:55:21.351812 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2235841 win 50320 18:55:21.351813 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2238561 win 47600 18:55:21.351814 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2241281 win 44880 18:55:21.351866 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2246721:2248081(1360) ack 124 win 46 18:55:21.352227 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 1 {2246721:2248081}> 18:55:21.352332 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2252161:2253521(1360) ack 124 win 46 18:55:21.352526 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 2 {2252161:2253521}{2246721:2248081}> 18:55:21.353288 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2257601:2258961(1360) ack 124 win 46 18:55:21.353430 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 3 {2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.353750 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2260321:2261681(1360) ack 124 win 46 18:55:21.353892 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2261681}{2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.354481 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2261681:2263041(1360) ack 124 win 46 18:55:21.354622 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2263041}{2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.355201 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2263041:2264401(1360) ack 124 win 46 18:55:21.355342 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2264401}{2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.355959 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2264401:2265761(1360) ack 124 win 46 18:55:21.356100 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2265761}{2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.356700 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2265761:2267121(1360) ack 124 win 46 18:55:21.356840 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2267121}{2257601:2258961}{2252161:2253521}{2246721:2248081}> 18:55:21.357629 IP dlcl-vip1.netcologne.de.http > 172.20.0.24.37125: . 2267121:2268481(1360) ack 124 win 46 18:55:21.357770 IP 172.20.0.24.37125 > dlcl-vip1.netcologne.de.http: . ack 2242641 win 50320 <nop,nop,sack 4 {2260321:2268481}{2257601:2258961}{2252161:2253521}{2246721:2248081}>

Comments