Mailing List Archive: NANOG: users

Thoughts on increasing MTUs on the internet

1 2 3

View All

Index | Next | Previous | View Threaded

gtb at slac

Apr 12, 2007, 11:16 AM

Post #26 of 64 (2982 views)
Permalink

RE: Thoughts on increasing MTUs on the internet [In reply to]

> Last I heard, the IEEE won't go along, and they're the ones who
> standardize 802.3.
>
> A few years ago, the IETF was considering various jumbogram options.
> As best I recall, that was the official response from the relevant
> IEEE folks: "no". They're concerned with backward compatibility.

As I remember it, the IEEE did not say "no" (that is not the
style of such standards bodies). Instead, they said something
along the lines of "We will consider any proposal that does
not break (existing) standards/implementations". And, to the
best of my knowledge, the smart people of the world have not
yet made a proposal that meets the requirements (and I believe
more than a few have tried to think the issues through).

There is absolutely nothing to prevent one from implementing
"jumbos" (if you can even agree how large that should be).
It just seems that whatever one implements will likely not
be an IEEE standard (unless one is smarter than the last
set of smart people).

Gary

randy at psg

Apr 12, 2007, 11:51 AM

Post #27 of 64 (2979 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

>> A few years ago, the IETF was considering various jumbogram options.
>> As best I recall, that was the official response from the relevant
>> IEEE folks: "no". They're concerned with backward compatibility.
>
> As I remember it, the IEEE did not say "no"

i was in the middle of this one. they said "no." the checksum becomes
much weaker at 4k and 9k. and ether does have errors.

randy

iljitsch at muada

Apr 12, 2007, 12:18 PM

Post #28 of 64 (2993 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On 12-apr-2007, at 20:15, Randy Bush wrote:

>> A few years ago, the IETF was considering various jumbogram options.
>> As best I recall, that was the official response from the relevant
>> IEEE folks: "no". They're concerned with backward compatibility.

> worse. they felt that the ether checksum is good at 1500 and not
> so good at 4k etc. they *really* did not want to do jumbo. i
> worked that doc.

It looks to me that the checksum issue is highly exaggerated or even
completely wrong (as in the 1500 / 4k claim above). From http://
www.aarnet.edu.au/engineering/networkdesign/mtu/size.html :

---
The ethernet packet also contains a Frame Check Sequence, which is a
32-bit CRC of the frame. The weakening of this frame check which
greater frame sizes is explored in R. Jain's "Error Characteristics
of Fiber Distributed Data Interface (FDDI)", which appeared in IEEE
Transactions on Communications, August 1990. Table VII shows a table
of Hamming Distance versus frame size. Unfortunately, the CRC for
frames greater than 11445 bytes only has a minimum Hamming Distance
of 3. The implication being that the CRC will only detect one-bit and
two-bit errors (and not non-burst 3-bit or 4-bit errors). The CRC for
between 375 and 11543 bytes has a minimum Hamming Distance of 4,
implying that all 1-bit, 2-bit and 3-bit errors are detected and most
non-burst 4-bit errors are detected.

The paper has two implications. Firstly, the power of ethernet's
Frame Check Sequence is the major limitation on increasing the
ethernet MTU beyond 11444 bytes. Secondly, frame sizes under 11445
bytes are as well protected by ethernet's Frame Check Sequence as
frame sizes under 1518 bytes.

---

Is the FCS supposed to provide guaranteed protection against a
certain number of bit errors per packet? I don't believe that's the
case. With random bit errors, there's still only a risk of not
detecting an error in the order of 1 : 2^32, regardless of the length
of the packet. But even *any* effective weakening of the FCS caused
by an increased packet size is considered unacceptable, it's still
possible to do 11543 byte packets without changing the FCS algorithm.

Also, I don't see fundamental problem in changing the FCS for a new
802.3 standard, as switches can strip off a 64-bit FCS and add a 32-
bit one as required.

randy at psg

Apr 12, 2007, 12:21 PM

Post #29 of 64 (2981 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

> It looks to me that the checksum issue is highly exaggerated or even
> completely wrong (as in the 1500 / 4k claim above). From
> http://www.aarnet.edu.au/engineering/networkdesign/mtu/size.html :

glad you have an opinion. take it to the ieee.

randy

randy at psg

Apr 12, 2007, 12:28 PM

Post #30 of 64 (2980 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

mark does not have posting privs and has asked me to post the
following for him:

---

To: Gian Constantine <constantinegi[at]corp.earthlink.net>
From: Mark Allman <mallman[at]icir.org>
cc: NANOG list <nanog[at]merit.edu>
Subject: Re: Thoughts on increasing MTUs on the internet
Date: Thu, 12 Apr 2007 11:47:35 -0400

Folks-

> I agree. The throughput gains are small. You're talking about a
> difference between a 4% header overhead versus a 1% header overhead
> (for TCP).

This does not begin to reflect the gain. Check out the model of TCP
performance given in:

M. Mathis, J. Semke, J. Mahdavi, T. Ott, "The Macroscopic Behavior of
the TCP Congestion Avoidance Algorithm", Computer Communication Review,
volume 27, number3, July 1997.
(number 35 at http://www.psc.edu/~mathis/papers/index.html)

The key point is that performance is directly proportional to packet
size. So, an increase in the packet size is much more than a simple
lowering of the overhead.

In addition, the newly published RFC 4821 offers a different way to do
PMTUD without relying on ICMP feedback (essentially by trying different
packet sizes and trying to infer things from whether they get dropped).

A good general reference to the subject of bigger MTUs is Matt Mathis'
page on the subject:

http://www.psc.edu/~mathis/MTU/

allman

--
Mark Allman -- ICIR/ICSI -- http://www.icir.org/mallman/

swmike at swm

Apr 12, 2007, 1:05 PM

Post #31 of 64 (2985 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On Thu, 12 Apr 2007, Joe Loiacono wrote:

> Large MTUs enable significant throughput performance enhancements for
> large data transfers over long round-trip times (RTTs.) The original

This is solved by increasing TCP window size, it doesn't depend very much
on MTU.

Larger MTU is better for devices that for instance do per-packet
interrupting, like most endsystems probably do. It doesn't increase
long-RTT transfer performance per se (unless you have high packetloss
because you'll slow-start more efficiently).

--
Mikael Abrahamsson email: swmike[at]swm.pp.se

jloiacon at csc

Apr 12, 2007, 1:31 PM

Post #32 of 64 (2982 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

owner-nanog[at]merit.edu wrote on 04/12/2007 04:05:43 PM:

>
> On Thu, 12 Apr 2007, Joe Loiacono wrote:
>
> > Large MTUs enable significant throughput performance enhancements for
> > large data transfers over long round-trip times (RTTs.) The original
>
> This is solved by increasing TCP window size, it doesn't depend very
much
> on MTU.

Window size is of course critical, but it turns out that MTU also impacts
rates (as much as 33%, see below):

MSS 0.7
Rate = ----- * -------
RTT (P)**0.5

MSS = Maximum Segment Size
RTT = Round Trip Time
P = packet loss

Mathis, et. al. have 'verified the model through both simulation and live
Internet measurements.'

Also (http://www.aarnet.edu.au/engineering/networkdesign/mtu/why.html):

"This is shown to be the case in Anand and Hartner's "TCP/IP Network Stack
Performance in Linux Kernel 2.4 and 2.5" in Proceedings of the Ottawa
Linux Symposium, 2002. Their experience was that a machine using a 1500
byte MTU could only reach 750Mbps whereas the same machine configured with
9000 byte MTUs handsomely reached 1Gbps."

AARnet - Australia's Academic and Research Network

>
> Larger MTU is better for devices that for instance do per-packet
> interrupting, like most endsystems probably do. It doesn't increase
> long-RTT transfer performance per se (unless you have high packetloss
> because you'll slow-start more efficiently).
>
> --
> Mikael Abrahamsson email: swmike[at]swm.pp.se

swmike at swm

Apr 12, 2007, 1:48 PM

Post #33 of 64 (2983 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On Thu, 12 Apr 2007, Joe Loiacono wrote:

> Window size is of course critical, but it turns out that MTU also impacts
> rates (as much as 33%, see below):
>
> MSS 0.7
> Rate = ----- * -------
> RTT (P)**0.5
>
> MSS = Maximum Segment Size
> RTT = Round Trip Time
> P = packet loss

So am I to understand that with 0 packetloss I get infinite rate? And TCP
window size doesn't affect the rate?

I am quite confused by this statement. Yes, under congestion larger MSS is
better, but without congestion I don't see where it would differ apart
from the interrupt load I mentioned earlier?

--
Mikael Abrahamsson email: swmike[at]swm.pp.se

David_Hankins at isc

Apr 12, 2007, 2:28 PM

Post #34 of 64 (2986 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

Hopefully I'll be forgiven for geeking out over DHCP on nanog-l twice
in the same week.

On Thu, Apr 12, 2007 at 11:20:18AM +0200, Iljitsch van Beijnum wrote:
> 1. It's no longer necessary to limit the subnet MTU to that of the
> least capable system

I dunno for that.

> 2. It's no longer necessary to manage 1500 byte+ MTUs manually

But for this, there has been (for a long time now) a DHCPv4 option
to give a client its MTU for the interface being configured (#26,
RFC2132).

The thing is, not very many (if any) clients actually request it.
Possibly because of problem #1 (if you change your MTU, and no one
else does, you're hosed).

So, if you solve for the first problem in isolation, you can
easily just use DHCP to solve the second with virtually no work
and probably "only" (heh) client software updates.

I could also note that your first problem plagues DHCP software
today...it's further complicated...let's just say it sucks, and
bad.

If one were to solve that problem for DHCP speakers, you could
probably put a siphon somewhere in the process.

But it's an even harder problem to solve.

--
David W. Hankins "If you don't do it right the first time,
Software Engineer you'll just have to do it again."
Internet Systems Consortium, Inc. -- Jack T. Hankins

jloiacon at csc

Apr 12, 2007, 2:31 PM

Post #35 of 64 (2985 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

I believe the formula applies when the TCP window size is held constant
(and maybe as large as is necessary for the bandwidth-delay product).
Obviously P going to zero is a problem; there are practical limitations.
But bit error rate is usually not zero over long distances.

The formula is not mine, it's not new, and there is empirical evidence to
support it. Check out the links for more (and better :-) info.

Joe

owner-nanog[at]merit.edu wrote on 04/12/2007 04:48:09 PM:

>
> On Thu, 12 Apr 2007, Joe Loiacono wrote:
>
> > Window size is of course critical, but it turns out that MTU also
impacts
> > rates (as much as 33%, see below):
> >
> > MSS 0.7
> > Rate = ----- * -------
> > RTT (P)**0.5
> >
> > MSS = Maximum Segment Size
> > RTT = Round Trip Time
> > P = packet loss
>
> So am I to understand that with 0 packetloss I get infinite rate? And
TCP
> window size doesn't affect the rate?
>
> I am quite confused by this statement. Yes, under congestion larger MSS
is
> better, but without congestion I don't see where it would differ apart
> from the interrupt load I mentioned earlier?
>
> --
> Mikael Abrahamsson email: swmike[at]swm.pp.se

dts at senie

Apr 12, 2007, 2:58 PM

Post #36 of 64 (3000 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

At 05:28 PM 4/12/2007, David W. Hankins wrote:

>Hopefully I'll be forgiven for geeking out over DHCP on nanog-l twice
>in the same week.
>
>On Thu, Apr 12, 2007 at 11:20:18AM +0200, Iljitsch van Beijnum wrote:
> > 1. It's no longer necessary to limit the subnet MTU to that of the
> > least capable system
>
>I dunno for that.

Indeed. I do hope the vocal advocates for general use of larger MTU
sizes on Ethernet have had in their careers the opportunity to enjoy
the fun that ensues with LAN technologies were multiple MTUs are
supported, namely token ring and FDDI. Debugging networks where MTU
and MRU mismatches occur can be interesting, to say the least.

It's not just a matter of receiving stations noticing there's packets
coming in that are too big. Depending on the design of the interface
chips, the packet may not be received at all, and no indication sent
to the driver. The result can be endless re-sending of information,
doomed to failure.

OSPF has a way to negotiate MTU over LAN segments to deal with
exactly this situation. I uncovered the problem debugging a largish
OSPF network that would run for weeks or months, then fail to
converge. Multi-access media benefits from predictable MTU/MRU sizes.
Ethernet was well served by the fixed size.

I have no issue with allowing for a larger MTU size, but disagree
with attempts to reduce everyone on the link to the lowest common
denominator UNLESS that negotiation is repeated periodically (with
MTU sizes able to both increase and decrease). If systemns negotiate
a particular size among all players on a LAN, and a new station is
introduced, the decision process for what to do must be understood.

An alternative is to limit everyone to 1500 byte MTUs unless or until
adjacent stations negotiate a larger window size. At the LAN level,
this could be handled in ARP or similar, but the real desire would be
to find a way to negotiate endpoint-to-endpoint at the IP layer.
Don't even get into IP multicast...

> > 2. It's no longer necessary to manage 1500 byte+ MTUs manually
>
>But for this, there has been (for a long time now) a DHCPv4 option
>to give a client its MTU for the interface being configured (#26,
>RFC2132).
>
>The thing is, not very many (if any) clients actually request it.
>Possibly because of problem #1 (if you change your MTU, and no one
>else does, you're hosed).

Trying to do this via DHCP is, IMO, doomed to failure. The systems
most likely to be in need of larger MTUs are likely servers, and
probably not on DHCP-assigned addresses.

>So, if you solve for the first problem in isolation, you can
>easily just use DHCP to solve the second with virtually no work
>and probably "only" (heh) client software updates.
>
>
>I could also note that your first problem plagues DHCP software
>today...it's further complicated...let's just say it sucks, and
>bad.
>
>If one were to solve that problem for DHCP speakers, you could
>probably put a siphon somewhere in the process.
>
>But it's an even harder problem to solve.

DHCP has enough issues and problems today, I think we're in agreement
that heaping more on it might not be prudent.

Dan

David_Hankins at isc

Apr 12, 2007, 3:09 PM

Post #37 of 64 (2975 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On Thu, Apr 12, 2007 at 05:58:07PM -0400, Daniel Senie wrote:
> >> 2. It's no longer necessary to manage 1500 byte+ MTUs manually
> >
> >But for this, there has been (for a long time now) a DHCPv4 option
> >to give a client its MTU for the interface being configured (#26,
> >RFC2132).
>
> Trying to do this via DHCP is, IMO, doomed to failure. The systems
> most likely to be in need of larger MTUs are likely servers, and
> probably not on DHCP-assigned addresses.

If you're bothering to statically configure a system with a fixed
address (such as with a server), why can you not also statically
configure it with an MTU?

--
David W. Hankins "If you don't do it right the first time,
Software Engineer you'll just have to do it again."
Internet Systems Consortium, Inc. -- Jack T. Hankins

dts at senie

Apr 12, 2007, 3:18 PM

Post #38 of 64 (2979 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

At 06:09 PM 4/12/2007, David W. Hankins wrote:

>On Thu, Apr 12, 2007 at 05:58:07PM -0400, Daniel Senie wrote:
> > >> 2. It's no longer necessary to manage 1500 byte+ MTUs manually
> > >
> > >But for this, there has been (for a long time now) a DHCPv4 option
> > >to give a client its MTU for the interface being configured (#26,
> > >RFC2132).
> >
> > Trying to do this via DHCP is, IMO, doomed to failure. The systems
> > most likely to be in need of larger MTUs are likely servers, and
> > probably not on DHCP-assigned addresses.
>
>If you're bothering to statically configure a system with a fixed
>address (such as with a server), why can you not also statically
>configure it with an MTU?

Neither addresses interoperability on a multi-access medium where a
new station could be introduced, and can result in the same MTU/MRU
mismatch problems that were seen on token ring and FDDI. The problem
is you might open a conversation (whatever the protocol), then get
into trouble when larger data packets follow smaller initial
conversation opening packets.

Or you can work with the same assumptions people use today: all
stations on a particular network segment must use the same MTU size,
whether that's the standard Ethernet size, or a larger size, and a
warning sign hanging from the switch, saying "use MTU size of xxxx or
suffer the consequences."

David_Hankins at isc

Apr 12, 2007, 3:31 PM

Post #39 of 64 (2986 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On Thu, Apr 12, 2007 at 06:18:56PM -0400, Daniel Senie wrote:
> Neither addresses interoperability on a multi-access medium where a
> new station could be introduced, and can result in the same MTU/MRU
> mismatch problems that were seen on token ring and FDDI.

Solving Ilijitsch's "#1" is a separate problem, and you can solve
them in isolation. If you chose to do so, "#2" is already solved
for all hosts where dynamic configuration is desirable.

--
David W. Hankins "If you don't do it right the first time,
Software Engineer you'll just have to do it again."
Internet Systems Consortium, Inc. -- Jack T. Hankins

will at harg

Apr 12, 2007, 4:17 PM

Post #40 of 64 (2981 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

perry at coders

Apr 12, 2007, 4:41 PM

Post #41 of 64 (2997 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

Iljitsch van Beijnum wrote:
>
> Dear NANOGers,
>
> It irks me that today, the effective MTU of the internet is 1500 bytes,
> while more and more equipment can handle bigger packets.
>
> What do you guys think about a mechanism that allows hosts and routers
> on a subnet to automatically discover the MTU they can use towards other
> systems on the same subnet, so that:
>
> 1. It's no longer necessary to limit the subnet MTU to that of the least
> capable system
>
> 2. It's no longer necessary to manage 1500 byte+ MTUs manually
>
> Any additional issues that such a mechanism would have to address?

I have a half completed, prototype "mtud" that runs under Linux. It
sets the interface to 9k, but sets the route for the subnet down to
1500. It then watches the arp table for new arp entries. As a new MAC
is added, it sends a 9k UDP datagram to that host and listens for an
ICMP port unreachable reply (like traceroute does). If the error
arrives, it assumes that host can receive packets that large, and adds a
host route with the larger MTU to that host. It steps up the mtu's from
1500 to 16k trying to rapidly increase the MTU without having to wait
for annoying timeouts. If anything goes wrong somewhere along the way,
(a host is firewalled or whatever) then it won't receive the ICMP reply,
and won't raise the MTU.

The idea is that you can run this on routers/servers on a network that
has 9k mtu's but not all the hosts are assured to be 9k capable, and it
will increase correctly detect the available MTU between servers, or
routers, but still be able to correctly talk to machines that are still
stuck with 1500 byte mtu's etc.

In other interesting data points in this field, for some reason a while
ago we had reason to do some throughput tests under Linux with varying
the MTU using e1000's and ended up with this pretty graph:

http://wand.net.nz/~perry/mtu.png

we never had the time to investigate exactly what was going on, but
interestingly at 8k MTU's (which is presumably what NFS would use),
performance is exceptionally poor compared to 9k and 1500 byte MTU's.
Our (untested) hypothesis is that the Linux kernel driver isn't smart
about how it allocates it's buffers.

list at satchell

Apr 12, 2007, 8:00 PM

Post #42 of 64 (3000 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

Steven M. Bellovin wrote:
> On Thu, 12 Apr 2007 11:20:18 +0200
> Iljitsch van Beijnum <iljitsch[at]muada.com> wrote:
>
>> Dear NANOGers,
>>
>> It irks me that today, the effective MTU of the internet is 1500
>> bytes, while more and more equipment can handle bigger packets.
>>
>> What do you guys think about a mechanism that allows hosts and
>> routers on a subnet to automatically discover the MTU they can use
>> towards other systems on the same subnet, so that:
>>
>> 1. It's no longer necessary to limit the subnet MTU to that of the
>> least capable system
>>
>> 2. It's no longer necessary to manage 1500 byte+ MTUs manually
>>
>> Any additional issues that such a mechanism would have to address?
>>
>
> Last I heard, the IEEE won't go along, and they're the ones who
> standardize 802.3.
>
> A few years ago, the IETF was considering various jumbogram options.
> As best I recall, that was the official response from the relevant
> IEEE folks: "no". They're concerned with backward compatibility.
>
> Perhaps that has changed (and I certainly) don't remember who sent that
> note.

No, I doubt it will change. The CRC algorithm used in Ethernet is
already strained by the 1500-byte-plus payload size. 802.3 won't extend
to any larger size without running a significant risk of the CRC
algorithm failing.

From a practical side, the cost of developing, qualifying, and selling
new chipsets to handle jumbo packets would jack up the cost of inside
equipment. What is the payback? How much money do you save going to
jumbo packets?

Show me the numbers.

saku+nanog at ytti

Apr 12, 2007, 10:20 PM

Post #43 of 64 (2984 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On (2007-04-13 00:17 +0100), Will Hargrave wrote:

> At LONAP a jumbo frames peering vlan is on the 'to investigate' list. I
> am not sure if there is that much interest though. Another vlan, another
> SVI, another peering session...

Why another? For neighbours that are willing to peer over eg. VLAN MTU 9k,
peer with them only over that VLAN. I don't see much point peering
over both VLANs.
What I remember discussing with unnamed european IXP staff was that
they were worried about loosing 'frame too big' counters. Since of
course then the switch environment would accept bigger frames even
on the 1500 MTU VLAN. And if member misconfigures the small MTU VLAN,
and calls to IXP complaining how IXP is dropping their frames (due
to sending over 1500bytes) IXP staff can't quickly diagnose the
problem from interface counters. I argued that it's mostly irrelevant,
since IXP staff can ping from IXP 'small mtu VLAN' the customer
they're suspecting sending too large frames, and confirm this
if router replies to a ping over 1500 bytes. But then again, I have
0 operational experience running IXP and it's easy for me to
oversimplify the issue.

> The fabric itself is enabled to 9216 bytes; we have several members
> exchanging L2TP DSL traffic at higher MTUs but this is currently done
> over private (i.e. member addressed) vlans.

This I believe to be biggest gain, tunneling, eg. ability to run IPSec
site-to-site while providing full 1500bytes to LAN.

> There are some other possible IX applications... MPLS springs to mind as
> another network technology which requires at least baby giants; what
> would providers use this for? Handoff of multiprovider l2/l3 VPNs?
>
> The other technology which sees people deploying jumbos out there is
> storage. Selling storage as well as transit over the IX? It could happen :-)
>
> --
> Will Hargrave will[at]lonap.net
> Technical Director
> LONAP Ltd
>
>
>
>
>

--
++ytti

saku+nanog at ytti

Apr 12, 2007, 10:22 PM

Post #44 of 64 (2984 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On (2007-04-12 20:00 -0700), Stephen Satchell wrote:

> From a practical side, the cost of developing, qualifying, and selling
> new chipsets to handle jumbo packets would jack up the cost of inside
> equipment. What is the payback? How much money do you save going to
> jumbo packets?

It's rather hard to find ethernet gear operators could imagine using in
peering or core that do not support +9k MTU's.

--
++ytti

neil at domino

Apr 13, 2007, 12:35 AM

Post #45 of 64 (2984 views)
Permalink

RE: Thoughts on increasing MTUs on the internet [In reply to]

Saku Ytti wrote:

> IXP peeps, why are you not offering high MTU VLAN option?
> From my point of view, this is biggest reason why we today
> generally don't have higher end-to-end MTU.
> I know that some IXPs do, eg. NetNOD but generally it's
> not offered even though many users would opt to use it.

Larger MTU size was something I did some work on back in the FDDI days and the benefits are significant. More than just CPU improvements. Throughput and server performance increased substantially also. But FDDI and the like didn't come cheap so little interest at the time. At the LINX a few providers did run larger MTUs during those FDDI days. We did some testing with SRP/DPT in Stockholm and London also and again it worked well but again not cheap. (we were looking at this for storage and exchange of cached content at the time)

Unfortunately I think the time where IXPs could make a difference might be past - and to make this happen tere needs to be more of a demand from the members of tose exchanges, its not just a case of turning on a vlan either the impact to the main fabric needs to be understood. Also atleast here in Europe many of the circuits into exchanges are Ethernet based also and I suspect many circuits into exchanges would require a lot of work to support Jumbos. And then again lots of circuits into customer premise are Ethernet based now also some on GFP based SDH systems, some ATM and other whacko technologies with dubious support for jumbo or larger frames.

Then there is the actual interface card support of large amounts of jumbos which in my experience is questionable based on a limitedl amount of testing though - Come back POS all is forgiven!

Regards,
Neil

michael.dillon at bt

Apr 13, 2007, 2:03 AM

Post #46 of 64 (2992 views)
Permalink

RE: Thoughts on increasing MTUs on the internet [In reply to]

> No, I doubt it will change. The CRC algorithm used in Ethernet is
> already strained by the 1500-byte-plus payload size. 802.3
> won't extend
> to any larger size without running a significant risk of the CRC
> algorithm failing.

I believe this has already been debunked.

> From a practical side, the cost of developing, qualifying,
> and selling
> new chipsets to handle jumbo packets would jack up the cost of inside
> equipment. What is the payback? How much money do you save going to
> jumbo packets?

I believe that the change is intended to apply to routers and the
ethernet switches that interconnect them in PoPs and NAPs and exchange
points. Therefore the cost of a small chipset modification is likely to
be negligible in the grand scheme of things.

As for numbers, it is not dollar figures that I want to see. I would
like the people who have jumbo packets inside their end-user networks to
run some MTU discovery and publish a full MTU matrix on all paths on the
Internet. That way we can all see where there is end-to-end support for
large MTUs and people who want to make buying decisions on this basis
will have something other than vendor assurances to show that a network
supports jumbograms.

--Michael Dillon

Valdis.Kletnieks at vt

Apr 13, 2007, 7:36 AM

Post #47 of 64 (2985 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On Fri, 13 Apr 2007 08:22:49 +0300, Saku Ytti said:
>
> On (2007-04-12 20:00 -0700), Stephen Satchell wrote:
>
> > From a practical side, the cost of developing, qualifying, and selling
> > new chipsets to handle jumbo packets would jack up the cost of inside
> > equipment. What is the payback? How much money do you save going to
> > jumbo packets?
>
> It's rather hard to find ethernet gear operators could imagine using in
> peering or core that do not support +9k MTU's.

Note that the number of routers in the "core" is probably vastly outweighted
by the number of border and edge routers. There's a *lot* of old eBay routers
out there - and until you get a clean path all the way back to the source
system, you won't *see* any 9K packets.

What's the business case for upgrading an older edge router to support 9K
MTU, when the only source of packets coming in is a network of Windows
boxes (both servers and end systems in offices) run by somebody who wouldn't
believe an Ethernet has anything other than a 1500 MTU if you stapled the
spec sheet to their forehead?

For that matter, what releases of Windows support setting a 9K MTU? That's
probably the *real* uptake limiter.

leigh.porter at ukbroadband

Apr 13, 2007, 7:52 AM

Post #48 of 64 (2989 views)
Permalink

RE: Thoughts on increasing MTUs on the internet [In reply to]

I don't think it matters that everything can use jumbograms or that every single device on the Internet supports them. Heck, I still know networks with kit that does not support VLSM!

What would be good is if when a jumbogram capable path on the Internet exists, jumbograms can be used.

This way it does not matter than some box somewhere does not support anything greater than a 1500 byte MTU, anything with such a box in the path will simply not support a jumbogram. How do you find out? Just send a jumbogram across the path and see what happens.. ;-)

--
Leigh Porter
UK Broadband

-----Original Message-----
From: owner-nanog[at]merit.edu on behalf of Valdis.Kletnieks[at]vt.edu
Sent: Fri 4/13/2007 3:36 PM
To: Saku Ytti
Cc: NANOG list
Subject: Re: Thoughts on increasing MTUs on the internet

On Fri, 13 Apr 2007 08:22:49 +0300, Saku Ytti said:
>
> On (2007-04-12 20:00 -0700), Stephen Satchell wrote:
>
> > From a practical side, the cost of developing, qualifying, and selling
> > new chipsets to handle jumbo packets would jack up the cost of inside
> > equipment. What is the payback? How much money do you save going to
> > jumbo packets?
>
> It's rather hard to find ethernet gear operators could imagine using in
> peering or core that do not support +9k MTU's.

Note that the number of routers in the "core" is probably vastly outweighted
by the number of border and edge routers. There's a *lot* of old eBay routers
out there - and until you get a clean path all the way back to the source
system, you won't *see* any 9K packets.

What's the business case for upgrading an older edge router to support 9K
MTU, when the only source of packets coming in is a network of Windows
boxes (both servers and end systems in offices) run by somebody who wouldn't
believe an Ethernet has anything other than a 1500 MTU if you stapled the
spec sheet to their forehead?

For that matter, what releases of Windows support setting a 9K MTU? That's
probably the *real* uptake limiter.

swmike at swm

Apr 13, 2007, 8:27 AM

Post #49 of 64 (2974 views)
Permalink

RE: Thoughts on increasing MTUs on the internet [In reply to]

On Fri, 13 Apr 2007, Leigh Porter wrote:

> What would be good is if when a jumbogram capable path on the Internet
> exists, jumbograms can be used.

Yes, and it would be good if PMTUD worked, and ECN, oh and large
UDP-packets for DNS, and BCP38, and... and... and.

The internet is a very diverse and complicated beast and if end systems
can properly detect PMTU by doing discovery of this, it might work.
Requiring the core and distribution to change isn't going to happen
overnight, so end systems first. Make sure they can properly detect PMTU
by use of nothing more than "is this packet size getting thru" (ie no
ICMP-NEED-TO-FRAG) or alike, then we might see partial adoption of larger
MTU in some parts and if this becomes a major customer requirement then it
might spread.

--
Mikael Abrahamsson email: swmike[at]swm.pp.se

smeuse at gmail

Apr 13, 2007, 8:42 AM

Post #50 of 64 (2984 views)
Permalink

Re: Thoughts on increasing MTUs on the internet [In reply to]

On 4/13/07, Valdis.Kletnieks[at]vt.edu <Valdis.Kletnieks[at]vt.edu> wrote:
>
>
> For that matter, what releases of Windows support setting a 9K
> MTU? That's
> probably the *real* uptake limiter.

Most, if not all. I have an XP box that has a GigE with 9k MTU.

--

-Steve

1 2 3

View All

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact lists@gossamer-threads.com