How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX...

Page 1 of 16

About the Author

Brad Hedlund is a Consulting Systems Engineer at Cisco Systems focused on Data Center
technologies with 13 years combined experience as a Cisco customer, Cisco Partner, and now as a
Cisco employee. While at Cisco, Brad has been awarded: "Top Engineer" (August 2007); "Systems
Engineer Sales Champion" (August 2008 & 2009); "CSE of the Quarter, US & Canada" (May 2009);
and "Top Data Center CSE of the Year, US & Canada" (August 2009). Brad Hedlund holds CCIE
#5530 (Routing & Switching) and VMware VCP certifications.

See All Posts by This Author

How to Calculate TCP throughput for long

distance WAN links
December 19th, 2008 • Related • Filed Under

So you just lit up your new high-speed link between Data Centers but are unpleasantly surprised to
see relatively slow file transfers across this high speed, long distance link — Bummer! Before you
call Cisco TAC and start trouble shooting your network, do a quick calculation of what you should
realistically expect in terms of TCP throughput from a one host to another over this long distance link.

When using TCP to transfer data the two most important factors are the TCP window size and the
round trip latency. If you know the TCP window size and the round trip latency you can calculate the
maximum possible throughput of a data transfer between two hosts, regardless of how much
bandwidth you have.

Here is how you make the calculation:

TCP-Window-Size-in-bits / Latency-in-seconds = Bits-per-second-throughput

So lets work through a simple example. I have a 1Gig Ethernet link from Chicago to New York with a
round trip latency of 30 milliseconds. If I try to transfer a large file from a server in Chicago to a
server in New York using FTP, what is the best throughput I can expect?

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 2 of 16

First lets convert the TCP window size from bytes to bits. In this case we are using the standard
64KB TCP window size of a Windows machine.

64KB = 65536 Bytes. 65536 * 8 = 524288 bits

Next, lets take the TCP window in bits and divide it by the round trip latency of our link in seconds.
So if our latency is 30 milliseconds we will use 0.030 in our calculation.

524288 bits / 0.030 seconds = 17476266 bits per second throughput = 17.4 Mbps maximum possible
throughput

So, although I may have a 1GE link between these Data Centers I should not expect any more than
17Mbps when transferring a file between two servers, given the TCP window size and latency.

What can you do to make it faster? Increase the TCP window size and/or reduce latency.

To increase the TCP window size you can make manual adjustments on each individual server to
negotiate a larger window size. This leads to the obvious question: What size TCP window should
you use? We can use the reverse of the calculation above to determine optimal TCP window size.

Formula to calculate the optimal TCP window size:

Bandwidth-in-bits-per-second * Round-trip-latency-in-seconds = TCP window size in bits / 8 =
TCP window size in bytes

So in our example of a 1GE link between Chicago and New York with 30 milliseconds round trip
latency we would work the numbers like this…

1,000,000,000 bps * 0.030 seconds = 30,000,000 bits / 8 = 3,750,000 Bytes

Therefore if we configured our servers for a 3750KB TCP Window size our FTP connection would be
able to fill the pipe and achieve 1Gbps throughput.

One downside to increasing the TCP window size on your servers is that it requires more memory for
buffering on the server, because all outstanding unacknowledged data must be held in memory should
it need to be retransmitted again. Another potential pitfall is performance (ironically) where there is
packet loss, because any lost packets within a window requires that the entire window be
retransmitted – unless your TCP/IP stack on the server employs a TCP enhancement called “selective
acknowledgements”, which most do not.

Another option is to place a WAN accelerator at each end that uses a larger TCP window and other
TCP optimizations such as TCP selective acknowledgements just between the accelerators on each
end of the link, and does not require any special tuning or extra memory on the servers. The

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 3 of 16

accelerators may also be able to employ Layer 7 application specific optimizations to reduce round
trips required by the application.

Reduce latency? How is that possible? Unless you can figure out how to overcome the speed of light
there is nothing you can do to reduce the real latency between sites. One option is, again, placing a
WAN accelerator at each end that locally acknowledges the TCP segments to the local server, thereby
fooling the servers into seeing very low LAN like latency for the TCP data transfers. Because the
local server is seeing very fast local acknowledgments, rather than waiting for the far end server to
acknowledge, is the very reason why we do not need to adjust the TCP window size on the servers.

In this example the perfect WAN accelerator would be the Cisco 7371 WAAS Appliance, as it is rated
for 1GE of optimized throughput.

WAAS stands for: Wide Area Application Services

The two WAAS appliances on each end would use TCP optimizations over the link such as large TCP
windows and selective acknowledgements. Additionally, the WAAS appliances would also remove
redundant data from the TCP stream resulting in potentially very high levels of compression. Each
appliance remembers previously seen data, and if that same chunk of data is seen again, that data will
be removed and replaced with a tiny 2 Byte label. That tiny label is recognized by the remote WAAS
appliance and it replaces the tiny label with the original data before sending the traffic to the local
server.

The result of all this optimization would be higher LAN like throughput between the server in
Chicago and New York without any special TCP tuning on the servers.

###

Tags: tcp, waas

There Are 38 Responses So Far. »

1.

Comment by Serge on 12 January 2009:

Excellent material, Brad, thank you very much. The only thing, which confuses me the most, it
that:

When using TCP to transfer data the two most important factors are the TCP window size and
the round trip latency. If you know the TCP window size and the round trip latency you can
calculate the maximum possible throughput of a data transfer between two hosts, regardless of
how much bandwidth you have.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 4 of 16

Why don’t we use one-way latency?

Or for reverse calculation, when we are using BW*RTT formula, what do we receive as a
result? Amount of data on the wire (in flight), so why do we need a double-sized buffer?

2.

Comment by Brad Hedlund on 12 January 2009:

Serge,
We use round trip latency because we need to account for the time it takes the TCP sender to
receive an acknowledgment from the receiver. In our example here, we want the server in
Chicago to continue sending data while the acknowledgment from the New York server is
traveling back to Chicago.

Make sense?

3.

Comment by Serge on 13 January 2009:

Well, yes, but …

Let’s assume that one-way latency is 15ms – that means that Chicago sent 1MB of data and
after 15 ms NY received it. Immediately it sends ACK, which is received by Chicago after
another 15ms. That means that we really have throughput 1MB per 30ms, but in fact half of the
time sender waits till the ACK come. Is it really the way TCP works?

4.

Comment by Brad Hedlund on 13 January 2009:

That is correct. Once the TCP window has been transmitted the TCP sender will stop
transmitting data until an acknowledgement is received. If we use one-way latency in our
calculations the WAN link would be idle 50% of the time while the sender is waiting for
acknowledgements from the receiver.

5.

Comment by Tingli Pan on 14 January 2009:

I vaguely remember that in tcp protocol, the sender won’t wait for the first acknowledgement
before sending the second one. It will send several packages continuously before getting the
first acknowledgement, thus to speed up the transfer rate.

6.

Comment by Brad Hedlund on 14 January 2009:

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 5 of 16

A fundamental principal of TCP is the “congestion avoidance window” which represents the
maximum amount of unacknowledged data. This window is precisely what we are using for the
calculations in this article.

Some variations and enhancements to TCP optimize the behavior of the congestion avoidance
window, such as dynamically adjusting its size in varying conditions. However the fundamental
concept of managing a maximum amount of unacknowledged data never changes.

Most standard Windows machines use a very standard TCP/IP stack without all the additional
enhancements.

You can read more about fundamental TCP behavior and its variations here:
http://en.wikipedia.org/wiki/TCP_congestion_avoidance_algorithm

7.

Comment by Emmanuel Courreges on 5 February 2009:

Tingli pan: you are referring to acknowledging TCP segments for which the size is usually
around your MTU so 1400 bytes, which may be delayed and sent on every other segment, but
that’s still acknowledged at the beginning of a 64kB burst of segments which is the window
size.
Hope that doesn’t add to confusion…

8.

Comment by Ned on 5 March 2009:

Hi, just want to confirm some behavior. If there is physical machines connected to switches at
1000Mb at either side of a WAN link which is only 100Mb than would the servers throttle
down to 100Mbps or would they continue sending at 1000 but due to the WAN link being only
100Mb the packets will be dropped. Will TCP Window size effect the rate at which the
machine send and receive packet. Please confirm.

9.

Comment by Brad Hedlund on 5 March 2009:

Ned,
Two possible things could happen here, assuming a standard TCP/IP implementation on each
server…

1) the latency of your 100mb WAN link is high enough and/or the TCP window is small
enough such that TCP windowing is never able reach 100mb of throughput.

2) if the latency is very low and the window sizes are large enough, TCP will ramp up to 100mb
and at which point congestion will occur and packets will begin to drop. When packet loss is
detected TCP will cut throughput in half and slowly ramp back up to 100mb again, and the
cycle will repeat. This is called the saw tooth effect.

10.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 6 of 16

Comment by Ned on 7 March 2009:

Hi thanks for replying. Can u pls explain why the devices will only send at 100Mb even though
their connection is set to a 1000Mb. The Window size from what I read is the number of byte
packets that can be sent before getting acknowledgement and is negotiated at session startup
and during the sessions it keeps changing. Correct?

Also another confusion is there a calculation to check how many packets need to be transferred
to get a throughput of 100Mb and 1000Mb given that the packet size is 1460(1500-20-20)
Bytes of data = 11680 bits. 1Gb = 1000000000b. Hence if each packet is 11680 bits than to
send 1000000000 bits it would 1000000000/11680 = 85616 packets of 1460 bytes each. Given
that the size of packet can only be 1460 and the window is usually 65KB=512000b only than it
would take approx 43 segments to fill the window size. So its not really possible to send
packets at 1Gb. Is this correct calculation because it looks like something is wrong. thanks you.

11.

Comment by Brad Hedlund on 9 March 2009:

Ned,

Can u pls explain why the devices will only send at 100Mb even though their
connection is set to a 1000Mb

TCP pays no attention to the physical LAN connection speed of the host (1000Mb in your
scenario). The only three factors that matter in TCP throughput are window size, latency, and
packet loss. Of course the underlying link speed sets the maximum possible throughput under
ideal conditions.

Given that the size of packet can only be 1460 and the window is usually
65KB=512000b only than it would take approx 43 segments to fill the window
size. So its not really possible to send packets at 1Gb.

Yes, it is possible. In your scenario, if the latency is low enough the sender may have received
the first ACK before all 43 segments had been sent, at which point the window size is
replenished by the amount of bytes acknowledged in the ACK. This is called a sliding window
and allows the sender to continuously send data at the rate of the link speed until there is packet
loss.

12.

Comment by lisa on 20 March 2009:

Brad,

Can you clarify one of the equations for me?

In the section on optimizing window size, you have

“1,000,000,000 bps * 0.030 seconds = 30,000,000 bits / 8 = 3,750,000 Bytes Therefore if we

configured our servers for a 3750KB TCP Window size …”

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 7 of 16

So you are taking 3,750,000 Bytes and dividing by 1000 to get 3750KB.

My confusion relates to what side of the fence ‘window size’ sits on – is it a data storage
number (since its size affects memory allocation and storage amounts), or is it a network
number (since its size affects the throughput of the link)?

I assumed it landed on the data storage side of the fence, so I was wondering why you didn’t
divide by 1024 instead of 1000, where the window size would turn out to be 3662.11 KB. Since
I’m not that familiar with networking, I want to make sure I know when to multiply/divide by
1000 vs 1024.

BTW Thanks for the excellent, clear explanation. It’s the type of thing that’s really helpful for
newbies like myself.

13.

Comment by Brad Hedlund on 22 March 2009:

Lisa,
You are right. To be 100% accurate I should have divided by 1024, not 1000.
Notice that I started the post by calculating how many bits were in a 64KB window size by
multiplying 64 * 1024.
When talking about bytes, as we are in TCP window sizes, we should be using the 1024 number
to represent a KB.
With serial communications such as in networking, when we are talking about bits, it is normal
to use 1000 as the number that represents a Kb (kilobit).
A TCP window size is more of a storage number than it is a bit rate number.

Nice catch. You get extra credit points

Cheers,
Brad

14.

Comment by jeff on 12 April 2009:

Brad,

Our branch and the main office are also in Chicago and NY. I‘ve done a lot of low level WAN
research using wireshark. What I found is that without any optimization the acknowledgement
sometimes coming back after 3 frames of 1460 bytes size, sometimes after 6, 12, 23, and 45.
The windows size is always the same – 64KB. How this can be explained?

I also know about so call Windowing mechanism that requests an acknowledgment after 6
frames. How this fit into the picture?

Now, in one of our tests we have been using the compressing within application. We achieved
30 times improvements in the transfer speed
The throughout was 4 MB/sec, which exceeds your calculation limit. How this can be explaned.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 8 of 16

15.

Comment by Bruce H. on 21 April 2009:

TCP-Window-Size-in-bits / Latency-in-seconds = Bits-per-second-throughput

Brad,

The calculation sounds plausible. However, we have a large ISP network with thousands of
routers, switches, layer 1 trasport devices, customer F/W’s etc, etc along one path. In our
sceanrio, are we supposed to modify the TCP window size on all devices? What an effort!
Shoud we standardize window size? Hmmm…

The only thing we can truly control is latency, in which case, you must look at each link to
reduce latency as much as possible. Also, along the path, the packet may hit 100mbps, 1000
mbps or 10Gbps links or higher. The link may also be saturated or over subscribed increasing
latency. It sounds like, reducing latency by standardizing (for example) all gbps links, low link
utilization, fast CPU’s at each link, having a clean, error-free network is the best way to reduce
latency, hence increase throughput.

What is the best practical method to get the highest throughput with a very large ISP type
network?

16.

Comment by Brad Hedlund on 3 May 2009:

Bruce,
Adjusting TCP window settings to improve throughput is only effective on the client and server
machines, not the intermediate devices carrying the traffic. Adjusting any such TCP settings on
your gear carrying the customer traffic will be a futile effort.

17.

Comment by Brad Hedlund on 3 May 2009:

Hi Jeff,

acknowledgement sometimes coming back after 3 frames of 1460 bytes size,

sometimes after 6, 12, 23, and 45. The windows size is always the same – 64KB.
How this can be explained?

TCP is a very conservative protocol. This is likely the TCP Slow Start mechanism. New TCP
flows will start slow and gradually ramp up until packet loss is detected, at which point the
Slow Start cycle repeats.

Now, in one of our tests we have been using the compressing within application.
We achieved 30 times improvements in the transfer speed

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK EX... Page 9 of 16

The throughout was 4 MB/sec, which exceeds your calculation limit. How this can
be explaned.

The 4MB/sec was likely the throughput as observed by the application because of the
compression. However, the actual load on the network should still fit within the TCP
throughput calculations.

18.

Comment by Mario on 15 May 2009:

Hi Brad,

First off, thank you for the well-written article and the replies posted above. It’s been a very
interesting read.

Regarding your last comment on the TCP slow-start mechanism, is there a way to optimize this
to cause TCP to send flows as quickly as possible from the start?
I realise this is a mechanism of TCP, but am interested to know if there is a calculation
available or setting to optimise this.

Thanks,
Mario

19.

Comment by Vaclav Molik on 20 May 2009:

Hi Brad, your article is really helpfull, thanks for it. Let me ask if you have also experience
with IP Sec? Consider you have MPLS VPN with access circuits mostly E1 or T1. But you
encrypt all traffic between HQ and specific branch using IP Sec tunnel. E.g. NY-HongKong or
NY-Moscow. What would you advice for the best utilization of available capacity at this case?
Thanks a lot, Vaclav.

20.

Comment by Brad Hedlund on 23 May 2009:

Mario,
WAN optimization appliances such as Cisco WAAS employ several optimizations for TCP
traffic over WAN links and one of those is bypassing TCP Slow Start mechanism and
employing large initial windows. The nice thing about using WAN optimization appliances like
WAAS is that no TCP modifications are required on the client or server machines.

Without WAN optimization appliances you can try loading a HS-TCP (high speed TCP)
compliant TCP/IP stack on your client and server machines.

http://www.faqs.org/rfcs/rfc3649.html

Cheers,
Brad

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 10 of 16

21.

Comment by Brad Hedlund on 23 May 2009:

Vaclav,

I have several customers with IPSec WAN environments that are using Cisco WAAS with great
success. Some are using the Cisco ISR router with the WAAS network module, others are using
a WAAS appliance deployed inline between the existing router and LAN switch.

Cheers,
Brad

22.

Comment by Alab on 11 June 2009:

Hello,
if one is using a low bandwidth WAN link in which the bandwidth-delay product (call it B1) is
significantly less than the TCP receive window what is the effective throughput formula?

Is it
(1) TCP-Window-Size-in-bits / Latency-in-seconds = Bits-per-second-throughput

or just
(2) B1

23.

Comment by Brad Hedlund on 14 July 2009:

In your case, if the bandwidth delay product is significantly less than the TCP window size then
throughput is constrained by the speed of the link, not by TCP.

Cheers,
Brad

24.

Comment by chris stand on 22 July 2009:

go grab “TCPOPTIMIZER” if you are running win2k or xp.

then you can turn on sacks. vista & win2k3 & win2k8 all provide this feature.

25.

Comment by invisible on 26 July 2009:

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 11 of 16

Hmm, if I remember correctly there are TCP/IP implementations allowing sliding TCP window
sizes. In other words, if either side started with 64K TCP window it does not mean that window
size will remain the same all the time – it will increase and decrease depending on underlying
conditions.

26.

Comment by cp on 27 July 2009:

Brad,

Thanks for the write-up. My question is with the BDP formula using actual metrics for RTT,
window size, and throughput.

A lot of people give example on how the BDP formula work to give the estimated throughput
including the ideal window size. Like in your example.

The problem that examples they provide are either vague and do not provide real world results
to determine some of these things.

Here is my example and what I have done:

CLIENT———-SERVER (http)

Downloading a 6.33MB file from the SERVER.

======================
WINDOW SIZE from BDP:
======================

First, what is the RTT. When I do a “ping” to the server I get 32ms for the RTT.

Second, during the file transfer (using Vista or XP) it tells me that my data rate around
270KB/s. Convert that to bits, that’s around 2.1Mbps. Which is correct because when I do
many bandwidth tests they give me the same thing. Perfect.

Therefore based on that info my RWIN size should be this:

Bw (270KB) x RTT (.032) = 8640 bytes.

Well, when I ran Wireshark during the download session my RWIN on my client, receiving the
download was 66,780 bytes. Well that’s not 8640 bytes. Not that it matters, but the SERVER
window size was 6816KB. Not close to 8640 bytes.

Let’s try this in a different way

======================
THROUGHPUT from BDP:
======================

What’s my throughput then,

RWIN (66,780 bytes) / RTT (.032) = 2,086,875 bytes (or 16.7Mbps)

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 12 of 16

Umm, my DSL is only up to 2Mbps, so that is NOT my throughput nor correct.

======================
LATENCY from BDP:
======================

And when I calculate the latency from the BDP formula:

Bw (270KB) / RWIN (66,780 bytes) = 4 seconds (4043 ms) for the RTT

Not true either.

I understand that the BDP is used to help engineers know what window size should be
optimized to fill-up or use efficiently a circuit (LAN or WAN). Well using actual data that I
shown above, I’m not getting results that prove that the BDP works in real world. Thus, there is
something I am not understanding at all and relating it to actual/estimated throughput results
that can be calculated.

Or I’m obtaining the wrong type of data for the window size (through Wireshark that I see in
the SYN packet and all the data sessions decreasing the RWIN size at data is being received),
RTT (using ping), or the throughput (by looking at the file transfer window I see on Vista or
XP).

Can you and others please help me fill in the gap on what is going on here?

FYI – it would also be super nice if there was a simple GUI APP (that can be installed on a
client and server of coarse) that would show what is the RTT, Throughput, and Window Size
for the connection between them. Heck maybe also the packet loss, etc. The numbers would be
variable, but that would be a useful tool for troubleshooting and understanding the performance
conditions. I don’t think there is anything simplified like that out there.

Thank you very much!

cp

27.

Comment by Brad Hedlund on 28 July 2009:

CP,

Your math is correct, and proves my statement from a few comments above when I said: “…if
the bandwidth delay product is significantly less than the TCP window size then throughput is
constrained by the speed of the link, not by TCP.”

The speed of your link is 2.1mbps

You could have the largest window size in the world and the lowest possible latency — and
running the BDP formulas as you have done will give you a very high max theoretical
throughput — but if your link speed is only 2.1 mbps, throughput cannot go higher than 2.1
mbps, obviously.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 13 of 16

So, what your math has told you is that your TCP window size needs no adjustment to improve
your throughput. The only adjustment that will make your file transfer go faster than 2.1 mbps
would be to get a faster link, OR, compress the TCP data at each end to provide the illusion of a
faster link to the servers and clients — Cisco WAAS.

Cheers,
Brad

28.

Comment by Brad Hedlund on 28 July 2009:

invisible,
You are correct. Infact, this is how the TCP slow start phase works, start out with a small
window and gradually increase throughput by increasing the window size. When packet loss is
detected, throughput is cut in half and ramps up again via adjusting window sizes.
There is usually a set limit of how large the window will go, which would be the “max window
size”. You can’t keep increasing the window size without a limit as larger window sizes require
larger memory resources on the host.

Cheers,
Brad

29.

Comment by Duke on 18 August 2009:

Dear Brad,
Thanks for your article. It really helps me a lot to improve network performance.
But there is still something I can’t figure out in an experiment. Hope you could help me out.
Thanks in advance.
Experiment enviorment:
A——B (A downloads from B) The bandwidth from A to B is 50Mbps and the latency is 22ms.
A and B are both Linux system with version 2.6.18.
BDP=50000*22/8=137.5 KBytes
I tuned the maximum number of tcp_rmem in client and tcp_wmem in server to BDP 137500
and to get about 1.2MBps downloading speed. From the formula
“throughput=window_size/RTT”, I should have get about 6.25MBps
(137.5K/22ms=6.25MBps) theoretically. I don’t know why I only get 1.2MBps. But when I set
the max number to 1375000 and 26214000 in client and server, the speed comes to 2.3MBps
and 4.42MBps.
So I guess if there is someting I don’t take into consideration when calculating throughput.

Thanks and regards,

Duke

30.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 14 of 16

Comment by Brad Hedlund on 19 August 2009:

Duke,
There are other factors, for example packet loss has a detrimental effect on TCP throughput.
The formulas discussed in this article calculating max *theoretical* throughput assume a best
case scenario of zero packet loss. Most well engineered networks have very low packet loss in
the .01% range, however some links may have higher packet loss for a variety of reasons (too
much congestion, lack of QoS, rate-limiting, etc.)
Your WAN bandwidth of 50Mbps is interesting, that’s an odd number that doesn’t match a
typical physical circuit speed. Why 50Mbps? Is it perhaps a physical link much master than
50Mbps (such as 100Mbps or 1GE) but something is rate-limiting the throughput to 50Mbps?

Cheers,
Brad

31.

Comment by Duke on 21 August 2009:

Thanks Brad.
Actually, 50Mbps is the SDH physicall link bandwidth.
There are no Qos and rate-limiting policies applied to each side of the routers.

32.

Comment by Duke on 22 August 2009:

Hi Brad,
If we set the window size to BDP, we could get max theoretical throughput. The max
theoretical throughput should be the bandwidth/8. Am I correct?
If we put the same client and server into another link of the same bandwidth but higher latency,
we should get the same max theoretical throughput(bandwidth/8) because the bandwidth is the
same. right?
The reason why I get much smaller throughput in higher latency is higher latency has more
packet loss or what?
Thanks.
Duke

33.

Comment by Deepak Vyas on 12 September 2009:

Can we change the tcp window size on router to get a better throughput?

34.

Comment by Brad Hedlund on 15 September 2009:

Deepak,
The TCP session state is between the client and server machines, the routers are just passing the

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 15 of 16

TCP packets between client/server untouched, therefore changing any settings related to TCP
on your routers will have no effect.

35.

Comment by CT on 15 September 2009:

I found this blog while researching how long should a 1.7G file transfer take from HQ (45M) to
a remote site (2xT1) over MPLS GRE. My file transfer test indicates that I am getting about 2
Mbps. How much should I accommodate for the overhead for GRE and T1 bundling? I already
have tcp-mss size on the GRE tunnel interface set to 1436 per one of the Cisco article’s I read…
1500-40 (TCP/IP header)-24(GRE header). When I remove this command from the tunnel
interface then the performance is even worse. So, are you saying that changing the MSS helps,
but changing the tcp window size on a router will not? Any suggestions on how I can optimize
my setup further so that I use full 3M? Or what data do I give to our ISP to prove that the traffic
is available to be sent but the pipe is not carrying it? Thanks in advance.

36.

Comment by Pete on 17 September 2009:

Hi

I’m wondering about the measurement of the RTT on a link with ping. What should the correct
packet size be in the ping when measuring the RTT ?

regs. pete

37.

Comment by Brad Hedlund on 29 September 2009:

Pete,
A very good question. To get the most precise and worst case scenario measurement for your
calculations you would want to know how much the latency the TCP stream will see when the
pipe is full. The most accurate way to do this would be to saturate the link with a UDP stream
of MTU sized packets and measure how long it takes for the packets to get from point A to
point B. Then do the same measurement from point B to point A, then add the two
measurements together for the worst case round-trip latency (full pipe, large packets).

Cheers,
Brad

38.

Comment by prabhu on 13 October 2009:

HI brad,
I am new to wireshark and linux tcp tuning. and i used BIC tcp algorithm
i underwent a Experiment..
I downloaded a 100mb of file and captured the packets uing wireshark analyzer.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009
How to Calculate TCP throughput for long distance WAN links : INTERNETWORK E... Page 16 of 16

From this i need to calculate the

1.Link Delay..
2.Bandwidth
3.Slowstart phase
4.Window size
And i need to tune the TCP parameters like rmem and wmem and again i need to download the
file measure these things.
i am pretty confussed abt this concept ..Can u please explain this..

What will happen when we tune the tcp parameters and what will happen to the window size.

http://www.internetworkexpert.org/2008/12/19/how-to-calculate-tcp-throughput-for-long-... 10/16/2009