Hack Usenix14
Hack Usenix14
Hack Usenix14
Lynne Salameh, Astrit Zhushi, Mark Handley, Kyle Jamieson, Brad Karp
University College London
Abstract pre-transmission idle periods under contention, further
reducing medium efficiency.
WiFi’s physical layer has increased in speed from
802.11b’s 11 Mbps to the Gbps rates of emerging In an effort to amortize the significant overhead of
802.11ac. Despite these gains, WiFi’s inefficient MAC medium acquisition over multiple data frames, 802.11n’s
layer limits achievable end-to-end throughput. The culprit MAC protocol batches multiple data frames into a single
is 802.11’s mandatory idle period before each medium aggregate MAC protocol data unit (A-MPDU), and in-
acquisition, which has come to dwarf the duration of a curs only a single medium acquisition for each such batch.
packet’s transmission. This overhead is especially punish- 802.11n further aggregates the LL ACKs for the data pack-
ing for TCP traffic, whose every two data packets elicit ets in a received A-MPDU into a single LL Block ACK.
a short TCP ACK. Even frame aggregation and block While batching helps one sender, TCP traffic is inherently
link-layer ACKs (introduced in 802.11n) leave signifi- bidirectional: a TCP receiver typically transmits a single
cant medium acquisition overhead for TCP ACKs. In TCP ACK packet for every pair of TCP data packets it
this paper, we propose TCP/HACK (Hierarchical AC- receives. Not only do TCP ACKs incur further expensive
Knowledgment), a system that applies cross-layer opti- medium acquisitions by the TCP receiver—they run the
mization to TCP traffic on WiFi networks by carrying risk of colliding with the TCP data sender’s transmissions
TCP ACKs within WiFi’s link-layer acknowledgments. as well.
By eliminating all medium acquisitions for TCP ACKs WiFi’s data frames elicit LL ACKs that the receiver
in unidirectional TCP flows, TCP/HACK significantly sends without contending for the medium, as other would-
improves medium utilization, and thus significantly in- be senders defer for an ACK frame’s duration after hear-
creases achievable capacity for TCP workloads. Our mea- ing a data frame. We observe that this LL ACK is an ideal
surements of a real-time, line-speed implementation for vessel for carrying TCP ACK information on the reverse
802.11a on the SoRa software-defined radio platform and path without incurring a costly medium acquisition. We
simulations of 802.11n networks at scale demonstrate that name this overall cross-layer approach—in which a single
TCP/HACK significantly improves TCP throughput on transmission of feedback by a lower-layer protocol addi-
WiFi networks. tionally carries feedback from a higher-layer protocol—
Hierarchical ACKnowledgment (HACK). Though apply-
1 I NTRODUCTION ing HACK to carry TCP ACKs in LL ACKs is conceptu-
In today’s WiFi wireless networks, each time a sender ally quite simple, a robust design to do so must address
wishes to transmit, it must first sense the medium to be several systems challenges. In this paper, we describe and
idle for a randomly chosen interval. These random delays evaluate such a design, TCP-over-HACK (TCP/HACK).
desynchronize would-be concurrent senders. To use a Our contributions in this work include:
concrete example, Enhanced Distributed Channel Access • We offer an analysis of the capacity of the 802.11n
(EDCA) in 802.11n [1] enforces an average idle period MAC protocol for TCP traffic as function of bit-rate,
of 110.5 µs before a frame’s transmission, whereas a and the throughput gains theoretically achievable by
1500-byte payload itself lasts only 80 µs at 150 Mbps. avoiding medium acquisitions for TCP ACK packets.
Each frame’s link-layer acknowledgment (LL ACK) con- • We describe TCP/HACK, a scheme that increases the
sumes further channel capacity. As the physical-layer WiFi MAC’s efficiency by encapsulating TCP ACK
bit-rate increases but the pre-transmission idle period re- information in WiFi LL ACKs. TCP/HACK fully sup-
mains the same, this inefficiency only worsens. If a 600 ports 802.11n’s batching of data packets and use of LL
Mbps 802.11n sender sent single frames in this fashion, Block ACKs.
it would only achieve 9% of the theoretical channel ca- • We show how to efficiently encode the full range of
pacity. Moreover, WiFi senders back off exponentially TCP ACK information (e.g., timestamp options, re-
after a failed transmission, and so incur even longer mean ceiver window changes) within LL ACKs.
• We identify potential pathological interactions between
The research leading to these results has received funding under the TCP’s reliability and congestion control mechanisms
EU’s 7th Framework Programme, grant n◦ 317756, n◦ 287581, and and WiFi’s LL reliability protocol that would limit
European Research Council grant n◦ 279976. We gratefully acknowl-
edge a hardware donation from the Microsoft Research Software Radio system throughput, and ensure that TCP/HACK avoids
Academic Program. such interactions, without any changes to any node’s
30 500
TCP implementation. TCP/802.11a TCP/802.11n
TCP/HACK TCP/HACK
• We offer an interface between the network device driver 25 400
Goodput (Mbps)
software and the network interface card (NIC) hard- 20 300
packets. 5
6 912 18 24 36 48 54
0
0 100 200 300 400 500 600
• Through an evaluation in simulation of up to 10 com- Phy Rate (Mbps)
status quo WiFi MAC—were delayed ACK not used, a TCP receiver 2 802.11n’s parameter names and values differ slightly (e.g., AIFS in-
would generate twice as many ACK packets, and the WiFi MAC would stead of DIFS); the overall scheme of per-medium acquisition overhead
incur significantly more medium acquisitions. does not.
WiFi MAC. But if TCP/HACK is to enclose TCP ACK TCP/HACK that improve robustness and performance.
information in LL ACKs, the host TCP implementation Finally, we consider the constraints of real-world systems
cannot possibly generate a TCP ACK for a newly received software and NIC hardware, as well as of lossy wireless
TCP data packet within SIFS. To accommodate typical links, and flesh out the design of TCP/HACK into a fully
host protocol stack processing delays, TCP/HACK must practical system.
allow the TCP ACK for a newly received TCP data packet In the interest of brevity, we describe the design of
to be enclosed within the LL ACK for a different TCP TCP/HACK in the context of an 802.11 client acting as
packet received later. Yet it mustn’t unduly delay the a TCP receiver while downloading via an 802.11 AP.
return of an ACK to the TCP sender (see “cross-layer Throughout, we refer to this downloader as the “client.”
nuances” below). Note, however, that TCP/HACK is a fully symmetric
design—both the design and our implementation of it
Efficient encoding of general TCP ACK information
also work on TCP uploads by an 802.11 client.
The WiFi MAC reserves time on the wireless medium
for a LL ACK to return after a data packet, so that other 3.1 HACK in Overview
senders’ transmissions do not collide with the LL ACK.
It is important that TCP/HACK encode TCP ACKs in LL Let us first consider how TCP/HACK works in the sim-
ACKs efficiently, to minimize the period of medium occu- pler case of 802.11a, without batching of packets into
pancy for these lengthened LL ACKs. The encoding for A-MPDUs. When a regular TCP client receives a TCP
TCP ACKs must be compact yet allow the full generality data packet, its network stack generates a TCP ACK and
of information that may potentially be found in a TCP enqueues it for transmission by the WiFi NIC.
ACK, (e.g., TCP timestamp options, changes in receiver’s Under TCP/HACK, a client does not immediately en-
advertised window, &c.) all of which is important to the queue a TCP ACK for transmission. Instead, the client
correct and efficient operation of TCP. compresses each TCP ACK and appends them to a com-
pressed frame that it builds. When the next data packet
Simplicity of NIC modifications TCP/HACK should from the AP arrives, the client encapsulates the com-
not require any in-NIC intelligence about TCP packet pressed TCP ACK frame within the returning LL ACK,
headers or other TCP protocol details. Both at clients and effectively avoiding all medium acquisitions for the corre-
APs, all TCP-aware processing must occur in the host sponding TCP ACKs. The AP recognizes an “augmented”
software. We set this goal to minimize the complexity LL ACK, which it decompresses, reconstitutes the en-
and thus the cost of the NIC, but also because we would coded TCP ACKs, and forwards them upstream.
like HACK to generalize to other higher layers than TCP Now let us consider 802.11n, where data packets can
such as SCTP [10] or DCCP [5]: if the NIC treats the be aggregated into a single batched A-MPDU, and link-
feedback to be appended to an LL ACK as opaque bits layer ACKs take the form of a Block ACK that includes a
that it needn’t understand, then HACK should generalize bitmap indicating which packets from the A-MPDU were
in this way. received. On “stock” 802.11n during a TCP download the
No changes to TCP TCP changes are difficult to stan- normal repeating pattern will then be:
dardize and difficult to deploy, as many widely used 1. one A-MPDU from AP to client containing TCP data
OSes ship with a single closed-source TCP implemen- 2. one Block ACK from client to AP
tation. Both at clients and APs, HACK-related function- 3. one A-MPDU from client to AP containing TCP ACKs
ality should be confined to the WiFi NIC’s device driver 4. one Block ACK from AP to client
(which is bound to the NIC’s hardware design—i.e., NIC To eliminate medium acquisitions for TCP ACKs in
hardware that supported HACK would routinely ship with 802.11n, we would like a TCP/HACK client to encap-
a driver supporting HACK). sulate all the TCP ACKs generated in step 3 in the Block
ACK sent in step 2, and thus avoid step 4 entirely. In prac-
Avoid pathological cross-layer interactions Finally, tice, the arrival of an A-MPDU containing a batch of TCP
it is important to note that TCP relies on a stream of data packets will cause the client’s OS to generate a burst
TCP ACKs reaching the sender to maintain steady packet of TCP ACK packets in step 3 after the Block ACK has
transmissions by the sender (and thus high throughput). departed for that A-MPDU. These TCP ACKs arrive at
TCP/HACK must not disrupt the timely return of correct the client’s transmit queue where they are compressed and
TCP ACKs to the sender. concatenated, waiting for the arrival of the next A-MPDU
from the AP. The client will then append this compressed
3 HACK D ESIGN frame to the Block ACK it sends the AP in step 2.
We first offer an overview of TCP/HACK’s design. We Although the description above is for downloads, the
then explore nuances of the cross-layer interactions be- design is in fact symmetric; we envisage TCP/HACK as
tween TCP and 802.11n, which motivate refinements to especially useful for wireless backup to LAN-attached
AP Driver AP NIC Client NIC Client Driver as HACK packets and no vanilla TCP ACK packets need
to be sent. But what happens if no further data packets
4 3 2 1
Tx Queue
arrive? The client cannot retain the TCP ACKs for too
long, or it will cause the TCP sender to time out and
TCP Data 1, 2 retransmit. Thus, after some time period, the client must
A-MPDU (TCP
Data 1, 2)
TCP Data 1, 2 send uncompressed vanilla TCP ACKs in the normal way.
4 3 In Figure 2, TCP ACKs 3 and 4 meet this fate, and are in
Block ACK TCP ACKs
Tx Queue Comp. 1 1, 2 turn Block-ACKed.
Such a design may often squander the opportunity to use 1 Compress Comp.
TCP ACK
Tx Queue Descriptor
HACK, but it has the virtue of seeming simple—until
c_addr Dst Addr 1
one considers the complexity of the NIC-network driver
Dst Addr 2
interface needed to implement it. .
.
The M ORE DATA Bit In Figure 2, initially there are four .
data packets queued at the AP. When the AP forms the TCP ACK Cache
first batch containing TCP data packets 1 and 2, it already DMA DMA
knows more data will be sent to that client, as it already Client NIC 2
has packets 3 and 4 in its queue. So long as the AP has
Set TCP/HACK Ready
more packets queued than will fit in a batch, it knows Set Tx Ready
that it is safe for the client to save up compressed ACKs
waiting for the next batch. The AP simply tells the client Wireless Medium
that there is more data coming by setting the M ORE DATA
bit in the 802.11 header of the A-MPDU.5 When the client Figure 3: Client-side TCP/HACK compressing a TCP
sees this flag, it latches this state and will not transmit ACK, ready to be sent on the link-layer acknowledgment
any more non-encapsulated TCP ACKs until the next data of the next frame.
packet arrives, when it can use HACK to send them.
Client The client’s driver needs to determine when it
3.3 HACK in Practice can use TCP/HACK and when it must send TCP ACKs
normally. In Figure 2, on receiving packets 1 and 2, the
In the preceding section, we have presented a concep- client’s NIC also passes the M ORE DATA state to the
tual description of TCP/HACK, but several questions driver. The client TCP stack acknowledges the data, gen-
concerning the practicality of this conceptual design re- erating TCP ACKs 1 and 2, and puts them in the transmit
main unanswered. First, how realizable is TCP/HACK queue at point 1 .
given current systems and hardware? In particular, how
Figure 3 shows what happens at points 1 and 2 from
should TCP/HACK’s functionality be divided between
Figure 2 in more detail. If the driver is not in the M ORE
a station’s network interface card (NIC) hardware and
DATA state, it simply enqueues these ACKs normally.
NIC device driver? Finally, what manner of compression
However, if M ORE DATA is set, it compresses the arriving
should TCP/HACK employ to reliably encode the TCP
TCP ACKs and creates corresponding buffer descriptors.
ACKs?
A separate buffer descriptor chain per destination address
is needed to match compressed TCP ACKs with Block
3.3.1 Driver and NIC Functionality
ACKs for that destination.
We realize TCP/HACK (including the M ORE DATA mech- At point 2 the driver DMAs the buffer descriptor
anism) with very few changes to a station’s 802.11 NIC. chain to the NIC. The NIC maintains this table of com-
The main strategy is to implement the bulk of TCP/HACK pressed TCP ACK descriptors separately from normal
within the NIC’s driver, as we demonstrate using the ex- transmission descriptors. Finally, the driver sets a flag in
ample shown in Figure 2. Our discussion is in the context the NIC to indicate that TCP/HACK is ready.
of a modern Linux wireless driver, such as the Atheros Figure 4 shows what happens when the next batch from
ath9k driver.6 the AP arrives at the client. If the TCP/HACK flag indi-
cates “ready,” the NIC uses the corresponding descriptors
AP (data transmission) The only modification needed to DMA the compressed TCP ACK frames to the card.
to the AP when transmitting data packets is to set the It concatenates these frames, and appends them to the
M ORE DATA flag when there are more packets remaining returning Block ACK at point 3 . Recall that the NIC
in the transmit queue for the same client. normally fires an interrupt when it receives data packets.
5 This bit exists in stock 802.11 to assist with power saving. HACK In this case, the interrupt must also indicate whether the
uses this bit irrespective of whether power saving is enabled. NIC succeeded in sending the compressed ACKs.
6 http://wireless.kernel.org/en/users/Drivers/ath9k This design also copes with the race condition where
the batch carrying packets 3 and 4 arrives with the M ORE Client Driver
DATA flag not set before the driver has succeeded in con- Rx Interrupt
More Data State
veying compressed TCP ACKs 1 and 2 to the NIC. In TCP/HACK Success State
this case, the TCP/HACK “ready” check will fail. The DMA
Client NIC
NIC sends a normal Block ACK and signals to the driver
a TCP/HACK failure in the receive interrupt. The driver
now is free to re-enqueue the TCP ACKs on the transmit Compressed TCP ACKs
4
3
4
✘ ✘ ✘ 8
5
Comp. Ack Block AC
Comp. Ack
State cleared K request
State cleared
Ack 4 123
ck Ack Block Compressed
Blo
TCP ACKs
CK +
(a) (b) Duplicates: Failure Block A TCP (not cleared)
p res s ed
to decompress com , 2, 3
K s 1
AC
Figure 5: Coping with loss of (a) Block ACKs and (b)
single LL ACKs by retaining TCP ACK state.
Figure 6: Retaining state: gap in sequence space.
Both clients
T Client 2
...
3 4 Block AC U
K request
✘ K+ T
Block AC ✘ CK +
C K s 1, 2 Block A 1, 2 H
TC P A C K s
TCP A
U
One Client
Block AC
K request
4 5
no more d T
✘ ata + syn
CK + c H
Block A 1, 2
C s
K CK + U
TCP A Block A 1, 2
C K s
TCP A
...
0 5 10 15 20 25
(continued right) Mean Goodput Mbps
Figure 8: SYNC bit for retaining state. Figure 9: TCP throughput with stock 802.11a (T), TCP
with HACK (H), and UDP (U) with stock 802.11a, with 1
repeatedly? Under normal 802.11n operation, the AP will and 2 clients.
continue to send Block ACK requests until it hits the retry
limit, when it will give up and send the next batch of data. are slightly lower.
The client does not know that the AP has failed to receive Testbed Our three wireless nodes each have four-core
the compressed TCP ACKs and, when it sees new data, it Intel Core i7 CPUs, between 8–24 GB of RAM, and a
would normally discard the previously retained TCP ACK PCI Express SoRa radio control board. One acts as the
state. In this case, the AP explicitly notifies the client that AP and the other two act as clients. We operate the SoRa
it has moved on by setting a SYNC bit in the next batch’s interfaces in ad hoc mode to eliminate periodic beacon
header. Upon seeing this bit set, the client doesn’t discard transmission. We run experiments on 802.11g channel 14
the compressed TCP ACKs but rather appends them to (2.484 GHz) in an open-plan office environment. We use
the next Block ACK, as shown in Figure 8. iperf to generate TCP data streams with a 1500 byte MTU
and send at 54 Mbps, the highest 802.11a rate.
4 E VALUATION
4.2 SoRa Results
We evaluate TCP/HACK through a combination of sim-
ulation in ns-3 and experiments with a real-world imple- Besides demonstrating a successful implementation as
mentation for the SoRa software-defined radio platform. evidence of TCP/HACK’s practicality, we wish to answer
We simulate TCP/HACK for 802.11n in ns-3, while our several questions experimentally:
SoRa implementation is for 802.11a, as the public SoRa • Are TCP/HACK’s capacity benefits in line with theo-
release does not support 802.11n. retical predictions?
• When an AP sends TCP flows to two clients, does TCP
4.1 SoRa Implementation over 802.11a suffer collisions between clients’ TCP
ACKs, and if so, does TCP/HACK offer a performance
We implemented TCP/HACK including the M ORE DATA
benefit partly by eliminating such collisions?
bit and ROHC compression for the SoRa user-level phys-
• Do TCP/HACK’s benefits come only from eliminating
ical layer on Windows 7. Hardware limitations of our
channel acquisitions and collisions, or are there other
SoRa radio boards require us to run 802.11a in the 2.4
overheads that TCP/HACK eliminates?
GHz band, but this does not affect protocol behavior.
One quirk of the SoRa platform bears mention. We Baseline Comparison Figure 9 compares the application-
have found that SoRa receivers sometimes return 802.11 level throughput achieved by TCP/802.11a and
link-layer ACKs later than the 802.11 specification’s ACK TCP/HACK for bulk downloads, with UDP/802.11a
timeout interval, causing spurious link-layer retransmits for comparison. Each bar shows a different experiment:
and backoffs. To avoid this performance hit, we increased sending to one or both clients, using TCP over HACK,
the 802.11 ACK timeout to accommodate SoRa’s late TCP over stock 802.11a or, as a control experiment,
LL ACKs. The net effect of these delayed LL ACKs is unidirectional UDP, which gives an upper bound on
that at 54 Mbps, our SoRa implementation only achieves usable capacity. The data is the mean over five different
87% of the theoretical throughput across all protocols. We 120-second runs; error bars show standard deviation.
confirmed though simulation that this change does not Client 1’s throughput is slightly less than Client 2’s be-
significantly affect the relative benefit of TCP/HACK over cause it suffers a greater packet loss rate, even when only
regular 802.11a, but the absolute performance numbers one flow is active. UDP’s unidirectional data minimizes
medium acquisitions, and achieves the greatest through- To understand other contributing factors in more detail,
put possible on SoRa with link-layer ACKs enabled. In we ran an experiment where the AP transmits 25 Mbytes
an ideal 802.11 MAC, UDP would achieve 30.2 Mbps; of data to a client using TCP/802.11 and TCP/HACK. By
on SoRa, UDP averages 26.5 Mbps across the three ex- fixing the amount of work we can compare both protocols
periments. SoRa’s link-layer ACK delays alone reduce in time. The first two columns of Table 2 show the number
the attainable throughput to 28.1 Mbps, and our UDP of TCP ACKs sent as well as how many bytes were in
measurements approach that figure. those ACKs. The next two columns show the same figures
If TCP/HACK encapsulated all TCP ACKs in LL for compressed ACKs, and the last column shows the
ACKs, it would achieve almost the same throughput as compression rates ROHC acheives.
UDP (though UDP’s packet headers are smaller). In prac- Reducing the number of transferred bytes is beneficial,
tice, TCP/HACK’s single-client throughput of 25.0 Mbps but TCP ACKs are treated as regular data when sending
(mean of C1 and C2) is very close to the UDP bench- over 802.11 wireless links and are sent at 54 Mbit/s in
mark. TCP/802.11a only achieves 19.4 Mbps in this sce- our experiments. LL ACKs, however, use the more robust
nario. TCP/HACK improves performance by 29% and 24 Mbit/s rate. To factor this in, we investigate how saved
32.2% in the one- and two-client cases respectively. Both bytes translate into saved transmission time, together with
TCP/HACK and TCP/802.11a are fair. TCP/HACK’s impact on channel acquisition time and
retransmission time.
UDP/ TCP/ TCP/
802.11a HACK 802.11a TCP Acquire LL ACK
no retries 99% 97% 87% ACK ROHC Channel overhead
Client 1
1 or more 1% 3% 13% TCP/802.11a 70 ms 0 1093 ms 456 ms
no retries 99% 98% 88% TCP/HACK 0.08 ms 13.1 ms 1.17 ms 0.46 ms
Client 2
1 or more 1% 2% 12% Table 3: TCP ACK time overhead breakdown for
no retries 99% 98% 86%
Both TCP/802.11 and TCP/HACK.
1 or more 1% 2% 14%
Table 1: Percentage of frames successfully sent on the Table 3 shows time taken to send TCP ACKs (TCP
first attempt (no retries) and after one or more retries, ACK), time to send compressed TCP ACKs (ROHC),
when the AP is sending to Client 1 and Client 2 alone, time spent waiting for channel before transmitting TCP
and both clients at the same time, using UDP/802.11a, ACKs (Channel) and extra time waiting for LL ACKs (LL
TCP/HACK, and TCP/802.11a. ACK overhead). From the table, we see that most savings
come from channel acquisition and LL ACK overhead.
Ideally LL ACKs are returned immediately after a SIFS
Where do TCP/HACK’s savings come from? time, but this is not always the case in the real 802.11 im-
We note with interest that TCP/HACK improves through- plementations. On SoRa we observe 37 µs on average
put more than predicted analytically in Section 2.1. That of additional LL ACK overhead, while on two differ-
prediction focused solely on saving medium acquisitions ent commercially-available wireless NICs (the Atheros
for TCP ACKs. In Table 1 we show the percentage of AR9300 and the Intel 5300) we measure 10.4-13.4 µs of
frames received after the first transmission, and the per- LL ACK overhead, on average. While TCP/HACK ben-
centage that required one or more retransmissions. We efits more from saving ACK overhead on SoRa than on
see that TCP/802.11a experiences far more link-layer re- the commercial cards, the benefit on commercial wireless
transmissions than TCP/HACK or UDP/802.11a. These hardware is still large. TCP/HACK not only eliminates
retransmissions occur because of collisions between TCP channel overheads, it also reduces collisions and any ad-
ACKs sent by clients and TCP data packets sent by the ditional LL ACK overheads incurred by the device.
AP. TCP/HACK obviates most (but not all) of these TCP SoRa and ns-3 Cross-Validation
ACKs, and so significantly reduces the number of retrans-
missions needed. TCP/HACK not only eliminates costly To cross-validate our SoRa implementation against the ns-
channel acquisition overheads, but by encapsulating TCP 3 simulator, we simulated 802.11a in ns-3 with the same
ACKs in LL ACKs, also incurs fewer collisions. packet loss rate as that observed on SoRa (12% and 2%
for TCP/802.11a and TCP/HACK, respectively). Since
ACK ACK ACKC ACKC Comp. ns-3 returns LL ACKs immediately after SIFS, whereas
count bytes count bytes ratio SoRa incurs additional delay, ns-3 running TCP/802.11a
TCP/802.11a 9060 471120 0 0 (1)
achieves 22.4 Mbit/s vs. SoRa’s 19.6 Mbit/s. After post-
TCP/HACK 10 520 9050 39478 12
processing to eliminate SoRa’s added LL ACK delay, we
Table 2: Conventional and compressed ACK counts, and observe SoRa throughput of 22 Mbit/s, which matches
compression rates of ROHC-compressed ACKs. simulation. Similarly, when simulating TCP/HACK in
ns-3, we get 28 Mbit/s vs. SoRa’s 25.5 Mbit/s. After ac-
UDP
counting for SoRa’s extra LL ACK delay, SoRa achieves
10 clients TCP/HACK More Data
27.7 Mbit/s, which matches simulation. TCP/Opp. HACK
TCP/802.11
4.3 Simulation Results
4 clients
We now examine how TCP/HACK interacts with frame
aggregation, with a larger number of clients than possi-
ble in our testbed. To this end, we implement A-MPDU
support and TCP/HACK in ns-3. We evaluate both the op- 2 clients
portunistic and M ORE DATA variants of HACK described
in Section 3.2 to verify that the the latter outperforms the
former as hypothesized. 1 client
We simulate multiple WiFi clients scattered randomly
within a circle of 10-meter radius centered on the AP. 0 20 40 60 80 100 120 140
Mean Aggregate Goodput (Mbps)
Our aim is to model the scenario where several clients
connect via 802.11n WiFi to a server located nearby on a
Figure 10: TCP goodput for different transmission
high-speed LAN. We present results modeling an 802.11n
schemes with 1–10 clients, and UDP for comparison.
single-antenna setup using data packet and link-layer
ACK bit-rates of 150 Mbps and 24 Mbps, respectively.
These collisions account for the lower measured goodput
The wired link between the server and the AP has a la-
than that predicted in Section 2.1.
tency of one millisecond and a bit-rate of 500 Mbps.
We note with surprise that Opportunistic TCP/HACK
To glean the benefits of the M ORE DATA scheme, we
does not significantly outperform TCP/802.11n: this most
would like AP’s transmit queue to contain at least 126
naı̈ve implementation of HACK sends few compressed
packets per flow. We choose this number so that the AP
TCP ACKs in LL ACKs, and mostly regular TCP ACKs.
may buffer of up to three batches of 42 packets per client,
It therefore does not achieve a TCP goodput closer to the
accounting for some variability in the A-MPDU size in
physical rate.
the presence of TCP retransmissions. To avoid adverse
“buffer bloat” effects [3], the transmit queue should not Role of M ORE DATA Bits We now turn our attention to
be too large in the case of one flow, but rather grow as the the bars labeled “TCP/HACK More Data” in Figure 10.
number of flows increases. A large buffer in our system We observe that the M ORE DATA variant of TCP/HACK
would cause an excessive loss of packets when slow start achieves the most pronounced throughput gain over un-
overflows the buffer, with or without TCP/HACK. With modified 802.11n. While simple, the M ORE DATA mech-
ten clients, the AP’s transmit queue would be 1260, which anism is crucial to TCP/HACK’s success in reducing
is reasonable since Linux drivers usually use buffer sizes medium acquisitions, and gives rise to goodput improve-
of 1000 packets. ments between 15% for one client and 22% for ten clients
TCP/HACK vs. TCP/802.11n To determine the benefit at the physical rate of 150 Mbps.
of TCP/HACK and its constituent parts, we compute the Lossy Environment We next evaluate TCP/HACK un-
aggregate goodput for TCP flows sending 1460 byte pack- der different SNR regimes. In addition to providing a
ets, averaged across five simulated runs per experiment. wider spectrum of comparison between TCP/HACK and
To mitigate phase effects with multiple clients, we stagger TCP/802.11n, these experiments will verify whether the
the starts of clients’ downloads. As such, we compute HACK protocol with the properties described in Sec-
the aggregate goodput over the steady-state portion of the tion 3.4 can indeed avoid any decompression CRC fail-
runs, once all the clients have more or less exited slow ures, or stalls due to recurring TCP timeouts.
start. We begin with a setup similar to that described above,
Figure 10 shows that UDP maintains a roughly con- and then place a single client at varying distances from the
stant goodput as the number of downloading clients AP in order to simulate a decreasing set of SNRs. In lieu
varies, as expected. As a unidirectional protocol, UDP’s of simulating bit rate adaptation explicitly, at each partic-
performance is minimally affected by the number of ular distance we simulate a download of a 100 MB file at
clients competing for the link. In contrast, the goodput of a rate selected from a range of 802.11n high throughput
TCP/802.11n decreases slightly as the number of down- rates. This range corresponds to rates which are achiev-
loading clients increases. Although the AP elicits TCP able using a 40 MHz channel, 400 ns guard interval and
ACK packets from clients in turn, there is still a chance one antenna. The corresponding LL ACK rates are chosen
that two or more clients’ TCP ACKs can collide, or that from the set of basic rates (6, 12 and 24 Mbps) accord-
a TCP ACK can collide with a data packet from the AP. ing to the rules outlined in the 802.11n specification. To
140 140
TCP/HACK Theor. TCP/HACK
150 Mbps
Average Goodput (Mbps)
TCP/802.11
120 135 Mbps Sim. TCP/HACK
120
60 80
60 Mbps
40 45 Mbps 60
30 Mbps
20
15 Mbps 40
0
% Improvement
20
20
10 0
0 50 100 150
0 Phy Rate (Mbps)
0 5 10 15 20 25 30
SNR
Figure 12: Theoretical and simulated TCP goodputs vs
Figure 11: Envelope of average TCP goodput for 802.11n physical rates.
TCP/HACK and TCP/802.11n under different SNR
and plot these and the analytical predictions in Figure 12.
regimes and physical rates. The lower graph shows
As we expect, simulated goodputs are lower than the cor-
TCP/HACK’s percent improvements over TCP/802.11n.
responding analytical predictions—the predictions do not
emulate a real system, we applied the 4 ms transmit op- model 802.11n collisions or retries, nor do they take into
portunity limit to all transmissions, therefore limiting the account TCP’s retransmissions and congestion control.
size of A-MPDU packets for experiments using lower Note, however, that the goodput improvement
physical rates. At each distance/physical rate combina- TCP/HACK offers over TCP/802.11n exceeds that pre-
tion, we computed the average TCP goodput (including dicted analytically. Since TCP/802.11n suffers more from
slow start) over five runs. collisions than TCP/HACK, its throughput suffers cor-
Figure 11 shows the average TCP goodput for respondingly more. TCP/HACK greatly reduces the col-
TCP/HACK and TCP/802.11n. It plots a separate dashed lision rate by eliminating medium acquisitions for TCP
curve per 802.11n physical rate for TCP/HACK. We use ACK packets. At 150 Mbps, TCP/HACK offers a simu-
these curves to compute the envelope (in black), which lated goodput improvement of 14%, vs. the 7% improve-
indicates the best goodput achievable by an ideal bit rate ment predicted analytically.
adaptation algorithm. Similarly we plot the correspond-
ing envelope for regular TCP/802.11n (the separate rate 5 D ISCUSSION
curves for TCP/802.11n are not shown). Both batching using A-MPDUs and TCP/HACK help to
Our simulations indicate that TCP/HACK functions reduce the time wasted on unnecessary WiFi medium
correctly in a lossy environment and does not elicit any acquistions. TCP/HACK relies on the M ORE DATA bit
decompression CRC failures. Moreover, TCP/HACK im- to know when it is safe to compress ACKs and wait
proves TCP goodput by an average of 12.6% across the for another packet on whose LL ACK to piggyback. A-
range of SNR values. Figure 11 shows that as the physical MPDUs require sufficient packets in the AP’s queue to
rate drops, the relative improvement increases slightly for gain efficiencies. With sufficient buffering at the AP and a
the cases where the transmit opportunity limit reduces the large window, both work well. In such cases the wireless
number of packets a station can possibly transmit in an ag- medium is busy, and efficiency is important. TCP/HACK
gregate. Recall that 802.11n uses aggregation to amortize can significantly reduce collisions when there are multiple
medium access costs, therefore we expect a better good- senders by turning bidirectional TCP flows into unidirec-
put gain for TCP/HACK over regular TCP at these rates. tional ones, reducing the number of contending hosts.
Similarly, as the physical rate increases past 90 Mbps, However, if the traffic patterns are such that queues do not
the overall improvement increases slightly to about 14%, build in the AP or clients, there won’t be enough packets
because the 802.11n medium access delays now consume to fill A-MPDUs or any remaining packets in the queue to
a larger portion of the transmission time relative to data. allow the M ORE DATA bit to be set. Neither mechanism
Analytical Predictions vs. Simulations How well does will work well in this case. Similarly, if an AP has very
the average TCP goodput measured in simulation match many clients, it may not buffer enough packets for each
that computed analytically in Section 2.1? We extract the client for either mechanism to work well.
highest achievable goodput at each physical rate for both Longer batches improve utilization, but monopolize
TCP/802.11 and TCP/HACK from the prior experiment, the medium for longer. 802.11e allows the AP to reduce
medium acquisition latency by specifying a shorter max- for the data packet just sent to the client” or “no ACK
imum batch duration through the transmit opportunity for the data packet just sent to the client.” As a result,
limit. In such cases, we would expect TCP/HACK to help these designs prevent the delivery of duplicate ACKs to
claw back some of the efficiency loss caused by limiting the TCP sender, and prevent the use of fast retransmit,
the maximum batch duration. leaving only inefficient TCP timeouts. Furthermore, this
Sending a TCP timestamp option in the last TCP ACK work took place before the introduction of 802.11n, and
of a batch would generalize the M ORE DATA mechanism. as a result, does not consider the interaction with frame
The TCP sender would echo it, and the client could use aggregation or block ACKs.
receipt of the echo as an implicit ACK-of-ACK. When
the client hasn’t yet received a timestamp echo, it can 7 C ONCLUSION
reasonably expect further data to arrive, and thus delay In this paper, we have described the design and imple-
sending TCP ACKs. We leave this for future work. mentation of TCP/HACK, a cross-layer acknowledgment
design for TCP and the 802.11 MAC that eliminates most
6 R ELATED W ORK of the expensive medium acquisitions that TCP ACK
One approach to amortizing medium acquisition over- packets require, significantly increasing TCP flows’ wire-
head across more data is narrow-band channelization. less throughput. TCP/HACK improves throughput further
Since the effective data rate on each subband is much when used with frame aggregation, yet offers significant
lower than that of wide-band 802.11, the time required throughput improvements without it. While frame aggre-
for MAC-layer contention becomes smaller relative to gation and other previous approaches reduce the cost of
the packet transmission time on a single subband, thus individual medium acquisitions [11, 2, 6], TCP/HACK
more effectively amortizing medium acquisition over- eschews many medium acquisitions entirely. It is thus
head across multiple packet transmissions. FICA [11] and complementary to these prior approaches. Our evalua-
WiFi-NC [2] take this approach. Both require redesigns tions in simulation and in a real-world implementation
of the physical and MAC layers. TCP/HACK is comple- confirm TCP/HACK’s throughput improvements.
mentary: combining the two systems should yield greater
R EFERENCES
medium efficiency than either system achieves alone.
WiFi-Nano [6] shortens the 802.11 contention slot time [1] IEEE Standard 802.11-2012. Mar. 2012.
to 800 ns. TCP/HACK is again complementary: while [2] K. Chintalapudi, B. Radunovic, V. Balan, M. Buet-
WiFi-Nano reduces medium acquisition overhead, our tener, S. Yerramalli, V. Navda, and R. Ramjee. WiFi-
proposal eliminates many medium acquisitions entirely. NC: WiFi over narrow channels. In NSDI, Apr 2012.
Maranello [4] is a link-layer design for 802.11 wireless [3] J. Gettys and K. Nichols. Bufferbloat: Dark buffers
networks that incorporates sub-frame granularity check- in the Internet. CACM, 55(1), 2012.
sums into link-layer acknowledgments, allowing the com- [4] B. Han et al. Maranello: Practical Partial Packet
municating pair to undertake partial packet recovery on Recovery for 802.11. In NSDI, 2010.
corrupted frames. Unlike TCP/HACK, Han et al. imple- [5] E. Kohler, M. Handley, and S. Floyd. Designing
ment Maranello partially on the firmware processor of a DCCP: Congestion control without reliability. In
commodity 802.11 NIC, thus requiring access to the as- SIGCOMM, 2006.
[6] E. Magistretti, K. Chintalapudi, B. Radunovic, and
sembly source code of the firmware processor. While
R. Ramjee. WiFi-Nano: Reclaiming WiFi efficiency
Maranello does not share the same networking goals
through 800 ns slots. In MobiCom, 2011.
as TCP/HACK, it does share systems context in terms
[7] Q. Pang, S. Liew, and V. Leung. Performance im-
of the hardware and software available to both designs.
provement of 802.11 wireless network with TCP
Like Maranello, TCP/HACK is realizable with very few
ACK agent and auto-zoom backoff algorithm. In
changes to the NIC itself.
Proc. IEEE VTC, June 2005.
Of prior work in reducing channel acquisition over-
[8] G. Pelletier, K. Sandlund, L.-E. Jonsson, and
head, Pang et al. [7] most closely resembles TCP/HACK,
M. West. RObust Header Compression (ROHC):
proposing that a client use a MAC-layer ACK to signal
A Profile for TCP/IP. RFC 6846, Jan 2013.
successful reception of TCP data. However, the designs [9] R. Rivest. The MD5 Message-Digest Algorithm.
they propose are only capable of communicating to the RFC 1321, April 1992.
AP when a client observes a TCP ACK for the same data [10] R. Stewart. Stream control transmission protocol.
packet just received. The authors do not mention the pos- RFC 4960, Sept. 2007.
sibility of the generation of a TCP ACK with a lower [11] K. Tan, J. Fang, Y. Zhang, S. Chen, L. Shi, J. Zhang,
ACK number after a loss, and the link-layer feedback and Y. Zhang. Fine-grained channel access in wire-
mechanism they propose is incapable of communicating less LAN. In SIGCOMM, 2010.
any information to the AP other than “cumulative ACK