Virtual Speed Test:

an AP Tool for Passive Analysis of Wireless LANs

Peshal Nayak ∗ , Santosh Pandey † , Edward W. Knightly ∗
∗ Rice University, USA, † Cisco Systems, USA

Abstract—Internet speed tests assess end-to-end network per- observable at the AP. We call our measurement based frame-
formance by measuring throughput for 10s of MB of TCP work virtual speed test. The speed test results obtained by
uploads and downloads. While such tests provide valuable a STA can vary over time depending on numerous factors
insights into network health, they are of little use to network
administrators since (1) the results are only available on the such as the number of active STAs, interference level, etc.
client that performs the test and (2) the tests can saturate Likewise, virtual speed test can enable the network manager to
the network, increasing load and worsening performance for dynamically track any given STA’s speed test results based on
other clients. In this paper, we present virtual speed test, a its own unique characteristics (e.g., via a dynamic dashboard).
measurement based framework that enables an AP to estimate Virtual speed test employs a novel L2 edge TCP model
speed test results for any of its associated clients without any
special-purpose probing, with zero end-user co-operation and to perform throughput estimation. The key challenge for the
purely based on passively observable parameters at the AP. AP to estimate these inherently bi-directional, end-to-end and
We implemented virtual speed test using commodity hardware, layer-4 throughputs, is that the AP only has a limited view
deployed it in office and residential environments, and conducted of the network. Since the AP is unaware of the presence of
measurements spanning multiple days having different network hidden terminals, interference from neighboring BSS to the
loads and channel conditions. Overall, virtual speed test has mean
estimation error less than 6% compared to ground truth speed STAs, etc. (which affect the STA’s queuing delays, NAV timers
tests, yet with zero overhead, and outcomes available at the AP. and packet retransmissions), the AP cannot estimate how long
it takes a STA to successfully transmit after it starts to attempt.
I. I NTRODUCTION Our design is motivated by the fact that since the WLAN is
the final hop for any TCP segment directed towards a STA,
TCP speed tests are end-to-end tests of network health and this duration can also be estimated by measuring the delay
are available via a plethora of online apps [1], [2], [3]. As part incurred between the transmission of a TCP segment on the
of the measurement process, a client performs an active TCP downlink to the reception of the corresponding TCP ACK on
download and an active TCP upload to a server to measure the the uplink from the STA. This TCP segment, therefore, can
download and upload TCP throughput respectively. Since more belong to any TCP flow (e.g., a Netflix video stream) and need
than 80% of current Internet traffic is transmitted via TCP [4], not be a part of a flow from a nearby server. To carry out these
the performance of numerous online applications is crucially measurements, the AP must identify TCP flows. To this end,
dependent on the maximum TCP throughput achievable over we leverage TCP’s inherent bi-directionality and packet size
an underlying network path. signatures to spot TCP flows. Specifically the fact that TCP
If a client’s speed test uses a nearby server (i.e., a server with flows involve TCP segment traversing on the forward path and
minimum possible latency to the AP), the WLAN becomes small sized TCP ACKs on the reverse path enables the AP to
the key part of the end-to-end path. Consequently, the results identify these flows and perform its measurements.
would be valuable to the network manager to assess WLAN Second, we develop a virtual speed test enabled AP (VST
performance and make decisions on network infrastructure AP) by using commodity hardware. We build APIs that enable
alterations to improve the quality of service experienced by the VST AP to passively collect a number of per packet
the end user. However, the results can only be seen by the end statistics and feed them into the L2 edge TCP model to
user and are unavailable to the administrator without seeking obtain throughput estimates. While virtual speed test does
end user co-operation. Moreover, regularly performing such not require collection of STA-side statistics, for validation
speed tests imposes additional traffic load on the network and purposes, we also implement APIs for data collection from
hence doing so can potentially disrupt user traffic and drain STAs associated with the VST AP for characterizing the
the battery of mobile devices. operating environment and for ground truth measurement.
In this paper, we make the following contributions. First, We deploy VST AP in two environments: an office located
we present a framework that enables an AP to estimate the inside a university building and an apartment in a residential
outcome of a speed test, i.e., the upload and download TCP complex. The VST AP is deployed in the office for a period
throughputs that any of its associated STAs should obtain of 2 days and in the apartment for a period of 7 days. Both
from a nearby server, yet, without any special-purpose probing, deployment settings are characterized by interference from
with zero co-operation of endpoints (i.e., the server and the non-BSS devices co-existing in the same frequency band,
client), and solely based on measurements that are passively human mobility and link diversity with respect to signal
 may or may not be in range of other APs. Likewise, STAs may
be hidden from each other or mutually in range. It is further
possible that a STA is in range of other APs which are not
in range of the AP that is serving it. The interference and
contention possibilities are further compounded by the need
Co-managed AP 1 Co-managed AP 2 to consider both downlink transmissions (AP to STA), uplink
transmissions (STA to AP), and mixes.
We do not make any assumptions about the PHY layer
capabilities of the AP or the STAs. For instance, the AP may
Non-managed AP have advanced physical layer capabilities such as multi-user
Fig. 1: Enterprise WLAN scenario: bold lines indicate connectivity while MIMO. Likewise, the AP can have any channelization strategy,
dotted lines indicate interference. e.g., dynamically bonding channels to 80 MHz as available.
propagation (i.e., LoS vs non-LoS paths) and supported PHY
B. Background on TCP upload and download speed test
rates. The office and the residential scenario cover a total
of 36 and 49 topologies respectively with a varying number Speed tests measure a client’s upload and download TCP
of STAs. Overall, the VST AP observes a total of 113,047 throughput from a server on the internet.1 If the speed test
TCP flows across both deployments. These TCP flows result happens from a nearby server or low latency server, the WLAN
from multiple applications running on end devices such as becomes the key part of this end-to-end path and the network
video streaming, music streaming, pdf downloads and email manager can use these results to assess WLAN performance.
activities. For validation, actual client-based speed tests are For the remainder of the paper, we focus on speed tests that
employed as ground truth. Virtual speed test demonstrates a happen from a nearby server. A speed test is user initiated and
high level of estimation accuracy compared with ground truth, the results are visible to the user at the end of the measurement.
with average estimation error under 6% for both upload and Speed tests primarily consist of two phases: a setup phase
download speed estimation. during which the speed test parameters are configured and a
Finally, we implement virtual speed test into ns-3’s source measurement phase which involves an active TCP upload and
code and perform extensive simulations to investigate operat- download.
ing conditions beyond those encountered in our field trials. Setup phase. The setup phase begins with a server selection
The simulation results concur with field trial conclusions process which can either be manual or app driven. If this
demonstrating estimation errors below 5%. is app driven, a server is selected by probing a pool of
To the best of our knowledge, virtual speed test is the first to available servers such that the backbone delay between the
estimate both upload and download TCP throughputs of STAs server and the AP is as less as possible to ensure a maximum
in the network by using passive measurement metrics at only TCP throughput [5] (with the ideal case being a server in
the access point, i.e., without any active probing, additional the same LAN as the AP). Since the goal is to measure
hardware infrastructure or user participation. the maximum TCP throughput, while running a speed test,
a STA is recommended to turn off other applications. Next,
II. V IRTUAL SPEED TEST: SCENARIO DESCRIPTION AND the client and server side TCP parameters are configured. The
PROBLEM FORMULATION exact mechanism used for performing this configuration differs
from one speed test application to another. A commonly used
A. Enterprise WLAN setup
procedure is to conduct a test download and a test upload
We consider an enterprise WLAN environment such as from the STA. For instance, for the Ookla speed test, the STA
illustrated in Fig. 1. As depicted, the network comprises of initially downloads or uploads a small file to estimate initial
multiple APs. While the network may use channelization, for throughput. Following this initial phase, the STA adjusts the
ease of exposition we consider only APs with at least partially file size, buffer size and number of parallel TCP flows (limited
overlapping channels such that they can potentially interfere to maximum of 8) to maximize the network connection usage
with each other. Moreover, we consider that in addition to while preventing congestion during the measurement phase
the managed infrastructure, there may be one or more non- [6].
managed WLANs that may be interfering. Such WLANs can Measurement phase. As shown in Fig. 2, the measurement
correspond to an LTE hot spot or a neighboring WLAN under phase consists of two sessions: an upload session and a down-
different administrative control. load session. The vast majority of speed test apps available
Ideally, all such networks should have sufficient physical online follow a flooding based mechanism in these sessions
separation to enable full spatial reuse for each AP (i.e., simul- which involves establishment of several parallel TCP flows
taneous transmission for each network). However, as depicted, between the server and a STA with a calculation of aggregate
the unwanted interconnectivity creates interference and con- throughput across all the flows [7]. This ensures that the results
tention among nodes. Moreover, inter-node connectivity can are robust to a small TCP window size (e.g., due to loss of a
form a complex relationship: while all STAs are necessarily
connected to the APs that they associate with, a particular STA 1 Unless stated explicitly, the terms client and STA are used interchangeably.
 wired wireless While the above may appear to be an exhaustive set
Server 3
STA3 of information for performance characterization, there are a
Backbone number of STA-side metrics that remain unobservable by the
Network AP
STA2
Server 2
AP. For instance, the AP does not know the STA’s idle times
TCP download
TCP upload or the STA’s defer times due to NAV especially when the STA
Server 1 STA1 is deferring to a non-BSS device. Since the network scenario
download
upload throughput throughput considers a complex inter-node connectivity which may lead to
Fig. 2: Illustration of an upload and download speed test: here STA 1 performs inter-cell interference, hidden terminals, etc., these parameters
a speed test. Server 1 is selected from a pool of servers consisting of server cannot be directly calculated based on the metrics mentioned
1, 2 and 3. Here server 1 has minimum backbone delay to the AP.
above. However, the throughputs that we want the AP to
TCP segment or a small advertised receive window size) [8]. estimate are inherently bi-directional, end-to-end and layer-
The number of parallel flows to be established is determined 4 and can indeed be degraded by the above factors. To this
in the setup phase. During the upload session, a STA performs end, we infer the impact of these unknowns using the above
an active upload to the selected server and measures the TCP AP-side observables with the help of techniques described in
throughput by averaging the total data transmitted end-to-end Section IV.
over the total time taken. During the download session, the
STA performs an active download and measures throughput III. L2 EDGE TCP MODEL
in a similar fashion. To enable an AP to estimate the upload and download
C. Virtual Speed Test: High level problem definition throughputs that a STA would obtain if it performs a speed
test, we develop a novel L2 edge TCP model that uses AP-side
Analogous to online speed tests, our goal is to realize a observables as inputs.
virtual speed test that enables an AP to estimate the TCP
download and upload throughput that a STA can achieve from A. Assumptions for mathematical analysis
a nearby server. As described in our network scenario, an
AP can have an arbitrary number of STAs associated with Here, we state the key assumptions that we make to capture
it and the AP should be able to estimate the throughput important aspects of the aforementioned speed test setup and
for any of the associated STAs. The speed test results of a measurement phases in our model.
STA can vary as driven by factors such as number of active In the measurement phase of an actual speed test, multiple
STAs, interference level, etc. Likewise, we target that virtual TCP flows are initiated between the server and the client so
speed test also tracks the speed test results for a given STA that the measurement phase is not bottlenecked by the number
based on its own unique characteristics. Note that the STA of circulating TCP segments. Instead, we model this by
does not perform the actual speed test. The AP is required to representing the packet flow dynamics by a single long lived
make the prediction using only passively collected information TCP flow with a maximum congestion window size of Wm
available on the AP side whereas no reports are available from which is large enough so that there are a sufficient number of
STAs and out-of-network APs. Further, we consider that no TCP segments circulating in the network. We further assume
additional commands can be required of STAs, e.g., STAs that this flow does not experience any permanent packet losses.
cannot be requested to send packets for testing purposes. This is not to say that collisions or packet errors do not occur
Moreover, STAs cannot be requested to download special on the wireless channel. Rather, packets lost on the wireless
purpose software or report STA-side measurements. Instead, channel are locally retransmitted by the MAC layer and we
we consider that by leveraging AP-side observables, the AP do account for these collisions and retransmissions in our
can estimate the following metrics [9]. analysis. We hereby refer to this modeled flow as the speed
Aggregate AP metrics. We consider that the AP can test flow, the STA under consideration as the target STA and
measure the airtime usage due to transmission and reception, the remaining STAs as non-target STAs.
defer time, contention time, idle time (no backlogged downlink In an actual speed test, the server selection process in the
traffic) as well as byte counts for downlink and uplink frames. setup phase selects a server with minimum latency to the AP
Per-STA metrics. Likewise, while the STAs do not report to reduce the impact of backbone elements on the measured
STA-side statistics, the AP can observe some per-STA metrics results. Consequently, we consider backbone congestion and
at the AP such as uplink RSSI and SNR, downlink and uplink delays as factors that do not impact the throughput. Also, recall
MCS and PHY parameters including use of advanced PHY that the parameters of the TCP flow used during the speed
features such as channel bonding, spatial multiplexing, multi- test are adjusted by the STA based on an initial measurement
user transmission and downlink retransmission statistics. performed to ensure that TCP does not drive the network into
Non-associated device metrics. Lastly, the AP may be in congestion.
range of a number of non-associated 802.11 devices that are
transmitting on a different BSS. When the AP is forced to B. Virtual end point representation
defer to a non-BSS device, it can record interferer air time The discussions in this sub-section are mainly in the con-
consumption. text of a download speed test. However, the arguments and
explanation are applicable to upload speed tests as well and TCP segment reaches head of AP’s
queue. AP begins to contend TX start TX end
will be generalized later. daccess,i dtx,i
In the network scenario of Fig. 1, there are no restrictions on
the traffic flows of non-target STAs and STAs in neighboring AP defer AP TX AP TX

BSSs and they may have UDP and/or TCP traffic going on collision
successful
STA 1 STA1 TX transmission
the downlink and/or the uplink. Further, the number of these
flows per device can also be variable and differ from STA STA 2 defer STA2 TX

to STA. Since we make no assumptions about the network

Fig. 3: Example timeline of a downlink transmission to depict daccess,i and
topology, interfering links or the type or number of flows, it is dtx,i . daccess,i and dtx,i denote the ‘access’ and ‘tx’ values respectively for the
not possible to state precisely the inputs for a queuing model. ith downlink transmission. daccess and dtx denote their mean values.
We remark that a majority of TCP models for Wi-Fi require Speed test flow Speed test flow
TCP segment TCP ACK
AP-side knowledge of network topology, interfering nodes
Backbone queues Forward queue
including those from neighboring BSSs, their traffic patterns,

TCP client
TCP server
PHY capabilities, data rates, etc. Removing this requirement
Sbf Svf
is vital to the realization of virtual speed test. Sbr Svr
The modeled speed test flow comprises both its TCP seg-
ments and TCP ACKs. First, we analyze the speed test flow by Reverse queue

considering the journey of a speed test flow segment from the Backbone Virtual end point

server to the target STA. On the forward path, a TCP segment Fig. 4: WLAN representation as a virtual end point consisting of a forward
experiences delays on the queues of devices on the backbone.2 and a reverse queue. The TCP client here refers to the socket level client and
When the packet enters the queue at the AP, it encounters is not to be confused with the physical STA itself.
another delay before reaching the head of the queue, part of in Fig. 4. We can think of TCP segments and TCP ACKs as
which arises from the AP serving non-speed test flow packets. jobs circulating in the network. The service time of each job
We denote the average amount of time the AP spends on non- is a sum of its ‘access’ term and its ‘tx’ term. E.g., for jobs
speed test flow packets prior to serving a speed test flow packet in the forward queue, the service time is a sum of daccess and
by V . Upon reaching the head of the queue, the AP begins dtx . Since we account for the ‘tx’ term in the service time
to contend to access the channel. It is possible that as the AP itself, the jobs themselves become indistinguishable. As we
counts down, the target STA or a non-target STA or another AP have not yet subsumed the effect of non-speed test flows, the
wins the channel, causing the AP to defer. It is also possible throughput of the virtual end point is not the same as that of
that a transmission from the AP fails either due to collision or the target STA in our WLAN.
poor channel quality, forcing the AP to double its contention In our second step, we account for the impact of non-
window size and re-contend and transmit (with the same or speed test flow packets on the throughput of this system
adapted data rates3 ). We denote the mean time the AP takes by inflating the service times of each queue to account for
to win the channel prior to a successful transmission by daccess the non-speed test flow packets. In essence, this inflation
as shown in Fig. 3. Notice that the value of this parameter makes the effective speed of each server as seen by the
can vary depending on the STA being considered as the target speed test flow packet in the virtual end point system the
STA. The average amount of time to transmit the TCP segment same as that in the original system where some server time
is represented by dtx . This includes any MAC and physical should have been consumed by non-speed test flow packets
layer overhead, MAC frame transmission time, all interframe as well. Consequently, on the forward queue, the service time
spacings and the MAC layer acknowledgement. Just like the is inflated by V . However, since the target STA has no other
TCP segment, the TCP ACK also faces a similar journey back uplink traffic while performing a speed test, the reverse queue
to the server. The terms uaccess and utx are defined in a similar service time requires no inflation. Similarly, we can subsume
manner for the target STA. the impact of cross traffic on the backbone queues into their
For our analysis, we represent the WLAN (AP and STAs) as respective service times.
a virtual end-point consisting of two queues: a forward queue
and a reverse queue. For now, let us assume that the non- C. Throughput analysis
speed test flows are non-existent and that only the speed test To analyze the throughput of the network shown in Fig. 4,
flow packets exist in the WLAN (we subsume the impact of we consider two cases. First we consider a case wherein TCP
non-speed test flow packets into the model parameters later). performs no ACK thinning. Consequently, in this case, each
With this consideration, we can treat the virtual end point TCP segment received by the STA results in the generation
as a black box replacing the WLAN that runs a speed test. of a TCP ACK. Next, we generalize this to account for the
The socket level TCP client (not to be confused with the case of ACK thinning with an ACK thinning ratio of n. In this
physical STA) runs on the virtual end point itself as shown case, the client generates a TCP ACK following the receipt of
2 A example cause of these delays is that due to cross traffic sharing a every nth TCP segment.
common queue on the backbone with the TCP segment. 1) No TCP ACK thinning: Ignoring the initial transient
3 The exact rate adaptation policy is vendor implementation dependent. stage during which TCP’s window size grows, the speed test
flow will reach a steady state wherein TCP operates at Wm . frame per transmission, the service times of both the forward
Consequently, the number of packets that are contained in the and reverse queue in the virtual end point stretch by an amount
speed test flow, which can either be TCP segments or TCP equal to (n − 1)×(daccess + dtx + V ) for the case of the download
ACKs, remain constant and the system behaves as a closed speed test. Here we inflate the service time of the reverse queue
queuing network with tandem servers and a constant number to account for the fact that the TCP ACK is not generated until
of jobs circulating inside it. the nth TCP segment is received. The numerator of Eq. (5)
Based on the aforementioned notations, the mean service should also be multiplied by n to compensate for the shrinking
time for the forward and the reverse queue in the virtual end of the total number of TCP segments. For the upload speed
point (Fig. 4) is given by: test, the service times stretch by (n−1)×(uaccess +utx ). However,
when the nodes transmit multiple frames per transmission,
Svf = daccess + dtx + V (1)
such an inflation is not necessary since the STA receives
Svr = uaccess + utx (2)
multiple TCP segments in a single downlink transmission and
Let S = Sbf + Sbr + Svf + Svr , Smax = there is no additional delay in the generation of a TCP ACK.
max(Sbf , Sbr , Svf , Svr ) and θ denote the throughput in These multiple frames may be transmitted using frame aggre-
terms of jobs per second. It can be shown [10] that gation in single stream transmissions (e.g., SISO) or by using
! multi-stream transmissions (e.g., MIMO) or a combination of
Wm 1 both frame aggregation and multi-stream transmissions. We
θ ≤ min , (3)
S Smax emphasize that this is possible since typical ACK thinning
Wm ratios of TCP are much smaller than the number of frames
where is an asymptotic bound for small values of Wm and
1
S that can be transmitted in a single transmission via the above
acts as an asymptotic bound for large values of Wm . The
Smax mentioned policies under 802.11 [11], [12], [13], [14].
cases of small and large here are relative to a critical value
In summary, the throughput in bits/sec is given by
Wm∗ which is the point at which the asymptotes cross each
other. Consequently, E[TCP segment size] × FAP
θdl = (6)
S max(Svf , Svr )
Wm∗ = (4) E[TCP segment size] × FSTA
Smax θul = (7)
To understand the physical relevance of the two components max(Svf , Svr )
of Eq. (3), let us consider two extreme case scenarios. Let where we denote θdl and θul as the download and upload TCP
us assume that Wm = 1 which makes the number of jobs throughputs respectively. FAP denotes the average number of
circulating in Fig. 4 the botteneck. The throughput, therefore, frames transmitted by the AP in a single downlink transmis-
is given by WSm . On the other extreme, if Wm is sufficiently sion to the target STA. For the case of the upload speed test,
large (again large as compared to Wm∗ ) to not bottleneck the we use FSTA instead.
system, then the slowest queue acts as a bottleneck. In this case Note that while calculating Svf and Svr for Eq. (6), dtx
the slowest queue always remains busy and in accordance with is the average time to transmit FAP number of TCP segments
1
the utilization law, θ = Smax . at the AP’s data rate and utx is the average time to transmit
Recall that due to the server selection process, Sbr and Sbf FSTA number of TCP ACKs at the target STA’s data rate. In
are not the bottleneck in the system. To understand the typical Eq. (7), this is reversed since the target STA is now the one
values that Wm∗ can take, let us consider the critical point transmitting TCP segments and the AP is the one transmitting
wherein Sbr = Sbf ∼ max(Svf , Svr ). Substituting in Eq. (4), the TCP ACKs. Svf and Svr further vary depending on which
2∗(Svf +Svr )
we will get Wm∗ = max(S vf ,Svr )
. The maximum value of Wm∗ STA is chosen as the target STA. Consequently, the AP has
occurs when Svf = Svr and thus max(Wm∗ ) = 4. In practice, to estimate these two parameters with respect to the particular
1 STA that is chosen as the target STA.
Wm  4 and consequently, we can see that θ ≤ Smax will act
as a asymptotic bound on the values of θ. In fact, we find in We remark that while the L2 edge TCP model needs to be
our experimental evaluation that for a typical speed test, the supplemented with AP-side measurements, it is not restricted
values of Wm is extremely large as compared to 4 and θ will by a requirement for AP-side knowledge of inter-node con-
tend to the bound yielding nectivity or an assumption on network traffic characteristics.
1 Next we show how the model parameters are estimated.
θ∼ . (5)
Smax IV. O BTAINING AP- SIDE MEASUREMENTS
2) TCP ACK thinning: Now, we extend the above to In this section, we show how the AP can measure all of the
the more general case of TCP ACK thinning. For an ACK parameters required for the above model, thereby enabling a
thinning ratio of n, we can view a maximum of only Wnm dynamic AP-side speed test estimate for each STA.
jobs circulating in the system and the remaining jobs can
again be accounted for by further inflating the service times A. AP-side estimation problem
of each of the queues (just as for non-speed test flows). We observe that Eq. (6) and (7) are independent of Sbr and
Consequently, when the wireless nodes transmit only one Sbf . To estimate θdl and θul at the AP, the key challenge is
computation of Svr , as the remaining parameters are based on does not have layer four visibility. To this end, we employ IP
common AP side observables described in Sec II-C. Recall addresses and size signatures as follows.
from Eq. (2) that Svr is composed of utx and uaccess . While IP address signature. Due to the inherent bi-directionality
the average uplink transmission time utx is known to the AP of TCP, the source and destination addresses for TCP segments
via per-STA metrics, the uplink access time uaccess is known traversing on the forward path are swapped for the correspond-
only at the STA side. Let tU,i
hq denote the time at which the ith ing TCP ACKs on the reverse path. This key factor enables us
uplink packet reaches the head of the STA’s queue, tU,i ts denote to distinguish individual TCP flows and separate them from
the start time corresponding to the successful transmission the remainder of the downlink and uplink traffic.
of this packet and tU,i
te denote the end time of this packet Packet size signature. Although the above signature
transmission. By definition, uaccess = E[(tU,i
ts − tU,i
hq )]. While the enables identification of a bidirectional flow, it does not aid
AP can observe tts for any uplink transmission, tU,i
U,i
hq remains in spotting the forward and reverse paths distinctly. While the
unknown. If the STA is assumed to be fully backlogged, the size of TCP segments on the forward path may fluctuate during
end time of the previous transmission can be approximated the course of a download, the reverse path is characterized
to be the time when the next packet reached the head of the by small TCP ACKs whose size remains fixed during the
queue. However, STA backlog is user activity dependent and is entire duration of the flow. Typically a TCP ACK is 20
not known to the AP. As a result, the AP cannot estimate uaccess bytes long [15]. Having distinctly identified the forward and
by a simple observation of packets received on the uplink. reverse paths, the AP can employ the uaccess estimation process
described in the previous sub-section.
B. Snooped handshakes for estimation of uplink access time
Suppose that the client is performing a TCP download V. I MPLEMENTATION AND E XPERIMENTAL
from a server (e.g., streaming a Netflix video). This can be M ETHODOLOGY
any server on the internet with any backbone delay to the In this section, we provide details of the virtual speed test
AP. The client will attempt to return a TCP ACK as fast as enabled AP (VST AP), field trial details and our ground truth
possible after reception of the corresponding TCP segment. procurement methodology.
This TCP ACK is “data” at layer 2. For now, consider a
case where there are no other flows on the uplink from the A. VST AP characterization
target STA and no ACK thinning. Since the WLAN is the VST AP runs on a Linux operating system and is factory
final hop for the TCP segment, upon reception of a TCP installed with 32 GB DDR4 SO-DIMM RAM, 2.4 GHz dual
segment, i.e., at the end of the AP’s successful downlink core CPU with a Gigabit LAN port. It is equipped with a
transmission (denoted by tD,ite ), the STA has the corresponding Ralink RT3070 off-the-shelf WiFi chipset. The radio card
TCP ACK and begins to contend. Consequently, in this case, supports IEEE 802.11b/g/n utilizing up to 40 MHz bandwidth
tU,i
hq = tD,i
te and thus the AP will have inferred a parameter and a peak PHY rate of 300 Mbps. To enable throughput
that is not directly observable. In essence, the delay incurred estimation, we build APIs that enable the acquisition of a
between the transmission of the segment to the reception of number of per packet statistics at the AP. Specifically, VST
the TCP ACK enables the AP measure how long it takes the AP collects packet timestamps (available on a nanosecond
STA to successfully transmit after it starts to attempt. Thus, granularity), source and destination IP addresses, frame sizes,
our general approach is to selectively sample TCP data-ACK and PHY rates using these APIs and feeds them into the L2
handshakes from any TCP download performed by the target edge TCP model to estimate the throughput. As described
STA and use them to drive a measurement based prediction earlier, the parameter estimation methodologies employ packet
of θdl and θul . We refer to such TCP flows as snooped flows. timestamps as a part of the computation process. While the
This can be generalized under a flow hypothesis (i.e., absolute value of these timestamps can be impacted by sys-
knowing that a given flow on the downlink is a TCP flow) tem dependent offsets, their post-subtraction residue becomes
by the following two cases. negligible as they have a low second moment. The STAs asso-
ACK queuing. This case occurs when the target STA has ciated with the VST AP are a mix of portable laptops running
other uplink flows whose packets get queued prior to the TCP on either Windows or Linux OS whose network interface card
ACK. Consequently, in such scenarios, tU,i hq = tU,i−1
te . In such supports 802.11b/g/n as well. While VST does not require
U,i−1
cases, we abuse the term tte to refer to the end time of collection of STA side statistics, for validation purposes, we
transmission of the immediately preceding uplink packet. also implement APIs and data collection capabilities for STAs
ACK immediate. However, if the target STA has no other associated with the VST AP to enable us to characterize the
uplink flow, it begins to contend as soon as the TCP ACK operating environment.
is queued. Consequently, tU,i hq = tD,i∗n
te where the superscript
‘D’ refers to a downlink transmission. B. Field trials
To study the estimation accuracy of virtual speed test, we
C. TCP flow inference deploy VST AP and STAs in two environments. The first
Because the layer four handshake is needed to estimate deployment is in an office located in a 3 storied building on
uaccess , it is crucial to identify this handshake at the AP, which a university campus. In this deployment, the VST AP and
its associated STAs co-exist with a university administered beta.speedtest.net speedtest.att.com Xfinity Iperf beta.speedtest.net speedtest.att.com Xfinity Iperf

Download Throughput (Mbps)

100 100

Upload Throughput (Mbps)

enterprise network and 2 BSS deployed in nearby offices. 80 80

We deploy a second network in an apartment in a 3 story 60 60

residential building primarily consisting of 1 or 2 bedrooms 40 40

per unit. These devices coexist with BSS from neighboring 20 20

apartments. In each environment, the traffic generated by co- 0 0

existing BSS is not controlled by us and is determined by user (a) Download speed test (b) Upload speed test
activity in those BSS. The VST AP deployed in the office Fig. 5: Iperf results as compared to exemplary online speed test applications.
performs measurements for a period of two days whereas the
residential scenario measurements are carried out for a period server in the same LAN as the AP. In practice, a server within
of one week. During the entire duration of deployment, the the same LAN may not be available in the server pool probed
VST AP observed a total of 113,047 snooped flows. These by these applications. Consequently, results obtained from a
flows are generated by STAs associated with the VST AP non-local server may be affected by backbone load and delay.
which are instrumented to run web applications such as video To explore this effect, we keep one 802.11n laptop associ-
streaming (via YouTube), music streaming (via Pandora), pdf ated with the AP and run multiple speed test applications 10
downloads (via IEEE Xplore) and sending and receiving times with a gap of 10 mins between each run. We attempt to
emails (via Gmail). The number of these applications running realize static WLAN conditions, i.e., with minimal activity
in parallel for each STA is also varied thereby generating on co-existing BSS, no device or environmental mobility,
cases of both single as well as mixed application traffic etc., in which the outcome of speed test applications should
with a varying number of parallel running applications. The remain stable. We compare these results with those obtained
STAs use Mozilla Firefox web browser for performing these by running iperf from a local server to remove backbone
online activities. Aside from the applications mentioned above, and server selection effects. Fig. 5 shows the values obtained
some of the STAs have Dropbox installed on them which for upload and download throughput from exemplary online
occasionally adds to the uplink traffic from these devices in speed test applications as well as iperf. Compared to iperf,
addition to that generated by email activities. the speed test applications demonstrate throughput reductions
It is important that virtual speed test is accurate in the and fluctuations due to backbone and server selection effects.
presence of variation of the empirical L2 edge TCP model On the other hand, the values obtained from iperf remain
parameters which are affected by traffic characteristics, active stable. Therefore, to obtain stable and more reliable ground
STA count, MAC and PHY statistics, etc. Post processing truth values in our experiments, we use the iperf tool.
traffic traces reveals that the application layer data size varied
VI. E XPERIMENTAL E VALUATION
from a minimum of 20 bytes to a maximum of 1.4K bytes.
Overall, the office and residential scenario covered a total of 36 In this section, we investigate the performance of virtual
and 49 different topologies respectively with a varying number speed test in the two deployment scenarios. In our experiments,
of STAs associated with the VST AP in each topology and the virtual speed test throughput estimates are obtained solely
VST AP made predictions for each associated STA. The total via AP measurements, i.e., by using AP-snooped TCP data -
number of associated STAs in a topology varied between 1 ACK handshakes from existing network traffic to estimate the
and 6 in the office scenario and 1 and 7 in the residential parameters required by the L2 edge TCP model. Immediately
scenario. The activity in the neighboring BSS varied in both following each throughput estimation, the ground truth is
the deployment. For instance, during approximately 30% of measured via iperf on instrumented clients. While network
the flows in both the office and the residential scenario, there traffic is composed of both download and upload traffic, virtual
was no activity on the neighboring BSS. On the other hand, the speed test only leverages download flows for uaccess estimation.
maximum portion of a TCP flow’s duration that a STA spent
deferring to neighboring BSS was 38.8% for the residential A. Uplink access time uaccess estimation
scenario and 51.4% in the office scenario. Further details Throughput and virtual service time are impacted by the
about the deployment can be found in our companion technical uplink access time. Recall that this model parameter is not
report available at [16]. directly observable at the AP. Rather, the VST AP leverages
snooped handshakes from TCP flows existing in the network
C. Iperf for ground truth procurement to estimate uplink access time. In our deployment, the VST
We compare the outcome of virtual speed test with that AP observes cases involving only single application traffic
obtained from online speed test applications [1], [2], [3] as as well as those with mixed application traffic involving
well as iperf. We focus on iperf for procuring ground truth concurrent TCP flows with a mix of downloads and uploads.
as online tests are subject to an additional and uncontrolled To correctly estimate the uplink access time, the VST AP
source of variation due to server selection. Namely, recall that needs to first identify TCP flows based on the address and size
during the setup phase, the server selection process of these signatures and then accurately identify the occurrence of the
speed test applications tries to minimize the backbone delay ACK queuing and ACK immediate regimes. We first explore
between the server and the AP with the ideal case being a the efficacy of the uaccess estimation method by comparing the
 1 1 60 60

Measured value of uaccess

Measured value of uaccess
Download Upload Download Upload

% Estimation Error
% Estimation Error
0.8 0.8 GT Est GT Est GT Est GT Est
(Mbps) (Mbps) (Mbps) (Mbps) (Mbps) (Mbps) (Mbps) (Mbps)
40 Best 40 Best
0.6 0.6 85.7 86.57 63.7 63.06 82.0 81.18 58.6 57.98
case case
Worst Worst
14.3 15.92 36.9 40.92 47.5 52.15 4.44 5.29
0.4 0.4 case case
20 20
0.2 0.2
y=x y=x
0 0 0 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Download Upload Download Upload
Ground truth value of uaccess Ground truth value of uaccess
(a) Office Scenario (b) Residential Scenario
(a) Office Scenario (b) Residential Scenario Fig. 7: Percent throughput estimation error statistics in the office and
Fig. 6: Scatter plot of uaccess measured and ground truth value in the office residential deployments. In the table, GT denotes the ground truth value and
and residential deployments. In each sub-figure, both the axis are normalized Est denotes the throughput estimated by virtual speed test.
with respect to the maximum ground truth value in that particular deployment. 100 100

% Estimation error
% Estimation error
80
estimates made by the VST AP using snooped flows with the 60
ground truth values obtained from the STA-side. 40
50

Fig. 6 shows a scatter plot of measured values of uaccess vs.

20
the ground truth value. For each deployment scenario, both
0 0
of these values are normalized with respect to the maximum 500 1000 1500 2000 2500 3000 500 1000 1500 2000 2500 3000
Observation set size Observation set size
ground truth value in that particular scenario. Fig. 6 shows
that as the ground truth values of uaccess varies due to numerous (a) Office: Download (b) Office: Upload
100 100
factors such as background traffic characteristics, active STA

% Estimation error
% Estimation error
count, etc., the points in the scatter plot continue to be closely
located to the identity line. Fig. 6 also demonstrates the 50 50

existence of homoscedasticity between estimates obtained by

VST AP and the ground truth value of uaccess obtained from
0 0
the STA-side. In essence, this shows that as the ground truth 500 1000 1500 2000 2500 3000 500 1000 1500 2000 2500 3000
Observation set size Observation set size
value of uaccess varies due to the numerous factors mentioned
previously, the estimated value of uaccess retains its accuracy. (c) Residential: Download (d) Residential: Upload
Fig. 8: Throughput estimation error as a function of the observation set size
available at the AP
B. Throughput estimation accuracy
The ultimate goal of virtual speed test is to estimate θdl and accuracy as follows. Since the backbone delay between the
θul . To evaluate throughput estimation accuracy, we calculate snooped flow’s server and the VST AP is inconsequential, we
the percent estimation error between the ground truth values initiate a single TCP flow from a local server to the target STA
obtained from iperf and the estimates of virtual speed test and treat it as a snooped flow. We control the observation set
|(θgt −θest )|∗100 size available at the VST AP by changing the download file
as %error = θgt where θgt denotes the ground
size and computing the percent throughput estimation error as
truth value of throughput and θest denotes the estimated value
before.
from virtual speed test. We remark that since the %error is
Fig. 8 depicts the mean estimation error as a function of
calculated after weighing by θgt , it is sensitive to the value of
the size of the observation set available at the VST AP. The
θgt , i.e., the same absolute error at a smaller ground truth value
mean estimation error demonstrates an exponential decrease
is bound to yield higher estimation error.
with an increasing set size. When the sample set size is on
Fig. 7 summarizes the percent throughput estimation er-
the order of a few 10s of samples, the estimation error is as
ror statistics in both the office and residential deployments.
high as 70%. The variance in this case is also very high. This
Overall virtual speed test shows a good match against ground
is due to the fact that the number of measurement samples are
truth values with a mean percentage error of 4.09% and 4.3%
insufficient to accurately characterize the model parameters.
for upload and download speeds respectively in the office
With an increasing sample set size, the precision in the model
scenario. In the residential scenario, these values are 5.51%
estimation increases as expected. Fig. 8 reveals that even when
and 2.9% respectively.
the observation set contains a few 1000s of samples (which
Virtual speed test estimates the target STA’s throughput by
could easily be obtained from observing the download of a
collecting measurements from its TCP traffic. Therefore, a key
research paper from IEEE Xplore), the mean estimation error
factor that determines the estimation accuracy of virtual speed
of virtual speed test is under 5% in both the deployments.4
test is the number of measurements that the VST AP can obtain
from the target STA’s TCP traffic which in turn depends on the VII. R ELATED W ORK
target STA’s online activity. On one extreme, a large number of Analytical Models. Prior analytical models can predict
observations obtained from either a single large file download throughput while incorporating various MAC and PHY layer
or a series of consecutive small file download results in refined aspects of Wi-Fi [17], [18]. Unfortunately, the models are
parameter estimation. We experimentally investigate the obser-
vation set size required to avoid a deterioration in estimation 4 ns 3 simulation details and results are in our companion tech report [16].
not applicable or extensible to our scenario as they require and the number of associated clients. Overall, virtual speed test
AP-side knowledge of network topology, interfering nodes exhibits high accuracy with mean estimation errors below 6%.
including those from neighboring BSS, their traffic patterns,
IX. ACKNOWLEDGEMENTS
PHY capabilities, data rates, etc. Obtaining this information
requires STA-side co-operation and regular reporting. On the This research was supported by Cisco and by NSF grants
other hand, virtual speed test enables throughput estimation CNS-1801857 and CNS-1642929.
with zero STA-side co-operation and no reporting. R EFERENCES
Active measurements. Active probing techniques such
[1] Ookla Speedtest. http://www.speedtest.net/. Accessed: 2018-05-27.
as [19], [20] involve usage of probing packets to estimate [2] AT & T Internet Speed Test. http://speedtest.att.com/speedtest/. Ac-
bandwidth. However, they impose additional traffic load on cessed: 2018-05-27.
the network that can disrupt user traffic or drain the battery of [3] Xfinity Speed Test. http://speedtest.xfinity.com/. Accessed: 2018-05-27.
[4] D. Murray, T. Koziniec, S. Zander, M. Dixon, and P. Koutsakis. An
mobile devices. Tools such as [21] require client-side software analysis of changing enterprise network traffic characteristics. In Proc.
to perform network analysis. On the other hand, virtual speed of IEEE APCC, 2017.
test performs passive measurement based estimation and hence [5] Ookla SpeedTest. How does the Begin Test button select a server?,
2012.
does not impose any additional traffic load on the network or [6] Ookla SpeedTest. How does the test itself work? How is the result
require any specialized client-side software. calculated?, 2012.
Passive measurements. Careful deployment of sniffers can [7] O. Goga and R. Teixeira. Speed measurements of residential internet
access. In Proc. of PAM, 2012.
be used to make passive observations to collect traffic traces [8] E. Altman, D. Barman, B. Tuffin, and M. Vojnovic. Parallel TCP
of various users to estimate throughput [22],[23]. Likewise, Sockets: Simple Model, Throughput and Validation. In Proc. of IEEE
collection of information from co-existing BSS can also enable INFOCOM, 2006.
[9] A. Patro, S. Govindan, and S. Banerjee. Observing Home Wireless
the AP to estimate throughput of its associated STAs [24]. Experience Through WiFi APs. In Proc. of ACM MobiCom, 2013.
However, such methods either require installation and mainte- [10] M. Harchol-Balter. Performance modeling and design of computer
nance of additional hardware or co-operation from neighboring systems: queueing theory in action. Cambridge University Press, 2013.
[11] D. Murray and T. Koziniec. The state of enterprise network traffic in
BSS for data collection. In contrast, virtual speed test requires 2012. In Proc. of APCC, 2012.
no additional infrastructure or any cooperation from among [12] R. Braden. RFC-1122: Requirements for internet hosts. Request for
co-existing APs. Comments, pages 356–363, 1989.
[13] IEEE Std. 802.11ac-2013. Enhancements for Very High Throughput for
Training via ground truth measurements. In this ap- Operation in Bands Below 6 GHz, 2013.
proach, followed by [9], [25], [26], network clients store [14] IEEE Std. 802.11n-2009. Enhancements for Higher Throughput, 2009.
empirical throughput of all TCP sessions and report them [15] P. Jon. Transmission control protocol–darpa internet program protocol
specification. Technical report, RFC-793, DARPA, 1981.
to the AP to build a database of TCP throughputs. This [16] Companion technical report, available at. https://www.dropbox.com/s/
coupled with AP-side records of wireless conditions during ruf3qh60bxdwhls/tech report.pdf?dl=0.
the TCP session (e.g., the session’s MCS, busy air time, [17] P. Nayak, M. Garetto, and E. W. Knightly. Multi-user Downlink with
Single-User Uplink can Starve TCP. In Proc. of IEEE INFOCOM, 2017.
and collision rate) enable the AP to predict throughput by [18] D. Miorandi, A. Kherani, and E. Altman. A queueing model for HTTP
correlating the current conditions with historical averages traffic over IEEE 802.11 WLANs. Computer Networks, 2006.
corresponding to similar conditions. However, this requires [19] K. Lakshminarayanan, V. N Padmanabhan, and J. Padhye. Bandwidth
estimation in broadband access networks. In Proc. of ACM SIGCOMM,
client-side reporting to obtain ground truth as network and 2004.
traffic conditions change, a requirement that is not allowed in [20] R. Kapoor, L. Chen, L. Lao, M. Gerla, and M. Sanadidi. Capprobe:
our problem formulation. A simple and accurate capacity estimation technique. In Proc. of ACM
SIGCOMM, 2004.
TCP flow analysis. TCP flow analysis has been leveraged to [21] K. Kim, H. Nam, and H. Schulzrinne. WiSlow: A Wi-Fi network
understand IP and TCP statistics such as segment reordering, performance troubleshooting tool for end users. In Proc. of INFOCOM,
duplication, etc. [27], identification of malicious attacks [28] 2014.
[22] L. DiCioccio, R. Teixeira, and C. Rosenberg. Impact of Home Networks
and for P2P Botnet detection [29]. In contrast, we utilize TCP on End-to-end Performance: Controlled Experiments. In Proc. of ACM
flow dynamics to measure L2 parameters to facilitate upload HomeNets, 2010.
and download throughput estimation for WLANs. [23] Y. Cheng, J. Bellardo, P. Benkö, A. Snoeren, G. Voelker, and S. Savage.
Jigsaw: Solving the Puzzle of Enterprise 802.11 Analysis. In Proc. of
ACM SIGCOMM, 2006.
VIII. C ONCLUSIONS [24] V. Shrivastava, S. Rayanchu, S. Banerjee, and K. Papagiannaki. Pie in
We presented virtual speed test - a measurement based the sky: Online passive interference estimation for enterprise wlans. In
Proc. of NSDI, 2011.
framework that enables an AP to continuously estimate TCP [25] C. Rattaro and P. Belzarena. Throughput prediction in wireless networks
speed test results for any of its associated STAs without using statistical learning. In Proc. of LAWDN, 2010.
any end-user co-operation, with no additional traffic load on [26] M. Mirza, K. Springborn, S. Banerjee, P. Barford, M. Blodgett, and
X. Zhu. On the accuracy of TCP throughput prediction for opportunistic
the network and solely based on passively obtained AP-side wireless networks. In Proc. of IEEE SECON, 2009.
observables. We deploy a VST enabled AP in a university [27] M. Mellia, A. Carpani, and R. Cigno. Tstat: TCP statistic and analysis
office and in a residential apartment characterized by a variety tool. In Proc. of Springer QoS-IP, 2003.
[28] Y. Chen and K. Hwang. TCP flow analysis for defense against shrew
of operating conditions including the presence of multiple co- DDoS attacks. In Proc. of IEEE ICC, 2007.
existing BSSs, link diversity in terms of signal propagation [29] L. Zhou, Z. Li, and B. Liu. P2P traffic identification by TCP flow
and supported PHY rates and variation in traffic characteristics analysis. In Proc. of IWNAS, 2006.