Perceptual-cognitive skill and the in situ performance of soccer

players

Author(s)
van Maarseveen, Mariëtte J.J.; Oudejans, Raôul R.D.; Mann, David L.; Savelsbergh, Geert
J.P.
DOI
10.1080/17470218.2016.1255236
Publication date
2018
Document Version
Final published version
Published in
Quarterly journal of experimental psychology
License
CC BY
Link to publication

Citation for published version (APA):

van Maarseveen, M. J. J., Oudejans, R. R. D., Mann, D. L., &
Savelsbergh, G. J. P. (2018). Perceptual-cognitive skill and the in situ
performance of soccer players. Quarterly journal of experimental
psychology, 71(2), 455-470.
https://doi.org/10.1080/17470218.2016.1255236

General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)
and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open
content license (like Creative Commons).

Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests,
please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the
material inaccessible and/or remove it from the website. Please contact the library:
https://www.amsterdamuas.com/library/contact/questions, or send a letter to: University Library (Library of the
University of Amsterdam and Amsterdam University of Applied Sciences), Secretariat, Singel 425, 1012 WP
Amsterdam, The Netherlands. You will be contacted as soon as possible.

Download date:11 Jul 2023

10.1080_17470218.2016.1255236
research-article2017
QJP0010.1080/17470218.2016.1255236The Quarterly Journal of Experimental Psychologyvan Maarseveen et al

Original Article

Quarterly Journal of Experimental

Perceptual-cognitive skill and the Psychology

2018, Vol. 71(2) 455­–470
© Experimental Psychology Society 2017
in situ performance of soccer players
https://doi.org/10.1080/17470218.2016.1255236
DOI: 10.1080/17470218.2016.1255236
qjep.sagepub.com

Mariëtte J. J. van Maarseveen1, Raôul R. D. Oudejans1,2,

David L. Mann 1 and Geert J. P. Savelsbergh1,2

Abstract
Many studies have shown that experts possess better perceptual-cognitive skills than novices (e.g., in anticipation,
decision making, pattern recall), but it remains unclear whether a relationship exists between performance on those
tests of perceptual-cognitive skill and actual on-field performance. In this study, we assessed the in situ performance
of skilled soccer players and related the outcomes to measures of anticipation, decision making, and pattern recall. In
addition, we examined gaze behaviour when performing the perceptual-cognitive tests to better understand whether the
underlying processes were related when those perceptual-cognitive tasks were performed. The results revealed that on-
field performance could not be predicted on the basis of performance on the perceptual-cognitive tests. Moreover, there
were no strong correlations between the level of performance on the different tests. The analysis of gaze behaviour
revealed differences in search rate, fixation duration, fixation order, gaze entropy, and percentage viewing time when
performing the test of pattern recall, suggesting that it is driven by different processes to those used for anticipation and
decision making. Altogether, the results suggest that the perceptual-cognitive tests may not be as strong determinants
of actual performance as may have previously been assumed.

Keywords
Anticipation; Decision making; Gaze behaviour; In situ performance; Pattern recall

Received: 29 October 2015; accepted: 20 October 2016

Perceptual-cognitive skills such as anticipation and deci- improve performance. However, it remains unclear what
sion making are crucial for successful performance in many might be the best way to measure perceptual-cognitive skill
complex dynamic motor tasks. For example, in aviation, to accurately reflect the demands of actual on-field perfor-
the military, when driving a car, and in sport, the ability to mance (Mann & Savelsbergh, 2015; Pinder, Headrick, &
pick up visual information and to select and execute an Oudejans, 2015; Williams & Ericsson, 2005), and this
appropriate action is key to high-level performance remains a significant barrier for scientists and practitioners
(Williams & Ericsson, 2005; Williams, Ford, Eccles, & who wish to better understand and improve high-level per-
Ward, 2011). Sports offer a unique, dynamic, and time-con- formance in dynamic motor tasks.
strained environment in which perceptual-cognitive skills Perceptual-cognitive skill as it is performed in motor tasks
can be examined. In team sports, like soccer, expert perfor- has typically been measured using simplified video-based
mance means choosing the correct action at the correct
moment and performing that course of action efficiently
and consistently throughout a match (Baker, Cote, & 1Department of Human Movement Sciences, Vrije Universiteit
Abernethy, 2003; Gréhaighne, Godbout, & Bouthier, 2001). Amsterdam, MOVE Research Institute Amsterdam, Amsterdam, The
The ability to measure the level of performance on these Netherlands
2Faculty of Sports and Nutrition, Amsterdam University of Applied
perceptual-cognitive tasks is crucial to better understand
Sciences, Amsterdam, The Netherlands
expert performance, and to identify the factors and underly-
ing processes that mediate successful performance Corresponding author:
(Williams & Ericsson, 2005). Accurate measures of percep- Mariëtte J. J. van Maarseveen, Department of Human Movement
Sciences, MOVE Research Institute Amsterdam, VU University
tual-cognitive skill could be used, for instance, for the pur- Amsterdam, Van der Boechorststraat 9, Amsterdam 1081 BT, The
poses of talent identification and development, and to Netherlands.
determine the efficacy of training interventions designed to Email: [email protected]
456 Quarterly Journal of Experimental Psychology 71(2)

tests in which participants do not move, but instead indicate The decoupling of perception and action provides a
their preferred action or response from a variety of options clear distinction between task designs in which partici-
either verbally or by way of a button press (e.g., Abernethy & pants are required to make actual movements (an action
Russell, 1987; Gorman, Abernethy, & Farrow, 2012; response) and those in which participants generally
Savelsbergh, Williams, Van der Kamp, & Ward, 2002). Using respond verbally or by a simplified movement like a but-
this method, clear differences have been revealed between ton-press (generally considered to be perceptual
experts and novices, and sometimes differences are studied responses). The two-visual system model of Milner and
within groups to discriminate those with relatively high and Goodale (1995) claims that action and perception rely on
low levels of perceptual-cognitive skill (e.g., Savelsbergh, two neuro-anatomically separate visual pathways within
van der Kamp, Williams, & Ward, 2005). Skilled performers the brain: The ventral “vision-for-perception” stream is
are consistently found to be superior on a variety of percep- thought to be used for perceiving what action a situation
tual-cognitive tasks including those designed to test (a) antici- affords, and the dorsal “vision-for-action” stream for the
pation, the ability to predict the outcome of another person’s visual guidance of that action. In a persuasive position
action on the basis of the pick-up of early visual information paper that examined the implications of the dual pathway
(e.g., Abernethy & Russell, 1987; Jones & Miles, 1978; model for research on anticipation, Van der Kamp, Rivas,
Savelsbergh et al., 2002; Williams & Ward, 2007); (b) deci- Van Doorn, and Savelsbergh (2008) suggested that much
sion making, the ability to select the best possible option from of the previous anticipation research had largely exam-
a variety of alternatives (e.g., Helsen & Pauwels, 1993; ined only the role of the ventral pathway because those
Vaeyens, Lenoir, Williams, Mazyn, & Philippaerts, 2007); studies had relied on video-based tests in which no actual
and (c) pattern recall, the ability to recall previously seen pat- movement had to be made. By excluding action from the
terns of play (e.g., Allard, Graham, & Paarsalu, 1980; Gorman participant response, Van der Kamp et al. (2008) claimed
et al., 2012; Van Maarseveen, Oudejans, & Savelsbergh, that most existing studies overlook the contribution of the
2015). In addition, differences in gaze behaviour are often dorsal system that is most likely to be relied on during
reported when these tasks are performed, generally showing actual performance. This distinction provides reason to
that experts use fewer fixations of longer duration than nov- believe that video-based tests of anticipation are likely to
ices (e.g., Mann, Williams, Ward, & Janelle, 2007), a finding under-represent (or even misrepresent) the true ability of
that has been interpreted to suggest that experts use a more skilled performers when performing in an actual perfor-
efficient search strategy when performing these tasks (Helsen mance setting (Dicks, Davids, & Button, 2009; Mann
& Pauwels, 1993). et al., 2007). The same could also be said for tests of deci-
Although the traditional video-based perceptual- sion making, where participants must perceive the situa-
cognitive skill tests offer a significant advantage in terms tion in order to decide an appropriate action to perform.
of their methodological rigour and control, it remains Therefore, it could be that decision making is also likely
unclear how well these tests might accurately represent the to be affected by the absence of an action response in the
on-field performance they are designed to sample (Mann same way that tests of anticipation might be. In support,
& Savelsbergh, 2015; Pinder et al., 2015; Williams & Oudejans, Michaels, Van Dort, and Frissen (1996) exam-
Ericsson, 2005). Recently, significant differences have ined safe road-crossing behaviour and showed that more
been found in both movement and visual behaviour when accurate decisions were made when people walked
comparing performance on traditional video-based tests towards the road than if they were standing still and mak-
with contexts that are likely to be more representative of ing the same decision. However, given that the recall of
the participants’ performance environment (Dicks, Button, briefly presented patterns of play is rarely required in the
& Davids, 2010; Pinder et al., 2015). For example, Dicks natural performance environment (Gorman, Abernethy, &
et al. (2010) showed that when compared to responding to Farrow, 2013; Williams & Ericsson, 2005) and that doing
a video simulation, soccer goalkeepers made more penalty so is unlikely to be coupled to an action, it could be that
saves and fixated earlier on the ball and for longer periods the impact that absence of action would have on a test of
of time in an in situ condition where actual interception pattern recall might be less pronounced than it would be
was required. Similarly, Mann, Abernethy, and Farrow for tests of anticipation and decision making. The test of
(2010) found that anticipation skill increased when partici- pattern recall is likely to be a highly perceptual test for
pants were required to make an actual movement rather which there might not be an equivalent test that would
than a simple verbal response when anticipating the direc- rely on a motor response.
tion of a cricket ball. In support, a meta-analysis of percep- The degree to which different perceptual-cognitive
tual-cognitive skill in sport has shown that expertise effects skills are related is an important topic of recent debate
are most apparent when participants have to perform genu- (Farrow, McCrae, Gross, & Abernethy, 2010). In particu-
ine actions under in situ task constraints rather than per- lar, it has been suggested that pattern recall may serve a
forming simplified responses in less representative functional role for facilitating anticipation and decision
conditions (Mann et al., 2007; Travassos et al., 2013). making. It has been claimed that athletes may use the
van Maarseveen et al 457

locations of players to anticipate the next state of the pat- perceptual-cognitive skill is whether performance on
tern of play and to make an appropriate decision in those tests predicts on-field performance. Existing studies
response to this evolving pattern (Farrow et al., 2010; have used the expert–novice comparison to show differ-
Gorman et al., 2012, 2013; Williams & Davids, 1995). ences between skill levels, and assumed that those percep-
This is a significant issue as it helps to reveal whether pat- tual-cognitive skills for which there are differences must
tern recall, anticipation, and decision making are inde- comprise an important element of expertise. It could be
pendent skills that should be acquired separately, or that some perceptual–cognitive skills are more related to
whether they are all related and underpinned by one the actual performance on the field than others, and this
underlying skill and thus similar cognitive processing could depend on how well the separate tests reflect the
(Gorman, Abernethy, & Farrow, 2015; North, Williams, processes that are needed for actual actions on the field.
Hodges, Ward, & Ericsson, 2009). Moreover, from a prac- Therefore, in some studies, the relative weight of factors
tical perspective, there would be no need to administer contributing to skilled performance have been exam-
multiple tests if they were to be assessing the same under- ined—for example, Ward and Williams (2003) assessed
lying attribute. The majority of research to date has exam- young soccer players using a multidimensional battery of
ined performance on the different tests of tests and found that anticipation and the use of situational
perceptual-cognitive skill independently (Williams & probabilities (i.e., expectations of what is likely to happen
Ward, 2007), with only a few studies having searched for next) were the best discriminating factors across the dif-
any relationship between those skills. One exception was ferent skill groups. However, this expert–novice approach
a study by Farrow et al. (2010) who examined correlations falls short of being able to provide direct evidence that
between the anticipation and pattern recall skill of expert, performance on those tests is related to on-field perfor-
intermediate, and novice rugby union players in line-outs. mance. Rather, superior performance could be a conse-
They found that pattern recall skill accounted for 40% of quence of experience in the game instead of being a
the variance in the anticipation task; however, when the contributing factor to expertise. As a result the relation-
level of expertise was accounted for they found that the ship between these perceptual-cognitive skills and actual
correlation between anticipation and pattern recall performance remains unclear (Ericsson, Patel, & Kintsch,
remained for the intermediate and novice players only, 2000; Ericsson & Smith, 1991).
and not for the experts. Farrow et al. consequently sug- In the current study, we sought to examine how well
gested that lesser skilled players use pattern recall when performance in a complex time-constrained motor task
attempting to anticipate an evolving pattern, but for could be predicted using representative tests of perceptual-
experts the contribution of pattern recall is diminished, cognitive skill. To do so we assessed the in situ perfor-
and the anticipation task is processed in a different mance of young talented soccer players in a small-sided
manner. soccer game and related it to their performance on separate
One possible way to better understand the degree to tests of anticipation, decision making, and pattern recall.
which different tests of perceptual-cognitive skill might Moreover we sought to determine the degree to which the
be related, and thereby the underlying processes relied three tests of perceptual-cognitive skill were related by
on when performing those tasks, is through the examina- exploring the correlations between the tests and the simi-
tion of gaze behaviour (Williams & Ericsson, 2005). In larity of the gaze of participants when performing those
1967, Yarbus first showed that gaze behaviour changes tasks. If performance on the tests of perceptual-cognitive
as a result of task requirements, even when the same skill were to be highly predictive of on-field performance
visual stimulus is viewed (in that case stationary images). then strong within-group correlations should be found
Similar results have been found within the sports domain; between the measures of perceptual-cognitive skill and
for example, Gorman et al. (2015) found differences in individual performance in the small-sided games.
the gaze strategies of skilled basketball players when Moreover, if performance on the three tests of perceptual-
watching the same video footage for the purposes of cognitive skills were to be highly correlated with each
decision making and pattern recall, and North et al. other, then similarities in gaze behaviour when performing
(2009) found differences in the gaze of soccer players those tasks would be expected. Instead, if the degree to
when watching video clips for the purposes of pattern which the skills were to overlap would be low then signifi-
recognition and anticipation. Differences in gaze behav- cant differences in gaze would be expected when partici-
iour between the various tests has been interpreted to pants were performing those tasks. Insight into the degree
provide support for the idea that different processes to which the perceptual-cognitive skills overlap and how
underpin these contrasting perceptual-cognitive skills well these tests represent in situ performance helps to
(North et al., 2009). reveal whether those skills are underpinned by different
To better understand and improve high-level perfor- cognitive processes, and may facilitate the development of
mance in dynamic motor tasks, the fundamental question an accurate method to evaluate performance in complex
of interest in establishing appropriate tests of time-constrained motor tasks.
458 Quarterly Journal of Experimental Psychology 71(2)

Figure 1.  Snapshot of video clip of the small-sided game. Players are located at their specific starting positions. To view this figure
in colour, please visit the online version of this Journal.

Experimental study (in press). In this system at any one point in time each
player has one of three roles: attacker with ball, attacker
Method without ball, and defender. For each role, the possible
Participants. Twenty-two highly talented female soccer actions and outcomes as well as the a priori determined
players from the national soccer talent team participated in number of points a player earns when performing that
the study (Mage = 16.3 years, SD = 1.1). They trained about action (and the consequent outcome) were determined by
15 to 20 hours a week and played in a high-level competi- two experts with over 25 years of experience in coaching
tion for men under 14 years of age and had an average of soccer at a national and international level (see Table 1).
9.8 years (SD = 2.3) of soccer experience. The experiment For example, when an attacker with the ball shoots at the
was approved by the local ethics committee of the research goal but the shot is saved by the goalkeeper, the player
institute, and all participants gave their written informed earns nine points. A slightly different approach was used
consent prior to the experiment; parental consent was pro- to evaluate the positioning of a player, with the duration
vided for players younger than 18 years. of time that the player was open or marked being regis-
tered and used to calculate the percentage of time a player
In situ test. The in situ test was identical to the one spent in each of the positioning categories (“Open, own
described by Van Maarseveen, Oudejans, and Savelsbergh half, centre”; “Open, own half, side”; “Open, opponents’
(in press). The test comprised 3 versus 3 small-sided games half, centre”; “Open, opponents’ half, side”; “Marked”).
(i.e., three attackers vs. two defenders and a goalkeeper) The overall score for positioning was calculated by multi-
because these games are considered to comprise the basics plying the percentage of time in each category by the
of the game of soccer according to the Dutch Royal Soccer number of points allocated to that specific category (Table
Association (KNVB; Dokter, 1993), and many more 1). For example, when a player was open in her own half,
behavioural observations are possible in a given period of in the centre of the field, for 30% of the total time, then
time when compared to an 11 versus 11 game (Davids, this player received 0.30 × 2 = 0.6 points for this position-
Araújo, Correia, & Vilar, 2013). Games were played on a ing category (for more details see Van Maarseveen et al.,
field of 40 m × 25 m (field dimensions were advised by the in press).
head coach). The six players started at specific locations The video footage of the in situ test was analysed frame
(Figure 1) and played according to the official soccer rules, by frame so that all actions and the consequent outcomes
including the use of the offside rule. In each test session were registered for each player on the field. Subsequently,
participants played five times in each of the playing posi- performance scores were determined by calculating the
tions. In total, eight test sessions were conducted across average number of points per trial that a player received in
4.5 months, resulting in a total of 733 trials, an average of each of the three roles, and summing those scores into an
34 trials per participant per playing position. The test ses- overall performance score. Van Maarseveen et al. (in
sions were video recorded using a Go-Pro Hero 3 camera press) validated the notational system on highly talented
(Black Edition, resolution 1920 × 1080 pixels, 30 Hz; Go- youth soccer players. Besides high content and ecological
Pro, USA) that was fixed on a 6.5-m high platform (Show- validity, they showed significant concurrent validity (i.e.,
tec LTB-200/6 Lifting Tower, The Netherlands) behind the correlation between the performance scores attained with
goal being defended by the attacking team. the notational system and judgments of the head coach;
The performance of the participants was assessed using τs > .397, ps < .05), construct validity (i.e., ability of the
the notational system designed by Van Maarseveen et al. notational system to discriminate the high- and low-skilled
van Maarseveen et al 459

Table 1.  Actions, outcomes, definitions, and allocation of points of notational system.

Role Action Outcome Definition Points

Attacker with ball
  Shooting The attacker shoots at goal and . . .  
  Goal . . . scores 12
  Blocked by defender . . . the shot is blocked by a defender 6
  Saved by goalkeeper . . . the shot is saved by the goalkeeper 9
  Post/crossbar . . . the ball hits the post or crossbar 9
  Wide/over . . . the ball goes wide or over the goal (within 1 m) 6
  Far wide/far over . . . the ball goes far wide or over the goal (more than 1 m) 0
  Passing The attacker passes the ball . . .  
  Successful, towards teammate in . . . and a teammate in a promising position receives the 5
promising position ball
  Successful, forward . . . forward to a teammate who receives the ball 2
  Successful, backward . . . sideways or backward to a teammate who receives the 1
ball
  Intercepted . . . and a defender or goalkeeper intercepts the ball 0
  Offside . . . towards a teammate in offside position 0
  Out of play . . . out of play 0
  Dribbling The attacker moves the ball, after receiving and prior to  
passing/shooting, (without a near defender) and . . .
  Maintain ball possession, towards . . . the attacker maintains ball possession and moves 5
promising position towards a promising position
  Maintain ball possession, forward . . . the attacker maintains ball possession and moves 2
forwards
  Maintain ball possession, to the . . . the attacker maintains ball possession and moves to the 1
side or backward side or backwards
  Ball possession lost . . . the attackers loses ball possession 0
  Offensive The attacker with ball and defender approach within 1 m,  
1:1 duel the defender is next to or in front of the attacker, and . . .
  Attacker wins and overtakes . . . the attacker wins the duel and overtakes the defender 5
  Attacker retains ball possession . . . the attacker maintains ball possession but does not 3
but goes back overtake the defender
  Defender plays ball out of play . . . the defender plays the ball out of play 2
  Defender wins ball possession and . . . the defender conquers ball possession and is able to 0
can continue directly continue to play immediately
  Defender wins ball possession but . . . the defender conquers ball possession and is not able 0
cannot continue directly to continue to play immediately
  Receiving The attacker receives the ball and . . .  
  Under control . . . controls it 1
  Out of control . . . does not control it 0
  Foul The attackers makes a foul 0
Attacker without ball
  Running The attacker off the ball accelerates or moves in another  
action direction than the flow of the game and . . .
  Defender follows, creating more . . . a defender follows the attacker, hereby creating more 2
space for ball carrier space for the ball carrier
  Got open on own half . . . the attacker gets open on his own half of the playing 2
field
  Got open on opponent’s half . . . the attacker gets open on the opponents’ half of the 4
playing field
  Wrong direction/timing . . . the attacker does not get open and the defender does 0
not follow him
  Offside The attacker is in offside position 0
  In promising The attacker is in a promising position—that is, inside the 7
position penalty box, and a 2-m-wide line from the attacker towards
the goal is open
(Continued)
460 Quarterly Journal of Experimental Psychology 71(2)

Table 1. (Continued)

Role Action Outcome Definition Points

  Foul The attacker off the ball makes a foul 0
  Positioning A 1-m-wide line from the ball carrier to the attacker off the  
ball is . . .
  Open, own half, centre . . . open, and the attacker off the ball is on his own half of 2
the field and in the centre
  Open, own half, side . . . open, and the attacker off the ball is on his own half of 1
the field and at the side
  Open, opponents’ half, centre . . . open, and the attacker off the ball is on the opponents’ 5
half of the field and in the centre
  Open, opponents’ half, side . . . open, and the attacker off the ball is on the opponents’ 3
half of the field and at the side
  Marked . . . marked by a defender 0
Defender
  Defensive The defender and attacker with ball approach within 1 m,  
1:1 duel the defender is next to or in front of the attacker, and . . .
  Attacker wins and overtakes . . . the attacker wins the duel and overtakes the defender 0
  Attacker retains ball possession . . . the attacker maintains ball possession but does not 2
but goes back overtake the defender
  Defender plays ball out of play . . . the defender plays the ball out of play 2
  Defender wins ball possession and . . . the defender conquers ball possession and is able to 6
can continue directly continue to play immediately
  Defender wins ball possession but . . . the defender conquers ball possession and is not able 4
cannot continue directly to continue to play immediately
  Defensive The defender accelerates towards the attacker with ball and  
pressure approaches within 2 m (and more than 1 m) and . . .
  Attacker goes forward . . . the attacker with ball moves forward 0
  Attacker goes backward . . . the attacker with ball moves to the side or backwards 3
  Towards 1:1 duel . . . the defender approaches to within 1 m, and a 1:1 duel 2
follows
  Intercepting The defender intercepts a pass and . . .  
  Under control . . . controls the ball 6
  No control . . . does not control the ball 2
  Blocking The defender blocks a shot at goal and . . .  
shot
  Defender got ball possession . . . the defender gains ball possession 5
  Defender got no ball possession . . . the attackers maintain ball possession 2
  Offside trap The last defender steps forward to put an attacker offside  
and . . .
  Well executed . . . the defender wins ball possession due to offside 3
  Not well executed . . . the timing is not correct, and thus the attackers −3
maintain ball possession
  Foul The defender makes a foul . . .  
  Inside penalty area . . . inside the penalty area −9
  Own half . . . on his own half −6
  Opponent’s half . . . on the opponents’ half −3

Note: Reproduced from Van Maarseveen et al. (in press) with permission.

players, ts > 2.505, ps < .05, rs > .69), and reasonably good Perceptual-cognitive skill tests
intra- and inter-observer reliability (intra: mean percentage Stimulus materials.  The test stimuli for the perceptual-
of error = 5.7%, correlation rs > .87, ps < .001; inter: mean cognitive skill tests were identical to those used by Van
percentage of error = 13.7%, correlation rs > .89, ps < .001, Maarseveen et al. (2015) and consisted of short video
except for one category of positioning r = .39, p < .05). Two clips (5 to 10 seconds) of similar 3 versus 3 small-sided
participants did not participate in the in situ test because of games to those experienced in the in situ test, but recorded
injury and therefore were excluded from the study. one year earlier. The video images were recorded using
van Maarseveen et al 461

the same camera set-up as that employed during the in the participants had to select the option that they thought the
situ tests—that is, an elevated camera behind the goal ball carrier in the video clip was going to perform at the
defended by the attacking team. The elevated filming moment of occlusion, and in the decision-making test, the
position was used to give a good overview of the situation participants had to select what they thought was the best
and to help the participants in perceiving depth (Mann, option for the ball carrier. In the pattern recall test, at the
Farrow, Shuttleworth, & Hopwood, 2009). The video moment of occlusion the video clips were replaced with an
clips ended at a decisive moment in the game (i.e., the image of a blank playing field. The participants were asked
onset of a shot, pass, or dribble). In order to mask irrel- to recall the last seen positions of the players and the ball by
evant distracting features (e.g., other players who did not dragging Xs, Os, and a small star towards the respective
participate), the area outside the playing field was made positions of the defenders, attackers, and the ball (see also
black using Adobe Premiere Elements 9 (see Figure 1). To Van Maarseveen et al., 2015). No instructions were given
ensure that the video clips contained structured game play about the speed of response, and hence no analyses were
exemplifying the participants’ level of play, two highly conducted on response times.
experienced soccer coaches (each held the highest coach-
ing qualifications in the country and had over 25 years of Data analysis. For the decision-making test, the correct
coaching experience at national and international level) responses were determined by two highly experienced
independently rated the video clips on a 10-point Likert- soccer coaches (taking into account the average playing
type scale (0 = completely unstructured, 10 = completely level of the participants) until consensus was reached for
structured), and only clips rated by both coaches with every trial. Response accuracy was calculated by the num-
scores 7 or higher were selected (cf. Gorman et al., 2012, ber of correct responses divided by the number of trials,
2013; North & Williams, 2008; North et al., 2009). for both the decision-making and the anticipation test.
Fourteen video clips were selected and were included in Since previous research on pattern recall tests have
three occlusion conditions in the anticipation and decision- shown that (a) experienced athletes anticipate the locations
making test: occluded at the moment of foot–ball contact, of the players further in advance of their actual finishing
and 100 ms (3 frames) prior to and 100 ms (3 frames) after point (Gorman et al., 2012; Van Maarseveen et al., 2015),
foot–ball contact, as is a common way to test anticipation and (b) the disruptive effects of the 2D perspective of the
and decision-making skill (cf. Williams, Davids, & video clip should be taken into account (Van Maarseveen
Williams, 1999). For the pattern recall test the moment of et al., 2015), we assessed anticipatory pattern recall scores
occlusion is arbitrary (as long as it occurs at a moment of and used two methods to correct for the perspective effects:
structured game play), and therefore only the 14 video real-world coordinates and geometric pattern features,
clips occluded at the moment of foot–ball contact were identical to those in Van Maarseveen et al. (2015). For the
used in this test. Two additional video clips were selected real-world coordinates method, the pixel coordinates were
as familiarization trials and were used in each test. first transformed into real-world coordinates (using Direct
Linear Transformation; Abdel-Aziz & Karara, 1971), and
Procedure. Participants performed the perceptual- then the spatial error of the recalled player positions was
cognitive skill tests while seated in front of a large screen calculated for the final frame of the video clip and for 60
(i.e., the distance between the participant and the screen subsequent frames. The smallest recall error was identified
was about 2.5 m) onto which a projector (ASK Proxima and was recorded as the anticipatory recall score. For the
C175, resolution 1024 × 768) displayed the video clips geometric pattern features method, the angles between the
with an image that subtended a viewing angle of approxi- three attackers and the angles between the three defenders
mately 23° horizontally and 18° vertically. The participants were calculated and compared to the answer templates of
wore SensoMotoric Instruments (SMI; Teltow, Germany) the final frame and the 60 subsequent frames. The smallest
Eye Tracking Glasses, a binocular eye tracking device that average error across the attackers and defenders indicated
recorded eye movements at 24 Hz. A one-point calibra- the anticipatory pattern feature score.
tion (as advised by the manufacturer) using a small cross Malfunctioning of the eye-tracker (e.g., calibration
in the centre of the screen was performed before starting problems) reduced the amount of gaze behaviour data.
each perceptual-cognitive skill test. Each test started with With our main focus being to analyse differences in gaze
instructions and two familiarization trials. The test video behaviour between the three perceptual-cognitive tests,
clips were displayed in random order, and the order of the gaze behaviour data of a particular video clip were only
tests was counterbalanced across participants. included in the analyses if they were available for all three
The video clips were displayed, and in the anticipation tests for a particular participant. This means that only
and decision-making tests the clips were replaced immedi- video clips occluded at the moment of foot–ball contact
ately afterwards with a response slide showing buttons for could be included to make valid comparisons across the
four possible options: shoot, dribble, pass to the left team- three test-types, as this occlusion condition was the only
mate, and pass to the right teammate. In the anticipation test, one used in the test of pattern recall). This resulted in a
462 Quarterly Journal of Experimental Psychology 71(2)

total of 264 trials (88 video clips × 3 tests) originating from fixation was on the ith area of interest. Gaze entropy can
13 participants. then be calculated using Ellis and Stark’s (1986)
The gaze behaviour was analysed frame by frame for equation:
the duration of the video clips. A fixation was defined as
gaze maintained on any area of the video display for a n  n 
period equal to or in excess of 125 ms or three sequential Entropy = ∑ p (i ) 

∑ p( j | i) log 2 p( j | i) 

frames (cf. Savelsbergh et al., 2002; Vaeyens, Lenoir, i =1 j =1

Williams, Mazyn, et al., 2007; Vaeyens, Lenoir, Williams,

& Philippaerts, 2007; Williams & Davids, 1998). The gaze In which p(i) is the zero-order probability of fixating on
behaviour of 30 randomly selected trials (i.e., 11%) was the ith area of interest (based on the percentage of total
recoded by the same experimenter to assess intra-rater reli- viewing time towards it), p(j|i) is the conditional probabil-
ability, and a second experimenter independently coded 35 ity of viewing area of interest j if the previous fixation was
random trials (i.e., 13%) to determine inter-rater reliability. on area i, and n is the number of areas of interest (i.e., 10
The intra-rater and inter-rater reliability both indicated in the current study). A higher entropy value represents a
good to almost perfect agreements (Hallgren, 2012), greater level of randomness in the gaze behaviour.
κ = .86 and κ = .79, respectively.
For each of the three tests, the four commonly used Statistical analyses. We performed some manipulation
dependent variables, search rate, fixation duration, per- checks to examine the internal validity of the perceptual-
centage viewing time, and fixation order, were calculated cognitive skill tests and any learning effects as a result of
for each trial and were then averaged to provide a mean watching the same video clips multiple times. For both the
value for each participant. Search rate was defined as the anticipation and decision-making tests, the accuracy scores
number of fixations per second, the mean fixation dura- of the three occlusion conditions (i.e., −100 ms, 0 ms, and
tion was determined per trial, and the percentage viewing +100 ms) were subjected to a repeated measures analysis of
time was calculated as the percentage of total viewing variance (ANOVA). To analyse whether there was a learn-
time spent on each of 10 areas of interest: attacker in pos- ing effect due to the repeated presentation of each of the 14
session of the ball (AB), attacker without ball (A), clips within one test, a repeated measures ANOVA was con-
defender (D), goal keeper (GK), ball (B), field/space (F), ducted on the accuracy scores of the first, second, and third
central spot in field/space (CF), attacker with ball closely presentation of the clips within the anticipation test and
marked by defender (AB/D), attacker without ball closely decision-making test separately. In addition, the accuracy
marked by defender (A/D), and other (O). The fixation scores of participants performing a test as the first, second,
order referred to the search strategy that was used by the or third test were compared for each perceptual-cognitive
participants and was calculated for each trial as the num- skill test using one-way ANOVAs to check whether there
ber of times per second that participants alternated their was any learning effect as a result of using the same video
gaze between the player in possession of the ball, some clips in all three perceptual-cognitive skill tests.
other area in the video clip, and back to the player in pos- Pearson’s correlation coefficients were calculated to
session of the ball (cf. Vaeyens, Lenoir, Williams, Mazyn, investigate the relationship between the performance
et al., 2007; Vaeyens, Lenoir, Williams, & Philippaerts, scores in situ and in the three tests of perceptual-cognitive
2007; Williams & Davids, 1998; Williams, Davids, skill, and for any relationship between the in situ perfor-
Burwitz, & Williams, 1994). mance scores and the gaze measures on the three tests.
To gain more insight into the visual search strategies of Also, a regression analysis was performed to examine
the participants, we analysed to what degree the gaze whether the in situ performance score could be predicted
behaviour was structured or randomly distributed by cal- by the perceptual-cognitive skill test scores. Moreover, we
culating gaze entropy (Allsop & Gray, 2014; Button, performed a median split on the in situ performance scores
Dicks, Haines, Barker, & Davids, 2011; Ryu, Mann, and used independent samples t-tests to see whether there
Abernethy, & Poolton, 2016) for each test for each partici- were any differences in how the best and worst performing
pant. To do this, we first calculated the number of fixation players in situ fared on the tests of perceptual-cognitive
transitions between the 10 areas of interest by producing a skill, and, vice versa, we performed median splits on the
first-order transition frequency matrix of p(i to j), in which performance scores of the tests of perceptual-cognitive
i represents the area of interest before the transition, and j skill and examined whether there were any differences in
represents the area of interest after the transition. Separate the in situ performance scores. Mean values for the gaze
matrices were calculated for each participant and for each behaviour variables search rate, fixation duration, fixation
test, and these were converted into conditional transition order, and entropy were compared across the three percep-
probability matrices of p(j|t), which gives a first-order tual-cognitive tests using separate three-way repeated
Markov process where the probability of fixating on the jth measures ANOVAs. Percentage viewing time was ana-
area of interest is calculated, given that the previous lysed using a 10 (area of interest) × 3 (perceptual-cognitive
van Maarseveen et al 463

skill test) ANOVA with repeated measures on both factors. expressed in pattern features, t(18) = 0.087, p = .932,
A Greenhouse–Geisser correction was applied to the d = 0.04. And vice versa, after performing median splits on
degrees of freedom when the assumption of sphericity was the performance scores of the perceptual-cognitive skill
violated. tests, no differences were found between the best and
worst performing players on the in situ test, ts < 0.960,
ps > .350, ds < 0.46.
Results
The correlations between performance on the three tests
Manipulation checks of perceptual-cognitive skill can also be found in Table 2.
Occlusion.  For the anticipation test, there was a signifi- Again there were no significant relationships between per-
cant effect of occlusion time on the accuracy scores, F(1.58, formance on any of the three tests (rs < .354, ps > .106).
33.16) = 10.351, p < .001, η2p  = .330. Pairwise comparisons The only significant correlation was a predictable one
revealed that the –100-ms occlusion condition was more between the two varieties of pattern recall score (r = .553,
difficult (M = 64.6%, SD = 10.5) than the 0-ms (M = 71.1%, p < .05)—that is, the pattern recall score expressed in real-
SD = 10.2) and +100-ms (M = 74.0%, SD = 9.2) occlusion world coordinates and the pattern recall score expressed in
conditions (p < .001; p < .05, respectively). There was pattern features.
no difference between the 0-ms and +100-ms occlusion The correlations between the in situ performance score
(p = .726). For the decision-making test, there was no sig- and the gaze behaviour variables of the perceptual-cogni-
nificant effect of occlusion on the decision-making test tive skill tests can be found in Table 3. Again almost none
scores, F(2, 42) = 0.554, p = .579, η2p  = .026. of the gaze variables were significantly related to in situ
performance, with the exception being a significant corre-
Learning effects.  No significant differences were found lation between the in situ performance score and the per-
between the accuracy scores of the individual video clips centage of time the participants watched the ball during the
that the participants saw for the first, second, or third time decision-making test (r = −.662, p < .05), indicating that
2
in the anticipation test, F(2, 42) = 0.319, p = .729, ηp  = .015, participants who scored high on the in situ test watched the
nor in the decision-making test, F(2, 42) = 1.144, p = .328, ball less during the decision-making test.
η2p  = .052. The order in which the three tests were pre-
sented had no impact on the results, with no significant Gaze behaviour
differences found between participants who performed Search rate.  The mean search rate (and SD) for each test
each test as the first, second, or third of the three tests is displayed in Figure 2A. There was a significant effect of
(anticipation test, p = .334, η2p  = .109; decision-making test on the mean search rate, F(2, 24) = 10.021, p < .001,
test, p = .646, η2p  = .045; or pattern recall test expressed η2p  = .455. Post hoc Bonferroni corrected pairwise com-
in real-world coordinates, p = .936, η2p  = .007, or pattern parisons revealed that the differences were largely a result
recall features, p = .409, η2p  = .090). Thus, there were no of the differences in gaze when performing the test of
learning effects during or across the tests as a result of pattern recall. Participants made significantly more fixa-
repetitively watching the same video clips. tions per second during the pattern recall test than they did
during the anticipation test (p < .05, d = 1.18), and the dif-
Relationship between performance on the in situ and percep- ference between the pattern recall test and the decision-
tual-cognitive skill tests.  The correlations between the in situ making test approached significance (p = .077, d = 0.80).
performance score and the scores for the anticipation, The difference between the anticipation test and the deci-
decision-making, and pattern recall tests can be found in sion-making test was not significant (p = .184, d = 0.44).
Table 2. There were no significant correlations between
the in situ performance score and any of the three tests of Fixation duration. The mean fixation duration (and
perceptual-cognitive skill (rs < .262, ps > .265). No signifi- SD) for each test is displayed in Figure 2B. There was
cant regression equation was found that could predict the a significant effect of test on the mean fixation duration,
in situ performance score on the basis of performance in F(2, 24) = 6.753, p < .05, η2p  = .360. Again the post hoc
the perceptual-cognitive skill tests, F(4, 15) = 1.074, Bonferroni corrected pairwise comparisons revealed that
p = .404. Furthermore, after a median split on the in situ the differences in fixation duration were largely a result
performance scores had been performed, the performance of fixations of shorter duration during the test of pattern
of the high- and low-performing participants was recall: The fixation duration was significantly shorter
compared on the perceptual-cognitive skill tests. There during the test of pattern recall than it was during the
were no significant differences between the best and worst test of anticipation (p < .05, d = 0.98), and the differences
performing players on the anticipation test, t(18) = 0.310, with the decision-making test approached significance
p = .760, d = 0.15, decision-making test, t(18) = −0.882, (p = .059, d = 0.68). The mean fixation durations during
p = .389, d = 0.42, and pattern recall test expressed in real- the decision-making test and anticipation test were not
world coordinates, t(18) = 1.309, p = .207, d = 0.62, or significantly different (p = .915, d = 0.26).
464 Quarterly Journal of Experimental Psychology 71(2)

Table 2.  Correlations between in situ score and anticipation, decision-making, and pattern recall scores.

Test 1 2 3 4
1 In situ score  
2 Anticipation .138  
3 Decision making −.204 .017  
4 Pattern recall real world coordinates .262 −.354 .273  
5 Pattern recall features .079 −.085 .306 .553*

Note: 1-4 across the top equal 1-4 reported in the first column.
*p < .05.

Table 3.  Correlations between in situ score and gaze behaviour on the anticipation, decision-making, and pattern recall tests.

Gaze variable Anticipation Decision making Pattern recall

Search rate .090 .103 .350
Fixation duration .031 .101 −.330
Fixation order .261 −.590 .207
Entropy .373 .468 .384
Area of interest
 AB .299 .190 .342
 A −.272 −.241 −.041
 D .028 .209 −.463
 GK −.208 .104
 B −.059 −.662* −.174
 F .068 .219 .281
 CF .074 .569 .004
 A/D .250 −.215 −.207
 AB/D −.040 −.223 −.373
 O −.267 −.030 .181

Note: Areas of interest: Attacker in possession of the ball (AB), attacker without ball (A), defender (D), goal keeper (GK), ball (B), field/space (F),
central spot in field/space (CF), attacker without ball closely marked by defender (A/D), attacker with ball closely marked by defender (AB/D), and
other (O).
*p < .05.

Fixation order. The mean fixation order (and SD) for in entropy between the tests of anticipation and decision
each test is displayed in Figure 2C. There was a significant making approached significance (p = .078, d = 0.54). The
effect of test on the mean fixation order, F(2, 24) = 6.5510, entropy during the decision-making test and the pattern
2
p < .05, ηp  = .353. Post hoc Bonferroni corrected pairwise recall test did not differ (p = .826, d = 0.27).
comparisons revealed the difference to be a result of sig-
nificantly fewer fixation shifts (from the ball carrier to Percentage viewing time.  The percentage viewing time
another location and back) in the test of pattern recall than per area of interest, separated for each test, is displayed
in the test of decision making (p < .05, d = 1.34). There in Figure 3. A significant main effect was found for area
2
were no differences in fixation order between the decision- of interest, F(9, 108) = 94.208, p < .001, ηp  = .887, but this
making and the anticipation test, and between the pattern was overridden by a significant area of Interest × Test inter-
2
recall test and the anticipation test (p = .433, d = 0.55; action effect, F(18, 216) = 11.835, p < .001, ηp  = .497. Post
p = .112, d = 0.94, respectively). hoc analyses revealed that once again the differences were
largely due to differences in the test of pattern recall, with
Gaze entropy.  The mean gaze entropy (and SD) for each participants looking less at the attacker with ball than they
test is displayed in Figure 2D. The test performed by the did during the tests of anticipation and decision making
participant had a significant effect on gaze entropy, F(2, (both ps < .001, ds > 2.26). Participants looked more at a
24) = 8.638, p < .05, η2p  = .419. Again the difference was central location in the visual field during the pattern recall
largely a result of a difference in the test of pattern recall, test than during the other tests (both ps < .05, ds > 1.55),
with gaze entropy being significantly higher, and thus and they tended to look less at the attackers without the
less structured, in the test of pattern recall than it was in ball during the pattern recall test than during the anticipa-
the test of anticipation (p < .001, d = 0.72). The difference tion test (p = .061, d = 0.97).
van Maarseveen et al 465

Figure 2.  Mean search rate (A), fixation duration (B), fixation order (C), and entropy (D) for the anticipation, decision-making, and
pattern recall tests. Error bars represent standard deviation; *p < .05, **p < .001.

Figure 3.  Mean percentage viewing time per area of interest for the anticipation, decision-making, and pattern recall tests.
Attacker in possession of the ball (AB), attacker without ball (A), defender (D), goal keeper (GK), ball (B), field/space (F), central
spot in field/space (CF), attacker without ball closely marked by defender (A/D), attacker with ball closely marked by defender
(AB/D), and other (O). Error bars represent standard deviation; *p < .05, **p < .001.
466 Quarterly Journal of Experimental Psychology 71(2)

General discussion pick up information to control their movements or actions.

According to the two-visual system model of Milner and
The aim of this study was to examine how well in situ Goodale (1995), excluding action from the participant
performance in a small-sided soccer game could be pre- response would diminish the contribution of the dorsal
dicted using video-based perceptual-cognitive skill tests of “vision-for-action” system (Van der Kamp et al., 2008).
anticipation, decision making, and pattern recall. We also Although the implications of the distinction between per-
examined the degree to which the three tests of perceptual- ception and action have previously been shown to be par-
cognitive skill were related by exploring the correlations ticularly relevant for anticipation (Dicks et al., 2009; Mann
between the tests and the similarity of the gaze of partici- et al., 2007; Van der Kamp et al., 2008), it seems reasona-
pants when performing those tasks. The findings reveal ble to expect similar implications for the test of decision
that the in situ performance of the soccer players could not making (see Oudejans et al., 1996). The current study did
be predicted by their performance on the tests of percep- not reveal expertise differences in any of the perceptual-
tual-cognitive skill. Moreover, even a median split of the cognitive tests. It is possible that expertise-related differ-
participants on the basis of their in situ performance score ences in performance on tests of anticipation and decision
failed to reveal any significant differences in performance making could be found if those tests incorporated suitable
on any of the three tests of perceptual-cognitive skill, and, movement responses. The same probably cannot be said
vice versa, median splits on the performance scores of the for the test of pattern recall, as there is unlikely to be an
perceptual-cognitive skill tests failed to reveal significant equivalent test to the one used here that would incorporate
differences in in situ performance scores. These findings an action.
indicate that the traditional video-based tests of anticipa- Third, in contrast to those previous studies that have
tion, decision making, and pattern recall may not be as shown perceptual-cognitive skill differences between lev-
strong a determinant of actual performance as has been els of expertise, the current study has shown that these
previously been assumed, and therefore caution is required video-based tests appear to be unsuitable to detect within-
at this stage in using them as conventional tests of talent in group differences between athletes of a comparable skill
dynamic time-constrained motor tasks. level. The expert–novice paradigm that is heavily relied on
There are a number of possible explanations for the in studies of expertise compares the performance of par-
lack of any relationship between performance on the in ticipants who possess very different levels of skill.
situ test of playing ability and on the video-based tests of However, in a within-group comparison the more subtle
perceptual-cognitive skill. First, it could be that the per- differences between more successful and less successful
ceptual-cognitive skills that were tested in this study are performers within a group are compared. It could be that
not necessary requirements of actual performance in game the video-based tests of the type used in this study are not
situations (see Ward, Williams, & Hancock, 2006; specific enough to detect these more subtle within-group
Williams & Ericsson, 2005) and consequently would not differences. Moreover, it could be that performance on the
reflect the processes required for optimal on-field perfor- perceptual-cognitive skill tests is a by-product rather than
mance. This is possible but seems unlikely given the con- a characteristic of expertise (though see Williams &
sistent finding of expert-related differences in performance Davids, 1995). This would suggest that caution is neces-
on these types of tasks (Abernethy & Russell, 1987; sary regarding the type of scenarios and tests in which
Gorman et al., 2012; Helsen & Pauwels, 1993; Savelsbergh these video-based perceptual-cognitive skill tests are used.
et al., 2002; Vaeyens, Lenoir, Williams, Mazyn, et al., Finally, it could be argued that the sensitivity of the in
2007; Williams & Ward, 2007). situ test of playing ability might be insufficient to pick up on
Second, it could be that the perceptual-cognitive skill any differences in skill level between the players. It could be
tests are not sufficiently representative of the actual perfor- that the measure of in situ performance is too broad, and
mance setting. The perceptual-cognitive skill tests are encapsulates other factors like speed, physical fitness, or
video based, and these video displays provide a less than motor skills. Or it could be that the in situ measure is not
veridical simulation of the visual information that is avail- sensitive enough to differentiate on-field performance.
able in the natural performance setting (Abernethy, Gill, However, Van Maarseveen et al. (in press) showed that both
Parks, & Packer, 2001; Dicks et al., 2009). Projecting 3D the concurrent validity and construct validity of the in situ
visual information onto a 2D display causes a loss of ste- performance measure were good in a homogeneous skilled
reoscopic depth information and a reduction in visual field group of soccer players—that is, the performance scores
and object size (Abernethy et al., 2001), and in this way it measured using the notational analysis system significantly
is difficult to adequately maintain the dynamic nature of correlated with the subjective judgments of a highly experi-
the situation (Mann et al., 2007). Furthermore, the partici- enced coach, and the notational analysis system demon-
pants in this study were required to respond to the video strated good ability to discriminate between the high- and
clips using a button-press on a keyboard, meaning they low-skilled players within the group. Therefore, it seems
were required to make a perceptual judgement and not to unlikely that the in situ performance measure is responsible
van Maarseveen et al 467

for the lack of any significant relationship between the pattern of play (Abernethy, 1988; Ryu, Abernethy, Mann,
scores of playing ability and perceptual-cognitive skill Poolton, & Gorman, 2013, 2015). The evidence for differ-
measured in this study. ences in the way that the tests were performed is less clear,
This study provides some evidence to suggest that the though, when comparing the tests of anticipation and deci-
tests of perceptual-cognitive skill are testing unique attrib- sion making, with no significant differences between any
utes that are not strongly related to each other. In particu- of the measures of gaze behaviour when those two tests
lar, pattern recall skill does not appear to be the were performed. There was only a borderline difference in
underpinning skill that supports anticipation and decision gaze entropy (p = .078), providing some suggestion that
making, as has been previously suggested (e.g., Farrow gaze was more structured when performing the test of
et al., 2010; Gorman et al., 2012, 2013; Williams & Davids, anticipation than it was when performing the test of deci-
1995). The outcome measures for performance on the sion making. On the basis of the measures of gaze it
three tests of perceptual-cognitive skill provide the best appears that the underlying processes responsible for
evidence to suggest that all three tests are different, with anticipation and decision making might be much less dis-
there being no significant correlations between perfor- tinct than that responsible when performing the test of pat-
mance on any of those three tests (ps > .106). This is con- tern recall.
sistent with earlier studies that have found no significant It does appear on balance, though, that participants did
correlation between the anticipation and pattern recogni- perform different tasks when performing the tests of antici-
tion skills of expert soccer players (North et al., 2009), and pation and decision making. The instructions to participants
between the anticipation and pattern recall skills of expert in the test of anticipation were to predict what would happen
rugby players (Farrow et al., 2010). Our findings highlight next in the clip, and in the test of decision making to choose
the need for a better understanding of the types of percep- the best option available to the ball carrier at the moment of
tual-cognitive skills required to attain expert performance, occlusion. It is possible, though, that the participants com-
and whether there are other attributes that may underpin pleted the anticipation test as they would the decision-mak-
those skills. For example, future research could incorpo- ing test, or, vice versa, completed the decision-making test
rate a test of long-term working memory to determine as they would a test of anticipation. Participants chose the
whether performance on any of the perceptual-cognitive same response on the tests of anticipation and decision mak-
tests is predicted by or related to long-term working mem- ing in only 65% of cases (SD = 10%), providing some sug-
ory (Ericsson & Kintsch, 1995). gestion that the tasks were done differently (participants
However, in contrast to the performance measures, the chose between four alternatives, and therefore the likeli-
evidence for unique attributes is less clear on the basis of hood of identical answers was 25% by chance). However,
the measurement of gaze when performing those tests. much stronger evidence that the tests were performed in a
Based on the original findings of Yarbus (1967) and more unique fashion was found in the lack of correlation between
recently on those in the sport domain (Gorman et al., 2015; the test scores for anticipation and decision making, and by
North et al., 2009), we reasoned that differences in gaze the fact that, as expected, we found a significant effect of
behaviour when performing the tests would provide sup- occlusion condition on the accuracy scores in the anticipa-
port for the idea that different underlying processes drive tion test, meaning that providing the participants with more
the way that the three different perceptual-cognitive tests information (i.e., a later occlusion condition) resulted in bet-
are performed (Gorman et al., 2015; North et al., 2009). ter accuracy scores, whereas in the decision-making tests
Gaze behaviour when performing the test of pattern recall we did not find an effect of occlusion condition. Thus in the
was clearly different to that when performing the other two decision-making test, providing the participants with more
tests, with significant differences found for each of the five information did not result in better accuracy scores, indicat-
measures of gaze behaviour (search rate, fixation duration, ing that they did not anticipate in the decision-making test.
fixation order, entropy, and percentage time spent viewing Overall, this implies that the participants approached these
the areas of interest) when compared to the way that the two tests differently and that these tests did not measure the
tests of anticipation and/or decision making were per- same quality.
formed. This provides strong evidence for the unique char- The findings of the present study highlight that percep-
acteristic being tested when performing a test of pattern tual-cognitive skill tests in their current form might not be
recall. During the pattern recall test the participants main- sufficiently representative of on-field performance to reli-
tained a high search rate, presumably to scan and memo- ably test for differences in skill between players of dynamic
rize the locations of the pattern elements as accurately as ball sports. Despite the findings of earlier studies that have
possible. They also looked more towards the centre of the shown video-based tests to be sensitive enough to pick up
field of view and tended to look less at the attackers than on group-based differences in skill, at present they seem to
during the anticipation and decision-making tests, proba- be less reliable for detecting within-group differences.
bly extracting information from outside the central area Therefore, the findings question the suitability of video-
using peripheral vision to get a better overview of the based perceptual-cognitive skill tests for studying
468 Quarterly Journal of Experimental Psychology 71(2)

perceptual-motor expertise (see Dicks et al., 2010), and Abernethy, B. (1988). Visual search in sport and ergonomics: Its
this suggests that caution is warranted when using these relationship to selective attention and performer expertise.
tests for talent identification or to evaluate the effective- Human Performance, 1, 205–235.
ness of interventions. Alternatives to the paradigms Abernethy, B., Gill, D. P., Parks, S. L., & Packer, S. T. (2001).
Expertise and the perception of kinematic and situational
employed in traditional laboratory studies have been pro-
probability information. Perception, 30, 233–252.
vided by recent technological advances such as mobile eye
Abernethy, B., & Russell, D. G. (1987). The relationship
tracking devices (Van Maarseveen et al., 2016; Pluijms, between expertise and visual search strategy in a racquet
Cañal-Bruland, Kats, & Savelsbergh, 2013), event-related sport. Human Movement Science, 6, 283–319.
visual occlusion goggles (Mann et al., 2010; Oudejans, Allard, F., Graham, S., & Paarsalu, M. L. (1980). Perception in
van de Langenberg, & Hutter, 2002), and virtual reality sport: Basketball. Journal of Sport Psychology, 2, 14–21.
(Bideau et al., 2010; Correia, Araújo, Cummins, & Craig, Allsop, J., & Gray, R. (2014). Flying under pressure: Effects of
2012). In order to accurately capture the perceptual-motor anxiety on attention and gaze behavior in aviation. Journal
performances of athletes, we suggest using in situ research of Applied Research in Memory and Cognition, 3, 63–71.
designs so that the task constraints represent as accurately Baker, J., Cote, J., & Abernethy, B. (2003). Learning from the
as possible the natural performance setting of the athlete experts: Practice activities of expert decision makers in
and actual movement responses are required. sport. Research Quarterly for Exercise and Sport, 74, 342–
347.
Bideau, B., Kulpa, R., Vignais, N., Brault, S., Multon, F., &
Conclusion Craig, C. (2010). Using virtual reality to analyze sports per-
formance. IEEE Computer Graphics and Applications, 30,
Our results show that the on-field performance of talented 14–21.
soccer players is not predicted by performance on a com- Button, C., Dicks, M., Haines, R., Barker, R., & Davids, K.
mon set of tests of perceptual-cognitive skill. The test of (2011). Statistical modelling of gaze behaviour as categori-
pattern recall appears to be driven by a different underly- cal time series: What you should watch to save soccer penal-
ing process from that used when performing tests of antici- ties. Cognitive Processing, 12, 235–244.
pation and decision making, with the results of the test of Correia, V., Araújo, D., Cummins, A., & Craig, C. M. (2012).
pattern recall being unrelated to those of the other two tests Perceiving and acting upon spaces in a VR rugby task:
and relying on significantly different gaze behaviour. Expertise effects in affordance detection and task achieve-
Although performance on the test of anticipation is unre- ment. Journal of Sport & Exercise Psychology, 34, 305–321.
lated to that on the test of decision making, gaze behaviour Davids, K., Araújo, D., Correia, V., & Vilar, L. (2013). How
small-sided and conditioned games enhance acquisition of
remains largely unchanged on the two tests providing
movement and decision-making skills. Exercise and Sport
some suggestion that the underlying processes when per-
Sciences Reviews, 41, 154–161.
forming those two tests are less distinct. In situ research Dicks, M., Button, C., & Davids, K. (2010). Examination of gaze
designs may be more suitable to accurately capture the behaviours under in situ and video simulation task con-
perceptual-motor performance of athletes so that the task straints reveals differences in information pickup for per-
constraints and response mode represent as accurately as ception and action. Attention, Perception & Psychophysics,
possible the actual skill and context in which the athlete is 72, 706–720.
engaged. Dicks, M., Davids, K., & Button, C. (2009). Representative
task designs for the study of perception and action in sport.
Acknowledgements International Journal of Sport Psychology, 40, 506–524.
Dokter, R. (1993). The Dutch vision on youth soccer. Zeist,
The authors would like to thank head coach Maria van Kortenhof,
Netherlands: Koninklijke Nederlandse Voetbalbond.
the other staff members, and the players of the CTO Amsterdam
Ellis, S. R., & Stark, L. (1986). Statistical dependency in vis-
Talent Team for their cooperation.
ual scanning. Human Factors: The Journal of the Human
Factors and Ergonomics Society, 28, 421–438.
Disclosure statement Ericsson, K. A., & Kintsch, W. (1995). Long-term working
No potential conflict of interest was reported by the authors. memory. Psychological Review, 102, 211–245.
Ericsson, K. A., Patel, V., & Kintsch, W. (2000). How experts’
ORCID adaptations to representative task demands account for the
expertise effect in memory recall: Comment on Vicente and
David L. Mann http://orcid.org/0000-0001-7476-6939
Wang (1998). Psychological Review, 107, 578–592.
Ericsson, K. A., & Smith, J. (1991). Prospects and limits of
References the empirical study of expertise: An introduction. In K.
Abdel-Aziz, Y. I., & Karara, H. M. (1971). Direct linear transfor- A. Ericsson & J. Smith (Eds.) Toward a general theory
mation from comparator coordinates into objects space coor- of expertise: Prospects and limits (pp. 1–38). Cambridge:
dinates in close-range photogrammetry. In Proceedings of Cambridge University Press.
the symposium on close-range photogrammetry (pp. 1–18). Farrow, D., McCrae, J., Gross, J., & Abernethy, B. (2010).
Urbana, IL: American Society of Photogrammetry. Revisiting the relationship between pattern recall
van Maarseveen et al 469

and anticipatory skill. International Journal of Sport locomotion on street-crossing behavior. Ecological
Psychology, 41, 91–106. Psychology, 8, 259–267.
Gorman, A. D., Abernethy, B., & Farrow, D. (2012). Classical Pinder, R. A., Headrick, J., & Oudejans, R. R. D. (2015). Issues
pattern recall tests and the prospective nature of expert and challenges in developing representative tasks in sport.
performance. The Quarterly Journal of Experimental In J. Baker & D. Farrow (Eds.), The Routledge handbook of
Psychology, 65, 1151–1160. sports expertise (pp. 269–281). London: Routledge.
Gorman, A. D., Abernethy, B., & Farrow, D. (2013). Is the Pluijms, J. P., Cañal-Bruland, R., Kats, S., & Savelsbergh, G. J.
relationship between pattern recall and decision-making P. (2013). Translating key methodological issues into tech-
influenced by anticipatory recall? The Quarterly Journal of nological advancements when running in-situ experiments
Experimental Psychology, 66, 2219–2236. in sports: An example from sailing. International Journal of
Gorman, A. D., Abernethy, B., & Farrow, D. (2015). Evidence Sports Science & Coaching, 8, 89–104.
of different underlying mechanisms in pattern recall and Ryu, D., Abernethy, B., Mann, D. L., & Poolton, J. M. (2015).
decision-making. The Quarterly Journal of Experimental The contributions of central and peripheral vision to exper-
Psychology, 68, 1813–1831. tise in basketball: How blur helps to provide a clearer
Gréhaighne, J. F., Godbout, P., & Bouthier, D. (2001). The teach- picture. Journal of Experimental Psychology: Human
ing and learning of decision making in team sports. Quest, Perception and Performance, 41, 167–185.
53, 59–76. Ryu, D., Abernethy, B., Mann, D. L., Poolton, J. M., & Gorman,
Hallgren, K. A. (2012). Computing inter-rater reliability for A. D. (2013). The role of central and peripheral vision in
observational data: An overview and tutorial. Tutorials in expert decision-making. Perception, 42, 591–607.
Quantitative Methods for Psychology, 8, 23–34. Ryu, D., Mann, D. L., Abernethy, B., & Poolton, J. M. (2016).
Helsen, W., & Pauwels, J. M. (1993). The relationship between Gaze-contingent training enhances perceptual skill acquisi-
expertise and visual information processing in sport. In J. tion. Journal of Vision, 16, 1–21.
L. Starkes & F. Allard (Eds.), Cognitive issues in motor Savelsbergh, G. J. P., van der Kamp, J., Williams, A. M., &
expertise (pp. 109–134). Amsterdam, Netherlands: Elsevier Ward, P. (2005). Anticipation and visual search behaviour
Science Publishers. in expert soccer goalkeepers. Ergonomics, 48, 1686–1697.
Jones, C. M., & Miles, T. R. (1978). Use of advance cues in pre- Savelsbergh, G. J. P., Williams, A. M., Van der Kamp, J., &
dicting the flight of a lawn tennis ball. Journal of human Ward, P. (2002). Visual search, anticipation and expertise
movement studies, 4, 231–235. in soccer goalkeepers. Journal of Sports Sciences, 20, 279–
Mann, D. L., Abernethy, B., & Farrow, D. (2010). Action speci- 287.
ficity increases anticipatory performance and the expert Travassos, B., Aráujo, D., Davids, K., O’Hara, K., Leitão, J., &
advantage in natural interceptive tasks. Acta Psychologica, Cortinhas, A. (2013). Expertise effects on decision-making
135, 17–23. in sport are constrained by requisite response behaviours
Mann, D. L., Farrow, D., Shuttleworth, R., & Hopwood, M. – a meta-analysis. Psychology of Sport and Exercise, 14,
(2009). The influence of viewing perspective on decision- 211–219.
making and visual search behaviour in an invasive sport. Vaeyens, R., Lenoir, M., Williams, A. M., Mazyn, L., &
International Journal of Sport Psychology, 40(4), 546–564. Philippaerts, R. M. (2007). The effects of task constraints on
Mann, D. L., & Savelsbergh, G. J. P. (2015). Issues in the meas- visual search behaviour and decision-making skill in youth
urement of anticipation. In J. Baker & D. Farrow (Eds.) The soccer players. Journal of Sport & Exercise Psychology, 29,
Routledge Handbook of Sports Expertise (pp. 269–281). 147–169.
London: Routledge. Vaeyens, R., Lenoir, M., Williams, A. M., & Philippaerts, R.
Mann, D. T. Y., Williams, A. M., Ward, P., & Janelle, C. M. M. (2007). The mechanisms underpinning successful deci-
(2007). Perceptual cognitive expertise in sport: A meta-anal- sion-making in skilled youth soccer players: An analysis of
ysis. Journal of Sport & Exercise Psychology, 29, 457–478. visual search behaviours. Journal of Motor Behavior, 39,
Milner, A.D., & Goodale, M.A. (1995). The visual brain in 395–408.
action. Oxford: Oxford University Press. Van der Kamp, J., Rivas, F., Van Doorn, H., & Savelsbergh, G.
North, J. S., & Williams, A. M. (2008). Identifying the critical J. P. (2008). Ventral and dorsal system contributions to vis-
time period for information extraction when recognizing ual anticipation in fast ball sports. International Journal of
sequences of play. Research Quarterly for Exercise and Sport Psychology, 39, 100–130.
Sport, 79, 268–273. Van Maarseveen, M. J. J., Oudejans, R. R. D., & Savelsbergh, G.
North, J. S., Williams, A. M., Hodges, N., Ward, P., & Ericsson, J. P. (2015). Pattern recall skills of talented soccer players:
K. A. (2009). Perceiving patterns in dynamic action Two new methods applied. Human Movement Science, 41,
sequences: Investigating the processes underpinning stimu- 59–75.
lus recognition and anticipation skill. Applied Cognitive Van Maarseveen, M. J. J., Oudejans, R. R. D., & Savelsbergh,
Psychology, 23, 878–894. G. J. P. (in press). System for notation analysis in small-
Oudejans, R. R. D., van de Langenberg, R. W., & Hutter, V. sided soccer games. International Journal of Sports Science
(2002). Aiming at a far target under different viewing con- & Coaching.
ditions: visual control in basketball jump shooting. Human Van Maarseveen, M. J. J., Savelsbergh, G. J. P., & Oudejans, R.
Movement Science, 21, 457–480. D. D. (2016). In-situ examination of decision making skills
Oudejans, R. R. D., Michaels, C. F., Van Dort, B., & Frissen, and gaze behaviour of basketball players. Manuscript in
E. J. P. (1996). To cross or not to cross: the effect of preparation.
470 Quarterly Journal of Experimental Psychology 71(2)

Ward, P., & Williams, A. M. (2003). Perceptual and cogni- Williams, A. M., Davids, K., Burwitz, L., & Williams, J. G.
tive skill development in soccer: The multidimensional (1994). Visual search strategies in experienced and inexpe-
nature of expert performance. Journal of Sport & Exercise rienced soccer players. Research Quarterly for Exercise and
Psychology, 25, 93–111. Sport, 65, 127–135.
Ward, P., Williams, A. M., & Hancock, P. A. (2006). Simulation Williams, A. M., & Ericsson, K. A. (2005). Perceptual-cognitive
for performance and training. In K. A. Ericsson, N. expertise in sport: Some considerations when applying the
Charness, P. J. Feltovich & R. R. Hoffman (Eds.), The expert performance approach. Human Movement Sciences,
Cambridge handbook of expertise and expert performance 24, 283–307.
(pp. 243–262). Cambridge: Cambridge University Press. Williams, A. M., Ford, P. R., Eccles, D. W., & Ward, P. (2011).
Williams, A. M., & Davids, K. (1995). Declarative knowledge Perceptual-cognitive expertise in sport and its acquisition:
in sport: A by-product of experience or a characteristic of implications for applied cognitive psychology. Applied
expertise? Journal of Sport & Exercise Psychology, 17, Cognitive Psychology, 25, 432–442.
259–275. Williams, A. M., & Ward, P. (2007). Anticipation and decision
Williams, A. M., & Davids, K. (1998). Visual search strat- making, exploring new horizons. In G. Tenenbaum & R.
egy, selective attention, and expertise in soccer. Research C. Eklund (Eds.), Handbook of sport psychology (pp. 203–
Quarterly for Exercise and Sport, 69, 111–128. 223). New York: Wiley.
Williams, A. M., Davids, K., & Williams, J. G. P. (1999). Visual Yarbus, A. L. (1967). Eye movements and vision. New York:
perception and action in sport. London: E & FN Spon. Plenum press.