Editorial

Perception
Seeing With the Ears 2015, Vol. 44(6) 610–612
! The Author(s) 2015
Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
Jan Koenderink DOI: 10.1177/0301006615594260
pec.sagepub.com

In recent talks, I mentioned how my artist friends often complain that their clients see with
their ears. It recently dawned on me that nobody understood what I said, or—worse—got the
wrong idea. The audience thinks of bionic devices (Proulx, Stoerig, Ludowig, & Knoll, 2008)
or bat echo location (Moss & Sinha, 2003)—not my intention at all. So what might this
‘‘seeing with the ears’’ be?
Do my artist friends mean anything comprehensible at all? I have long experience in art
galleries and overheard many conversations. I learn from both listening and watching. Here
is a typical event. A customer indicates interest in a work, the artist is interested in selling.
Artists have to eat too, and theirs is not an easy living. The customer hardly looks at the
work—although standing in front of it—but instead puts questions to the artist. What is the
title? What does the work mean? Why was it made? The customer’s ‘‘understanding’’ of the
work is at the level of discursive thought. Many artists are of the ‘‘conceptual’’ persuasion
today, so this often fits well. But consider true ‘‘visual art,’’ then the poor artist has no clue.
If the customer isn’t even looking at the work, then what to say? All there is to ‘‘say’’ is in full
view, so here is another customer who ‘‘looks with the ears’’! The artist attempts to shock the
customer out of this apathetic state by suggesting that the title is irrelevant, beauty
has nothing to do with art, and so forth. Obvious truths, but not exactly helpful. At this
point, the gallery owner steps in to save the situation. The kick-in-the-ass method of the artist
scares potential customers away, an undesirable situation for all involved!
To the visual artist, words cannot replace vision. The image ‘‘speaks for itself’’ or is
meaningless junk. However, the image does not speak, it points. In the East, one finds the
well-known instance of ‘‘Hotei pointing at the moon.’’ A monkey will look at his finger but
animals do not understand pointing (Miklósi & Soproni, 2006). A human child starts to
understand pointing at the age of about a year (Tomasello, Carpenter, & Liszkowski, 2007).
Pointing is an immensely interesting topic, I’ve worked on it on and off (Koenderink & van
Doorn, 1998; Koenderink, van Doorn, Kappers, & Todd, 2004; Koenderink, van Doorn, &
Lappin, 2003; van Doorn, Koenderink, & Wagemans, 2013; Wagemans, van Doorn, &
Koenderink, 2011).
Now look at Figure 1. If you recognize it as a depiction of ‘‘Hotei pointing’’ and read on,
then you have looked with your ears. That is to say, you have seen the image in discursive
thought. You looked at the finger. The painting ‘‘is about’’ the nature of dynamic lines and
their interrelations, it is first of all a visual object (Fiedler, 1887; Meinong, 1899) that cannot
be expressed in words. This demo perhaps explains the artist saying that the customer ‘‘looks
with the ears.’’ Discursive thought that dominates over visual awareness is a remarkably
common form of apatia. Most people suffer from it when throwing a cursory glance.
Such cases are interesting from an academic perspective because they show up the chasm
between discursive thought and visual awareness. Is this relevant to vision research? It is a
hairy topic.
Koenderink 611

Figure 1. Fugai Ekun (1568–C.1654). Hotei pointing (small part, tonal scale adjusted).

Visual awareness happens to you, it is not something you do. Awareness is ‘‘presentation.’’
It is your current reality. Science has no clue as to how this ‘‘works.’’ It cannot because there
are no causal relations between awareness and physical or physiological processes, at least
none that I ever heard of. That my pain is ‘‘really the firing of C-fibers’’ sounds like an
embarrassing cop-out (Place, 1956).
The miracle of visual awareness is my major incentive to pursue research in vision. Is it a
necessity? No, not at all. People who study shark electro-reception (Kalmijn, 1971) have no
clue of ‘‘what it feels like’’ to be aware of an electromagnetic field, yet this does in no way
prevent them from doing great science. It is irrelevant whether you know how to see in order
to be an eminent vision scientist. Some experts in color vision are color blind. I know
colleagues who instead of ‘‘seeing shape from shading’’ notice gradients of gray level.
Seeing with the ears is all that is needed in science, one might consider it an advantage.
I would agree, yet I remain convinced that being able to see is of some value, at least to some
people. Might there be two kinds of ‘‘vision science’’ then? One would be behavioristic, vision
‘‘science’’, whereas the other would be experimental phenomenological, ‘‘vision’’ science.
Although both are vision science, they are worlds apart.
I once experienced this when I had set up a demo with Andrea van Doorn that had to
convince an important colleague—we were only small fry—who kept telling the world that
we had it all wrong. We went to great pains to make the demo absolutely compelling.
Surprise then, when the great man looked at it cursorily and spoke I see it and I don’t
believe it! Ears prevailed over eyes. For us, visual awareness is our reality, we felt that we
were confronted with an alien being. I still feel like this when philosophers refer to awareness
as judgement. What are they talking about?
Once you dig this, you will perceive what often separates otherwise equally eminent
‘‘vision scientists’’ and you will understand why they make each other’s life as miserable as
they possibly can in their function of editor or reviewer. This is fully
understandable—reasonable is perhaps not the correct word—from either perspective. One
person will be convinced that some demo will shake the world, the other does not even look
at it but demands a generous sprinkle of p-values. I think both are just a little bit wrong
because either aspect is at least desirable. If visual awareness is ignored, there is no vision.
If p-values are ignored, there is no science. With ‘‘no vision,’’ I mean that human vision would
be known in the same sense as we understand the electrical sense of sharks. With ‘‘no science’’
I mean that awareness cannot be shared because it is the ultimate subjective fact.
There are various consequences that greatly matter in daily life. For instance, from a
strictly scientific perspective visual objects—and consequently visual art—do not exist.
612 Perception 44(6)

Properly speaking, Figure 1 illustrates a sheet of dried wood pulp with a sparse covering of
carbon particles in some particular distribution. This is a perfectly consistent view. A
consequence is that all art is conceptual art, properly enjoyed with the ears. Illustrating
the story of Hotei pointing is gilding the lily. The inventory of contemporary art museums
suggests that this has become a majority position.
It is only in arcane fields like experimental phenomenology that one considers ‘‘intentional
inexistent’’ entities (Brentano, 1874) such as ‘‘visual objects.’’ Perhaps fortunately, there is no
law against that. There are more dangerous beliefs than holding that I see with my eyes. You
may be able to think of a few. I feel perfectly at home in this state of delusion.

References
Brentano, F. (1874). Psychologie vom empirischen Standpunkte [Psychology from an empirical
perspective]. Leipzig, Germany: Duncker & Humblot.
Fiedler, K. (1887). Der Ursprung der künstlerischen Thätigkeit [The source of artistic endeavour].
Leipzig, Germany: Hirzel.
Kalmijn, A. J. (1971). The electric sense of sharks and rays. Journal of Experimental Biology, 55,
371–383.
Koenderink, J., & van Doorn, A. (1998). Exocentric pointing. In L. R. Harris, & M. Jenkin (Eds.),
Vision and action (pp. 295–313). London, England: Cambridge University Press.
Koenderink, J., van Doorn, A., Kappers, A., & Todd, J. (2004). Pointing out of the picture. Perception,
33(5), 513–530.
Koenderink, J., van Doorn, A., & Lappin, J. (2003). Exocentric pointing to opposite targets. Acta
Psychologica, 112, 71–87.
Meinong, A. (1899). Über Gegenstände höherer Ordnung und deren Verhältniss zur inneren
Wahrnehmung [Higher order objects in perceptual awareness]. Zeitschrift für Psychologie und
Physiologie der Sinnesorgane, 21, 187–272.
Miklósi, Á., & Soproni, K. (2006). A comparative analysis of animals’ understanding of the human
pointing gesture. Animal Cognition, 9, 81–93.
Moss, C. F., & Sinha, S. R. (2003). Neurobiology of echolocation in bats. Current Opinion in
Neurobiology, 13, 751–758.
Place, U. (1956). Is consciousness a brain process? British Journal of Psychology, 47, 44–50.
Proulx, M. J., Stoerig, P., Ludowig, E., & Knoll, I. (2008). Seeing ‘where’ through the ears: Effects of
learning-by-doing and long-term sensory deprivation on localization based on image-to-sound
substitution. PLoS One, 3(3), e1840.
Tomasello, M., Carpenter, M., & Liszkowski, U. (2007). A new look at infant pointing. Child
Development, 78(3), 705–722.
van Doorn, A., Koenderink, J., & Wagemans, J. (2013). Exocentric pointing in the visual field.
i-Perception, 4(8), 532–542.
Wagemans, J., van Doorn, A., & Koenderink, J. (2011). Measuring 3D point configurations in pictorial
space. i-Perception, 2(1), 77–111.