Quad Jpeg
Quad Jpeg
Quad Jpeg
July, 2012
JPEG: the quadruple object
_______________________________
2
JPEG: the quadruple object
Abstract
The thesis, together with its practice-research works, presents an object-
oriented perspective on the JPEG standard. Using the object-oriented
philosophy of Graham Harman as a theoretical and also practical starting
point, the thesis looks to provide an account of the JPEG digital object and
its enfolding within the governmental scopic regime. The thesis looks to
move beyond accounts of digital objects and protocols within software
studies that position the object in terms of issues of relationality,
processuality and potentiality. From an object-oriented point of view, the
digital object must be seen as exceeding its relations, as actual, present and
holding nothing in reserve. The thesis presents an account of JPEG starting
from that position as well as an object-oriented account of JPEG’s position
within the distributed, governmental scopic regime via an analysis of
Facebook’s Timeline, tagging and Haystack systems.
3
JPEG: the quadruple object
Acknowledgments
A lot of human and unhuman objects connected in this project. Thanks
are due first to the Arts and Humanities Research Council (AHRC) for
funding the research and Birkbeck, University of London for awarding me
the studentship. More recently, thanks to my employer, Winchester School of
Art, University of Southampton for freeing up my time to meet my deadline.
Without these connections this mesh would never have appeared.
I have been fortunate throughout this process to connect with some of the
people I have written about. Thanks to Graham Harman for a chat at
Paddington Station and emails letting me know I was accessing the right
qualities. Thanks too to Jane Bennett, Timothy Morton and Levi Bryant for
their support and interest in how my work connected with theirs. Particular
thanks to Ian Bogost for an early connection with Alien Phenomenology and
just the right word for the methodology I was struggling towards.
This object would not have the form it does without my supervisors.
Thanks to Tim Markham for ruthlessly ensuring the sensual dimensions did
not overwhelm any real qualities my work might have. Particular thanks to
Jussi Parikka who has been more than an academic supervisor. Generous
with his time, challenging with his feedback and personally supportive when
the object withdrew from all access - whatever value there is in this object is
credit to him.
Finally thanks to those human objects who have made my mesh what it is:
Sean Brierley, Niall Martin, the Wils, the Bug and the Bird, for seeing me
through the process and reminding me of the importance of the real.
4
JPEG: the quadruple object
Table of Contents
Introduction! 8
Conclusion 68
The connection 92
I object: relationality 95
I object: processuality 96
5
JPEG: the quadruple object
Conclusion 150
6
JPEG: the quadruple object
Conclusion 175
Epilogue! 181
Bibliography! 191
Also included:
• One 4GB SD memory card
7
JPEG: the quadruple object
Introduction
When confronted with photography, Roland Barthes suddenly becomes
poetic. In Camera Lucida, he struggles for the right language:
I remember making a print. It was always called that: prints were “made”.
I put the negative in the enlarger, projected it onto the easel for a specified
number of seconds, my hands made shadows to “dodge” parts of the image,
hold light back from hitting the paper. My hands then made a narrow
aperture, “burning” more light into particular sections of the print. I slid the
piece of paper into the dish and gently agitated the clear liquid over its
surface. An image gradually appeared. If all went well, the blacks deepened
while the whites remained clear and the greys neatly spaced out between. If
not, I would take the paper out of the dish and breathe on an area of the
8
JPEG: the quadruple object
1. I use the term “mesh” in preference to the more common “assemblage”. Timothy
Morton uses the term in his discussion of the ecological thought. He says, “mesh doesn’t
suggest a clear starting point” (2010a, p. 29) and “[t]he mesh is made of insubstantial
stuff, and its structure is very strange” (ibid p. 35). The term is particularly useful when
looking to develop an object-oriented account. As Morton continues, “the mesh isn’t
bigger than the sum of its parts” (ibid p. 35). Such a term therefore keeps the focus on
objects not some meta context or field and does not preclude an account of nested
objects.
It is important to note, that Morton’s “mesh” is not an holistic term. When he talks of
everything connected within a mesh, each of the objects has its own specificity and
reality. This is a flat ontology, not a holistic whole. Drawing on the image of Indra’s net
and its jewels reflecting in jewels, He says, “[t]otal interconnectedness isn’t holistic....
Indra’s net implies that large and small things, near and far things are all ‘near’” (ibid p.
40).
Manuel De Landa also uses the term “mesh” notably in A New Philosophy of Society
where he compares assemblages and totalities (2006, pp. 8-25), even going as far as to
speak of a flat ontology (ibid p. 28). Graham Harman says that De Landa’s ontology is “a
good ontology” (2010, p. 170). But, while applauding what he see as De Landa’s
“realism”, Harman fears that for De Landa, “realities are never fully actualized even in the
physical realm, let alone in our minds” (2010f, p. 171). Assemblages are an emergence
(ibid, p. 184). I will come on to address Harman’s insistence on actualism (in comparison
to Levi Bryant’s object-oriented ontology, in my discussion of potentiality.
2. I use the term “imaging” throughout this work to draw attention to a wider field of
picturing of which “photography” is now a part. Such imaging includes montages and
remixes in Photoshop; mechanic images captured by CCTV; visualisations and
infographics; and, as I shall discuss, mash-ups. For discussion of the changing nature of
the “visibility economy”, see Kember (2008) in terms of issues of “becoming”; Murray
(2008) on “everyday aesthetics”; Jackson (2009) on young people and visual new media
cultures; Palmer (2010) on archives and neo-liberalism and Schwarz (2011) on “the
negotiated panopticon”. More generally on the growth of “ubiquitous media”, see
Featherstone (2009); Schick & Malmborg (2010) and Deuze (2011). For a broader
discussion of “personal media”, see Lüders (2008).
9
JPEG: the quadruple object
the orange safelight and the paper have gone but there is, for me as
photographer, still ritual, craft. I still read the light and chose my exposure. I
still see in terms of Adams’ zones, now as histograms. I’m still part of a mesh
of human and unhuman objects. I still chose the decisive or indecisive
moment. Light still becomes information, a RAW “negative” that Adams
used to refer to as a musical score, waiting to be performed in the darkroom.
I load a disc into my iPad rather than a film strip into my enlarger but my
fingers still dance, dodge and burn areas on the haptic interface rather than
over the paper.3
3. Jenni Mäenpää and Janne Seppänen call this digital photo editing space an
“imaginary darkroom” (2010, p. 454). I will come on to argue that while it is certainly a
space of imaging and imagining, it is also real and material.
4. It is important to note that there is no simple linear teleology in play here. The
contemporary digital imaging scopic regime must be seen in complex relations with that
of the sort of “Decisive Moment” photography that Henri Cartier-Bresson practised
(1999) or any other “way of seeing”. There are continuities and discontinuities.
10
JPEG: the quadruple object
This project is about that new player, in particular one aspect of software:
the software standards that enable that technosocial mesh. At the moment of
taking and sharing something happens to turn light into data and more
significantly social data. To rewrite Kodak’s slogan, when I press the button,
software does the rest. JPEG photography is a complex ecology of human
and unhuman objects connecting the photographer, the camera, the silicon
and battery, the factories and poisoned workers, the card and the router,
Web 2.0 businesses, servers and the power that runs them, the carbon burnt
to keep those searchable archives running, the “friend” and searcher, the IP
lawyer and countless other actants. This project is about those objects 6 and
the complex, inaccessible relations and connections that make up digital
imag(in)ing.
6. Along with Harman (and Bruno Latour), I use the term “object” rather than “thing”.
Harman discusses how Heidegger (1975) distinguishes between the Thing which
“things” i.e. does something, “stands independently in itself”, and the ‘”object” which is
what we represent to ourselves (2010a, p. 24). For Heidegger, “’[o]bject’ is a negative
term, used to describe entities only in their presence-at-hand. But thing’ is a positive
term referring to entities in their proper reality” (Harman, 2007, p. 129). Harman stresses
that these are not to be seen as separate objects but rather dimensions of an object
(something I pick up when discussing “sensual” and “real” objects). Harman’s point
which he develops in The Quadruple Object (2011g), when he brings Heidegger and
Husserl together in his own fourfold, is that the thing/object has both dimensions. His
use of the term object is meant to cover both. For other accounts of Things, see Brown
(2001; 2004).
7. I use the term “apparatus” rather than “camera” so as to include my “mash-up” and
screengrab works but also to bring in the sense of “dispositif” used by Foucault (1980)
and Giorgio Agamben. Agamben says, “[f]urther expanding the already large class of
Foucauldian apparatuses, I shall call an apparatus literally anything that has in some
way the capacity to capture, orient, determine, intercept, model, control, or secure the
gestures, behaviors, opinions. or discourses of living beings” (2009, p. 14). For a
discussion of text-as-apparatus, see Weight (2006).
11
JPEG: the quadruple object
apparatus and practice is “weird”.8 It is real. It does things, but we can’t see
it or touch it. Its traces and connections are everywhere: in dot JPEG image
files, in social media archives and search engine and hard drive caches, in
data-mining strategies and surveillance practices, in business models. But
like Keyser Söze, the mysterious figure in Bryan Singer’s film The Usual
Suspects (1995), JPEG just slips out of sight. It withdraws.9
This work sets out to address JPEG as a standard enfolded in software and
hardware meshes at play in contemporary imaging, as an imag(in)ing
protocol which like transmission protocols enable network effects.10 More
than that however, it seeks to explore how JPEG plays its part in the regime
of governmentality with which the social, distributed imaging in spaces such
as Facebook, are implicated. I come on to explore how JPEG is enfolded in
Facebook’s tagging and Open Graph technologies, its datamining strategies
and working as a governmental “relationship engine”. I set out to understand
that weird object, its nature, its workings and its relationship to dimensions
of contemporary governmentality, the practices of governmental rationality,
ordering, “conduct of conduct” and the relationship of self to self that
characterise contemporary regimes of power.11 These were my research
questions.
8. I use the term “weird” in the same sense in which Jussi Parikka discusses the “weird
objects of network culture” (2011a, p. 268 see also Hertz & Parikka (2010)). the term is
also used by Douglas Hoftstadter in his discussion of Artificial Intelligence. He says, “we
will have a very hard time deciding when and if we are dealing with an AI program, or
just a ‘weird’ program” (1999, p. 680).
9. A character in the film, Roger “Verbal” Kint, says “the greatest trick the Devil ever
pulled was convincing the world he didn’t exist”.
10. Technically JPEG is a compression standard, rather than a protocol which usually
refers to common rules enabling network transmission. JPEG could perhaps be seen as
as social or imag(in)ing protocol, a common standard enabling the networking of images
and abiding by (or in OOP terms, connecting with) the transmission protocols of the
Internet. Throughout this work I use the term “protocol” in that wider sense of a technical
standard of compression written into software and hardware meshes but also a
networking standard enabling (social) imag(in)ing and as we shall see scopic
governmentality.
11. These issues, derived from the work of Michel Foucault, are discussed in more detail
in the JPEG: the governmental object chapter.
12
JPEG: the quadruple object
Bryant and in particular Graham Harman use the term. As I discuss, I use
this framework as not just an ontological starting point but also a
methodology. While I do not position this as a philosophical work, I do
argue that object-oriented ontology offers a powerful way of addressing
weird objects such as JPEG.
13
JPEG: the quadruple object
12. There is some disagreement around the best term for perspectives that start from
and work with objects. Harman prefers the term “object-oriented philosophy” while Levi
Bryant, Timothy Morton and Ian Bogost prefer “object-oriented ontology”. Following my
use of Harman’s quadruple structure I choose to use his terminology, hereafter
abbreviated to OOP.
13. As I discuss in this chapter and the next, I am not looking to develop a philosophical
work or account of JPEG. Rather I position OOP as not just an ontology but also a
method. Rather than engaging in debates about whether OOP is valid as philosophy, I
look to ask whether it has anything to offer to software studies and to practice.
14
JPEG: the quadruple object
Like Camera Lucida there is something very personal about this work. It
arises from and works through my practice as a photographer. Its object is
clear: JPEG, yet that object, like Barthes’ photo of his mother has a complex
form, character and power. The central question is simple: “what is JPEG?”
like Barthes’ “what is Photography?”. Yet both questions open up a complex
mesh of connections, relations and actants in play. Barthes’ mission took
him outside his comfort zone of language. Mine has taken me outside the
comfort zone of both “pure” practice and “pure research”. Finally, like
Camera Lucida this work is modest and tentative: Note sur le protocole.
14. In my conclusion I talk of how the form of this project and report are themselves
object-oriented and how I created the work in that way in order to explore the critical
potential of object-oriented approaches to academic writing and publishing.
15
JPEG: the quadruple object
One way to understand the “digital imaging pipeline” is via its chemical
equivalent. I use the term “chemical” rather than “analogue” because I want
to avoid debates about a digital-analogue divide. The issue here is not
whether one deals in discrete one and zero steps and the other a smooth
curve, but rather the way in which encoding works. The issue within which
JPEG is important is the difference between action of light on silver halides
within a chemical process, and light on silicon within a digital process.
15. This term “pipeline” is not simply a metaphor. Rather it is the recognised technical
term for addressing the ways in which digital imaging works and, as I understand it, the
way the various objects connect. See for example Ramanath, Snyder, Yoo & Drew (2005)
and Gonzalez & Woods, (1992).
16. For the recognised FAQs around JPEG, see Lane (1999a; 1999b).
16
JPEG: the quadruple object
through the negative onto a paper coated with a similar silver halide
emulsion, the exposed areas (black in the negative) stop light hitting the
paper, while unexposed areas (clear in the negative) let light hit the
crystals.17 What was light in the scene and black in the negative become
light in the print and vice versus. There are other technologies (or objects as I
would refer to them) in play. Lenses (or in my case pinholes); camera
apparatuses including the shutter and aperture assembly; enlargers; film and
paper as well as corporate objects such as Kodak as well as their failed
business strategy.18 Some of these of course are also in play in digital
photography. What is different is the encoding - the journey of light through
latent image or data to visible image.
To work with objects: the first object in the digital imaging pipeline is the
sensor. In digital photography these are two main types: CCD (charge-
coupled device) and CMOS (complimentary metal-oxide semiconductor)
sensors.19 Sensors are effectively an array of silicon, solar or photovoltaic
cells. When light hits one of these cells, some of its energy is absorbed by
the silicon, knocking electrons loose which are forced to flow in a particular
direction creating a current: photons become electrons, light become
electricity.
17. For the sake of clarity I focus on basic black and white photography rather than
colour imaging or reversal (slide) photography.
18. As I shall argue, just because an object or an object connection “fails” does not
mean it is not an object. The collapse of Kodak in 2012 can be understood in object-
oriented terms as its business strategy object failing to connect with other objects in the
imaging mesh. Latour famously explores such “failed” object relations in Aramis (1996).
19. There are other types of sensor such as the Olympus Live MOS sensor and the
Sigma Foveon DP sensor discussed by Bogost (2012a, pp. 69-71) see pp. 65-66.
17
JPEG: the quadruple object
CCD and CMOS sensors have either a red, green or blue filter over each
cell, essentially making the cell only sensitive to red, green or blue light.
These are arranged in a Bayer mosaic pattern consisting of two green, one
red and one blue filter - designed to match the bias of human perception of
colours.
The sensor reads the amount of charge from each cell (what comes to be
known as pixels). These electrical charges need to be collected and
organised before they can be processed by other software objects. A CCD
sensor handles this differently than a CMOS sensor. In a CCD sensor, a
control circuit causes each capacitor to transfer its contents to its neighbour
with the final output read at one corner of the array. In a CMOS sensor, each
pixel/cell is accompanied by several transistors that amplify and move the
charge using more traditional wires. Thus each pixel can be read
individually.20
20. CCD sensors are generally seen as more expensive and power hungry but also
having higher sensitivity and being capable of delivering higher quality. CMOS sensors
tend to be found in mobile phone cameras
21. Of course OOPs would be clear that we have already been dealing with a whole
series of nested objects in terms of the sensor, but for clarity’s sake I outline the key
actants.
22. A bit in computer terms has a value of on or off, one or zero. A two-bit ADC would
divide the information from the sensor into levels 00, 01, 10 and 11. An 8-bit sensor can
divide it into 256 levels from 00000000 to 11111111.
18
JPEG: the quadruple object
information becomes the RAW data file that is written to the camera’s
storage medium.23
Because each pixel/cell only senses one wavelength of light (red, green or
blue), the information making up the “latent image” needs to be interpolated
so that the image can represent the amount of red across the whole image
not just on those bits where the filter measured the red light. To do this a
“demosaicing algorithm” averages the values from the closest surrounding
pixels to assign a “true colour” to each pixel. This data can be encoded as a
visible colour image file. It is here where JPEG comes in.24
To concentrate just on the JPEG processing of that RAW feed of data, the
digital imaging pipeline continues in four steps: Sampling, Discrete Cosine
Transform (Cabeen & Gent, 1998), Quantization and Huffman Coding
(Miano, 1999, p. 44; Haas, 2008). At the end, the light-as-data is a JFIF
image, commonly know as a “jpeg photograph”.
The pixel data is first converted from RGB to YCbCr colorspace. The JPEG
standard is principally about compression. Its role in the imaging pipeline is
to reduce the amount of data in the file - hence its importance in the early
days of the Internet when bandwidth was at a premium. Part of the work of
compression is the move from RGB to YCbCr. Storing image data in both
RGB and YCbCr colorspaces demands three channels of information - in
RGB: red, green and blue; in YCbCr: luminance and two chrominance, blue
and red (Miano, 1999, p. 6). Both allow a full range of colours but in RGB,
23. This RAW data/file is not “pure”. Each camera has its own way of writing the RAW
data, its own format. RAW converters (the part of software that interprets and renders
that data as image within other software) have to know the various formats Olympus,
Nikon, Canon etc use in order to “make sense” of that data, see Fraser (2004). The
evocatively named “digital negative” DNG standard was developed to solve
incompatibility problems and also provide a standard that retained all the RAW data in a
standard format that could form the basis of archives.
24. For accounts of the history and development of JPEG, see Palmer (2011) and
Wallace (1992).
19
JPEG: the quadruple object
each channel is sampled at the same frequency while in YCbCr, this can be
varied. The Y component contributes most information to the visible image
and JPEG therefore assigns more weight to that component and reduces the
amount of information in the Cb and Cr channels, thus reducing the amount
of information and so the file size. As John Miano explains:
The next step in JPEG encoding is “Discrete Cosine Transform” (DCT). First
the YCbCr image data is divided into 8x8 blocks called data units.25 DCT
does not actually compress or throw information away, it merely readies the
data/information for that to happen in the next step by sorting the
information which can safely be discarded. Rather than record the individual
values of each Y, Cb and Cr component over an 8x8 block, we can average
the values for each block and record how each pixel differs from that
average value. DCT takes the set of values in each data unit and transforms it
into a set of coefficients to cosine functions with increasing frequencies
(Miano, 1999, pp. 77-90). In effect DCT arranges the digital information
ready for compression by finding the frequency of each value - in lay terms
the most frequent tone or colour values.
26. This is why JPEG compression is referred to as “lossy compression” because data is
lost.
20
JPEG: the quadruple object
Having discarded data from the RAW data file, JPEG’s final step is to
create a visible (JFIF) file. This is achieved through Huffman coding. Like
DCT, Huffman coding takes the set of values in each data unit and
transforms it into another set of values. Unlike the DCT, Huffman coding is
lossless - no further information is discarded. Rather this process saves
further space by assigning shorter codes to the most frequently used values.
Like Morse code, Huffman Coding assigns shorter codes to the most
frequently occurring values (vowels have shorter Morse code symbols than x
or z) according to a Huffman table. As Calvin Haas explains:
Having started as light photons, being turned into electrical charge and
from there into data, the resultant information has been sorted and
compressed by JPEG into a file ready to be written (potentially alongside a
RAW file) to the camera’s memory. JPEG wraps the compressed data within a
27. Jesse D. Kornblum identifies how quantization tables can be used for image source
identification within digital forensics (2008). While issues of JPEG and digital forensics as
well as “steganography” (the potential to hide data within JPEG-encoded images) are
dimensions of JPEG’s relation to broader regimes of power and governmentality, they are
not the focus of this work which aims to look at the more prosaic position of JPEG as
standard within imaging, its standard connection with other digital and governmental
objects. For accounts of steganography, see Johnson & Jajodia (1998); Provos &
Honeyman (2001); Fridrich (2004) and Fridrich, Pevný & Kodovský (2007).
21
JPEG: the quadruple object
One such marker is the APP marker which hold application-specific data.
These are used by software or applications to add additional information
beyond what is demanded by JPEG. An encoder that uses JPEG can specify
particular information within an APP marker. This is important when it
comes to the two most widely used JPEG-encoded file formats.
JPEG does not define a file format. As Miano says, “it says nothing about
how colors are represented, but deals only with how component values are
stored” (1999, p. 40). Other file formats such as TIFF can compress using
JPEG. JPEG can therefore write more than one sort of data/image file. The
two most common follow the JFIF (JPEG File Interchange Format) (Hamilton,
1992) and the EXIF (Exchangeable Image File Format) (CIPA, 2011)
standards. The two standards are very similar with EXIF allowing the addition
of specific metadata tags but not allowing colour profiles. Most cameras
encode to an EXIF file while imaging applications use JFIF. Technically JFIF
and EXIF use different APP markers (APP0 and APP1). In practice most photo
applications use JFIF and include the metadata from the APP1 marker.28
Other markers provide space in the file for comments, details of the width
and height and number of components in the image as well as the Huffman
and quantization tables.
As I shall discuss in The JPEG object in theory and The JPEG object in
practice chapters, this “family of compression algorithms” (Lane, 1999a,
n.p.) can be addressed as an object in Harman’s terms not only in terms of
its existence in paper standards documents but also in terms of its “weird”
quadruple existence within the digital imaging pipeline. Clearly however, it
is possible to address this whole pipeline (or indeed the chemical imaging
pipeline) through OOP.
28. Strictly speaking this goes against the standard with both JFIF and EXIF demanding
that their marker is first in the data-stream. As with much software, this demand is
fudged.
22
JPEG: the quadruple object
What OOP (at least in the Harman version I explore) offers is firstly a
refusal to leave that focus on objects - to refuse to talk of systems,
assemblages, meshes or contexts as anything other than objects. Secondly
Harman’s OOP refuses to characterise those objects as defined by their
relations. Rather they have an existence and, in Jane Bennett’s terms a
“vitality”, that exceeds their relations (2010a). Thirdly, those objects are not
processes. They are not in flux. Rather, change is matter of new objects
formed in new object connections. Finally, from an OOP perspective, the
objects in play in the digital imaging pipeline do not hold anything back.
They do not harbour potential. They are fully present in their connections
not harbouring potential, or somehow waiting to “become”.
23
JPEG: the quadruple object
24
JPEG: the quadruple object
Atget walked the streets of Paris from 1897 until his death in 1927 with
his view camera and a particular sensibility. He was a working hack. He
took pictures to sell as “documents for artists” in the nearby town of
Montparnasse. At the same time as Russian constructivists and Italian
Futurists were feting speed as technique, source material and inspiration,
Atget plodded around Paris like the fabled flâneur and his turtle,30 unfurling
his equipment, waiting as the light encoded Paris as information and then
waiting while light encoded it again as an Albumen print.
The long exposures meant the pictures were often devoid of people but
that is not what makes his sensibility object-oriented. Rather it is the ghostly
traces of humans occasionally caught in the doorways, on street corners or
reflected in windows alongside the rags ‘n refuse31 of Paris that make Atget
object-oriented.
30. For discussion of the flâneur and online media, see Featherstone (1998) and
Atkinson & Willis (2007).
31. The term comes from Walter Benjamin’s account of the fragments collected in The
Arcades Project (2002, pp. N1a,8). This is discussed further in my conclusion.
32. For Bryant “[t]he democracy of objects is not a political thesis to the effect that all
objects ought to be treated equally or that all objects ought to participate in human
affairs. The democracy of objects is the ontological thesis that all objects, as Bogost has
so nicely put it, equally exist while they do not exist equally” (2011a, p. 19).
25
JPEG: the quadruple object
For Atget, as for anyone walking the city, Paris withdrew. Yet he
encountered it. There was a sensual dimension to Paris that Atget and his
camera connected with and it was the withdrawn reality that made the
sensual so powerful. His images are a trace of those encounters. Our
encounter with them echoes that withdrawal/access tension as we enjoy the
coffee table book or search online, as the images resonate or evoke and yet
something, somewhere withdraws.
26
JPEG: the quadruple object
When Atget wrote “Va Disparaitre” on the back of 41 Rue Broca in 1912,
it was a note to himself and us that the building would soon disappear.
Literally it would suffer the fate of La maison no 5 de la rue Thouin on the
10th August 1910. It would soon have its day of demolition with its new
rubble objects nestling next to a blurred ghostly figure and a boy who seems
to have stepped out of a Diane Arbus picture. But Atget is not nostalgic. Nor
is he simply a recorder of passing time. His object-oriented sensibility knows
that 41 Rue Broca disappears in another sense. As an object it disappears
from access at the moment of taking and viewing in 1912 as much as it does
a century later. Just as Atget knew it was sensually present for him and for us
then and now, it was also out of reach. It had already, inevitably and
irrevocably disappeared. It had withdrawn.
Atget’s Paris, often the name for the books published about him, as an
object then and now for the Paris tourist board, Eurostar, Woody Allen, my
iPhone and the wheel of Mark Cavendish’s bike is never fully there. It is not
just a brand, an ideology, a metaphor or a sign, but it is also never fully
accessible as a real object. Those objects encounter its sensual dimensions
as did Atget, his camera and the light that fell on the Door knocker (1909) or
the House of pleasure (1921) and rendered his albumen prints. Atget’s
object-oriented sensibility emerges from his willingness to sink into that
mesh of objects and sensual/real connections, to refuse the correlationist
agenda offering him a subject as opposed to object position or some
privileged access to Paris’ objects.34
34. The term “correlationist” is Quentin Meillassoux’s (2009) and is discussed in The
JPEG Object in Theory chapter.
27
JPEG: the quadruple object
35. The term is Mark Marino’s (2006). Later he says, “Critical Code Studies names the
practice of explicating the extra-functional significance of source code. Rather than one
specific approach or theories, CCS names a growing set of methodologies that help
unpack the symbols that make up software” (2011, n.p.). See also Mackenzie &
Vurdubakis (2011)
36. I use the term “network” in a Latourian rather than a technical sense. For studies of
interfaces, see Bratton (2008) and Chun (2008).
37. For an alternative account of the materiality of finance systems, see Miyazaki (2005)
and on algorithms and trading, see Lenglet (2011). For a discussion of the relation
between “data derivatives” and risk within the context of border security systems, see
Amoore (2011).
28
JPEG: the quadruple object
Latour in the first part of Prince of Networks (2009c) makes clear, actants in
networks is a powerful model: objects connect. Where OOP differs is in
demanding that objects are not defined by their relations and that relations
are matters of objects not of exterior context. Objects’ character and power
exceeds their relations and the networks or meshes within which they are
enfolded. As my own work will show, exploring JPEG as having an
existence, character and power beyond its relations allows us to see how
governmental issues of data-mining are best addressed as a matter of the
JPEG-object connecting with a search algorithm-object within another,
specific object. Objects relate within objects not within contexts or fields.
This is the heart of “object-orientation”, a refusal to leave an account of
specific objects even when building a critique of networks. It is this refusal
(or perhaps more positively, focus) that enabled me to engage in my
particular imaging and build my particular critique.
For some seminal work in software studies, including the first discussions
of protocol, this is not the case. Objects are best addressed in terms of
relationality.
38. Of course there is more to Manovich’s work than this text and indeed more to The
Language of New Media than his account of the object, notably his discussions of
representation, cinema and realism. My concern is not to provide a full account of these
arguments but to draw out Manovich’s particular, and influential take on the object. For
his broader argument on the focus of software studies, see (2008; 2011b) In terms of
particular software, see (2005a; 2006; 2011a). On cinema, see (2000) and Manovich &
Kratky (2005). For his work on data visualisation, see (2010) and Manovich & Douglass
(2009).
29
JPEG: the quadruple object
He draws attention to the term’s use in computer science and industry and
how that aids in a discussion of “computerized culture”. Objects for
Manovich are products. They are the discs, films, data files, software
environments, interfaces,40 webpages and sites, even networks and digital
images,41 Manovich adds real value by understanding these as material.42
He says:
39. Other critics have looked at concrete non-art digital objects such as gadgets and
even digital fetishes particularly in terms of mobile. See for instance Itō, Okabe &
Matsuda (2005); Richardson (2005); Goggin (2006); Levy (2006); Hawk, Rieder &
Oviedo (2008); Ling (2008); Green & Haddon (2009) and Reading (2009).
40. An issue he had addressed as early as 1995 when “virtual reality” (VR) was all the
rage. In (1995a) he sought to find a way to address the specificity of that object through
an analysis of the “screen” through which it worked. For a more media archaeological
account of the screen, see Gere (2006).
41. In (1992) Manovich had discussed the question of “what is a picture?” including the
computational picture in his review of pictorial semiotics. He concluded that “the study of
the mechanism of pictorial signification and the understanding of social forces
responsible for these mechanisms should constitute a joint project”. Here is an early sign
not only of Manovich’s interest in the specificity of the object but also his location of that
object within its relations and context.
42. Lisa Gitelman questions whether Manovich is truly comfortable with the idea of
materiality: “Even the most astute and exacting critics of cyberculture tend to signal a
certain ambivalence about the bodies that electronic texts have, judging at least from
the frequency with which the word material appears between scare quotes. Lev
Manovich (2001, pp. 45, 48) writes that the ‘basic, “material” principles of new media
[are] numeric coding and modular organization,’ and that hardware and software have
“material” as well as “logical principles.”” (Gitelman, 2008, p. 96).
30
JPEG: the quadruple object
44. He says: “The theory and history of cinema serve as the key conceptual ‘lens’
though which I look at new media” (2001, p. 9).
45. In (1997, n.p.) Manovich had said, “[a]ll information becomes encoded in one code;
all cultural objects become computer programs, something which is not only seen,
heard or read, but first of all stored and transmitted, compiled and executed”.
Wendy Hui Kyong Chun has problems with this stress on computerisation. She argues
that the “problem with ‘software studies’ or transcoding, however, is this privileging of
software as readable text; it ignores the significance of hardware and extramedia
representation because it only moves between software and interface. Also, this notion
of transcoding perpetuates the idea that software merely translates between what you
see and what you cannot see, effectively erasing the many ways in which they do not
correspond” (2006, p. 18).
For Chun computation makes differences rather than simply registering them. Simply
opening a file can change it (see (Parikka & Sampson, 2009)). For Chun, “[t]he problem
with Manovich’s notion of transcoding is that it focuses on static data and treats
computation as a mere translation. Programmability does not only mean that images are
manipulable in new ways but also that one’s computer constantly acts in ways beyond
one’s control. To see software as merely “transcoding” erases the computation
necessary for computers to run” (Chun, 2005, p. 46).
She also criticises his account of new media spaces as ones of navigation arguing that
such spaces are unnavigable insofar as users have no control over the path their (or
anyone’s data) takes (2006, p. 46). In her most recent work she says the notion of
transcoding treats computation as “a mere translation” and focuses on static data rather
than issues of programmability and computation (2011, p. 91).
31
JPEG: the quadruple object
Manovich says that “the new qualities of ‘digital media’ are not situated
‘inside’ the media objects. Rather, they all exist “outside” - as commands
and techniques of media viewers, email clients, animation, compositing,
and editing applications, game engines, and all other software
‘species’” (2011b, n.p.). When Manovich talks of the inside and and outside
of media objects, he is using the terms in a different way to Harman. Here
he is drawing attention to what he sees as a field of relations external to
objects enabled by software inside those objects. For an object-oriented
perspective, as we shall see, this is to move beyond objects and also to fail
to account for software itself as an object. The objects Manovich raises as
worthy of his formalist analysis derive their interest and their power from
47. It is perhaps interesting to see Manovich’s latest work around data visualisation as a
willingness to engage more fully with a wider field of objects. In (2002, n.p.) he wrote:
“Along with a Graphical User Interface, a database, navigable space, and simulation,
dynamic data visualization is one of the genuinely new cultural forms enabled by
computing”. He has since extended beyond considering visualisations as just cultural
forms to approaching them as something more problematic. In (2010, n.p.) he
suggested we “define information visualization as a mapping between discrete data and
a visual representation”. Here the visualisation is not a cultural form but a mapping,
something beyond both data and representation - a weird object perhaps.
32
JPEG: the quadruple object
48. Mark B. N. Hansen makes a similar critique when he says: “As I see it, digitization
requires us to reconceive the correlation between the user’s body and the image in an
even more profound manner. It is not simply that the image provides a tool for the user to
control the ‘infoscape’ of contemporary material culture, as Manovich suggests, but
rather that the ‘image’ has itself become a process and, as such, has become
irreducibly bound up with the activity of the body” (2004, p. 10). More generally on
convergence and media, see Jenkins (2006); Jenkins & Deuze (2008) and specifically in
terms of convergence and media work, Deuze (2009).
33
JPEG: the quadruple object
The Object Reader (2009b).49 His broader aim of showing how networks are
sites of particular control and governmental discipline, enabled by particular
configurations of software, depends on tracing the protocol-object as a key
player in (as well as evidence of) those relations. Galloway needs protocol to
be an “object” in order to be able to show that protocol does things in the
world. Furthermore his perspective demands that object to be an object-in-
relations in order to explore his real target, the “control society” as outlined
by Gilles Deleuze (1992).50
49. This object-focus can be contrasted with accounts of the values and power relations
apparent in the Internet’s technical working explored by those looking at technical code.
Andrew J. Flanagin, Craig Flanagin and Jon Flanagin say: “A technical code analysis,
therefore, reveals the underlying assumptions and choices that become built into
technologies, which would otherwise remain largely obscure” (2010, p. 180). Here the
focus is on the broader system.
50. See Best (2010) for a discussion of perceptions of the “control society”. For a
different account of the relation between “disciplined subjects” and interactive media,
see Barry (2001). As I will argue, an object-oriented approach does not deny relations
and can add real value to to a discussion of governmentality. The difference is that for
OOP those relations do not define the object.
34
JPEG: the quadruple object
51. The relation between code and language (and ideology) has been a consistent
theme within software studies. Matthew Fuller and Andrew Goffey talk about the “logic of
programmed hardware and software... as something that more closely approximates the
order of language” (2009, p. 142) while Adrian Mackenzie insists that “[o]ne way to resist
an abstracting turn away from software is to attend to its code-like structure” (2006, p.
3). Michael Mateas’ discussion of “weird languages” which “tease apart phenomena
present in all coding activity” (2006 p. 274) and Nick Montfort’s discussion of
programming languages (2006) are further examples. Others have sought to address
the form and ideology of that linguistic form. Hayles stresses performativity: “Code that
runs on a machine is performative in a much stronger sense than that attributed to
language. When language is said to be performative, the kinds of actions it ‘performs’
happen in the minds of humans, as when someone says ‘I declare this legislative
session open’ or ‘I pronounce you husband and wife’. Granted, these changes in minds
can and do result in behavioral effects, but the performative force of language is
nonetheless tied to the external changes through complex chains of mediation. By
contrast, code running in a digital computer causes changes in machine behavior and,
through networked ports and other interfaces, may initiate other changes, all
implemented through transmission and execution of code” (2005, p. 50).
Galloway (2006b) and Chun (2005) have discussed the relationship between code and
ideology with Galloway arguing that “software is not merely a vehicle for ideology;
instead, the ideological contradictions of technical transcoding and fetishistic
abstraction are enacted and ‘resolved’ within the very form of software itself” (ibid p.
319).
It is not my intention to enter these broader debates about software and ideology or even
code and language but merely identify the particular way in which Galloway articulates
protocol as language, as a special kind of machinic language not in terms of its internal
structure but rather its workings and role.
35
JPEG: the quadruple object
Galloway argues that protocols such as TCP/IP and DNS act as a “political
technology” (2004, p. 115) which encapsulate the complex enfolded
relationships between protocol and what Deleuze (1992) and Michael Hardt
and Antonio Negri (2000) identify as control societies.52 Where one (TCP/IP)
is to do with distribution, connection and decentralisation, the other (DNS)
is built around centralisation and control.53 For Galloway this dialectic is
productive in allowing him to characterise the Internet as distributed, not
decentralised but at the same time a space of control. In order to hold this
tension, Galloway positions the protocol object (as rules and language) as
the foundation of that control. He says his book “aims to flesh out the
specificity of this third historical wave [Deleuze’s control society] by
focusing on the controlling computer technologies native to it” (2004, p. 3).
Drawing a parallel between the panopticon and protocol, he continues,
“[p]rotocol is to control societies as the panopticon is to disciplinary
societies” (ibid p. 13).
52. For Galloway an analysis of protocol is a way of opening up the study of networks to
reclaim the issue of power from an account of networks that have vied between what he
calls gee-whizz accounts of progress (2004, p. 18) and descriptions of networks as
cyberutopian arenas of anarchist freedom. Chun of course argues that “control and
freedom are not opposites but different sides of the same coin” (2006, p. 71). As noted
above, my concern here is not to engage with debates about networks but it is important
to acknowledge that the network is Galloway (and Thacker’s) object of investigation.
Although Galloway is keen to move from “traditional” graph theory models of networks,
he still draws heavily on the pervasive metaphor of the rhizome, drawn from Deleuze and
Guattari (2004) (Galloway & Thacker, 2007, p. 29).
David Berry criticises Galloway and Thacker’s extended use of network as a metaphor
(2008b, p. 367), a criticism echoed by Greg Boirarsky (2006, p. 108). The rhizomamatic
metaphor is also criticised by Conley (2009) and Buchanan (2009).
Given that, in Protocol at least, Galloway is talking of technological systems this is
perhaps unfair. But even when dealing with the technosocial systems that are the focus
of The Exploit, it is the very independent nature (in Galloway’s terms, the rule-based
nature) of protocol that allows for exploit.
53. Recent attempts by the US Government to tackle Internet “piracy” have focused on
using DNS as away of blocking sites that media owners complain are facilitating illegal
media sharing (Lee, 2012).
36
JPEG: the quadruple object
37
JPEG: the quadruple object
they are deeply social. “By formal apparatus I mean the totality of
techniques and conventions that affect protocol at a social level, not simply
a technical one” (ibid p. 55). That social enfolding however is drawn in
terms of a formalist analysis. He says: “I move beyond the hard science of
protocol and begin to consider it from the perspective of form. That is: How
does protocol function, not as a material machine, but as an entire formal
apparatus? What techniques are used by and through protocol to create
various cultural objects? How can one define protocol in its most abstract
sense?” (ibid p. 53). My argument is that by moving “beyond the hard
science”, leaving the technical and particular realm of the object, one fails
to address the specificity of the object and also the particular material and
machinic ways these governmental relations are built.
55. For an interesting fictional exploration of such network struggle, see Roberts (2011).
38
JPEG: the quadruple object
56. Galloway argues that protocol allows new forms of politics and political struggle. His
is not the only framework to explore the connections between politics and technology,
Joss Hands for instance sees the digital as a space of resistance (2010) while Jodi Dean
argues that the digital is such a “noisy” space that it “hinders the formation of strong
counter-hegemonies” (2005, p. 53). What is particular about Galloway and Thacker’s
account is that they start from the object. The issue is however whether that object
remains the point of focus or merely the way into a broader field of relations, the control
society that it somehow fuels. An object-oriented account would look across the techno-
politcial space and its flat realm of objects, seeking to reconfigure object connections
including ”noise”, rather than looking for a particular key determinant or a power full
object to resist. I open up these questions in my conclusion.
57. In his analysis of the TV series 24, Galloway discusses the show’s theme of “the
circumvention of protocol,” or hacking. “In the control society informatic systems are
always in a state of ‘self-exploitation’ and are defined not as an integral object but as a
flexible network of command and control, which only becomes realized through its own
transgression by another informatic force” (2007a, p. 19). Here again a protocological
system carries within itself the potential for its unpicking. Earlier, Galloway had said
“resistance during the post-modern age forms around the protocological control forces
existent in networks” (2004, p. 160) and again, “techno-resistance is not outside protocol
but at its center” (ibid p. 176).
58. This theme of sites of struggle and the possibility for critique and intervention
appears in Galloway’s praise for Mehdi Belhaj Kacem as an intellectual willing to extend
the boundaries of critique, to be “a self-styled outsider, a trickster, an autodidact, or, in
his own words, an ‘anti-scholastic,’ an ‘anti-philosopher’” (2009c, n.p.).
59. Galloway had previously discussed the virus in (2005, p. 24-27). Thacker had also
previously addressed the relationship between computation and biology (2004). For a
wide discussion of new biologies and their relation to economics and power, see papers
collected in Zylinska (2011).
39
JPEG: the quadruple object
61. This perspective can be read alongside McKenzie Wark’s positioning of the hacker
as one who takes a system’s tools and turns it against power (2004; 2006).
62. Tiziana Terranova Criticises Galloway and Thacker’s terminology drawing on Sadie
Plant (1997) to argue for a less masculinist language. “In the case of network conflict,
what seems important to the authors is not so much resistance as ‘impulsion’, ‘a thrust’
and even a ‘hypertrophy’. And yet if the ontology of networks is that of relations - that is,
as Sadie Plant has argued, a feminist ontology - why centre its political tactics around
such masculine ‘thrust’? What about those processes of topological and ethical
‘invagination’, which also seem necessary for the purposes of collecting, nurturing and
consolidating antagonistic network forces” (2009, p. 49).
63. Galloway had made a similar point in an earlier article: “The goal, then, is not to
destroy technology in some neo-Luddite delusion, but to push it into a state of
hypertrophy, further than it is meant to go. Then, in its injured, sore, and unguarded
condition, technology may be sculpted anew into something better, something in closer
agreement with the real wants and desires of its users” (2005, p. 30).
40
JPEG: the quadruple object
64. Galloway and Thacker discuss the model of networks built around “edges and
nodes” as it developed from mathematical graph theory (2007, pp. 31-35). Morton offer
a different perspective on edges and nodes in his view of the mesh: “Each point of the
mesh is both the center and edge of a system of points, so there is no absolute center or
edge” (2010a, p. 29).
65. In his discussion of interfaces, Galloway uses the language of exploits to explore
political art making. He discusses World of Warcraft’s interface as “awash in
information” (2009d, p. 945) overwhelming an aesthetic of the window or door. Galloway
connects this to the idea of “incoherent politics”, the deterritorialization of Deleuze:
“[T]he game displays an aesthetic of incoherence in that it foregrounds the apparatus
(statistical data, machinic functions, respawn loops, object interfaces, multithreading,
and so on), while all the time promoting a particularly coherent politics (protocological
organization, networked integration, alienation from the traditional social order, new
informatic labor practices, computer-mediated group interaction, neoliberal markets,
game theory, and so forth)” (ibid p. 951). On spam, see also Parikka (2007).
66. In a discussion of the later Baudrillard’s interest in games and play, Galloway writes
of how for the French theorist, “both sovereignty and resistance are gamic” (2007b, p.
377). It is not surprising that, with his interest in gaming (2006a), Galloway should draw
connections between his concept of the exploit and the metaphor of play, going as far
as to identify our contemporary moment as one of “ludic capitalism” (Galloway, 2009d,
p. 932).
It is also interesting that Galloway chooses a verb for the title of his study rather than the
more usual noun “games” (Wardrip-Fruin, 2009; Tanenbaum & Tanenbaum, 2010) or
even the journal Games and Culture. This focus on the process and practice rather than
the object of the game may seem to go against my argument that Galloway is a theorist
of the object but, just as protocol is for him important in terms of its object-work, so
games are important in terms of how and where they function as particular forms.
See also Galloway’s discussion of Guy Debord’s game Djambi (2009a).
41
JPEG: the quadruple object
The issue, from an object-oriented point of view, is that the object is once
again drawn in terms of relations. Echoing Adrian Mackenzie’s account of
software as a “neighbourhood of relations” (2006, p. 169) and his
exploration of wirelessness through a Jamesian focus on network as “nothing
but concatenated conjunctive relations” (2010, p. 121),67 Chun says:
“Treating software as as a thing means treating it, again, as a neighborhood,
67. James argues that, “what really exists is not things made but things in the
making” (1996, p. 263) .
42
JPEG: the quadruple object
68. Chun discusses Brenda Laurel’s (1993) work where human computer activity is
understood as a “designed experience” akin to theatre. Here software can be seen as
“characters’” defined by their action.
43
JPEG: the quadruple object
70. Galloway echoes this stress on process when he says: “[O]bjects are always derived
from a preexisting copy (loaded) using various kinds of mediative machinery (disk
drives, network transfers). They are displayed using various kinds of virtuation
apparatuses (computer monitors, displays, virtual reality hardware). They are cached.
And finally, objects always disappear. Objects exist only upon use. They are assembled
from scratch each time and are simply a coalescing (of their own objectness)” (2004, p.
74).
44
JPEG: the quadruple object
Like Chun, Matthew Fuller explores the object in terms of its working. He
begins his account of media ecologies by asserting that objects should be
“understood to mean processes embodied as objects, as elements in a
composition” (2007, p. 1). Here relationality is reframed in terms of
processuality. Like Chun, Fuller is keen to problematise the “form–content
dichotomy and places objects and processes in a constellation of
interrelations” (ibid p. 46). The ecologies such as pirate radio and digital
artworks that Fuller investigates are matters of objects in relations but those
objects are characterised as dynamic, as processural.
Discussing the Cctv - world wide watch artwork,72 Fuller says: “In the
comprehension of the image as an image in real time comes also that of the
image as a process. Digital images - especially in such a visibly raw, low-
bandwidth state - demand to be understood as a computational and
algorithmic process” (ibid p. 156). This is not just that the image is the
outcome of a process. Rather it is a process. The object, whether the image,
the image file or even the imaging standard are moments of becoming - the
rendering, the writing to disc, the encoding. As with Hansen’s account, the
image object is in movement as the code runs and renders and as the data is
read by humans or software. The computational and algorithmic process is
one of flux, with the object shifting and changing as it relates and works. Just
as the code runs, so does the image. The objects share a processuality.
This stress on the process nature of software and software objects appears
in Fuller’s earlier work on Microsoft Word (2003). What was innovative
about this work (and his role in developing software artworks with I/O/D and
Mongrel (1998)) was the move from dealing with Word just as a commodity
to the software as processural, as setting in motion practices, subject
72. http://www.irational.org/cctv
45
JPEG: the quadruple object
positions and new relations.73 Here software became, took on new forms as
it worked (as a machine) and as a cultural, political, artistic and ideological
practice. When Fuller programmed the “Web Stalker” he was using code’s
processural nature to reconfigure the mesh: “A processual opening up of the
web that whilst it deals at every link with a determinate arrangement has no
cut-off point other than infinity” (1998, n.p.).
Later when Fuller edits Software Studies: A Lexicon, which can be read in
part as a collection of papers on software objects, the “stuff of
software” (2008, p. 1), he presents the collection’s analytical and political
power as “not to stage some revelation of a supposed hidden technical truth
of software, to unmask its esoteric reality, but to see what it is, what it does
and what it can be coupled with” (ibid p. 5). Here “what it is” and “what it
does” are intrinsically linked. This arises not just from his stress on the
importance of understanding (and even practicing) programming (ibid p. 10)
but from the nature of the code-object itself as processural.
73. Manovich takes a similar approach to another software package, Adobe Photoshop,
seeking to “understand how software applications shape our worlds and our
imaginations” (2011a). In a similar vein Paul Brafield discusses the relation between a
software package (Adobe After Effects) and a cultural habitus (2010).
74. Galloway and Thacker also reference Whitehead when they say that “networks exist
through process” (2007, p. 62).
46
JPEG: the quadruple object
specifically in his broader study of digital fictions and games (2009) where
he looks to understand the Tale-Spin/Mumble text/game/story-generation
system through studying processes in play. He says: “Studying processes [...]
focuses on the design and operation of the parts of the mechanism” (ibid p.
164). The focus is on the objects, the parts of the mechanism but those need
to be understood as things that are designed but also, always in operation.
Wardrip-Fruin opens up the game and gaming mesh as well as broader
technosocial relations of surveillance and governmentality (ibid pp.
200-203) by working with objects, but objects as processes - or perhaps
processes as objects.
Far from focusing on the specifics of the “design and operation” of objects
as Platform Studies does,75 studying processes shifts the emphasis to both
relations and one dimension of those objects, their operation. This neglects
the work of the object when it is not running or when it fails to run.
When Wardrip-Fruin says that “the internal processes of digital media are
designed artifacts, like buildings, transportation systems or music
players” (ibid p. 156), he is drawing attention to this dual nature of his
object, the continual binary oscillation of process-object.76 The processes-as-
objects Wardrip-Fruin calls “operational logics”, are not just the instances of
software running but the components of the computational mesh itself. They
are “distinctive”, “’operationalized’ models” of other forces and practices
such as human language or motivation (ibid p. 4). This idea of processes-as-
objects or oscillation fudges the question, neither dealing with the specifics
of JPEG compression through objects such as the Huffman table, nor the
particularities of JPEG as object, an actant that is powerful beyond its work
as process.
75. A term coined and a book series edited by Nick Montfort and Ian Bogost (2009). See
also Gillespie (2010).
76. Matthew Kirschenbaum makes a similar point when he discusses the “duality” at the
heart of digital mechanisms: product and process (2008, p. 15).
47
JPEG: the quadruple object
It is not just Galloway and Thacker who have used this potentiality as a
way of positioning the object as critical tool. Although perhaps not a
“software studies” scholar, Vito Campanelli uses the idea that digital objects
harbour a potentiality to explore the DivX and MP3 experience (2010).77
Here the particular codecs set in motion particular aesthetic (as well as
socio-political) experiences as legitimate or “pirated” media is encoded,
decoded, streamed or downloaded. His broader target of the web aesthetic
77. For a discussion of the MP3 protocol from an industrial (rather than technical/
material) and psychoanalytic perspective, see Sterne (2006).
48
JPEG: the quadruple object
To speak of potentiality in software is not just to engage with its uses - the
sort of potential of, and struggles around, free/libre and open source software
(FLOSS) or creative commons licenses to destabilise political and economic
relations (Lessig, 2002; Berry, 2004; Zittrain, 2004; Weber, 2005; May, 2006;
Berry, 2008a; Chopra & Dexter, 2008; Kelty, 2008; Garcelon, 2009; Milberry
& Anderson, 2009) as well as the potential of networks to rewire societies
and markets (Benkler, 2002a; 2002b; 2004; 2007). Rather it is to approach
potentiality as an ontological issue. From this perspective, the software
object (as a process enfolded in relations) harbours a potentiality as part of
78. This specifically network aesthetic is different in focus to the digital aesthetic sought
and discussed by Sean Cubitt (1998). For an historical perspective on new media
aesthetics see Aristarkhova (2007); as a site of struggle, see Blackwell & Dodgson
(2010) and in terms of aesthetics and user experience, see Engholm (2010). Specifically
on code and aesthetics, see Cox, McLean & Ward (2001). In terms of digital aesthetics
and affect, see Parisi & Terranova (2001). For a discussion of aesthetics and HD (High
Definition) technologies in video as an issue of materiality, see Flaxton (2011). For a
practice-research discussion of mobile aesthetics, see Baker, Schleser & Molga (2009).
79 As I will discuss, although they may be actual and present that does not mean that
every dimension is accessible.
49
JPEG: the quadruple object
50
JPEG: the quadruple object
80. Christian Metz is usually credited with first using the concept “scopic regime” (1981).
Of course the term “scopic” has a different genealogy, taking in Lacan’s “scopic field”
and the split between the eye and the gaze in The Four Fundamental Concepts of
Psychoanalysis (1986, pp. 67-78), through feminist film theory (Mulvey, 2009) and on to
Slavoj Žižek’s exploration of the gaze of the object and his realisation that “I can never
see the picture from the point that it is gazing at me” (1989, p. 8) (an idea picked up by
W. J. Mitchell in What do Pictures Want (2007)). It was Metz however who, while not
rejecting the psychoanalytic basis of the concept, arguably broadened its reach, from
the “scopic field” to the “scopic regime”.
51
JPEG: the quadruple object
81. For Crary the scopic and the scopic apparatus are only one dimension of the
workings of power. The “scopic” just as “technology” must not be allowed to be the only
actant on the stage. “I do not believe that exclusively visual concepts such as ‘the gaze’
or ‘beholding; are in themselves valuable objects of historical explanation” Crary argues
(2001, p. 3). In this later book he uses the term “perception” as a way of exploring how a
subject has come to be defined “in more than the single-sense modality of sight, in
terms also of hearing and touch and, most importantly, of irreducibly mixed modalities
which, inevitably, get little or no analysis within ‘visual studies’” (op cit.).
52
JPEG: the quadruple object
“[W]hat determines vision at any given historical moment,” Crary says, “is
not some deep structure, economic base, or worldview, but rather the
functioning of a collective assemblage of disparate parts on a single social
surface. It may even be necessary to consider the observer as a distribution
of events located in different places” (1990, p. 6) .83 It is this mesh, the
complex enfolding of subjectivity and technology, that he is looking to
unpack. His history is not one of “the observer” as opposed to “technology”
but rather observer-technologies. Those observer-technological objects carry
their potential disciplinary power into a field of relations actualised and
articulated in different ways in different historical moments.
82. For another media archaeological account of the kaleidoscope, see Christie (2007).
84. See for example Erkki Huhtamo’s account of media archaeological topoi, the
“temporary manifestation of a persisting cultural tradition” (2011, p. 41).
53
JPEG: the quadruple object
54
JPEG: the quadruple object
realised in the text but in a new actant network, the technosocial space or
mesh where Alan Turing and Strachey worked, struggled with their gay
identities and marked out the fields of computation. Wardrip-Fruin looks
towards processes as active players because they harbour potential, become
in specific historic conjunctures.
The issue for these media archaeologists is not simply to historicise media
or media studies but to address the potentiality within historically located
media objects as realised in particular meshes. For Harman however such a
pictures of objects as holding something back is to move away from
“actuality”. For Harman conceiving of objects as always, inevitably fully
present and actualised allows for an expanded, multi-faceted account of the
object and its relations.
55
JPEG: the quadruple object
Coming from film studies, Sean Cubitt has approached the new
distributed spaces of motion imaging such as YouTube through the H.263
codec which is enfolded with the Flash video (.flv) format. He says: “There is
no internet without the standardisation of internet protocols” (2008, p. 46).
Cubitt picks up on Mackenzie’s short account of motion imaging codecs in
Fuller’s Software Studies: A Lexicon where Mackenzie argues that: “codecs
structure contemporary media economies and cultures in important ways [...
they] catalyze new relations between people, things, spaces, and times in
events and forms” (Mackenzie 2008, p. 48). In particular Cubitt draws a
connection between video codecs’ “transform compression” and “motion
estimation”, the technique it uses to compress but also render motion, and a
“relational ordering that articulates realities together that previously lay
further apart” (2008, p. 45).86 Cubitt has continued this interest in standards,
drawing connections between colour space standards and an emergent 3D
scopic regime (2010).
While Cubitt’s demand that film studies engage with the codecs and
protocols that are now so important to the industry as well as the cultural
practices and relations that run through spaces and businesses such as
YouTube, Google and cloud computing (Cubitt, Hassan & Volkmer, 2011), is
important, those objects remain components, actants in a troupe rather than
the focus themselves. H.263 and HSV, LAB and RGB are enfolded with
corporate interests (Adobe) and telecoms and non-governmental bodies
(ITU, ISO etc.). He uses these relations as a way of mapping global and
neoliberal relations and discourses of the public sphere, at the same time
rendering those protocol objects as in an almost Latourian fashion, defined
by those relations. The determination may be more than one way but the
object does not exceed its relations.
At the same time those codecs are drawn as objects exhibiting a form of
dynamism, enabling processes of visualisation and imagining as well. They
are framed as processural in terms of how they work. Here compression
86. Fuller’s collection is interesting insofar as the series of very short chapters, all focus
on very specific, even technical aspects of “software”: the copy function; import and
export; the code library; the function; the interrupt or object orientation. Aside from
Mackenzie however, the protocols and standards are surprisingly absent.
56
JPEG: the quadruple object
While both Cubitt and Mackenzie have certainly engaged with protocols,
they have arguably not approached those codecs and standards as specific
objects requiring an account of their position and nature as objects. Rather
they have been addressed as components in a computational, visual or
techno-social mesh. They are defined and positioned by their relations with
other objects, their becoming and their potential to realise new meshes. Just
as Galloway positions TCP/IP in terms of its relation (as rules) to control
societies, so Mackenzie locates wireless standards in terms of broader fields
of experience87 and Cubitt draws colour space standards as elements in a
politically and economically charged scopic field and the history of the
“standard observer”.
Specifically in terms of the JPEG codec, Palmer argues that: “the JPEG
format is part of the new computational logic of photography” (2011, n.p.).88
For Palmer, JPEG needs to be approached as a rhetorical form. Following
Manovich’s linguistic turn, Palmer traces the ideological workings of JPEG as
a matter of coding, with JPEG a powerful component in the processes of
encoding at play in digital imaging. For Palmer, “the JPEG is rhetorically tied
to the idea of democracy in an age of distributed imaging, in which the
87. Stephen Jones and George Thiruvathukal also discuss WiFi in terms of hardware in
their discussion of the Ninetendo Wii (2012, p. 11).
88. Technically of course JPEG is not the format but the standard that enables the JFIF
or EXIF format.
57
JPEG: the quadruple object
image has been spatialised in global databases” (op cit.). He argues that “its
persuasive power, lies in this same invisibility” (op cit.). While I would agree
(and come on to discuss, JPEG’s enfolding within governmental rhetorics), I
would argue that it is precisely JPEG’s visibility (admittedly in terms of its
sensual rather than withdrawn dimensions - see next Chapter), that is
important. JPEG’s ubiquity as object in multiple connections depends on its
sensual visibility and accessibility.89 I look to an object-oriented focus on
the object itself not its rhetorical or linguistic workings as a way into
addressing its power.
89. Palmer points to the fact that Chun makes a similar claim, seeing software as “a
powerful metaphor for everything invisible that generates visible effects, from genetics to
the invisible hand of the market” (Chun, 2011, p. 2).
58
JPEG: the quadruple object
59
JPEG: the quadruple object
90. There is an interesting parallel with Harman’s willingness to account for imaginary as
well as “real” objects.
60
JPEG: the quadruple object
This account of the object also has practice implications. In their work on
“zombie media” and “circuit bending”, Parikka and Garnet Hertz explore
the creation of the “punctualized object”. They say:
61
JPEG: the quadruple object
Within software studies itself the three themes often appear enfolded
together. At first sight Berry’s The Philosophy Of Software (2011b) would
appear to offer an object-oriented account of software objects. After all,
62
JPEG: the quadruple object
91. The term is in quotes here to highlight Latour’s reluctance to be seen as developing
such a framework (Latour, Harman & Erdélyi, 2011).
63
JPEG: the quadruple object
92. Robert Gehl says: “the archival capacity of Web 2.0 allows for new centralizations of
power, hidden away beneath the abstractions of the smooth Web 2.0 interface” (2011, p.
1240). While I would certainly argue for the importance of addressing power and the
archive, I would contend that one needs to address decentralisation and circulation of
governmental power through objects rather than look for a central location.
64
JPEG: the quadruple object
affordances they set in motion. One cannot account for the seminal position
of VCS and VCS gaming without seeing how the objects connected.93
93. As noted above, Galloway (2006a) has also written on games and Thacker has
published in a journal that looks at “gaming” in a wider sense (2001). Although Galloway
is once again keen to explore how games relate to broader control systems and
practices, he does this via a formalist account of the game using a film studies
methodology. The material form, platform or object remains secondary. Thacker too uses
games as way into exploring his themes of the relationship between the body and
technology. Again simulation and gaming are the focus. The material platforms,
technologies, consoles, gadgets and objects are not the focus. For a specifically media
archaeological approach see Huhtamo (2005).
94. For a more ANT-derived account of “technical objects” see Akrich (1992) where
although “technical objects [...] simultaneously embody and measure a set of relations
between heterogenous elements” (1992, p. 205) the stress is on the relations rather than
the object.
65
JPEG: the quadruple object
relation to it but also its connections with Sigma’s brand, business, Flickr
groups and social media imag(in)ing practices. A Foveon imaging pipeline
creates different images but also different networks.95
95. A similar argument could be made around the emerging “light field” camera
technologies http://www.lytro.com also discussed by Bogost (2012b) as well as my
stereo photography, see Epilogue.
96. I discuss the critique of OOP as formalist in The JPEG Object in Theory chapter.
97. In their Afterword on Platform Studies (2009, pp. 145-150), the authors lay out a
vision for a discipline that takes such details seriously. Using a diagram reminiscent of
Galloway’s account of the four nested layers of the Internet suite (2004, p. 39), and
Mackenzie’s similar exploration of the ISO’s seven-layer Open Systems Interconnection
Reference Model in relation to wireless technologies (2010, pp. 99-100), Montfort and
Bogost outline “the five levels of digital media” (2009, p. 146). Platform studies positions
itself as filling a gap: “If code studies are new media’s analogue to software engineering
and computer programming, platform studies are more similar to computing systems
and computer architecture, connecting the fundamentals of digital media work to the
cultures in which that work was done and in which coding, forms, interfaces, and
eventual use are layered upon them” (2009, p. 147).
66
JPEG: the quadruple object
Bogost’s list of objects in play is, like Latour and Harman, fundamentally
inclusive:98
For Bogost systems are also units. “[A] unit is a material element, a thing.
It can be constitutive or contingent, like a building block that makes up a
system, or it can be autonomous, like a system itself. Often, systems become
units in other systems” (ibid p. 5 emphasis in original).99 Here Bogost echoes
Harman’s willingness to address even meshes at the scale of objects.
Bogost’s systems are just another form of unit. As an example, when he
discusses intellectual property (IP) and games, IP is not some context or
structure or background. It is a specific object/unit. “Like component
software, game engines are IP.100 They exist in the material world in a way
that genres, devices, and clichés do not” (ibid p. 56).101 Intellectual property
is a key component in the mesh and so is a unit in play. One cannot
understand video games, their cultural position or the workings of the
industry unless that object too is on the table. Just as I argue with respect to
imaging standards, “[g]ame engines are no more transcendental than genres,
in the sense that one cannot play a game engine but only a game that
encompasses and integrates that engine to create a work” (ibid p. 57).
98. Bogost coined the term “Latour Litany” to refer to Latour’s lists of objects. See for
instance: “Golden Mountains, phlogiston, unicorns, bald kings of France, chimeras,
spontaneous generation, black holes, cats on mats, and other black swans and white
ravens will all occupy the same space-time as Hamlet, Popeye, and Ramses II” (Latour,
1999, p. 161). For a software Latour Litany creator, see http://www.bogost.com/blog/
latour_litanizer.shtml.
99. Wardrip-Fruin draws a parallel between Bogost’s term and his own account of
“operational logics” (2009, p. 17). As I have argued however I see Wardrip-Fruin’s stress
on processuality as a move away from a more object-oriented focus.
101. Of course Harman’s OOP would admit genres, devices, and clichés to the realm of
objects.
67
JPEG: the quadruple object
Conclusion
I have sought to show that accounts of the software object and even
protocol have worked within the three themes of relationality, processuality
and potentiality. These have positioned the software, protocol or standards
object as processes enfolded within powerful meshes harbouring a potential
for critical or governmental engagement. The stress in these accounts is on
the field of becoming and relations.
68
JPEG: the quadruple object
In her later work What Remains (2003), Mann was again accused of
objectification. As the omniscient artist-subject she had violated another
taboo, turning decomposing corpses in an FBI scientific facility into
“bodies”, objects of her art. Once again, she as subject turned her camera
on objects.
69
JPEG: the quadruple object
Mann’s use of not just ancient cameras and lenses but also antique
processes is also a part of her object-oriented sensibility. The glitches her
technologies introduce, like the cliched inclusion of the photographer’s
shadow, inscribed the technology-object across the image. The “failed”
coating of the wet collodion plate, the dust, the refracted light in the ancient
lens, are themselves objects connecting with each other, with Mann, the
thing being photographed, the gallery, the art market and... Mann is open to
those objects. More, she embraces them and their connections.
In The Quality of the Affection, 2006 an image part of the Proud Flesh
series (Ravenal, 2010), Mann’s naked husband Larry is an object. Posed,
arranged, positioned he sits with his back to Mann’s camera, a parabola
intersecting with a dark shadow on which other objects - a glass, a pencil
shape rendered illegible by the camera. One encounter with the work places
70
JPEG: the quadruple object
Sally, Larry and the glass within a field of relations interwoven within the
history of nude art, gender, the body (Larry suffers from adult muscular
dystrophy) and power. But The Quality of the Affection, 2006 is an object-
work. It is created through and with objects - not in some value-laden sense
of powerful subject, powerless objects and the gaze - but as a series of
objects connections within the frame, at the moment of exposure, the
moment of viewing and the moment of analysis. Larry’s back, the lights
falling on his spine, the glass, the streaks in the collodion, the glass, the
gelatin in the print, the Aperture Foundation and the Virginia Museum of
Fine Arts, Sally, her lens... connect and reconnect within the image object,
the art object, the photographic exposure objects, the family object... This is
not a photo of objects by a subject but a photograph with objects, by an
object.
71
JPEG: the quadruple object
102. Objects “such as Styrofoam and plutonium that exist on almost unthinkable
timescales” (Morton, 2010a, p. 19).
104. Bennett is clear that things do not have to be impressive or somehow deserve our
attention. Anything is an object and can be lively. I would agree with Matthew Tiessen
(interestingly a practice-research artist-ontologist) who says: “[I]f nature and things have
to be exceedingly impressive to deserve our consideration we’re left repeating the
expectations that gave rise to our lack of recognition for thing-power in the first place. In
response to Bennett’s concerns about fear and respect my modest proposal is that
things be encountered from a position of responsive humility – a position that recognizes
that things are all we’ve got, whether they command respect or not” (2010, p. 234
emphasis in original).
72
JPEG: the quadruple object
unicorns, characters in books and protocol but also points to the fourfold
character of objects that allow me to develop a JPEG-based object-oriented
photography.
105. Harman uses a curious negative way of framing his rules. One could perhaps
reframe these objects as positive statements: “an atom is as much an object as is a
skyscraper”; “an electron is as much an object as is a piano”; “a soul is as much an
object as is cotton candy”; “helium is as much an object as is plutonium” and
“mountains are as much objects as are hallucinated mountains”.
73
JPEG: the quadruple object
74
JPEG: the quadruple object
worthy of study and necessary to account for. This is not the simple point
that media and cultural studies made when it said that Homer was as worthy
of study as Homer or that we needed an account of tattoos as well as Titian.
Nor is it the far earlier establishment of literary or art history studies arguing
for their objects being worthy alongside the natural sciences. This is not a
flattening of hierarchies and categories for political, professional or
academic interest. It is a metaphysical statement that all objects are in play
whether we like it or not. One could debate whether cosine is a natural
“thing”, a Platonic form, but from an OOP perspective such a debate is
meaningless. The DCT demonstrably is present within JPEG and does
something in the digital imaging pipeline.
This is not to say that Harman sees every object as equally real. His whole
perspective with real and sensual objects (see below) is designed to explore
these different dimensions (2012a). What Harman is looking to leave out of
analysis is the idea of any kind of “non beings”. If things are at work, then
they are objects. JPEG is not imaginary but it is certainly difficult to see or
find. It is a standard written or maybe woven into software and hardware
meshes as well as business strategies and grandmother’s doting over a new
baby. But even if JPEG was not “real”. Even if the idea that a standard that
compressed data efficiently and effectively was an elaborate Capricorn One-
like conspiracy perpetrated by mad scientists, Adobe and Google, it
wouldn’t matter. JPEG would still be worthy of study because it was still at
play in people’s photography, their photographic consumption and their
75
JPEG: the quadruple object
In some ways JPEG is an ideal candidate to explore these rules. It does not
really have a size; it is both simple in its role but complex in its form; it is
ephemeral in its working but durable in its enfolding within imaging; it is
unnatural and it is clearly real within the digital imaging pipeline but
simultaneously unreal in its presence within the business plans of photo-
network start-ups where it is designed to reassure venture capitalists of
interoperability and flexibility 106.
106. Instagram’s API documentation says: “You must first save your file in PNG or JPEG
(preferred) format” (Instagram, 2011). This is not just instructions to engineers, it is a
statement for the whole of the Web 2.0 community - including investors and partners,
that Instagram works with the standards users (for which read customers) use.
Facebook purchased Instagram in the lead up to its IPO in 2012 secure in the
knowledge Instagram images and imagers would connect seamlessly with its services,
technologies and I will argue governmental rationality.
107. While Harman consider his position as “realist”, Bryant prefers to characterise his
as “materialist” (2012a; 2012c), as I discuss below. In fact in a “personal”
correspondence on Facebook he told “me” (the scare quotes signify my unease about
the nature of the interaction and the correct terminology for it): “I find myself increasingly
shying away from OOO and moving into the Bennett:materialist camp sans
vitalism” (2012b, n.p.).
76
JPEG: the quadruple object
The first critical response to objects asserts that they are not
fundamental. All of the dogs, candles, and snowflakes we
observe are built of something more basic, and this deeper
reality is the proper subject matter for philosophy (2011g, p.
8).
108. This form of reductive materialism must be distinguished form the vitalist or
objected-oriented materialism that I come on to discuss in relation to Bennett and
Bryant.
77
JPEG: the quadruple object
109. JPEG’s presence in a patent and in the software programmed onto a camera chip
are different in kind of course, but from an OOP perspective they are equally important
see the discussion of Raskolnikov above.
78
JPEG: the quadruple object
110. In The Quadruple Object and in his study of Meillassoux (2011h), Harman
discusses various forms of correlationism and why, in his view, they are philosophically
and logically indefensible. My aim here is not to enter into those subtleties but merely
highlight the more general human-object formulation as a problematic.
111. I will come on to argue this human-object, JPEG-object relation happens within
something that should also be seen as an object.
112. Relationism, the idea that objects must be understood in terms of their relation to
other objects has a long history e.g. Gottfried Leibniz’s argument contra Newton, that
time and space must be seen in relation to material events in the universe (Futch, 2008);
Georg Simmel’s “Wechselwirkung”, or the “reciprocal actions and effects” (Milà, 2005, p.
44) and Karl Mannheim’s sociology of knowledge which held that: “there are spheres of
thought in which it is impossible to conceive of absolute truth existing independently of
the values and position of the subject and unrelated to the social context” (1955, p. 79).
79
JPEG: the quadruple object
head on for arguing that an object’s power is defined by its field or network
of relations or accidents. JPEG from this perspective is power-full because it
is part of a scopic regime, apparatus or project. It is more powerful than
Google’s alternative WebP because it is in networks with Flickr, Facebook,
Adobe, Nikon, data-mining and domestic memory management. JPEG is
either an image in the mind or just its effect on other things: a bit player.
Harman argues that to follow this through means the object disappears. He
says:
But that house, that JPEG, is real. There is a unity to it and not just in the
standards written and posted by its developers. It is in relations - within my
imaging and apparatuses as well as within the governmental scopic regime -
but that does not exhaust its reality, its unity. The correlationist and relationist
projects overmine the object. Empiricist or process philosophies undermine
it. Both refuse to deal with objects in their multiple dimensions,
connections, presence and reality. In terms of my project, they skip over
JPEG entirely. Harman argues that undermining and overmining are not
mutually exclusive but partners in shifting the focus from objects. “[E]very
undermining philosophy needs an overmined component as a supplement,
and vice versus” (ibid p. 14). A materialist call to find the basic building
blocks then locates those blocks (atoms, sub-atomic strings or Huffman
tables) in terms of relations or qualities - hardness, resistance, statistical
chance, probability etc. Similarly a relationist picture of flux and becoming
still demands a fundamental perceiver such as Whitehead’s God to correlate
all perceptions. In both cases “autonomous objects are [...] excluded as a
80
JPEG: the quadruple object
113. Harman discusses this missing piece as “the third table” alongside A.S.
Eddington’s (1928) “scientific table that is mostly empty space and made up of rushing
subatomic particles, and the table of everyday life” (2011j, n.p.; 2012c).
114. Karen Barad argues that “the primary ontological unit is not independent objects
with independently determinate boundaries and properties but rather what Bohr terms
‘phenomena’ [...] the ontological inseparability of agentially intra-acting
components” (2007, p. 33). For her, phenomena have a reality. Harman would agree.
Where he would differ is in Barad’s sense of objects needing something more. Barad
cites Judith Butler’s argument that matter is “fully sedimented with discourses” (1993, p.
29), arguing “’Things’ don’t preexist; they are agentially enacted and become
determinately bounded and propertied within phenomena. Outside of particular agential
intra-actions, ‘words’ and ‘things’ are indeterminate. Matter is therefore not to be
understood as a property of things but, like discursive practices, must be understood in
more dynamic and productive terms-in terms of intra-activity” (Barad, 2007, p. 150).
Here it is when objects are “enacted” that they become things. For Harman, there is no
necessity for anything outside the object. Objects do not lie in wait or in potentiality for
enactment.
115. Harman is clear that this real/sensual objects is not the the same as a primary/
secondary qualities distinction. He argues that what are seen as primary qualities,
physical properties such as mass, shape, position are “not deep enough to qualify for
the status of ‘primary’”. These qualities are no more “real” or basic than so-called
secondary “mere qualities, such as ‘sweet’, ‘red’, and so forth” (2009c, p. 195).
81
JPEG: the quadruple object
116. Harman prefers to use the term “sensual” for this dimension of objects. He says:
“Husserl uses ‘intentional’ to refer only to the unified objects of consciousness, while
excluding the shifting surface qualities of things from the intentional domain. So-called
‘sense data’ are not intentional for Husserl, precisely because they are not object-
oriented. For this reason, a new unified term is needed that covers both the enduring
objects of consciousness and the overly specific facades through which they are always
manifest” (2009c, p. 136).
82
JPEG: the quadruple object
117. Inevitably when dealing with objects as actants, one slips into anthropomorphic or
worse anthropocentric discourse speaking of unhuman objects as “perceiving” when
attempting to explore the way they connect. Bennett is not ashamed of the often very
anthropomorphic language she uses. “We need to cultivate a bit of anthropomorphism -
the idea that human agency has some echoes in nonhuman nature - to counter the
narcissism of humans in charge of the world” (2010a, p. xvi). Harman echoes this when
he says that “a bit of anthropomorphism may be needed to overcome
anthropocentrism” (2011a, n.p.). Bogost explores similar issues as he discusses
“metaphorism” and the “unknowable inner lives of units” (2012a, pp. 61-84).
118. It is important to note that just as with the “distinction” between RO and SO, so
when Harman talks of real and sensual qualities, he is not saying that one is more
important, fundamental, basic or even real than the other. Rather they must be seen as
two dimensions at work within a unified object.
119. It is important to note that for Harman these real or eidetic qualities are not
universal. These are not the ‘eternal objects’ of Whitehead. Rather they are always
particular to an individual object. When I press the button and encounter the sensual
JPEG (that dimension to JPEG that I work with), I encounter a particular running of JPEG,
in a particular moment, within a particular apparatus (JPEG’s SQ). I also encounter
JPEG’s RQ, its particular digital imaging pipeline that make JPEG particular.
83
JPEG: the quadruple object
84
JPEG: the quadruple object
120. It is important again to note that Harman (and I) are not denying relations. What he
is drawing attention to here is that those relations do not define the object.
121. Harman is at pains to distance his reading of Heidegger from what he sees as a
view that Heidegger’s withdrawn realm is a “deeper and unified system of
reference” (2011g, p. 35) which he would see as a case of “undermining”. Objects
withdraw not into some field or monastic lump of being but into themselves, into “private
interiors, barely able to relate at all” (ibid p. 36). The reason we cannot reach JPEG, the
reason it slips through our fingers and all we are left with are its traces in JFIF or EXIF
files or our sensual encounters with its instantiations, is because, as with all objects,
JPEG “does its work in the cosmos”. It has a reality beyond any relations or particular
instantiations. This reality is not located in the specifications of the Joint Photographic
Experts Group. It has a metaphysical reality: its status as object.
122. Here Harman draws on his debt to Aristotle in his debate with Plato regarding forms
and substance (Harman, 2011c).
85
JPEG: the quadruple object
real JPEG does not. It still has an ontological reality, an object status.
Although the object withdraws from access, Harman argues that the RO is in
relation (or “tension” as he calls it) with the object’s sensual and real
qualities. The shifting sensual qualities we encounter as we walk around the
Velodrome or the particular Huffman table the software encounters in a
particular imaging moment, cannot just be phenomenal. These qualities
must emerge from something real. The object may withdraw or be
inaccessible but it “emits sensual qualities into the sphere of presence,
despite being withdrawn in its own right” (ibid p. 49). There is a relation
between the RO and the object’s SQ. Similarly, the RO is not an empty unit.
It is in tension with real qualities (RQ), those essential features that make the
object what it is. As a cyclist I encounter the Velodrome. I ride a particular
geometry in a particular temperature-controlled space. Those essential
qualities define this as the Olympic Velodrome not the National Cycling
Centre Velodrome. Those qualities are connected with the withdrawn RO. If
the RO did not have those specific but real qualities, it would be
indistinguishable from any other withdrawn object. In the case of protocol,
JPEG is not an empty unit. It has real qualities Colour Transforms, Huffman
Coding, DCT, the things that make JPEG compression the object it is. The
withdrawn RO must have a relation to those qualities or JPEG would be no
different than any other standard... and it is.123
This is Harman’s fourfold: Real Object (RO), Sensual Object (SO), Real
Qualities (RQ) and Sensual Qualities (SQ). These are four poles to the
unified object, four dimensions that allow us to explore objects without
recourse to fields of relationality, potentiality or process. To map JPEG as a
quadruple object is to see:
123. Harman draws on Leibniz’s argument that: “monads must have qualities, otherwise
they would not even be beings” (Leibniz, 1989, p. 216). Or in other words, each monad
“needs a multitude of qualities to be what it is, to differ from other monads rather than be
interchangeable with them” (Harman, 2011g, p. 49). Harman’s whole quadruple model is
designed to show how the qualities (real or sensual) are not all there is to an object.
They are part of the picture but not the whole story.
86
JPEG: the quadruple object
124. This theme has been developed over the past few years in Harman’s work. This
chapter concentrates on that presented in The Quadruple Object but earlier workings
can be see in (2009c, pp. 214-221; 2010b; 2010d). See also Harman’s developing
thinking in (2010c).
125. As I discuss in the JPEG: the governmental object chapter, the “Open Graph” is a
term coined by Facebook for its particular implementation, and it argues, extension of
the term “social graph”, a term from graph theory designed to encapsulate the network
relations between people (Facebook, 2012).
87
JPEG: the quadruple object
126. Although JPEG (in its EXIF file format in particular) is related to issues of
geolocation and geolocative metadata, I chose not to focus on these issues in my
practice as I was concerned with JPEG encoding, not with the file formats and the
broader issues of metadata.
127. This is of course not to suggest that physical distance or space does not exist,
merely that thinking in terms of objects gives us a new way of understanding what that
means for us and other objects.
88
JPEG: the quadruple object
89
JPEG: the quadruple object
stateless digital objects,128 let alone protocols, can be seen as a matter of the
JPEG RO and its tensions with its particular profiles SQ.
128. Of course this idea of “stateless” does not preclude legal battles over digital
objects see for instance struggles over Wikileaks, Pirate Bay etc.
129. These real (essential) qualities are different to the sensual qualities we encounter in
particular instantiations or profiles of JPEG. A specific Huffman table is a sensual quality
an object encounters. “Huffman coding” or “Discrete Cosine Transform” as a
mathematical formula or algorithm is “real”. It cannot be encountered in its totality. While
this would seem to have Platonic formal overtones, as has been noted Harman owes far
more to Aristotle. These formulae are not in the realm of forms, or abstract or
phenomenal. They are real, present and are dealt with in terms of substance.
90
JPEG: the quadruple object
relations) and the Real Qualities (the algorithms and mathematical formulae
that JPEG uses), what Husserl calls the eidos. We may never be able to fully
access the Velodrome’s geometry or Discrete Cosine Transform, as a
mathematical formula in their complete reality, as opposed to their particular
sensual instantiations for us, but we can intellectually identify them and
recognise them as distinguishing characteristics.
For Harman these relations between the four different poles of the object
are in “tension”: what he calls fission and fusion. The relations between a SO
(the Velodrome or JPEG we encounter) and its particular profiles (SQ) that
we experience as time is characterised by moments of rupture (2011g, p.
103). At the moment of passing the Velodrome at dusk we realise that
unified kernel has transient sensual qualities as the light sparkles on the
wood panelling - “I never noticed that before”. At the moment of encoding
light-as-data we (or other software actants) are faced with various
compression settings, the transient qualities, profiles or instantiations. The
tension breaks: fission. The same break can happen in the tension between
the SO and its RQ (eidos). The intellectual labour of paring down JPEG to
find its real qualities. this “reverse engineering” through “theory” is a matter
of fission, breaking down JPEG (as I do in the Digital Imaging Pipeline
chapter). It is this process of fission I look to explore in my RAW/JPEG
apparatus.
91
JPEG: the quadruple object
The other two relations, between the RO and its SQ (which we know as
space) and its RQ (which we know as essence) are also best understood as
tensions but this time characterised by “fusion”. Real objects and their
sensual qualities only meet when they are fused (ibid p. 105). The real,
withdrawn, inaccessible velodrome has sensual qualities in particular
locations as I circle it. Its sensual qualities fuse with it as I look or take a
photograph. Even in the non-locational space of JPEG, the SQ of JPEG that I
engage with, the particular profiles, settings or instantiations, are fused to
“JPEG” so I can image with it or write about it. Those SQ must have a
connection but that connection is made or fused (locally in my camera or
globally across the networks) with each use. Similarly, the relation between
the RO and its RQ is again a matter of fusion. This essence is “fused” as the
always withdrawn protocol/standard that we can never fully grasp is brought
together with those RQ (the mathematical laws and algorithms) that can be
intellectually grasped but never exhaust the totality of the object. That
essence is fused, for Harman, through an outside entity, a mediating term. As
we shall see, objects meet others within objects. They effect each other
within/through objects which fuse them. Huffman coding or Discrete Cosine
Transform (as JPEG’s RQ) are fused as a part of “JPEG” (the RO) by and
within the Photoshop object as it encodes the data. They are fused as part of
the JPEG standard that a Web 2.0 pitches as its new business model: “Our
social network will use JPEG so users can easily upload, share and see each
other’s images...” As we shall see, it is this process of fusion I look to explore
in my mash-up apparatus.
The connection
As I have noted, Harman’s framework of autonomous, actual objects does
not preclude the sort of actant networks that Latour talks about, and the sort
of techno-social, computational meshes addressed by software studies. In
fact the power of Harman’s quadruple object is that it offers a powerful way
of addressing the relations between human and unhuman actants, the sort of
relations within which JPEG is enfolded, that characterise scopic
governmentality and that I notice and work with within my own practice.
92
JPEG: the quadruple object
Harman is clear that these relations do not define the object. Rather his
model looks for an object-oriented account of relations.
130. Harman discusses the history of the idea of a necessary mediator in terms of
“occasionalism” (2009c, pp. 112-116; 2011g, pp. 70-72). He calls Latour the first
“secular occasionalist” (2009c, p. 159).
131. It is important to re-emphasise that real and sensual objects are not separate
objects but rather different dimensions of the same object.
93
JPEG: the quadruple object
JPEG object in the literature, there are other ways of approaching digital,
even “weird” objects that are productive and creative. My argument is not
that Harman is somehow more “true” than other accounts, but merely that
his framework of objects connecting with objects allows us to map network
objects-meshes like Facebook and digital imag(in)ing and its protocols in
ways that raise new questions and open new practices.
But this mediation extends beyond just the position of the human
observer. Real unhuman objects act as the mediator. The datamining
algorithm on a Facebook server that reads the metadata or even the faces in
a JPEG-encoded image file has a sensual dimension. It is a SO insofar as it it
is present to human or unhuman consciousness or access. Similarly that data
file has a sensual dimension that can be read.132 The two connect. We know
they do because we see the ads served on our page or the Friends suggested.
That connection happens within a RO, an object that has hidden
dimensions, a deeper totality that is not available to full access. The Open
Graph is more than a Facebook marketing term or even ideology. It is an
object with a real dimension. Its reality as governmental actant (as I shall
discuss) is deeper and more inaccessible than those dimensions present to
my or any other object’s consciousness. It is this RO within which the
algorithm (SO) and the image data (SO) connect. The connection is within
objects not in some wider field, some psychological, semiotic or capitalist
132. Both actants have a real dimension too. There is always inevitably more to their
nature and power than a particular profile, instantiation or access.
94
JPEG: the quadruple object
I object: relationality
Harman’s focus on withdrawn real objects in tension with accessible
sensual qualities is, for Steven Shaviro, incompatible with contemporary
relational culture.
133. Harman often talks about these connections as happening within the “molten” core
of another object (2009c, p. 215). In The Quadruple Object he phrases it slightly
differently: “any relation immediately generates a new object” (2011g, p. 117 emphasis
in original). The point is the same. The “digital imaging pipeline”, “photography” or
“business plan” as objects are the site of connection. The enfoldings, relations and their
governmental implications are located at the scale of objects.
95
JPEG: the quadruple object
I object: processuality
Shaviro’s reading of Whitehead also demands a focus on processuality. He
holds that “Becoming is the deepest dimension of Being” (2009, p. 17) and
argues that Harman’s focus on actuality leads not only to an analysis devoid
of relations but also of change. Without an account of objects as matters of
134. Here he differs from Manuel De Landa who speaks of the exteriority of relations
(2006) where relations are exterior to the terms they connect.
135. Facebook’s photo storage system, see the JPEG: the governmental object chapter.
96
JPEG: the quadruple object
136. It is important to note that not everyone would agree that Deleuze is a “process
philosopher”. My aim here is not to engage with Massumi’s reading of Deleuze but rather
consider his implicit critique of OOP.
137. Shaviro too critiques OOP for what he sees as its failure to deal with change:
“Harman tends to underestimate the importance of change over the course of time, just
as he underestimates the vividness and the extent of relations among entities” (2011, p.
285).
97
JPEG: the quadruple object
For Harman a position derived from Whitehead and even from Latour
alone cannot account for objects and change. Whereas Shaviro accuses
OOP of denying relations and the possibility of change, Harman argues that
an account of objects as constituted by their relations actually prevents an
account of change. If there is nothing beyond relations, there is no “surplus”.
138. Harman reads Whitehead’s “occasions” as a way of understanding the object, not
rejecting the idea of entity. Where they differ, he says, is that Whitehead sees those
entities as clusters of relations whereas Harman works toward a non-relational model of
objects. He says: “The reason they can be called ‘occasions’ is because ‘the notion of
an unchanging subject of change is completely abandoned’. An entity is not a durable
substance undergoing accidental adventures in time and space: instead, ‘actual entities
“perpetually perish”’. They do not lie behind their accidents, qualities, and relations like
dormant substrata, but are ‘devoid of all indetermination’ (Whitehead, 1978, p. 29).
Actual entities are fully deployed in every instant and then instantly perish, attaining
‘objective immortality’ not by persisting over time (impossible for Whitehead) but by
giving way to closely related yet new actual entities. In Prince of Networks I showed that
the same holds for Latour” (2011i, p. 294 emphasis in original). Clearly this is a particular
reading of Whitehead, one that Shaviro and many others would disagree with. My aim
here is not to discuss the validity of that reading, but rather use it as a way of clarifying
Harman’s own position with respect to objects and change.
98
JPEG: the quadruple object
“Every object would be exhausted by its current dealings with all other
things” (ibid p. 295). But objects do have more. There are dimensions that
are hidden, withdrawn. Where Latour and Whitehead may argue that change
is possible as objects become and perish (Whitehead) or enter new alliances
(Latour), this demands that change is a series of discrete steps - new
occasions or new configurations. For Harman this moves away from a strict
actualist focus on the object to either advocating a second realm of objects -
the “eternal objects” of Whitehead (1978, p. 61)) or a realm of potentiality
beneath objects (the “plasma” of Latour (2005, p. 244). Harman refuses to
imag(in)e anything beyond the actual object. For him an object, if conceived
as deeper than its relations, can account for change and networks without
recourse to something else.
99
JPEG: the quadruple object
I object: potentiality
Levi Bryant, a strong advocate of objects as the starting point for
philosophy has questioned Harman’s rejection of potential, his insistence
that objects hold nothing back. He says:
100
JPEG: the quadruple object
139. Bryant explicitly connects this idea of the split to Lacanian thought (2011, p. 131).
140. Bryant is more comfortable with Deleuze than Harman, arguing “[n]o one has
explored this anterior side of substance - in the transcendental, not the temporal, sense -
more profoundly than Gilles Deleuze” (2011a, p. 94). Harman, meanwhile argues that
objects prior to their relations are not “virtual”. Rather they have a definite character prior
to entering relations. That character can be described as a matter of objects and
qualities. He says: “The much-discussed difference between potential and virtual, so
often wielded like a billy club in our time by Deleuzian hooligans, is irrelevant here - both
terms fail Latour’s standard of concreteness in exactly the same way” (2009c, p. 101). It
is not my concern here to engage with their respective readings of Deleuze and debates
around the potential and the virtual (Bryant, 2011a, pp. 94-104; 2011b; Harman, 2010f)
but rather to address Bryant’s argument that the rejection of potential undermines
Harman’s account of the actual. Bryant is not uncritical of Deleuze. He says: “What we
thus get in Deleuze’s thought is a sort of vertical ontology of the depths. Rather than
entities or substances interacting with each other laterally or horizontally, we instead get
an ontology where difference arises vertically from the depths of the virtual” (2011a, p.
100). As with all object-oriented approaches, any tendency towards depth, foundations
or context is a move away from objects.
101
JPEG: the quadruple object
142. Bryant also uses the terms “susceptibility” to translation using Latour’s idea of the
network relations that objects undergo (2011a, pp. 115-116)
143. He would perhaps appreciate Morton’s feting of John Clare’s “absolute ‘therenes’”
in his poem Mouse’s Nest (Morton, 2010a, p. 50) but perhaps not Gertrude Stein’s
comment about Oakland: “there’s no there there” (1971, p. 239).
102
JPEG: the quadruple object
potentiality as a retreat from the scale of objects.144 For him, “[t]he recourse
to potentiality is a dodge that leaves actuality undetermined and finally un-
interesting; it reduces what is currently actual to the transient costume of an
emergent process across time, and makes the real work happen outside
actuality itself” (2009c, p. 129). For Harman nothing must replace the actual
and the concrete. There are no ”hidden overlords: whether they be potential,
virtual, veiled, topological, fluxional, or any adjective that tries to escape
from what is actually here right now” (op cit.).
144. This goes to a broader Bryant-Harman argument around how objects touch. For
Bryant objects can and do touch and that is how they “unleash the forces of another
object” (2011a, p. 71). The acorn’s coiled, potential to become an oak tree, a missile,
food or an artwork are unleashed as it touches the soil, a child’s catapult, a squirrel or a
canvas. JPEG’s coiled, potential to become an imaging standard, a data-mining tool or a
social convention are unleashed as it touches in-camera software, a Facebook algorithm
or an Instagram API. Harman is not against connection, let alone power-relations. What
he says however is that objects cannot touch. Because they have a “real” dimension that
withdraws form all access, they cannot touch except within objects, through a mediating
object (RO with RO through a SO; SO and SO though RO). The acorn and the catapult
connect in a weapon object. JPEG and the software within a Open Graph object. Each
connection is different, actual, specific and now. For Harman, if objects could touch
there would be no need for mediating objects or what Latour calls “translation”. For
Harman this is philosophically difficult, not taking Heidegger seriously enough. In terms
of my project, it is difficult because it fails to adequately account for mediating objects
such as the Open Graph.
103
JPEG: the quadruple object
I object: formalism
Victor Burgin identifies two “pitfalls awaiting the art theorist with no grasp
of semiology, ‘the temptation to treat the work of art as a purely formal
construction’ [... and a] focus [..] on the internal life of the autonomous
object” (1986, p. 1). Burgin picks up on a powerful tradition of anti-
formalism within media and cultural studies that arguably has a new
relevance when object-oriented approaches demand that everything starts
from (and in Harman’s case perhaps) finishes with objects.
146. Harman would of course say that “exists” and “works” are two different things,
objects exist regardless of whether or how they work, I include both here because my
concern in this work is with the relations (work) of JPEG which arises from its ontological
existence.
104
JPEG: the quadruple object
possible” (1976, p. 138).147 Had OOP been around in the early days of
media and cultural studies148 it would doubtless have faced charges of
formalist fetishization of the object. How, therefore does OOP stand against
the charge of formalism? Firstly one must show that OOP is formalist and
then that formalism is, in itself a bad thing.
OOP demands that the object is the core focus of analysis and
interpretation. Particularly in Harman’s case, there is never any need to leave
the object and look to a wider field, plasma, process or realm of becoming.
The famous Latour litanies with which OOP is littered are testament to the
belief that we can do philosophy and media analysis by concentrating on
objects. Formalist approaches to literary texts began with a similar focus on
objects, rather than the later preoccupation with systems. “The Formalists
started out by seeing the literary work as a more or less arbitrary assemblage
of ‘devices’, and only later came to see these devices as interrelated
elements or ‘functions’ within a total textual system” (Eagleton, 1996, p. 3).
These devices, discrete, particular formal components were the target of
analysis because it was such elements (sound, imagery, rhythm, syntax,
metre, rhyme, narrative techniques etc.) that did the work, turning ordinary
language into literary language with all its effects.149 “’[L]iterariness’ was a
function of the differential relations between one sort of discourse and
another; it was not an eternally given property” (Eagleton, 1996, p. 5
emphasis in original). It was this argument that powered the development of
structuralism’s focus on systems where, by looking at how the system was
put together, one could address its workings and power relations.
147. Williams famously critiqued Marshall McLuhan along these lines. In Television:
Technology and Cultural Form, he said: “The work of McLuhan was a particular
culmination of an aesthetic theory which became, negatively, a social theory: a
development and elaboration of formalism which can be seen in many fields, from
literary criticism and linguistics to psychology and anthropology, but which acquired its
most significant popular influence in an isolating theory of ‘the media’” (2003, p. 129). Of
course Harman would be happy to be associated with McLuhanism. He says: “[N]o one
in the twentieth century, not even Heidegger, does as much as the McLuhans to retrieve
the metaphysics of objects as a viable medium” (2009a, p. 122),
148. Let’s leave aside Harman’s claims that OOP can be traced back throughout the
history of Western and Eastern philosophy.
149. For an account of narrative in new media see Murray (1998) and Bassett (2008).
105
JPEG: the quadruple object
There are clear parallels with OOP. The focus on “devices” and specific
components mirrors OOP’s single-minded commitment to objects. Here
ecosystems, computer games, indeed the whole universe is made up of
objects connecting between or within objects. What we perceive as systems,
meshes or assemblages are really just components clashing, connecting or
relating. Just as language is not an eternal given property neither is the mesh,
or, in my case the governmental scopic regime. In both frameworks the gap
between objects is important. For Harman it is the sensual-real difference
and the way the fourfold allows differential connection (what he calls
“vicarious causation” (2009b; 2009c, pp. 146-147)) that characterises the
mesh of objects. Sensual can only connect with real, real only with sensual.
Objects withdraw from us and from each other.150 That is what drives the
mesh, creates new objects and new relations. It is the making strange of
language in literature, the gap between everyday discourse and that of the
novel, the withdrawal that creates art and culture. And by focussing on the
technical devices, one can see that, unpick it, critique it and create it.
150. This parallel is perhaps unsurprising bearing in mind Husserl’s influence on the
Russian Formalists (Eagleton, 1996, p. 51).
106
JPEG: the quadruple object
151. I would of course say that while I dismiss the idea of a “referent” external to the
object, I hold to the fact that those objects have real dimensions and qualities.
107
JPEG: the quadruple object
I object materialism
While I find an OOP account of JPEG a persuasive and stimulating one in
terms of its capacity to remain actual, specific and particular while still
mapping the complex computational and governmental networks within
which JPEG operates, it is possible to see OOP as having a mystical
dimension that undermines a materialist critique. It is important to re-
emphasise that when I say “materialist” I am not referring to the
undermining philosophy that Harman criticises, one that looks for something
more basic and fundamental than objects, but rather a sensibility within an
ontology to the world of material things.152 My project and the distributed
imag(in)ing regime and mesh that it seeks to explore is deeply material. From
the digital rubbish that Jennifer Gabrys (2011) and Ned Rossiter (2009;
2011) discuss, through the carbon footprint of cloud computing (Cubitt,
Hassan & Volkmer, 2011) to the digital detritus stockpiled, as we shall see, in
Facebook’s Haystack, there is nothing immaterial about the digital mesh.153
While OOP provides a provocative way of mapping the nature of the objects
in play, the way they are accessible and inaccessible and the ways they
connect, it arguably underplays a very real materiality at play in both the
sensual dimension we encounter and the real dimension forever withdrawn.
152. Bryant refers to his own work as “object-oriented materialism”, saying: “The
materialist need only claim that all entities are materially embodied, not that all entities
are reducible to elementary parts” (2012c emphasis in original) See also n. 107 above.
His debate with Harman is less to do with rebalancing OOP/OOO to focus on material
objects than it is to do with their broader debate about potentiality (see above). As I shall
discuss, Janee Bennett is aware of the issues around “materialism” and undermining
that Harman identifies. When questioned on her relation to other forms of materialism in
her interview with Gulshan Kahn she says: “Mechanistic materialism does not attract me;
it implicitly returns us to the status of consummate agents who run the machine” (Khan,
2009, n.p.)
153. This of course does not even touch on the materiality of scopic and computational
apparatuses discussed by Crary, Jay and those working in media archaeology (see
above). For discussion of materiality and the digital image, see Sassoon (2004) and on
materiality and technology, see Küchler (2008). I come on to discuss the seemingly
immaterial software objects as in some sense “material” below.
108
JPEG: the quadruple object
154. A loose collection of authors who share a concern for expanding the conception of
the material to explore issues of agency (see the papers collected in Coole & Frost
(2010b) and Miller (2005a) as well as Parikka (2011b). For an overview, see Packer &
Wiley (2012) and for a feminist perspective, see Van der Tuin & Dolphijn (2010). Diana
Coole and Samantha Frost even make a point of talking of new materialisms in the plural
(2010a). See also the work of Karen Barad who says: “Language matters. Discourse
matters. Culture matters. There is an important sense in which the only thing that doesn’t
seem to matter anymore is matter”. (2007, p. 132). For a Latourian approach to
materialism, see Bencherki (2012). For one drawing, as does Harman, on Alfonso Lingis,
see Introna (2009). It is not my intention to explore the different ontological positions, let
alone the specific case studies, that have emerged under the label “new materialist” but
rather to explore the connections between a particular politics-based articulation of that
concern. It is important to note that as with speculative realism, there is a concern
among its proponents that “new materialism” does not simply substitute one orthodoxy
or hierarchy with another. As Daniel Miller says: “Having dethroned the emperor’s
culture, society, and representation, there is no virtue in enthroning objects and
materialism in their place. The goal of this revolution is to promote equality, a dialectical
republic in which persons and things exist in mutual self-construction and respect for
their mutual origin and mutual dependency” (2005b, p. 34). This concern for moving
beyond representation can also be seen in the work of Nigel Thrift (2008), Brain
Massumi (2002) and Rosi Braidotti (2002) among others. There are of course other
strands to “new materialism” particularly as it appliers to media. Parikka (2012) for
instance traces the links to “German Media Theory” (See also Siegert (2007)). My
decision to focus on Bennett is because of her ontological project to redraw our
conception of the object as material. While many have taken a materialist look at media,
there are few who have take a materialist and object-oriented approach to ontology.
155. Needless to say, some would query whether one can address “electron flow” and
“neoliberal faith” as objects. As I have argued, following Bogost and Harman, one can.
Furthermore the question should perhaps be not whether we can but what happens
when we do.
156. Miller is also fond of the Latour Litany: “We start with the need for a theory of stuff
as material culture [...] that can account for every kind of stuff: bodies, streaming videos,
a dream, a city, a sensation, a derivative, an ideology, a landscape, a decay, a
philosophy” (2010, p. 54). More broadly on approaches to “material culture”, see Hicks
& Beaudry (2010)
109
JPEG: the quadruple object
both, objects are the focus. Like Harman she rejects the idea of objects as
signs and demands an account of objects as more than the human object
correlation. But Bennett adds a particular concern for the material that is
underplayed in Harman’s more philosophical account.
Bennett echoes Daniel Miller’s argument that semiotics can be “as much a
limitation as an asset” (Miller, 2010, p. 12) when looking at “the minutiae of
the intimate” (ibid p. 41), the “stuff” or things people have, use and (in
object-oriented terms) connect with (Miller, 2008).157 The objects in her
gutter are not some instantiation of an industrial process or structure. Of
course the glove was made in particular social and economic system under
particular modes of production. Its story can be read as one of globalisation
and capitalism. It can be read as the trace or representation of those
historical processes. But Bennett argues that the discourse of representation,
of tracing the power and meaning of things as signs, falls short of what is
needed. She says:
For Bennett, like Harman, there is also more to objects than the human-
object correlation. Objects are material. But that materiality is lively and
157. A similar nuanced account of things can be seen in Sherry Turkle’s account of
“evocative objects” (2007) discussed by Harman (2008b). A concern for things could
also be traced back through Arjun Appadurai’s exploration of how things “move in and
out of the commodity state”(1986, p. 13) and Pierre Bourdieu’s account of the role of
everyday things in socialization (2008). Similarly Alfred Gell sought to move beyond
semiotics in account of how artworks “appear as, or ‘do duty as’, persons” (1998, p. 9).
110
JPEG: the quadruple object
active.158 Bennett’s objects are real and located. They are presences in the
world but they “call to us” and have a form of agency, “agentic capacity”,159
a “thing-power” that animates the seemingly inert. Bennett draws on a
history of vitalism (Fraser, Kember & Lury, 2005; Greco, 2005) in particular
the work of Hans Driesch, a early twentieth century vitalist.160 Driesch
developed the concept of “entelechy” as a similar animator to explain what
he saw as the question of the enfolded character of nature.161 Bennett says:
“Entelechy is born in the negative spaces of the machine model of nature, in
the ‘gaps’ in the ‘chain of strictly physico-chemical or mechanical
events’” (2010a, p. 70). She is keen to stress that Driesch does not see this
animating force in terms of a soul or even simply a “vital energy”. Rather it is
located within materiality and its possibilities. Where Bennett moves beyond
Driesch is in refusing to see matter as “so passive and dull that it could not
possibly have done the tricky work of organizing and maintaining morphing
wholes. [For Driesch] sometimes this matter is infused with entelechy and
becomes life, and sometimes it is not and coagulates into inorganic
158. Shaviro draws attention to a similar idea of liveliness in his discussion of Gwyneth
Jones’ novel The Universe of Things (2010). He says: “[I]f we are to accept the
ontological dignity of things, and do not reduce them to being just the illusory effects of
quantum fields, then I think that we need to accept some sort of non-dualistic neo-
vitalism, or what Jane Bennett calls vital materialism: the idea that ‘every thing is
entelechial, life-ly, vitalistic’ (Bennett, 2010a, p. 89)” (2010, p. 15 emphasis in original).
159. See also the work of Barad around materialism (2003) and “agential
realism” (2007).
160. Bennett discusses Driesch and later Henri Bergson in terms of Kant’s insistence on
the “unbridgeable chasm between life and ‘crude matter’” in the Critique of Judgement
(Kant, 1987, sec. 81, #424) and his invoking of a “formative drive” or Bildungstrieb
(2010a, p. 65). She argues that Driesch and Bergson extend this idea by allowing the
agentic force to be present in matter not just in organisms. This point is echoed by
Morton who argues that we should “abandon all variation of Romantic vitalism - that is,
believing in a vital spark separate from the material organization of life forms” (2010a, p.
68).
161. Driesch, whose work began with scientific experimentation, can perhaps be seen
as engaging in practice-research.
111
JPEG: the quadruple object
machine” (ibid p. 75). For Bennett, matter is always “vital”162 and that vitality
lies outside the human-object correlate.
One might criticise Bennett for mystical language (a similar charge could
be addressed towards Harman’s talk of sensual and withdrawn dimensions).
Peter Gratton has said: “Bennett’s position would seem to leave us bereft of
any politics worthy of the name, and the reader may worry Bennett has
brought us either to the edge of some pan-psychic New Age philosophy, or
worse, to a nihilism that renders meaningless all human actions and
common praxis” (2010b, p. 159).163 But beneath Bennett’s language is
exactly the sort of reality and material concern that a purely philosophical
account lacks. While both Harman and Bennett would talk of trees, gloves,
servers, CCDs and JPEG, Bennett is more likely to argue that we need to take
particular account of their concrete as well as ontological existence.
Bennett’s stress on how the quadruple object erupts into the world, how its
sensual and real dimensions and qualities connect in material as well as
ontological terms, challenges us to look at the CCD object as metaphysically
fourfold but also material. What is more, Bennett’s willingness to see vibrant
materiality across the object spectrum means we can address objects like
JPEG which, as we have seen, while seemingly abstract, virtual and unreal,
are clearly actants and objects in material conditions of connection with
software, the State, my camera and me as photographer. Harman is clearly a
realist, Bennett adds a form of object-oriented materialism to that account of
the real.
162. Bennett also draws on Hernri Bergson’s “élan vital”, a force, an “inner directing
principle” (Bergson, 1998, p. 76) that underlies his idea of how life and matter are not
fixed categories but tendencies of a cosmic flow. There is perhaps an interesting parallel
with the idea of active and reactive forces discussed by Deleuze in his book on
Nietzsche (1986, pp. 39-71).
112
JPEG: the quadruple object
164. In her turn, Bennett argues that Harman too readily associates “lump ontology” with
Deleuze (Gratton, 2010a). As I discuss, others within object-oriented philosophy, notably
Bryant, are, like Bennett, willing to read Deleuze as concerned with objects that are not
exhausted by their relations, as a philosopher of objects not just fields. Bryant who
unpicks what he sees as Deleuze’s “schizophrenia: between monism and
pluralism” (2011a, pp. 94-104), even goes as far as to list Deleuze as one of the “heroes
of onticology” alongside Harman (2011a, p. 27). Bryant argues that “Deleuze’s concept
of the virtual provides us with the means of thinking substance as structured without
being qualitative” (2011a, p. 31). For Deleuze as for Harman, substance is a proper
subject for philosophy and, what is more, again like Harman, there is more to an object
than its qualities.
For Harman however - as in his debate with Bryant - this is not enough. One needs to
reject potentiality in order to remain at the scale of the object. He says: “Contra Deleuze
we must champion individual, actual things as the protagonists of philosophy” (2011i, p.
292).
113
JPEG: the quadruple object
In 1976 New York Times critic Hilton Kramer and MoMA curator John
Szarkowski famously agreed that William Eggleston’s style was “perfect”. For
the curator, Eggleston’s saturated colour was a “snapshot aesthetic” taken to
an extreme, perfectly attuned to a saturated imagespace and postmodern
sensibility. To the critic, the images were indeed perfect: “[P]erfectly banal,
perhaps. Perfectly boring, certainly” (Kramer, 1976). The important point
about their reading of the show was that point of agreement, “perfectly
banal”. Eggleston embraces the banal by working with and through objects.
His modestly entitled Guide (2002) is no catalogue to the exhibition,
monograph of an oeuvre or photobook. More like a child’s I Spy book or a
throwaway pamphlet sold with an admission ticket, the Guide makes no
pretence to be anything other than a tour of objects in Eggleston’s South.
165. The loaded terms keeps cropping up in the work of Eggleston. One of his
collections was called The Democratic Forest, (1989) another (and the title of his
retrospective at the Whitney Museum in 2008-9 was The Democratic Camera (Sussman,
2008).
114
JPEG: the quadruple object
His refusal to title or date his images is more than just a Zen refusal of
labels - fingers pointing at the moon. It is a sensibility towards the “this”, a
willingness towards objects, a positive statement about their withdrawal yet
very real accessibility as sensual presences, understood in terms of objects
not fields of relations or time. When he rounds a corner and something
catches his eye he does not dance around looking for the perfect position to
take the image or shoot different frames or compositions. He raises his
camera and... He says:
115
JPEG: the quadruple object
116
JPEG: the quadruple object
166. This report focuses on my photographic practice but I have also explored what
Kenneth Goldsmith has called “uncreative writing” (2011) including Tweeting and
Facebook posting James Joyce’s Finnegan’s Wake backwards, 140 characters at a time
allowing it to interweave or entangle with other status and link objects. A second project
involved feeding algorithmically generated advertising copy as status-objects back into
Facebook’s Wall and so the Open Graph where it would form the source material for new
algorithmically generated ad-objects.
167. I make no claim for the originality of OOPh. Bogost has argued that Gary
Winogrand can be read as object-oriented: “Garry Winogrand made photographs of the
things themselves. Lots of them [...] His works are not commentaries, they are precisely
the opposite. Garry Winogrand makes photographs not to capture what he sees, but to
see what he will have captured. That’s what it means to take photographs to see what
the world looks like in photographs [...] It’s too hard for most viewers to take Winogrand’s
project seriously, because they’re too busy looking for social commentary in his
photographs to see them for what they are: pictures that help their viewers see things in
pictures. The object-oriented ontology project is just as simple, yet still just as hard: to
see things in pictures and everywhere else too. To see the world of things as things in a
world, rather than our world, with things in it” (2011, n.p.). My photo interludes in this
work are a similar project.
117
JPEG: the quadruple object
with the vibrant materiality of the objects I photograph, photograph with and
through.169
There are two strands to my OOPh practice. The first (upon which the
project I discuss here - an exploration of using Harman’s quadruple
framework as analytical and creative method - is built), is about
photographing objects in “meshes” not in the sense of objects in context or
with other objects as a background but in terms of how what are sometimes
seen as a hierarchy of objects are best seen, and photographed as flat.
169. OOPh as a practice concerned with materiality and actuality can be seen as
engaging with Morton’s broader conception of ecology. As he says: “Art is ecological
insofar as it is made from materials and exists in the world” (2010a, p. 11).
118
JPEG: the quadruple object
Olympic brands, security and “legacy”. But the ring is not just a sign of
something more. Nor is it just the trace of historical and political economic
processes or human impact on “the environment”. There is no background
or context here. The ring does things in the world at an ideological scale but
also at a material scale. It rusts or leaks chemicals into the soil (alongside the
toxins released by 2012 excavations (Wells, 2009; Chapman & Wells,
2010)). But when the ring-object is addressed in a flat ontology with the
grass, the long-forgotten ironworks, the canalboat, its moorings licence,
British Waterways and its internal memos proposing changes to the rules for
the Olympics (Griffiths, 2011) - that network of human and unhuman
actants is real, present, actual, power-full and governmental.171
The second OOPh practice is based around remixing objects 172. Here I
use my mobile phone screen as an imaging apparatus remixing data objects
with the “view” or image on the screen. I use an augmented reality app that
overlays data from web searches and databases across a camera view as an
imag(in)ing apparatus, screengrabbing the view as an image (encoded by the
iPhone using the PNG standard, but re-encoded through JPEG when
171. The key distinction here between OOP and classical ANT is that for Harman these
“separate” objects exceed their relations. They have an actuality beyond the network.
Their power is not dependent on the network or their relations. Morton also draws
attention to the importance of actuality within object-oriented creative practice when he
draws on Giorgio Agamben’s phrase (2007, p. 49) to position “ecological art” as an art
of “whateverness” (2010a, p. 105).
119
JPEG: the quadruple object
173. While this practice features in my mash-up apparatus, in terms of this project and
its research aim of addressing JPEG, I concentrate primarily on my use of a (pinhole)
camera to engage with the first form of OOPh. I chose to do this because, as I discuss, I
was able to narrow down the range of object-actants in play. The latter practice was
premised on multiplying the number of actant-objects at work.
174. This allows it to extend into the field of machinic vision as I explore in terms of my
mash-up apparatus’ imaging whether a human is involved or not.
120
JPEG: the quadruple object
121
JPEG: the quadruple object
have an agentic capacity as they connect and reconnect. They are not
passive tools of a photographic overlord but vital players in an imaging
encounter. The human photographer is one object but so are other hardware
and software objects... including JPEG.
JPEG imaging
The particular practice I discuss here emerges from that broader OOPh
practice. It focuses on using OOPh to explore one of the objects in play,
JPEG. The practices or works I discuss were designed to explore JPEG and
also OOP as a method for investigating digital objects. I crafted the works
(picking up on the language Bogost uses for his own practice-research, see
below) to allow me to understand JPEG’s existence and working (remember
that for OOP, these are different), to expose and explore its quadruple form.
In order to do this, I took photographs with a digital pinhole camera and a
network, mash-up apparatus. I processed the data in a digital darkroom and
produced a series of Photo eBooks.
177. An Olympus E420 DSLR with the lens removed and a home-made pinhole body
cap.
122
JPEG: the quadruple object
through to eBay listings for pinhole surveillance cameras, the simple action
of light and aperture has never been outside power. Even when particular
technologies, devices and brands are stripped away, leaving just a light and a
hole, imaging is still scopic and therefore powerful. To imag(in)e with
pinhole is to remain part of that historical enfolding, to foreground the
connection to optics, optical toys and scopic problematics.
178. As noted above (n. 5) my digital camera used an electronic, rather than a
mechanical shutter.
123
JPEG: the quadruple object
179. Anne Wilkes-Tucker reports Mann as saying of digital: “There is nothing whimsical
about digital. No ‘gifts’ occur. It’s too predictable” (2009, p. 175). I would respectfully
disagree. I looked to be open to those gifts by embracing the accident through
handholding the camera as well as the broader vagaries of dust on the sensor. A
commercial vendor of pinholes for DSLRs says: “If you have a Digital Single Lens Reflex
camera, you should use a ‘No Dust’ pinhole or zone plate body cap. ‘No Dust’ means
the pinhole or zone plate is surrounded by a very opaque black area on film; the pinhole
or zone plate area is clear on the film. No dust can enter the camera -- dust on the
sensors can be a problem with digital photography” (pinholeresource, 2011). As well as
the desire to put nothing between the light and the sensor, it was that “problem” I looked
to embrace.
180. For another photographer who sees the pinhole aesthetic as offering a way into
exploring memory, see Ess (2001).
181. Such a concern is apparent in recent work in sound and visual art around “the
glitch” (Menkman, 2011a; 2011b). See also the work of Phillip Stearns (2012) and
Amerika’s Museum of Glitch Aesthetics (http://glitchmuseum.com/).
182. I use the term apparatus to refer not just to the material device but the scopic mesh
or actant-network. See my discussion of the work of Crary. The “apparatus” is more than
the camera. It is the scopic mesh of human and unhuman, hardware and software
actants.
124
JPEG: the quadruple object
183. I use the term “unvisible” rather than “invisible” to draw attention to the fact that it
was not that they could not be seen but rather that they were seen differently, outside the
mesh of social imaging connections. RAW-encoded files appear as a broken icon or as
a failed action. For discussion of the “invisible”, see Birchall (2011).
184. A second version of the apparatus used a similar arrangement of JavaScript, HTML
and API objects to pull my JPEG-encoded project images from Flickr and load them
alongside other JPEG-encoded “2012 images” in a Webpage (and the browser’s cache).
The user could then swap the images and create different juxtapositions, or dialectic
images as Benjamin might call them.
125
JPEG: the quadruple object
necessary through JPEG ready to be found by the search engines and ready
to appear in the screengrabs.185
185. This use of screengrabs as a form of imaging developed from earlier experiments
with screengrabs and searches. While this apparatus raises issues of copyright,
caching, search data trails and privacy, specifically around protocol, these are not the
concerns of this project. Rather I look to use it to explore the quadruple structure of the
JPEG object and its workings.
126
JPEG: the quadruple object
through JPEG, RAW and WebP187 standards); the interlude pages encoded
through JPEG, DNG and WebP188 as images/image files; and the “live”
mash-ups. The “books” I created were different in terms of their affordances
as they were read and potentially read/written on different devices and in
different spaces. What is important in terms of my practice-research was that
they connected or failed to connect with the protocol-objects in different
ways. Those images or live mash-up images encoded through JPEG were
visible, and usable, even shareable and embeddable - inter-operative and
flexible across all the eBooks.189 JPEG had connected and enabled object
connections. Those encoded through RAW or WebP were not.190 The formats
could not connect with these images, rendering gaps in the Book, fails,
broken icons etc. The reader, her devices, social spaces etc could not
connect.
188. While I could have encoded the interlude pages with the Olympus version of the
RAW standard I used in my camera by photographing each layout and saving as a
RAW-encoded .orf file, I chose to use a different RAW standard to save the text images
as DNG files. See n. 23 above.
189. As I discuss in the Appendix, the JavaScript necessary for the mash-ups to work/
connect meant that Kindle eBook could not access the JPEG-encoded images pulled in
through the mash-ups.
190. As I discuss in the next chapter, Facebook attempts to re-encode image files
uploaded to its service with JPEG. It can (or chooses to) do this with certain format files
but not others and of course it cannot do it with “live” images such as my mash-ups.
127
JPEG: the quadruple object
191. Of course the Web is arguably not open in terms of issues of access, digital divides
and the necessity of proprietary technologies to access even open and creative-
commons content. See also Jonathan Zittrain’s discussion of “appliancized proprietary
networks” (2008, p. 25) and Eli Paris’ work on the “filter bubble” (2011)
192. More generally on eBooks, see Eraso, Ludovico & Krekovic (2006) and Ludovico &
Muller (2007; 2008).
128
JPEG: the quadruple object
Why practice?
Before exploring how I used this practice as a basis for exploring JPEG
and governmentality and what I found about the nature of the quadruple
JPEG object it is important to address why I approached my questions about
the nature and workings of JPEG through practice. I chose not to approach
194. I am conscious of the loaded nature of this term. My decision to use it here is not to
engage with debates about the nature of the political or even the politics of software or
publishing but merely to draw attention to my desire to acknowledge and work with and
through the real-world powerful nature of eBook platforms.
195. When Apple released their new iBooks format and iBooks Author software, there
was considerable controversy over the End User License Agreement which, in its first
iteration, said that any eBook created by the software regardless of final format could
only be sold through Apple’s iStore (Foresman 2012).
129
JPEG: the quadruple object
196. In the guise of a long-suffering PhD supervisor Bruno Latour tells his rhetorical
student: “If I were you, I would abstain from frameworks altogether. Just describe the
state of affairs at hand” (2004, p. 64).
197. For such an account of imaging and Flickr, see Cox (2008).
130
JPEG: the quadruple object
It was only through the practice of using JPEG that its nature and workings
could become “visible”. Even exploring other photographers’ work would
not have given me access to all the objects in play within the “cameras” and
processes and practices of photography. It would not have allowed me to
assess to what extent Real-Sensual poles and relations of fusion and fission
are really at the heart (sic) of JPEG and JPEG imaging. By making my own
practice my laboratory, I had access to the full panoply of objects (including
myself as imager) as well as the full sets of connections.
198. I am conscious of the problem of using the term “purely” as if such an abstract,
neutral, object thing was possible. I use the term here to imagine a position where the
research question were addressed using theory alone.
131
JPEG: the quadruple object
complex, and dynamic all at the same time” (ibid p. 113 fig. 4.5).199 For
Sullivan, practice-research is best understood using both visual and
conceptual metaphors. It is a dynamic jigsaw and the sort of “complex
adaptive system” scientists use to talk of non-linear dynamics in natural
systems and social scientists and cultural historians have used to both
picture and also account for processes in social systems.200 For Hazel Smith
and Roger Dean the metaphor is one of a circle and a web (2009). Where
Sullivan pictures a folding/unfolding movement, here the image is one of an
“iterative cyclic web” with a “research phase” and a “practice phase”
connected, repeated and ratcheting each other up as a project moves
forward.201
Bogost too uses a metaphor in his call for new ways of doing, what is for
him object-oriented, research. He discusses the “practice of constructing
199. Sullivan can be seen as located within the same discourse as Victor Burgin (2006)
and Desmond Bell (2008) in seeing practice-research as a political struggle for arts’
legitimacy. For Burgin practice-research emerged within an institutional and historical
frame. Sullivan starts with an historical account of how art has always created new
knowledge (ibid pp. 3-31) and when he gets onto the issue of contemporary discussions
of practice-based and practice-led research, this is located in terms of responses to the
OECD’s Frascati Manual an internationally recognised guide for standards in research
and development used to help develop policies and practices which includes a
framework for defining research activity (ibid p. 74). He continues this contextualisation
by discussing the “academic art world” (ibid pp. 79-82) before presenting his own
model. That model emerges from an account of practice-research as a political-
economic and historical form located in particular material and professional relations.
200. Within complexity theory complex adaptive systems are poised on the edge of non-
linear chaos. The arguments is that the whole is more than the sum of its components. At
large and small scales, systems have characteristics that are the same. They are “scale
free”, “self similar” or fractal. Within complex systems, small units, or “actants” (whether
atoms, neurones, ants, populations, share dealings, bits within a computer etc) interact
in complex ways and generate particular states. These states or “attractors” are poised
on the “edge of chaos”. The system settles but only temporarily around an attractor only
to be moved on to another (not necessarily higher) level of organisation. “Attractors” act
to stabilise these systems at particular moments which are never fixed but always in
process. For introductions see Holland (1995); Urry (2005) and Johnson (2009). For its
use in cultural analysis, see for instance De Landa’s attempt to write a non-linear history
(2000) where social structures (whether material or non-material, human or not) emerge
from complex historical processes that cannot be traced to a founding essence or
dynamic. Rather De Landa argues, ideas of social causality must include an
understanding of the sort of feedback mechanisms that scientists find at work in chaotic
and complex adaptive systems.
201. Smith and Dean position their metaphor in relation to Deleuze and Guattari’s
rhizome (2004) “in which any point can be linked to any other and there are ‘multiple
entryways and exits’” (2009, p. 21).
132
JPEG: the quadruple object
Mark Amerika too “practices” philosophy. His “remix the book” project
(2011b) consists of a book but also an “open content platform”, a space for
“digital remixes of many of the theories generated in the print book [it]
features the work of artists, creative writers and scholars for whom the
practice and theory of remix art is central to their research interests” (2011c,
n.p.). Again these remixes are not, for Amerika, illustrations or even
responses to his philosophy. They are not even distinct philosophical
statements themselves. As objects his book (itself a collection of fragment-
objects) and the remixes connect and reconnect within new objects. He
talks of “artwork as a spontaneous and continuous theory-to-be” (op cit.
emphasis in original).
202. Bogost acknowledges his debt to Harman’s “the carpentry of things” (2005) an idea
Harman borrowed in turn from Alphonso Lingis (1998).
133
JPEG: the quadruple object
203. In a sense it is also live coding (Brown, 2007) insofar as the mash-up is continually
remade as new screengrabs are pulled in.
204. At this stage of my research I had yet to encounter Harman’s work which, as I have
discussed stimulated a particular theoretical and practical approach.
206. I narrowed the focus even further, as far as my own imaging went, by working in a
particular liminal space, the Greenway, a footpath and cycleway on the embankment
containing the Northern Outfall Sewer and being remade as a pathway to the Olympic
site as well as part of the Crossrail developments.
134
JPEG: the quadruple object
207. For broader discussions of surveillance and the militarisation of urban spaces, see
Graham (2011). In terms of the militarisation of online space, see Deibert (2003). For a
different account of the machinic and the urban see Amin & Thrift (2002). Specifically
around the digital and the urban, see Featherstone (1998) on mutable spaces and
flânerie; Atkinson & Willis (2007) on overlapping realities; de Souza e Silva (2007) on
mobile hybrid spaces; McQuire (2008) on architecture and contributions to Sutko & de
Souza e Silva (2009) on urban location and gaming. For accounts of urban development
around 2012, see Gold & Gold (2007) and Pointer (2009). For photographic explorations
of urban space, see Burgin (1996); Atkins and Sinclair (1999) and the work of Stephen
Gill (2004; 2007a; 2007b). For an historical account of visualising urban space and time
see Clarke & Doel (2007). In terms of the gaze and modern urban imaginaries, see
Jansson & Lagerkvist (2009). For a history of the rhetorics around surveillance, see
Levin, Frohne & Weibel (2002).
208. As I discuss (Caplan, 2010), such metadata can of course be “fooled”, manipulated
or otherwise Exploit-ed. As noted above (n. 126) my concern in this project was not with
the geolocative metadata capabilities and implications of JPEG. Their use here was
simply to define searches.
135
JPEG: the quadruple object
136
JPEG: the quadruple object
The report
JPEG as an object
My practice-research indicates that JPEG can be seen as an object.209
JPEG has a unity. This can be seen in its technical structure (as outlined in
the Digital Imaging Pipeline chapter), in the documents and RFCs online but
also in my practice. As I built my mash-up or eBook I could “call on” a
unified JPEG to encode screengrabs and render them visible. Regardless of
whether that encoding issued as a JFIF/EXIF image file, was accessed by me
or other software, there was unified JPEG-object-actant in play. I will come
on to address the relations within which I found it at work, but my practice
indicated that a unified JPEG had an existence outside those relations.
Regardless of whether JPEG is ever called (within Photoshop, my camera or
my mash-up) it has a presence and existence. If it did not, I could not have
built my apparatuses. Following Harman’s Latourian stress on objects as
actants, doing things in the world, again JPEG can be seen as an object.
JPEG does things. It encodes light as data, data as accessible data. What is
more, as I come on to discuss in the next chapter, JPEG does governmental
work. It enables image data to be viewable, shareable, linkable and
efficiently archived as part of Facebook’s Open Graph. My attempt to use the
RAW standard to do the same work indicates that standard-protocols such as
JPEG are active creators of Open Graph-friendly data and data points. One
could of course argue that JPEG is not the only object-actant doing that
governmental work. Other protocols, hardware and software objects are also
in play and necessary for the digital imaging pipeline to work. All my
practice shows is that JPEG is a player in that mesh and is unified and
exceeds any relations.
209. This does not of course exhaust the ways in in which JPEG can be understood (or
possibly imag(in)ed). My project is not claiming that objects is the only way of
understanding protocols and standards, merely that it is an under-developed way of
seeing such digital objects.
137
JPEG: the quadruple object
actants in the mesh. As I planned and built the various apparatuses, I had to
work with specific accessible instantiations and adumbrations, different
profiles.210 In order that the mash-up could sample the distributed
imaginings, my own photography could place the RAW and JPEG objects
next to each other and collide my eBook formats, I needed to access
dimensions of the JPEG standard (not just the JPEG/JFIF images) present to
me, my camera, my browser. Specifically the plans I made and the devices I
constructed were premised on access. As those plans were realised and the
apparatuses worked, that sensual dimension became apparent. These
encounters did not access the totality of JPEG, merely an instantiation,
particular configurations of the transforms, codings and tables that form
JPEG. I experienced these sensual dimensions as there as surely as I knew
that the totality of JPEG lay beyond my and my other actants’ access.
210. Interestingly, this term is used in imaging software and digital photography to refer
to different colour spaces configured for different screens and devices.
211. Another interesting legal-governmental phrase highlighting the hidden but powerful
operations in play.
138
JPEG: the quadruple object
If there was more to JPEG than the sensual accidents and profile that I and
other objects encountered, then it must have a presence and existence when
I (or any other object) stopped paying attention to it. My practices shows that
the unitary JPEG had that existence.213 This was most apparent in my mash-
up apparatus. The service I used to screengrab image searches and encode
them as image files to be sent back to my installation existed regardless of
whether I or anyone or anything else called it to run. The JPEG object
remained enfolded in that software service regardless of whether my
JavaScript called it or not. Similarly the JPEG decoding object had a
presence within the browser and eBook reading software regardless of
whether it was in use. This was not some potentiality. It was not waiting to
be realised, somehow non-present or unreal. It was actually built into
software and systems but accessible only as instantiations or profiles.
Similarly Save As in Photoshop or the in-camera software before or after I
pressed the button had the JPEG on hand (present at hand). Whether I or any
other object boot up those systems and access the sensual JPEG is a separate
issue. My apparatuses and my practices depend on something real, if forever
withdrawn. What became particularly apparent as I imaged was that JPEG
not only withdrew from me but from other objects in my apparatuses. When
I pressed the shutter button and set the digital imaging pipeline in play or
139
JPEG: the quadruple object
loaded my webpage I did not reach the totality of JPEG, but neither did any
of the other objects. The silicon chip in the camera or the hard drive on
which Photoshop ran, the JavaScript call or the Google algorithm accessed a
dimension of JPEG - all that was needed to do their work. They had to
encounter an instantiation in order to render search results. They had to
encounter a profile in order to be able to render a file.
JPEG’s qualities
My practice-research also indicated that JPEG was in tension with real and
sensual qualities. As I worked as programmer, imager or reader, JPEG
withdrew but it did not dissolve into a indistinguishable mass. It had real
qualities. It was different from other real objects and protocols. It used
Huffman Coding, DCT etc. These algorithms and mathematical formulae,
whole inaccessible (except through intellectual work) were there in my
work. If they were not I could not have developed a RAW/JPEG parallel
imaging apparatus, there would have been no distinction to highlight.
140
JPEG: the quadruple object
214. For a different account of time and software, explored through artworks see Raley
(2008). On distraction and experience of time, see Cubitt (2007) and more broadly on
time and new media, see Lee & Liebenau (2000) and Leong, Mitew, Celletti & Pearson
(2009). For an exploration of time compression and visuality, through the work of Virilio,
see Bartram (2004) and more generally on mobility and time and space, see Green
(2002).
215. The timestamp-object is not fixed or necessarily “true”. That EXIF metadata can be
edited. The time of the image or the geolocation can be changed. Of course forensic
methods can be used to show that such changes have been made but that does not
change the fact that the time-object is not related to some fixed temporality.
141
JPEG: the quadruple object
When a reader opens my eBook and turns 219 to a page with a live feed,
she experiences a window into multiple spaces (just as if I had embedded a
216. This is not to suggest that this is the only object tension in play or setting
experiences of time in motion. Other imaging objects were similarly split and so enfolded
in those experiences.
217. This is not to institute a correlationist agenda, privileging human relation to time.
Object-oriented approaches do not deny the human and her experience (in relation to
time), they simply refuse to privilege it.
218. For a different account of space and software, see Kitchin & Dodge (2011) who
identify new flow spaces enfolded with and in many ways dependent on, information and
software. For another account of flow and new media, see Hepp, Krotz, Moores & Winter
(2008). For work around locative media and visualisation of space, see Tuters & Varnelis
(2006). See also Zook & Graham (2007) on the relationship between code and place;
Stromer-Galley & Martey (2009) on the relationship between online space and offline
norms; McGarrigle (2010) on the influence of situationism in locative media and art, and
Lapenta (2011) on locative and augmented reality “geomedia” and collective image
production as commodified objects of exchange. As noted above in terms of time, such
discussions are, from an object-oriented perspective, built around a correlationist
agenda, discussing as they do our (human) reframing of space in and through code.
219. An interesting term in terms of my iPad and Kindle’s different haptic interface
(whether serving eBooks or Evernote/Facebook app-books) in contrast to those eBooks
on the Web.
142
JPEG: the quadruple object
live CCTV camera or webcam, but with the added complexities of the times
discussed above).220 In terms of the mobile eBooks, these spaces are further
overlaid (or underlaid or inlaid) by the spaces of reading - in the Olympic
stadium, in France, in the office of my examiner etc. Like a form of
augmented reality, multiple reading and writing spaces appear. The
experience of space set in motion by my apparatuses is not simple, or stable.
JPEG plays a crucial role in that spatial experience, rendering window views
visible, rather than the broken unvisible, perhaps opaque view of a RAW-
encoded window. The same experience of multiple spaces occurs when I
scroll back through the images on my camera screen.221 It is not just
different times I encounter but also different spaces, not just where the
image was taken but, when viewing on my phone’s Facebook app, the server
space in the US or wherever the Haystack server is, in whoever’s jurisdiction
and tax regime it sits.
These complex spatial dynamics and spaces are not just a matter of
different geolocations where images were taken but also where that data is
held, cached and read, where it is encoded and decoded potentially in
multiple spaces simultaneously. These dynamics, for Harman, are the result
of the tension between a withdrawn standard (RO) which is some sense
locationless (”in” the US, UK, the Web, the standards body) and those
profiles which I and other objects encounter as pages are turned, image-data
accessed/decoded, screengrabs encoded, Timeline’s rendered. The tension
between the withdrawn JPEG (RO) - that object that always exceeds our
access and its relations, the unfathomable object always just out of reach in
my apparatuses but ensuring that they work - and the (SQ) the particular
Huffman table or settings accessible to me and other objects, sets up those
220. For discussion of the “window” and discourses of space, see Friedberg (2006). For
an account of the relationships between screens and space/experience, see Cubitt
(2010). For a materialist exploration of the screen, see Patterson (2010).
221. The fact that I can scroll back through and see RAW-encoded data and so
encounter particular spaces is purely because my camera embeds a visible JPEG -
encoded preview into the RAW data. The RAW-encoded data remains unvisible, the
window opaque. Pavel Büchler discusses the implication of the camera’s “real-time Live
View” on the rear screen for our understanding of time and space, concluding “the
uninterrupted flow of information on the ‘live’ preview screen blocks the view of the
moment ‘out there’” (2010, p. 17).
143
JPEG: the quadruple object
144
JPEG: the quadruple object
This sense of an eidos and essence to JPEG was not only apparent to me.
It also appeared for other objects in my apparatuses which worked by and
through a series of dependencies. The browser object depended on JPEG-
encoded data to create a visual page and RAW-encoded data for an
unvisible one. Facebook’s algorithms depended on JPEG’s compression
working to make the millions of uploaded pictures manageable (as I discuss
in the next chapter). These unhuman actant-objects were created with the
knowledge that Huffman Coding and DCT were at work, just as they were
built in the knowledge that an iPad user would see and interact with them in
a particular way. The engineer and designers’ categorical intuition was built
into those objects. They “knew” the eidos of JPEG. But just as those
engineers did not, could not and did not need to know the full essence of
JPEG - the governmental implications and powers - so those unhuman
objects too remained unaware of that hidden essence.
For Harman the tension between the poles within the quadruple object
are matters of union and/or fracture. As I have said above, I find his account
theoretically convincing but I needed to see if that framework fitted with my
imaging practice.
145
JPEG: the quadruple object
146
JPEG: the quadruple object
147
JPEG: the quadruple object
digital imaging pipeline one sees objects and objects nested within objects
as their different dimensions connected.
What is also apparent from my practice is that the mediating object within
which those connections happen follows Harman’s logic. As noted above,
he argues that: “[r]eal objects can touch only through the medium of an
intentional object, and intentional objects can touch only through the
medium of a real one” (2009c, p. 208). My research indicated that this
asymmetrical relation appears in practice. The real, withdrawn dimensions
of the JPEG object and the real browser software object could not touch.
There was always an excess, something held back. There were dimensions of
each object that were not accessible to each other: JPEG’s governmental
nature (as I discuss below), the browser’s position as a software assemblage
with legal, political, IP and disciplinary implications for instance around
cookies and datatracking (Elmer, 2002). These dimensions were never fully
present or accessible not because they were in someway hidden by design
so much as withdrawn by nature. But JPEG and the browser clearly did
connect. What became clear was that they were mediated by a sensual
object, an accessible browser interface object, the “window” on my iPad or
in my eBook. It was this sensual object that served as the object and plane of
connection. Similarly with JPEG’s sensual dimension, a particular
instantiation connected with a particular profile of the camera hardware, a
particular material dimension of the chip and sensor accessible to electrical
charges and data as information. This encounter was mediated through a real
object, the camera’s hardware. This object had hidden depths, withdrawn
dimensions which these objects could not and need not access but that real
object served as the asymmetrical plane of connection for the accessible
sensual objects.
148
JPEG: the quadruple object
It may seem odd to argue that my practice bore out my argument for a
materialist strain of OOP. After all, I was looking at and working with
immaterial protocols and standards, developing immaterial eBooks and
image works and even building cameras from immaterial software
apparatuses (such as screengrabbing). With all those seemingly immaterial,
digital and machinic objects, where was the materiality? My practice
indicated that those objects had and have a form of materiality through
which their vibrant agentic capacity worked. The first and most obvious
sense in which that appeared was in the carbon footprints of those
technologies and the material traces of their manufacture. JPEG, JavaScript,
Facebook’s Social Graph are not carbon-neutral. They can never be fully
immaterial. They are always enfolded within devices, technologies and
chemicals, just as they are always enfolded in labour relations and
globalisation. But in a second sense the objects I used in my work had a
materiality. That materiality was different than Bennett’s discarded glove or
my iPad but the light-as-data-as-information flowing through my apparatuses
and the global network was real and material like the photons that
transgressed the Olympic Fence.224 That data excited sensors. That
information excited algorithms. That information was bought and sold in
marketplaces225. It was “farmed” and stockpiled in archives. This is not just
an extended metaphor. The information, the software standards I was
224. In a number of images, I simply opened the pinhole to the sky over the Fence. For a
discussion of photography and the sky, see Beck (2011).
225. Had I chosen to sell my Kindle and iBooks publications, that data and JPEG-
encoded data would have had a material position within the information economy. Even
as a “free” Book, that information had a position within that economy, not least in terms
of its position within my PhD, my career and my University department’s REF.
149
JPEG: the quadruple object
working with had a material charge as they worked and connected. Their
agentic capacity and quadruple form was enfolded with the material
technologies of distributed imaging but also the material physics of
information and data.
Conclusion
By approaching JPEG as an object and then building and using
apparatuses through objects, I found that objects, even “weird” ones like
JPEG, did have real and sensual dimensions, that the relations between those
dimensions were tensions characterised by fission and fusion and that while
particular and discreet, objects did connect, relate if you will, but within
other objects. While I hold that an object-oriented, quadruple account of
JPEG provides not only an imaging but also an analytical methodology, it is
important to show how an account of JPEG based around relationality,
processuality and potentiality fails to account for my experience in practice.
At first sight my practice seemed to show that the JPEG object has a
processural nature. After all it was apparent in its running or instantiations.
What became clear though was that the running was a matter of new
tensions and connections not the smooth movement of a continuing process.
Just as the evidence of my apparatuses within the mesh shows that objects
connect, so the different JPEG encodings within the mash-up appeared as
new objects. Each screengrab JFIF pulled in, each frame in the stream as a
result of JPEG’s running was an object but so was the specific JPEG
arrangement that gave rise to it. The dynamic form and content of the mash-
up process was a matter of specific, discrete objects not the flow of process.
150
JPEG: the quadruple object
A processural account of the JPEG object could not account for the specific
discrete operations of encoding in the apparatuses. Once again it is not that
there is no “process” but rather than this must be understood as a matter of
objects.
151
JPEG: the quadruple object
152
JPEG: the quadruple object
actants in a complex mesh of objects. The working class lift operator and
waitress or the society aristocrat or movie mogul are objects alongside a
Santa Claus sign or a fur stole. These objects are not semiotic markers of an
underlying class relation any more than the human is an archetype. They are
all objects in the complex assemblage of 1950s America connected and
connecting not at some external representational scale but in real world
materiality of serving drinks, being ignored by commuters or forging social
and business networks. Frank is not external to this. He too is an actant. His
shadow or gaze is woven into these object relations as it falls on windows or
is returned suspiciously. There is no objective recorder or photo-journalistic
position. There is only the position of object.
The image-objects are not somehow different to the objects in the images.
They are not more or less than those objects. They are just different. The
photographs (or the reproductions of the prints of the negatives...) are objects
now positioned in new object-relations with the bookmark on my desk, my
words on the screen, the image search, the print-out of my chapter, the code
of my own images, the protocols enabling those images.
153
JPEG: the quadruple object
that withdrawal that Frank could work. It was the fact that those objects were
all equal ontologically and photographically that enabled him to take this
photograph and make it work with all the others in The Americans. Most
importantly each object was actual. It was not defined by its relations to any
other, a plasma or a potentiality. The jukebox. The DNA in the baby. The
wooden chair leg were all real, material and vibrant regardless of any other
object. But there were connections. They connected with each other in the
heart of other objects. The real baby object connected with the sensual floor
object (a dimension of the floor object) within another object - the cafe-baby
object that Frank connected with as he pressed the button and exists now as
part of Cafe - Beaufort, South Carolina, the Google search for “Robert Frank”
and this chapter. The objects are not just compositional building blocks, they
are ontological ones too. And Frank’s practice depends on them so he could
create image-objects.
154
JPEG: the quadruple object
While of course Facebook is focused on textual data as raw material for its
Open Graph (and therefore advertising strategy), image-data, particularly in
the massive quantities it is processing, is increasingly important.
226. Daniel Rosenberg and Anthony Grafton trace the history of the timeline as a visual
and discursive device, although the book was written before Facebook’s particular
imag(in)ing of the idea (2012).
155
JPEG: the quadruple object
In the Facebook Open Graph API, photos are an object which developers
can access alongside any other data on Facebook.227 This is particularly true
in terms of how Facebook allows and encourages users to add metadata or
tags to that scopic datastore. Tagging has always been important to
Facebook. Bret Taylor, formerly co-creator of Google Maps and the Google
Maps API, CEO of FriendFeed and now CTO at Facebook said:
With this patent, as Goncalo Ribeiro puts it: “Static pictures have
essentially been made social” (2011, n.p.). The patent moves from discussing
tags within photos (as metadata) to how that facilitates interaction, social
connections and the broader work of the Open Graph:
227. Facebook began using the term “Open Graph” for its particular understanding, and
arguably exploitation of, the human “social graph” network at its f8 conference in 2010
(CBS News, 2010).
Facebook refers to the data it holds and allows developers to access and build
applications around (and so feed new data back to Facebook) as “objects”. Clearly this
is drawn from the use of the term in computing rather than philosophy. With that said, it
would be possible to consider Facebook’s Open Graph and its Open Graph API from an
object-oriented philosophical perspective where everything is an object connecting with
other objects within objects. Such a broader account is beyond the remit of this project.
228. For a discussion of tagging as a motor of social relations, see Rubinstein (2010).
For discussion of tagging and privacy from the point of view of technical design, see
Besmer & Richter Lipford (2010) and on patterns in collaborative tagging, see Cattuto,
Loreto & Pietronero (2007). For a semiotic discussion of tagging, see Huang & Chuang
(2009).
156
JPEG: the quadruple object
Here again, for Facebook, photos are more than images, they are objects
whose object-connections in the form of the broader Open Graph are
dynamic and integral to the business. In a patent for the new Timeline,
photos and “relationships” are clearly connected. Claim 5 states: “the social
timeline further comprises photos of the members connected in
relationship” (Sittig & Zuckerberg, 2010, col.11 line 41-43).
157
JPEG: the quadruple object
another” (ibid col.9 lines 17-18) is entwined with the user’s photographic
data and metadata on the site.229
229. A final proof that imaging is central to Facebook’s strategy can be seen in the fact
that other companies are seeking to challenge its legal position. In 2011, FotoMedia
issued a writ against Facebook (as well as MySpace, Tagged and Memory Lane)
arguing it held patents that allowed users to “upload, tag, and share digital
media” (FotoMedia, 2011). Journalist Joe Mullin (2011) claims that FotoMedia is a
“patent troll” a company that buys up or acquires patents and then exploits the US’
confusing and arguably lax patent system (Blumberg & Sydell, 2011) by taking
companies to court for infringement. Regardless of the status or legitimacy of FotoMedia
or its claims, what is clear is that photos, photo management, photo tagging and photo
sharing are central to broader IP and business battles and Facebook’s position.
Furthermore software, standards in the form of patents and protocols are at the heart of
that. Similar proof comes in the shape of Facebook’s pre-IPO purchase of Instagram.
230. It is beyond the scope of this work to address the debates around Foucault’s work
on government and indeed the subtle shifts in his conception of power. For the former
see Keenan (1982); Burchell, Gordon & Miller (1991); Barry, Osborne & Rose (1996);
Rose (1999); Lemke (2001; 2011); Bratich, Packer & McCarthy (2003); Jessop (2006);
Gane (2008) and Dean (2009) and on the latter Nealon (2008). In terms of spatial
rationality, see Rose-Redwood( 2006); Crampton & Elden (2007) and Huxley (2006:
2007). For a critique of the concept as “top down” and marginalising struggle, see Kerr
(1999). Government is also a theme picked up by Guins (2009) as discussed above. For
a Foucauldian archaeology of photography, see Bate (2007). In terms of governmentality
and film, see Grieveson (2009). For an early use of Foucault to discuss the Internet in
terms of legal discourse, see Boyle (1997).
158
JPEG: the quadruple object
231. For an early use of Foucault to explore what became known as “social media”, in
this case Usenet, see Aycock (1995). For more general explorations of governmentality
software and identity, see Elmer (2002; 2004; 2008) and Cheney-Lippold (2011). For a
discussion of databases, identity and power, see Aas (2004).
159
JPEG: the quadruple object
Before I look at how JPEG is connected to these issues and how an object-
oriented account of JPEG can help us address them, it is important to draw
out more particularly, even technically the software instantiations of
governmentality within Facebook. In order to do this I return to two
Facebook patents and a technical paper from the company on photo storage
and search.
232. Eli Pariser’ discussion of what he calls the “Filter Bubble” (2011) and Jonathan
Zittrain and John Palfry’s work on internet filtering (2007) point to the material and
technical underpinnings of these social and cultural practices of management of
information and the self.
160
JPEG: the quadruple object
161
JPEG: the quadruple object
What is important about this phrasing is that “data items [are] associated
with at least one relationship between users”. This is the core of Facebook’s
business: forming a bridge between data and relationships, turning
relationships into data (objects). The social timeline is “generated” from the
data, and itself becomes a data object within the “relationship storage
module” (ibid col. 3 lines 54-56). Furthermore, “a different social timeline
can be generated for different communities associated with the user” (ibid
col. 4 lines 4-6). Here relationships are dynamically created by user or by
machinic action.
162
JPEG: the quadruple object
233. See also Geoffrey Bowker and Susan Leigh Star’s discussion of categories and
standards (2000).
163
JPEG: the quadruple object
database” (ibid col. 5 lines 11-12). The governmentality here is more than
simply labeling and positioning (or self-labeling and self-positioning) of
population as data or information. It is the creation of a self-sustaining
human-unhuman machine of governmentality. The relationship engine
continually generates, or helps us generate, new orders, new connections,
new relationships which are fed back into the engine as new data objects
open to yet more orderings, connections and relationships. Ultimately of
course this Open Graph becomes the engine for advertising and data-mining
as well as surveillance.235
Images and imaging are a key part of that engine, as evidenced in both
the timeline and tagging patents. They act as vehicles for ordering and
connecting and relationships - self and self to self-management.
235. Even leaving aside any potential state uses of the Social Graph, Facebook is happy
to talk of “monitoring” as a core component of its technologies: “The monitoring module
tracks one or more user’s activities on the social network environment. For example, the
monitoring module can track the user’s interaction with one or more items of digital
media, such as digital images, news stories, other users’ profiles, email to other users,
chat rooms provided via the social network provider, and so forth. Any type of user
activity can be tracked or monitored via the monitoring module. The information, digital
media (e.g., digital images), people, groups, stories, and so forth, with which the user
interacts, may be represented by one or more objects, according to various
embodiments. The monitoring module may determine an affinity of the user for subjects,
other user’s digital images, relationships, events, organizations, and the like, according
to users’ activities” (Zuckerberg, Sittig & Marlette, 2011, col. 4 lines 44-58). For
discussions of monitoring and databases, see Andrejevic (2009) and Parry (2011). For a
discussion of the US government’s “Project Carnivore” surveillance programme (a
project critiqued by Galloway and his Radical Software Group, http://r-s-g.org/carnivore)
in terms of concepts of governmentality, see Ventura, Miller & Deflem (2005).
164
JPEG: the quadruple object
236. Harman says: “While the term ‘black box’ is not of Latour’s own invention, he
deserves much of the credit for importing it into philosophy. A black box is any actant so
firmly established that we are able to take its interior for granted” (2009c, p. 33).
165
JPEG: the quadruple object
The secret to effective photo storage and retrieval and so running of the
“relationship engine” is metadata - creating, finding and serving scopic data
points to enable new relationships.238 Leaving aside issues surrounding the
company’s controversial exploration of face-recognition,239 what Facebook’s
Timeline, tagging and other relationship services and practice do is deal with
metadata. Users connect “images-taken-on-my-birthday”, “images-tagged-
with-Charlie’s name”, “images-in-my-eBook-album”. This metadata can be
organised, connected and governed, by me or by software. The key problem
for Facebook and the most important aspect of Haystack, is managing or
governing that metadata as a way not only of finding and serving images but
also enabling relationships. The engineers identified that the existing system
was slowing down because of the amount of (governmental) metadata
237. The engineers report that “in Haystack, each usable terabyte costs ~28% less and
processes ~4x more reads per second than an equivalent terabyte on a NAS
appliance” (Beaver et al., 2010, n.p.). Here financial and user experience cost are seen
as equivalent. It is interesting to note the governmental discourse of waste and efficiency
that runs through Beaver et al.’s paper.
238. This separation of data and metadata can be approached from an object-oriented
point of view with engineers as well as philosophers and media critics working with
distinct, actual objects and their connections. Needless to say, however, that is beyond
the scope of this project.
239. For discussion, see Pidd (2011). For a broader discussion of the legal implications
of facial recognition technology, see McClurg (2007) and for a more technical discussion
of the possibilities of finding images of people in crowd scene photographs through
visual and contextual cues including time-stamps, see Garg et al. (2011).
166
JPEG: the quadruple object
associated with each image-object and the financial, storage and speed of
access costs involved in having to access all that metadata each time an
image was searched for or rendered. Their solution was to “keep[..] all
metadata in main memory, which we make practical by dramatically
reducing the per photo metadata necessary to find a photo on disk” (ibid p.
1). By making image objects in the system easier to find, resources could be
saved for the social metadata that enables the relationship engine to function
and new governmental relationships to be set in motion.
240. The authors continually elide between “photo” and “file” or “data”. Such language is
clearly interesting in terms of the ontology of image/data objects and issues of
discourses of representation. I use Facebook’s language here while recognising it as
problematic.
167
JPEG: the quadruple object
241. Vajgel, one of the Haystack engineers, also says: “The main requirement for the
new tier was to eliminate any unnecessary metadata overhead for photo read
operations, so that each read I/O operation was only reading actual photo data (instead
of filesystem metadata)” (2009, n.p.). Of course the system is not reading the JPEG-
encoded “photo data” so much as data about that “photo”. The metadata being left out
is about the filesystem.
242. There is an interesting object-oriented parallel with the way JPEG compresses
space and data through DCT and Huffman coding.
168
JPEG: the quadruple object
stream CDN nodes fail and need to refetch content” (ibid p. 4).243 The Cache
includes copies of the most frequently requested files. The Directory (note
again that this is materially located within a software/hardware mesh, it
exists somewhere in Facebook’s server farms) does not store “images” but
rather keeps track of the logical to physical mapping, any free space
available on the volumes and the metadata necessary for the browser to
construct the URL for an image.
243. JPEG-encoded files (or, as Beaver et al. refer to them, “photos”) are cached after
retrieval from the Store or CDN only if the request comes directly from a user and the
photo is fetched from a write-enabled i.e. more current Store machine (As Directory and
Store “machines” become full, they become read-only). The authors report that “photos
are most heavily accessed soon after they are uploaded and filesystems for our
workload generally perform better when doing either reads or writes but not both. Thus
the write-enabled Store machines would see the most reads if it were not for the Cache.
Given this characteristic, an optimization we plan to implement is to proactively push
recently up-loaded photos into the Cache as we expect those photos to be read soon
and often” (2010, p. 5).
169
JPEG: the quadruple object
Facebook server (for a page including an image). That server, contacts the
Directory which creates a URL for the JPEG-encoded image file held in the
Store and/or the Cache and/or the CDN. 245 The URL that is used to locate
the file destined to be Liked on the user’s page takes the form:
It is designed to enable Haystack to “find” and render the file (via the
JPEG decoder built into the browser).
245. The aim of Haystack is to make sure that images are held, and available for
connection, in whichever space is most beneficial to the user and the relationship
engine.
170
JPEG: the quadruple object
As has been noted the engineers think in object terms. The lawyers’
discourse too is object-oriented. The “modules”, “machines”, “databases”,
“fields”, “needles”, “files”, “data” and “metadata” as well as the tagging,
ordering and connecting associations detailed by both are objects on the
pages but also on material servers. Where Harman’s framework adds value is
firstly in opening up the scope of the objects that make up the engine and
secondly in explaining how they connect as a matter of objects not some
wider field. In short it enable us to see Facebook’s governmentality as a
matter of objects not as the outcome or result of an external governmental
rationality.
From Harman’s perspective what we see in Haystack, the Timeline and the
Open Graph is objects connecting within objects. The objects discussed
above and familiar to the engineers and lawyers are joined by the Facebook
user-object, the photographer-object, the Facebook brand-object, the
“friend”-object, the “Like” object (Gerlitz & Helmond, 2011) - a whole
collection of objects within and without Facebook. All have a real
dimension that withdraws. We can never access the totality of the
“relationship storage module”, the “needle” or the “identifier”. We
encounter its sensual dimension as we (human user, software algorithm,
image data etc.) expand energy on it. But those objects have real dimensions
outside of our relation to them. The Haystack has a reality “beneath” the
171
JPEG: the quadruple object
What Harman would add is that those object never fully connect. Real
can never connect with real, sensual with sensual. Objects connect through
mediating objects. There are “real” Facebook users and there are real
Facebook “users”. From an object-oriented point of view, both the flesh and
blood human and the social media “user” have a reality. As objects they
have a real dimension that withdraws and can never be grasped in its
totality. The relationship engine (and Facebook’s business depends on
connecting them. One has real money to spend, one a data position to be
sold and managed). These real objects connect within an object. They are
mediated by a sensual object, one that can be accessed. In this case they
connect (or are connected) within the Open Graph and Timeline (sensual)
objects. These objects, while also having real dimensions that cannot be
fully accessed also have a sensual dimension that is available for access as
long as energy is expended on it - literally in terms of the user being logged
in and also ontologically in terms of objects relating to it.
172
JPEG: the quadruple object
The question then become how does the JPEG object fit into this mesh.
My aim is not to argue that JPEG is the only protocological object in play
within Facebook, its business and IP struggles or their governmental or
disciplinary effects. Facebook connects with other standards objects. Users
can upload GIF, PNG and TIF-formatted files. But JPEG does have a
privileged position. In order to explore that, I turned to my practice.
173
JPEG: the quadruple object
250. I use the term “unsocial” in the same way I use the term “unvisible” (see n. 183) to
draw attention to the sense in which the data is somehow “outside” a particular social
mesh. It could be “social” within a different mesh.
174
JPEG: the quadruple object
Conclusion
JPEG, not just JPEG-encoded files, is enfolded with Facebook, its
technologies, material infrastructures, IP, datamining, advertising and
business practices. As such, as it connects with the panoply of objects in
play in the relationship engine’s Open Graph, it has governmental
implications. So much is clear from from an analytical and a practice-
research point of view. To see that with OOP eyes is not to position JPEG as
251. One could of course discuss the connection between JPEG and/or RAW-encoded
image files and Facebook’s software but the JPEG and RAW standards also connect or
“fail” to connect within the (social) digital imaging pipeline and so the imag(in)ing mesh.
An example is those photo library applications that enable direct uploads to Facebook.
Here JPEG’s presence as an encoder within the software connects with Facebook’s
upload software.
175
JPEG: the quadruple object
176
JPEG: the quadruple object
177
JPEG: the quadruple object
252. It should be noted that this paper, with its more Bryant-like embracing of protocol
“hold[ing] within their black boxes” a potential for exploit, was written before I explored
the critique of potentiality outlined above. As my practice moved on, so did my thinking.
I would now argue that the position of exploit is not dependent on hidden potential but
rather can be accounted for in terms of actuality and object connections.
178
JPEG: the quadruple object
Similarly I would argue that the object-oriented focus of this project (in
terms of theory, practice and methodology) has, like Benjamin’s mosaic,
through connecting and reconnecting object-fragments, problematised the
writing of media. As I discuss in my JPEG Object in Practice chapter,
practice-research claims to explore new ways of writing (and judging)
media. Amerika’s remix the book project and Bogost’s software works have
challenged the dominant (and lately powerfully governmental) discourse of
what counts as research. By working with and through objects, by allowing a
panoply of actants a place in my work and submitting a fragmentary mosaic
of objects, I look to follow their lead but also follow Benjamin in making the
form of my work as important as the content.253 Like Benjamin I look to my
253. For discussions of the relationship between Benjamin’s method and imaging/
visuality, see Buck-Morss (1989). Also see Gunning (2003) on Möbius strip-like
topographical views; Dubow (2004) on a particular Judaic conception of vision, and in
terms of Benjamin’s archive, see Leslie & Marx (2007). On montage as a method in The
Arcades Project, see Doherty (2006).
179
JPEG: the quadruple object
Perhaps the reason that The Arcades Project and Camera Lucida have such
a pull for academics is because they act from within the academic mesh to
overwhelm them by their own logic. Like Galloway and Thacker’s Exploit it
is not a gesture but a “swarm, the flood” (2007, p. 98). There is too much in
The Arcades Project and Camera Lucida. There are too many objects.
Benjamin’s form and Barthes’ tone while remaining within their respective
meshes overwhelm not just the reader but the mesh itself. Like a virus they
set off new connections, configure new objects, replicate and reconfigure
“history” and “photography”. Similarly I look to my photos, mash-ups,
eBooks, interludes and chapters to collide and connect within other objects,
to disturb as “dialectical images”. My aim is to analyse objects, work with
objects and use objects as a way of doing practice-research. By approaching
those objects as having a fourfold character, as withdrawing from access but
at the same time real, connected and present; by writing through and with a
flat mosaic of objects, I look to open up a space for those connections and
dialectics, question the scale of analysis and the scope of praxis and so
create an Exploit.
180
JPEG: the quadruple object
Epilogue
As part of my experiments with JPEG, I looked for an imaging practice that
was “beyond JPEG” not just in the sense of being analogue, but in terms of
being outside the governmental meshes I identified. I used a 1950s stereo
camera, the Belplasca and the last remaining rolls of Kodachrome 64 slide
film (the film and processing were discontinued in November 2010) to take
OOPh images.
The scopic object and experience was particular and unique, it could not
connect with JPEG.
I include a unique stereo Kodachrome slide with each copy of this thesis.
181
JPEG: the quadruple object
I include the eBooks as objects that, like all objects, must be addressed in
their specificity and actuality. As objects they have a reality that exceeds
their relations within my work. The objects on the “memory card” continue
as experiments. Following Bogost’s idea of “carpentry” they are also
theoretical works themselves - not illustrations of my theory nor just source
material for it, nor even a record of my own practice. As with the images and
imaging apparatuses, they were and remain the vehicles of the theory. I
“crafted” a range of eBooks as objects where JPEG and RAW/WebP
standards in authoring and viewing software, servers and hardware meshes
connect or fail to connect with other objects. It was that practice that
opened up the black box of JPEG for me to explore (and on which I report).
The book objects attached here continue to do the same philosophical work
for objects encountering them, connecting with them or even using them as
imag(in)ing apparatuses.
the eBooks
I chose to use the OOPh sections of this work as the material for my
eBooks. This obviously included the images, mash-ups and remixes I crafted
182
JPEG: the quadruple object
within my practice-research but also the text that forms the “interludes”. In
order to remain focused on images and imag(in)ing I chose to render/encode
that text as image files.254
During this project I created and worked with other images and mash-ups
such as the geolocation and augmented reality screengrabs I discuss in my
report. These do not form part of the eBooks although as images I have
added to Facebook’s Haystack, Flickr and my Blog, they may end up as parts
of the image-search mash-up.256
254. There is an interesting parallel with the way vector-software Adobe Illustrator can
trace text and make it a vector “object”.
255. The JavaScript for the mash-ups was made available at http://tutorialzine.com/
2010/02/photo-shoot-css-jquery and http://www.zurb.com/playground/rapid-prototyping-
with-flickrbomb as well as http://jquery.com
183
JPEG: the quadruple object
Technically in terms of the Kindle (.mobi) and the iBooks (.ibooks) files,
both formats are variations on the standard epub format. Issues surrounding
these formats, standards and protocols are beyond the scope of this project
which is around how JPEG connects or fails to connect with these meshes.
My experience of creating quadJPEG.mobi and quadJPEG.ibooks is however
instructive.
Kindle (quadJPEG.mobi)
In order to create an eBook for the Kindle (using the .mobi format257) I
created a simple HTML page with HTML <img src> links to the three sets of
differently encoded image files. The Kindle eReader available in the UK (not
the Kindle Fire tablet which is unavailable in the UK) cannot read (or
connect with, as an OOP approach would have it) JavaScript files so if I
wanted my HTML file to compile as a .mobi format eBook, I could not
include the HTML code necessary to make my mash-ups visible. Although
the Kindle could have connected with the JPEG-encoded objects brought
into the mash-up, the “failed” connection with the JavaScript meant the
Flickr, Google, Bing, Yahoo image searches and the Artviper server-objects
were inaccessible.
Having created the HTML file (and the two XML files necessary to encode
the eBook) I attempted to connect those objects with Amazon’s kindlegen
software. When the software “reached” the first RAW-encoded file and
attempted to connect with the non-JPEG objects, it “failed”. In order to
257. The Kindle uses files encoded using the .mobi and .azw standards, both
proprietary to Amazon and each of which allows DRM “protection” for books sold
through Amazon’s store. It has also developed a .kf8 standard for its new Kindle Fire.
184
JPEG: the quadruple object
encode the eBook I needed to remove the non-standard objects and the links
to them. When I did I could encode the eBook.
I then tried using the Open Source calibre software to convert the HTML
package. This eBook software object could connect with the HTML file and
while it too “failed” to decode the RAW/WebP-encoded objects, it did not
fail to create a viewable eBook. That eBook - the object that my Kindle
object connected with - includes the JPEG-encoded but not the RAW and
WebP-encoded objects. It is not that the Kindle device could not connect
with them. It did not even get the chance as other software intermediary
objects failed to connect and so did not include them. It is this calibre-
encoded Kindle file that I sideloaded 258 onto my Kindle and include on the
memory card. As a further experiment to see how JPEG-encoded, RAW-
encoded and WebP-encoded objects connect or fail to connect with the
Kindle-object, I sideloaded the complete set of images into a folder called
quadJPEG within a pictures folder on the device. In a little-known feature of
the Kindle, this creates a picture viewer. By pressing Alt-z, this quadJPEG is
added as a book on the home screen. When this is opened, the device-
object, operating system-object, screen-object connect with the JPEG-
encoded files, decoding and rendering them visible, but fails to connect with
the other digital objects.
iPad (quadJPEG.ibooks)
The iBooks format is an Apple-adapted version of the epub standard
(Glazman, 2012) and the iPad can read epub books. I could have re-
encoded my HTML, CSS and JavaScript “package” as an epub file. I chose
however to use Apple’s new iBooks Author WYSIWYG software to create an
eBook. By using this software (positioned as Apple’s attempt to popularise,
and arguably dominate, the creation as well as the consumption of
eBooks)259 I could explore how the JPEG object connected with an
authoring programme object as well as the browser and server software
258. To sideload means to transfer files directly onto a device as a mounted disc or
volume rather than using the App/Kindle store etc. These issues of “official” and
“unofficial” channels could of course be explored as a matter of objects but again, they
are beyond the scope of this project.
185
JPEG: the quadruple object
objects in the other eBook experiments. As I used the software, I found that I
could add RAW-encoded .orf files to the page. When I disassembled the
quadJPEG.ibooks file (essentially a zipped epub archive), I found that the
software had stripped the JPEG preview embedded in the Olympus .orf file
and used that as the image file. It could not do this with the WebP or DNG-
encoded files as there was no JPEG-encoded preview.260 Again, in terms of
my practice-research, it was the JPEG protocol object that was facilitating
connections, visibilities and the wider proprietorial and governmental
connections within which iBooks as brand, business and strategy are
enfolded.
260. One can choose to embed a JPEG-encoded preview when saving a .dng file. I
chose not to.
261. https://www.facebook.com/pages/Jpeg-the-quadruple-object/227885703990297#
262. https://www.evernote.com/pub/theinternationale/jpegthequadrupleobject
186
JPEG: the quadruple object
187
JPEG: the quadruple object
Of course it is impossible to set out every way in which the digital objects
I have “crafted” will be used and connected. A reader may read them on a
Mac or a PC; on Linux or Android; unpack them and recompile them for
different devices or simply never connect them with any other device. I will
however outline what a human-Apple-Amazon object (a reader with a Mac,
an iPad and a Kindle) might experience:
1. When the disc is plugged in, a folder appears on the desktop which
contains 3 files and 2 directory/folders: quadJPEG_web; quadJPEG.mobi;
quadJPEG.ibooks and the folders quadJPEG_web_files and pictures.
2. If the user clicks on the quadJPEG_web file, the webpage book opens.
This consists of a series of HTML pages, JavaScript files and image files all
held in the quadJPEG_web_files folder.
3. Depending on the browser the reader uses, she will see some image
files and not others. In OOP terms some object connections will “work”
others will “fail” depending on whether the various protocols involved in
the digital imag(in)ing pipeline of which the eBook is now a part, have
“worked” or “failed”.264
263. Of course the card could also be loaded into/connected with a camera. The
software in the camera would then display some of the encoded image files. Depending
on the model of the camera and the standards it uses, these could include the RAW-
encoded files as well as the JPEG-encoded files.
264. The terms “worked” and “failed” are, as discussed above, used advisedly see pp.
173-175 above.
188
JPEG: the quadruple object
6. If the reader has access to an Amazon Kindle, she can sideload the
quadJPEG.mobi file. When this book is opened on the Kindle devices
available in the UK (i.e. not the full colour Kindle fire), the book and its
JPEG-encoded images will appear in 16 shades of grey. The reader can
also sideload the pictures folder (containing all three sets of images) onto
the device, press alt-z and then open the quadJPEGslideshow “eBook”.265
These various scopic and imag(in)ing experiences (if the reader chooses to
use the apparatuses to screengrab 2012 imagin(in)ings and upload them to
265. The reader can also zoom in on the images: q = zoom in; w = zoom out; e = reset
zoom; c = actual size; f = full-screen; r = rotate; nav controller = pan and page forward
and back = cycle through images. The Kindle does include a WebKit based browser
and so could access, if not fully decode and render, an online version of the eBooks, but
my concern is with the Kindle as an eBook reader/imag(in)ing apparatus. Once again
there are interesting issues about eInk display objects and screen refresh technologies
which OOP could approach but are beyond the scope of this project.
189
JPEG: the quadruple object
the social web to appear in the mash-ups for instance266 ) of “success” and
“failure”, visibility and unvisibility, sociality and unsociality are the work of
quadruple objects connecting. JPEG is one of the objects in play here as it
was in my original OOPh. These eBooks are philosophical carpentry insofar
as they “do philosophy” every time they are read - or refuse to be read. My
practice in crafting them and yours in reading them are moments of object-
oriented practice-research.
266. To screengrab on an iPad click the Power and Home buttons simultaneously. The
png-encoded file will be added to the Photo stream. To screengrab on a Kindle, press alt
+ shift + G. The gif-encoded file will be added to the documents folder. These can then
be uploaded from the iPad or via a PC/Mac to the social web.
190
JPEG: the quadruple object
Bibliography
Aas, K. F. (2004). From Narrative To Database. Punishment & Society, 6(4),
379-393.
Agamben, G. (2009). What is an apparatus? and other essays (S. Pedatella &
D. Kishik, Trans.). Stanford, Calif.: Stanford University Press.
Amin, A. & Thrift, N. (2002). Cities: Reimagining the urban. Cambridge, UK;
Malden, MA: Polity.
Amoore, L. (2011). Data derivatives. Theory, Culture & Society, 28(6), 24-43.
191
JPEG: the quadruple object
Baker, C., Schleser, M., & Molga, K. (2009). Aesthetics of mobile media art.
Journal of Media Practice, 10(2&3), 101-122.
Barad, K. (2007). Meeting the universe halfway: Quantum physics and the
entanglement of matter and meaning. Durham, NC: Duke University Press.
192
JPEG: the quadruple object
Barry, A., Osborne, T., & Rose, N. S. (1996). Foucault and political reason:
Liberalism, neo-liberalism and rationalities of government. London: UCL
Press.
Bassett, C. (2008). The arc and the machine: Narrative and new media.
Manchester: Manchester University Press
Beaver, D., Kumar, S., Li, H. C., Sobel, J., & Vajgel, P. (2010). Finding a
needle in Haystack: Facebook’s photo storage. In Proceedings of the 9th
USENIX conference on operating systems design and implementation.
Beck, J. (2011). Signs of the sky, signs of the times: Photography as double
agent. Theory, Culture & Society, 28(7-8), 123-139.
Benjamin, W. (1997). One-way street and other writings (E. Jephcott & K.
Shorter, Trans.). London; New York: Verso.
193
JPEG: the quadruple object
Benkler, Y. (2002a). Coase’s penguin, or Linux and the nature of the firm.
Yale Law Journal, 369-446.
Berry, D. M. (2008a). Copy, rip, burn: The politics of copyleft and open
source. London: Pluto Press.
194
JPEG: the quadruple object
Berry, D. M. & Moss, G. (2005). The libre culture manifesto. Free Software
Magazine. Retrieved March 22, 2011, from http://
www.freesoftwaremagazine.com/articles/libre_manifesto.
Best, K. (2010). Living in the control society: Surveillance, users and digital
screen technologies. International Journal of Cultural Studies, 13(1), 5-24.
Blumberg, A. & Sydell, L. (2011, July 22). When patents attack. [Web page].
Retrieved October 18, 2011, from http://www.npr.org/blogs/money/
2011/07/26/138576167/when-patents-attack.
Bogost, I. (2010). A slow year: game poems. Highlands Ranch, CO: Open
Texture.
195
JPEG: the quadruple object
Bogost, I. (2011). Seeing things: Video and transcript of my talk at the third
object-oriented ontology symposium. [Web page]. Retrieved September 28,
2011, from http://www.bogost.com/writing/seeing_things_1.shtml.
Bowker, G. C. & Star, S. L. (2000). Sorting things out: Classification and its
consequences. Cambridge, MA: MIT Press.
196
JPEG: the quadruple object
Brassier, R., Grant, I. H., Harman, G., & Meillassoux, Q. (2007). Speculative
realism. Collapse: Philosophical Research and Development, 3, 307-449.
Bratich, J. Z., Packer, J., & McCarthy, C. (2003). Foucault, cultural studies,
and governmentality. New York: State Univ of New York Press.
Brockie, A. (2011, October 13). The Guardian iPad edition: ‘Put the
newspaper on the iPad’. [Web page]. Retrieved October 27, 2011, from
http://www.guardian.co.uk/media/gallery/2011/oct/13/guardian-ipad-
edition-design-evolution.
197
JPEG: the quadruple object
Buchanan, I. (2009). Deleuze and the internet. In M. Poster & D. Savat (eds),
Deleuze and new technology. (pp. 143-60). Edinburgh: Edinburgh University
Press.
Butler, J. (1993). Bodies that matter. Abingdon, Oxon; New York: Routledge.
Cabeen, K. & Gent, P. (1998). Image compression and the discrete cosine
transform. [Web page] Retrieved September 13, 2011, from http://
online.redwoods.cc.ca.us/instruct/darnold/laproj/fall98/pken/dct.pdf.
198
JPEG: the quadruple object
Cattuto, C., Loreto, V., & Pietronero, L. (2007). Semiotic dynamics and
collaborative tagging. Proceedings of the National Academy of Sciences of
the United States of America, 104(5), 1461-4.
CBS News (2010). Facebook: One social graph to rule them all. [Web page].
Retrieved March 16, 2012, from http://www.cbsnews.com/stories/
2010/04/21/tech/main6418458.shtml.
Chopra, S. & Dexter, S. (2008). Decoding liberation: The promise of free and
open source software. Oxford: Routledge.
Chun, W. H. K. (2006). Control and freedom: Power and paranoia in the age
of fiber optics. Cambridge, MA: MIT Press.
199
JPEG: the quadruple object
CIPA (2011). Exchangeable image file format for digital still cameras: Exif
version 2.3. Retrieved September 14, 2011, from http://www.cipa.jp/english/
hyoujunka/kikaku/pdf/DC-008-2010_E.pdf.
Clarke, D. B. & Doel, M. A. (2007). Shooting space, tracking time: The city
from animated photography to vernacular relativity. Cultural Geographies,
14(4), 589-609.
Cox, G., McLean, A., & Ward, A. (2001). The aesthetics of generative code.
Retrieved October 22, 2010, from http://www.generative.net/papers/
aesthetics/.
200
JPEG: the quadruple object
Cubitt, S., Hassan, R., & Volkmer, I. (2011). Does cloud computing have a
silver lining? Media, Culture & Society, 33(1), 149-158.
201
JPEG: the quadruple object
Deuze, M. (2011). Media life. Media, Culture & Society, 33(1), 137-148.
Dubow, J. (2004). Outside of place and other than optical: Walter benjamin
and the geography of critical thought. Journal of Visual Culture, 3(3),
259-274.
202
JPEG: the quadruple object
Ellison, N. B., Steinfield, C., & Lampe, C. (2007). The benefits of Facebook
friends: Social capital and college students use of online social network
sites. Journal of Computer-Mediated Communication, 12(4), 1143-1168.
Eraso, M., Ludovico, A., & Krekovic, S. (eds.) (2006). The mag.net reader:
Experiences in electronic cultural publishing. Spain: Arteleku-Diputacion
Foral de Gipuzkoa.
Ernst, W. (2005). Let there be irony: Cultural history and media archaeology
in parallel lines. Art History, 28(5), 582-603.
Facebook (2012). Open graph. [Web page] Retrieved March 16, 2012, from
https://developers.facebook.com/docs/opengraph/.
Featherstone, M. (1998). The flâneur, the city and virtual public life. Urban
Studies, 35(5), 909-925.
203
JPEG: the quadruple object
Flanagin, A. J., Flanagin, C., & Flanagin, J. (2010). Technical code and the
social construction of the internet. New Media & Society, 12(2), 179-196.
204
JPEG: the quadruple object
Fraser, M., Kember, S., & Lury, C. (2005). Inventive life. Theory, Culture &
Society, 22(1), 1-14.
Fridrich, J., Pevný, T., & Kodovský, J. (2007). Statistically undetectable JPEG
steganography: Dead ends challenges, and opportunities. In Proc. ACM
MM&S Workshop, Dallas, TX, September 20-21, pp. 3-14. Retrieved
February 2, 2012, from http://ws2.binghamton.edu/fridrich/
publications.html.
Fuller, M. & Goffey, A. (2009). Towards an evil media studies. In J. Parikka &
T. D. Sampson (eds.), The spam book: On viruses, porn, and other anomalies
from the dark side of digital culture. (pp. 141-59). Cresskill, N.J.: Hampton
Press.
205
JPEG: the quadruple object
206
JPEG: the quadruple object
Garg, R., Ramanan, D., Seitz, S. M., & Snavely, N. (2011). Where’s Waldo:
Matching people in images of crowds. In IEEE Conference on Computer
Vision and Pattern Recognition (CVPR).
Gehl, R. W. (2011). The archive and the processor: The internal logic of web
2.0. New Media & Society, 13(8), 1228-1244.
Gerlitz, C. & Helmond, A. (2011). Hit, link, like and share. Organizing the
social and the fabric of the web in a like economy. Paper presented at the
DMI mini-conference, January 24-25, 2011, at the University of Amsterdam.
Retrieved April 16, 2011, from http://www.annehelmond.nl/2011/04/16/
paper-hit-link-like-and-share-organizing-the-social-and-the-fabric-of-the-
web-in-a-like-economy.
207
JPEG: the quadruple object
Gillespie, T. (2010). The politics of platforms. New Media & Society, 12(3),
347-364.
Gitelman, L. (2008). Always already new: Media, history and the data of
culture. Cambridge, MA; London: The MIT Press.
Glazman, D. (2012). IBooks author, a nice tool but.. . [Web page] Retrieved
May 19, 2012, from http://www.glazman.org/weblog/dotclear/index.php?
post/2012/01/20/iBooks-Author-a-nice-tool-but.
Google (2011). WebP: A new image format for the web. Code.Google.Com
[Web page]. Retrieved September 29, 2011, from http://code.google.com/
speed/webp/index.html.
Graham, S. (2011). Cities under siege: The new military urbanism. London;
New York: Verso.
208
JPEG: the quadruple object
Guins, R. (2009). Edited clean version: Technology and the culture of control
(Kindle ed.). Minneapolis, MN: University of Minnesota Press.
Halpin, H., Robu, V., & Shepherd, H. (2007). The complex dynamics of
collaborative tagging. In Proceedings of the16th international WWW
conference May 8-12, 2007, Banff, Alberta, Canada.
209
JPEG: the quadruple object
210
JPEG: the quadruple object
Harman, G. (2011b). Another DeLanda book that came out quietly. [Web
page]. Retrieved November 8, 2011, from http://
doctorzamalek2.wordpress.com/2011/01/22/another-delanda-book-that-
came-out-quietly/.
Harman, G. (2011f). Not sure why this keeps resurfacing. [Web page].
Retrieved May 18, 2011, from http://doctorzamalek2.wordpress.com/
2011/01/25/not-sure-why-this-keeps-resurfacing/.
211
JPEG: the quadruple object
Harman, G. (2012b). What I’m writing about tonight. [Web page]. Retrieved
March 9, 2012, from http://doctorzamalek2.wordpress.com/2012/03/07/
what-im-writing-about-tonight/.
Harman, G. (2012c). The third table: Der dritte tisch (100 notes-100 thoughts
documenta 13). Ostfildern: Hatje Cantz.
Hawk, B., Rieder, D. M., & Oviedo, O. O. (2008). Small tech: The culture of
digital tools. Minneapolis, MN: University of Minnesota Press.
212
JPEG: the quadruple object
Hepp, A., Krotz, F., Moores, S., & Winter, C. (2008). Connectivity, networks
and flows: Conceptualizing contemporary communications. In A. Hepp, F.
Krotz, S. Moores, & C. Winter (eds.), Connectivity, networks and flows:
Conceptualizing contemporary communications. (pp. 1-12). Cresskill, NJ:
Hampton Press.
Hicks & Beaudry (2010). The Oxford handbook of material culture studies.
Oxford: Oxford University Press.
213
JPEG: the quadruple object
Introna, L. D. (2009). Ethics and the speaking of things. Theory, Culture &
Society, 26(4), 25-46.
Itō, M., Okabe, D., & Matsuda, M. (2005). Personal, portable, pedestrian:
Mobile phones in Japanese life. Cambridge, MA; London: The MIT Press.
Jacobson, D., Brail, G., & Woods, D. (2011). APIs: A strategy guide.
Farnham: O’ Reilly.
214
JPEG: the quadruple object
Jenkins, H. (2006). Convergence culture: Where old and new media collide.
New York: NYU Press.
215
JPEG: the quadruple object
Kelty, C. M. (2008). Two bits: The cultural significance of free software and
the internet. Durham, NC: Duke University Press.
Kerr, D. (1999). Beheading the king and enthroning the market: A critique of
Foucauldian governmentality. Science & Society, 63(2), 173-202.
Kramer, H. (1976). Art: Focus on photo shows. The New York Times, May 28,
1976.
216
JPEG: the quadruple object
Lane (1999a). JPEG image compression FAQ, part 1/2. [Web page] Retrieved
December 21, 2010, from http://www.faqs.org/faqs/jpeg-faq/part1/.
Lane, T. (1999b). JPEG image compression FAQ, part 2/2. Faqs.Org [Web
page]. Retrieved September 14, 2011, from http://www.faqs.org/faqs/jpeg-
faq/part1/.
Latour, B. (1996). Aramis, or, the love of technology (C. Porter, Trans.).
Cambridge, MA; London: Harvard University Press.
Latour, B., Harman, G., & Erdélyi, P. (2011). The prince and the wolf: Latour
and Harman at the LSE. Winchester; Washington: Zero Books
Law, J. (2004). After method: Mess in social science research. London; New
York: Routledge.
Lee, H. & Liebenau, J. (2000). Time and the internet at the turn of the
millennium. Time & Society, 9(1), 43-56.
217
JPEG: the quadruple object
Lee, T. B. (2012). Even without DNS provisions, SOPA and PIPA remain
fatally flawed.[Web page]. Retrieved April 27, 2012, from http://
arstechnica.com/tech-policy/news/2012/01/even-without-dns-provisions-
sopa-and-pipa-remain-fatally-flawed.ars.
Leong, S., Mitew, T., Celletti, M., & Pearson, E. (2009). The question
concerning (internet) time. New Media & Society, 11(8), 1267-1285.
Leslie, E., & Marx, U. (2007). Walter Benjamin’s archive: Images, texts, signs.
London; New York: Verso.
Lessig, L. (2009). Remix: Making art and commerce thrive in the hybrid
economy. New York: Penguin.
Lessin, S. W. (2011, September 22). Tell your story with timeline. [Web
page]. Retrieved October 18, 2011, from https://www.facebook.com/
blog.php?post=10150289612087131.
Levin, T. Y., Frohne, U., & Weibel, P. (2002). Ctrl [space]: Rhetorics of
surveillance from bentham to big brother. Cambridge, MA; London: The MIT
Press.
Levy, S. (2006). The perfect thing: How the iPod shuffles commerce, culture
and coolness. New York: Simon & Schuster.
218
JPEG: the quadruple object
Mackenzie, A. (2006). Cutting code: Software and sociality. New York: Peter
Lang.
219
JPEG: the quadruple object
Mann, S. (2003). What remains. Boston Mass., New York; London: Bullfinch
Press.
220
JPEG: the quadruple object
Manovich, L. (2007). What comes after remix? [Web page]. Retrieved April
30, 2012, from http://remixtheory.net/?p=16.
Manovich, L. (2011b). There is only software. Retrieved April 30, 2011, from
http://www.manovich.net/DOCS/Manovich.there_is_only_software.pdf.
Manovich, L. & Kratky, A. (2005). Soft cinema. Cambridge, MA; London: The
MIT Press.
Marino, M. (2006). Critical code studies. [Web page]. Retrieved October 12,
2010, from http://www.electronicbookreview.com/thread/electropoetics/
codology.
Marino, M. (2011). Critical code studies [Web page]. Retrieved January 27,
2012, from http://www.criticalcodestudies.com.
221
JPEG: the quadruple object
May, C. (2006). Escaping the trips trap: The political economy of free and
open source software in africa. Political Studies, 54(1), 123-146.
McClurg, A. J. (2007). In the face of danger: Facial recognition and the limits
of privacy law. Harvard Law Review, 120(7), 1870-1891.
McQuire, S. (2008). The media city: Media, architecture and urban space.
London; Thousand Oaks, Calif.: SAGE Publications.
222
JPEG: the quadruple object
Miller, D. (2011). Tales from Facebook. Cambridge, UK; Malden, MA: Polity.
Montfort, N. & Bogost, I. (2009). Racing the beam: The Atari video computer
system. Cambridge, MA; London: The MIT Press.
223
JPEG: the quadruple object
Morton, T. (2010b). Thinking ecology: The mesh, the strange stranger, and
the beautiful soul. Collapse, (VI), 265-293.
Nakamura, L. & Chow-White, P. (2011). Race after the internet. New York:
Routledge.
224
JPEG: the quadruple object
Pariser, E. (2011). The filter bubble what the internet is hiding from you.
Harmondsworth: Penguin.
Parisi, L. & Terranova, T. (2001). A matter of affect: Digital images and the
cybernetic re-wiring of vision. Parallax, 7(4), 122-127
Parr, M. & Badger, G. (2004b). The photobook, A history volume ii. London:
Phaidon.
225
JPEG: the quadruple object
Ramanath, R., Snyder, W. E., Yoo, Y., & Drew, M. S. (2005). Color image
processing pipeline. Signal Processing Magazine, IEEE, 22(1), 34-43.
Ravenal, J. B. (2010). Sally Mann: The flesh and the spirit. New York;
London: Virginia Museum of Fine Arts, Aperture Foundation.
226
JPEG: the quadruple object
Ribeiro, G. (2011, May 20). Facebook awarded patent for tagging photos
and digital media. [Web page]. Retrieved October 18, 2011, from http://
www.redmondpie.com/facebook-awarded-patent-for-tagging-photos-and-
digital-media/.
Rose, N. S. (1999). Governing the soul: The shaping of the private self.
London; New York: Free Association Books.
227
JPEG: the quadruple object
Shaviro, S. (2010). The universe of things. Retrieved March 12, 2011, from
http://www.shaviro.com/Othertexts/Things.pdf.
Sinclair, I. (2012). Ghost milk: Calling time on the grand project. London:
Hamish Hamilton.
Singer, B. (1995). The usual suspects (Writer and Director). [Motion Picture]
USA.
Sittig, A., and Zuckerberg, M., 2010. Facebook Inc.. Managing information
about relationships in a social network via a social timeline. U.S. Pat.
7,725,492.
Smith, M. (2008). That visual turn: The advent of visual culture. Interview
with Martin Jay. In M. Smith (ed.), Visual culture studies: Interviews with key
thinkers. (pp. 182-8). London; Thousand Oaks, Calif.: SAGE Publications.
228
JPEG: the quadruple object
Stearns, P. (2012). Year of the glitch. [Web page]. Retrieved February 25,
2012, from http://yearoftheglitch.tumblr.com.
Steichen, E. & Sandburg, C. (Eds) (1983). The family of man. New York: The
Museum of Modern Art. (Original work published 1955).
Sterling, B. (2005). Shaping things. Cambridge, MA; London: The MIT Press.
Sterne, J. (2006). The MP3 as cultural artifact. New Media & Society, 8(5),
825-842.
Sullivan, G. (2010). Art practice as research: Inquiry in visual arts (2nd ed.).
London; Thousand Oaks, Calif.: SAGE Publications.
Tagg, J. (1992). Grounds of dispute: Art history, cultural politics, and the
discursive field. Minneapolis, MN: University of Minnesota Press.
Tagg, J. (2009). Crime story: Walker Evans, Cuba and the corpse in a pool of
blood. Photographies, 2(1), 79-102.
229
JPEG: the quadruple object
Tanz, J. (2011). The curse of cow clicker: How a cheeky satire became a
videogame hit. [Web page]. Retrieved February 13, 2012, from http://
www.wired.com/magazine/2011/12/ff_cowclicker/all/1.
Taylor, J. (2012). Drones to patrol the skies above olympic stadium. [Web
page]. Retrieved February 13, 2012, from http://www.independent.co.uk/
news/uk/crime/drones-to-patrol-the-skies-above-olympic-
stadium-6267107.html.
Tofts, D. & McCrea, C. (2009). What now?: The imprecise and disagreeable
aesthetics of remix. Fibreculture, (15).
Tuters, M. & Varnelis, K. (2006). Beyond locative media: Giving shape to the
internet of things. Leonardo, 39(4), 357-363.
230
JPEG: the quadruple object
Urry, J. (2005). The complexity turn. Theory, Culture & Society, 22(5), 1-14.
Van der Tuin, I. & Dolphijn, R. (2010). The transversality of new materialism.
Women: A Cultural Review, 21(2), 153-171.
Ventura, H. E., Miller, J. M., & Deflem, M. (2005). Governmentality and the
war on terror: FBI project carnivore and the diffusion of disciplinary power.
Critical Criminology, 13(1), 55-70.
Weber, S. (2005). The success of open source. Boston: Harvard Univ Press.
231
JPEG: the quadruple object
Zittrain, J. (2008). The future of the internet. (Kindle ed.). London: Allen
Lane.
Zittrain, J. & John G. Palfrey, J. (2007). Access denied: The practice and
policy of global internet filtering (Oxford Internet Institute, Research Report
No. 14, June 2007). Oxford: Oxford Internet Institute.
232
JPEG: the quadruple object
Zuckerberg, M., Sanghvi, R., Bosworth, A., Cox, C., Sittig, A., Hughes, C.,
Geminder, K. and Corson, D., (2010). Facebook Inc.. Dynamically Providing
a Newsfeed About a User of a Social Network. U.S. Pat. 7,669,132.
Zuckerberg, M., Sittig, A. and Marlette, S., (2011). Facebook Inc.. Tagging
Digital Media. U.S. Pat. 7,945,653.
233