My research explores the phenomenon of distraction in relation to mobile digital communications and ubiquitous computing. I am interested in the capture of attention; the relationship between online and offline experiences; compulsive usage of internet-enabled devices; and practices of data gathering and computation that characterise digital engagement.

One way of thinking about the distraction that digital social technologies can invoke is to consider the change in focus that might take place when receiving a notification on a mobile device. Attention is drawn toward the device issuing the notification by audible, haptic or visual cues, and once engaged with the device, focus then shifts to the software system that sent the notification. Interface design then has a role in maintaining the user’s focus and keeping them interacting with the social media system. In this instance, there seems to be a clear shift from offline to online experience: the smartphone vibrates, the user responds, and becomes engaged with online activity at the expense of their co-present offline setting.

Interruptive distraction of this type is not the only way in which focus moves between offline and online realms, and in fact, the boundary between the two conditions is often considered both permeable and contestable. For example, is a player of the popular mobile game Pokémon Go online or offline when they are hunting virtual characters in real physical space?1 What, in this case, actually counts as ‘being online’? More provocatively, can a person be online without even being aware of the fact?

Writers such as Jurgenson2 and Boesel3 see online presence as a condition that cannot be avoided. In some of the cases they discuss, online presence lies beneath the threshold of attention, such as when being sensed by networked cameras or tracked by beacons or cookies while browsing the web.4 In other cases, online presence occurs outside of the autonomy and agency of the affected party. An example of this is when someone without a social media profile of their own is tagged in a photo by someone else: they become subject to the face-tracking algorithms and profiling that accompany online presence even if this data can’t immediately be connected with their name, location, or other characteristics.5 Boesel identifies that not connecting is an act of personal agency, while not being connected is different, relying instead on the agency of other actors.6

Regardless of whether engagement happens passively or intentionally, interaction with most digital systems generates a data trail that can be aggregated from different sources and combined into a unique profile. This leaves the individual open to targeted advertising or other methods of categorisation. Public consciousness often looks to data as a measure of objectivity or truth: the data doesn’t lie. If this data is produced passively by the user, the consequences of its leverage by marketers can be unsettling: a retailer correctly inferring that a customer is pregnant from their purchasing data before the customer had revealed this to their family is just one example of this.7

Since the production of data is so significant to digital social technologies, it makes sense to think about how these systems encourage users to stay engaged and to continue producing data. The palette of design techniques for creating ‘addictive’ apps includes tricks common to advertising such as offering the illusion of control or mastery, or invoking motivations such as the desire to acquire status or skill.8

Among these techniques, gamification—the use of game design elements in non-game contexts—is an important concept that describes a number of different design approaches.9 One aspect of gamification is the production of quantified measures in an attempt to bring about bids for status or social capital through competition. This can be seen in the scoreboards of fitness apps which allow users to share the progression of their exercise regimes, encouraging informal competition between users, but is also present across social media in which numerated likes, favourites or retweets provide measures of popularity or engagement.10

At the level of physical interaction with devices, gamified design often encourages repetitive physical motions, such as simple swipes or thumb-pulls. These are often coupled with feedback that generates repeated activity in the user of the system to create what is known in app design circles as a compulsion loop. Sometimes called ‘ludic loops’, these are behaviour patterns that involve repeated swiping, scrolling or tapping, and are characterised by constant repetitive switching between certainty and uncertainty.11 The loop in behaviour occurs when the uncertainty is resolved temporarily by a reward of some sort, but at a frequency that can’t be predicted reliably by the user. This generates a desire to repeat the activity to identify a pattern or seek a kind of closure.

The most obvious examples of this principle can be found in computerised gambling, where pressing a button either produces a win—which may be financial or just the positive feedback of beeps and flashing lights—or a lose, but at a seemingly random frequency. It can also be found in the refreshing of an email account or social media feed: repeatedly ‘pulling the lever’ to see whether new messages pop up.12 The ‘infinite scroll’ commonly found in social media software capitalises on this.

The creative work I have been producing inquires into these ideas and debates. Invisible Layers features an animation of a disembodied hand restlessly carrying out precisely the type of repetitive physical motions that characterise compulsive device usage.13 In this work, the hand carries out an endless series of ‘swipe’ and ‘pinch zoom’ gestures that reveal a composite of landscape data and infrastructural maps. The piece uses terrain data from the United States Geographical Service and map data from the crowd-sourced Open Street Map to compile intricate, layered views of the locations of the head offices of major smartphone manufacturers. The piece asks questions about the materiality of digital experience by focusing on the infrastructural elements in the Open Street Map database. The ‘pinch zoom’ gesture proposes a relationship with digital images that can’t be replicated with images in material form. Zooming is not something that the eye can do without the help of optical lenses or algorithms, and the factor of zoom caused by the gesture would induce vertigo if not bounded by the tiny edges of the smartphone screen. The ‘pinch zoom’ gesture provides an illusion of mastery and control over digital images that extends by inference to non-digital images in the imagination of the user. In the relentless dismissal of these complex map images, the ‘swipe’ gesture is depicted as a restless, repeated compulsion loop.

The work in the exhibition, Mock Objects, exposes questions emerging when user interfaces are unable to rely on visual design alone to invoke compulsive usage.14 The piece uses Augmented Reality (AR), a computer vision technique in which live video is embellished with 3D-rendered contextual material and presented to the user as a seamless mix between the two. To create the illusion of a 3D model existing in the same representational space as the user’s surroundings, the video scene needs first to be analysed to make sense of its spatial characteristics. In some systems, the video will be checked for an AR marker—typically a monochrome square similar in appearance to a QR code— so that 3D positioning can be calculated from the orientation of this marker.

The work consists of a modified version of the debugging output of a popular JavaScript AR library, js-aruco, which shows the algorithm’s best guess of the orientation of a marker in the video scene.15 Since the estimated position and angle of the marker changes with each frame of the video, the image flickers from one orientation to another. The restless image is evidence of the software’s indecision about how it understands what it sees.

Viewers may be able to infer that the movement is generated from human activity, since even after processing, there is still an identifiably organic character to the motion. The piece highlights an instance of algorithmic uncertainty, making the measurement of this uncertainty a major formal component of the work. While highly visually reductive and utilitarian in appearance, the piece makes visible a process of computation that is contingent and inconclusive.

...

1 More information about Pokémon Go can be found at the Nintendo Pokémon website, <http://www.pokemon.com/uk/pokemon-video-games/pokemon-go/> [accessed 13 July 2016].

2 Nathan Jurgenson, ‘Digital Dualism versus Augmented Reality - Cyborgology’, <https://thesocietypages.org/cyborgology/2011/02/24/digital-dualism-versus-augmented-reality/> [accessed 8 July 2016].

3 Whitney Erin Boesel, ‘A New Privacy Pt. I: Distributed Agency & the Myth of Autonomy’, <https://thesocietypages.org/cyborgology/2012/05/21/a-new-privacy-pt-i-distributed-agency-the- myth-of-autonomy/> [accessed 8 July 2016].

4 Finn Brunton and Helen Nissenbaum, ‘Vernacular Resistance to Data Collection and Analysis: A Political Theory of Obfuscation’, First Monday, 16.5 (2011) <http://firstmonday.org/ojs/ index.php/fm/article/view/3493> [accessed 6 November 2015].

5 Chris Davies, ‘Facebook “Shadow Profiles” Detail Non-Members, Prompt Investigation’, SlashGear, <http://www.slashgear.com/facebook-shadow-profiles-detail-non-members-prompt- investigation-21189885/> [accessed 14 July 2016].

6 Boesel, ‘A New Privacy Pt. I’.

7 Charles Duhigg, ‘How Companies Learn Your Secrets’, The New York Times, 16 February 2012 <http://www.nytimes.com/2012/02/19/magazine/shopping-habits.html> [accessed 13 July 2016].

8 Rob Sturgess, ‘Design for Addiction’ (presented at Are We All Addicts Now?, Peer UK, Hoxton Street, London, 2016).

9 P. J. Rey, ‘Gamification and Post-Fordist Capitalism’ in The Gameful Word: Approaches, Issues, Applications, ed. by Steffen P. Walz and Sebastian Deterding (Cambridge, Mass. and London: MIT Press, 2015), pp. 277–296.

10 Benjamin Grosser, ‘What Do Metrics Want? How Quantification Prescribes Social Interaction on Facebook : Computational Culture’, 2014 <http://computationalculture.net/article/ what-do-metrics-want> [accessed 15 July 2015].

11 Douglas Heaven, ‘Engineered Compulsion: Why Candy Crush Is the Future of More than Games’, New Scientist, 222.2971 (2014), 38–41.

12 Tristan Harris, ‘How Technology Hijacks People’s Minds — from a Magician and Google’s Design Ethicist’, Medium <https://medium.com/@tristanharris/how-technology-hijacks-peoples-minds- from-a-magician-and-google-s-design-ethicist-56d62ef5edf3#.79qttxh2g> [accessed 23 May 2016].

13 Michael Day, Invisible Layers, June 2015 <http://michaelday.org.uk/invisible-layers> [accessed 14 July 2016].

14 ‘In object-oriented programming, mock objects are simulated objects that mimic the behaviour of real objects in controlled ways.’ ‘Mock Object’, Wikipedia, the Free Encyclopedia, <https://en.wikipedia. org/wiki/Mock_object> [accessed 15 July 2016].

15 Juan Mellado, Js-Aruco - JavaScript Library for Augmented Reality Applications (2015), JavaScript <https://github.com/jcmellado/js-aruco> [accessed 14 July 2016].