Using data about wealth, social housing and austerity collected with local research partners and community groups the app shows the Serpentine Galleries shaped and abstracted by the actual forces of its social environment, charting real-life inequality through virtual means. This warped simulacrum of the physical building is designed to restore a social vision to an often unseen stark class reality, demanding the visitor leave their comfort zone and question their surroundings. The research partners include Architects for Social Housing, Disabled People Against Cuts, Reclaiming Our Futures Alliance, Unite the Union Hotel Workers’ Branch and The Voice of Domestic Workers.
Actual RealityOS is an open source digital tool for data visualisation that brings together augmented reality, immersive audio and strategies of data collection and mapping for mobile devices. The app considers the symbiotic relationship between technology and power. Nowhere is this relationship more clear than in the decision making power imbued within technologies like artificial intelligence and predictive modelling now used to ascertain access to services like insurance, housing and social benefits.
Core to these technologies is their use of data and machine learning as a, perceived, neutral means of assessment when data, in its mediation and representation, is political and often reflects the inherent biases of societal power structures. Actual RealityOS uses the potential of augmented reality to create a tool for visualising otherwise unseen information: the nascent immersive technology allows users to see a real-time composite of virtual imagery against their physical environment through the camera of a phone or tablet. Focusing on the UK and, more specifically, the Serpentine’s location in the Royal Borough of Kensington and Chelsea, the app exposes inequality data – modelled, sonified and as testimonial – concerning wealth, income, social housing, domestic and hospitality workers’ rights, and the impact of austerity measures.
Working with a coalition of research partners and data researchers to aggregate not only national survey data from the public domain but data mined through Freedom of Information Act requests, data collected through community-led initiatives, and personal testimonies as a form of data. The app reconciles macro statistical information with micro data sets and the voice of individuals that form a nuanced, human-centred approach to data and the way that it is collected and used.
The architecture of the Serpentine Sackler Gallery becomes the base metric onto which the data is mapped to the external facade of the building by overlaying, in augmented reality, a warped virtual simulacra that charts the stark reality of inequality at 1:1 scale. The building is surrounded by a digital perimeter of personal testimony in contrast to the abstraction of statistical representation while a data-driven soundtrack guides you. The audio is produced through a process of data sonification in which statistics are represented as sound.
Created by Jules Laplace and Bethany Barrett using an automated tool that maps data points to harmonic scales which are then played through a Behringer DeepMind synthesiser, the inclusion of sound augments reality in another, non-visual, form that presents another perspective of both data analysis and social experience
About Power Walks
Influenced by the Situationists' dérive, the Power Walks programme brings to life the Actual RealityOS data, highlighting issues around social housing, low wage work, histories of resistance and the accessibility of the city. The three walks and a tour have been created in conversation with campaigns, community groups, and organisations local to the areas surrounding the Serpentine, and takes place throughout the duration of Hito Steyerl's Power Plants exhibition. Each walk and tour tell a story of the hidden and not so hidden inequalities from the point of view of those most affected.