Augmented reality environment as social and design activism and urban games.

 

Abstract

This paper presents landscape as the continuous interface between the physical urban form and the ever-expanding use of digital devices and their content. It investigates contemporary attitudes toward digital public spaces, such as mainstream media facades, interactive art installations, and mobile apps. Media-infused landscapes could, if handled properly, transfer the public domain back from corporate ownership to public authorship.

Article Text

Introduction

The cultural and social interactions within the public realm, and to some extent the physical shape of urban spaces, are defined by contemporary electronic culture. Electronic devices augment our daily lives and the ways societies function. Video cameras oversee public safety, sensors track daily commutes, and wireless communication interconnects individual nodes into broader networks. At the same time, individual users with mobile devices extend the functionalities of these data networks through location-based and personal content to form user-centred and crowdsourced data landscapes. Peer-to-peer user-powered networks allow for direct, yet often anonymous communication that leads to new forms of social participation. This communication can be active and driven by users’ intentionally contributed content, or it can be passive and casual (opportunistic), as with navigational systems that automatically track users’ activities and pass them on to other participants.

These electronic networks provide unique opportunities for creativity and respond to our new expectations of globally connected, locally situated lives. They also start addressing Bertolt Brecht’s criticism about the limitations of the traditional media communications to an open electronic and bi-directional communication. As Brecht pointed out, “radio is one-sided when it should be two-. It is purely an apparatus for distribution, for mere sharing out. So here is a positive suggestion: change this apparatus over from distribution to communication.” [1] While radio and television have evolved little to satisfy Brecht’s aspirations, video games and other electronic technologies provide an effective apparatus to move media from distribution and sharing to communication, collaboration, and collective authorship.

This new urban dimension is enabled by ubiquitous mobile devices and willing participants who see the benefits of digitally enabled living. Always on, location-aware smartphones serve as portals to enter and navigate these multimodal landscapes. Geographic data, personal preferences, and audio-visual narratives merge into a single data-based landscape that extends the conventional definition of public spaces. Unlike users of past media, participants in these e-landscapes are both consumers of the media culture-location continuum and its creators. Due to their bidirectional operability, mobile devices serve both as receivers and as originators of data, thus satisfying Brecht’s proposition. However, a number of significant issues remain relatively unanswered and under-researched, particularly in the context of urban form. How do new electronic interactions cause societies to redefine physical and social structures of everyday lives? To what extent does digitality inform physicality, and is physicality rooted in digital worlds?

2. Towards City2.0

The recent progression from Web 1.0 to Web 2.0 (Web squared) is indicative of broader changes in the way people act and set expectations toward the surrounding world. Concepts behind Web 2.0 have been adapted to other disciplines, such as finance, management, and education. In a 2.0 paradigm, businesses benefit from users’ feedback, increased knowledge sharing, [2] and more effective marketing reaching a broader customer base. Similarly, in education, the Web 2.0 paradigm shifted the focus from presentation to participation, from access to information to access to people—teachers and classmates—effectively reframing the role of the faculty in academic teaching from knowledge source/expert to facilitator of learning. [3] Massive Open Online Courseware (MOOC), such as Coursera, often utilizes a crowdsource approach, with course participants serving as evaluators of classmates’ work. In all these examples, the focus of businesses and academia changes from knowledge-source centred to participant centred. However, in this new framework, consumers (students and users) are moving away from pure consumption and becoming content producers as well.

Similarly, the concepts behind Web 2.0 framework port into urban environments and the public realm. The correspondence between Web 2.0 and urban spaces is denominated by a common framework—social networks. Urban spaces are no longer exclusively defined as a distinct collection of physical buildings but as a dynamic network of inhabitants who actively contribute to the space’s image. The traditional concept of a city and its mental image as defined by Kevin Lynch [4] is no longer sufficient. The formative elements such as landmarks or nodes may still apply in a media-enhanced city, but they become more virtual and ephemeral than in a traditional interpretation. Furthermore, they transform from collectively shared to individually experienced; from permanent to ephemeral.

These elements can be no-longer-existing buildings that persist in residents’ memory or media creations such as the “Sex and the City” tour. In the latter, the tour organizers define city landmarks by situating them in the context of the TV show. These landmarks are often not-easily-recognizable structures that emerge into their visibility as the result of a random “15 minutes of fame” associated with a popular TV show. This acquired accidental prestige is used to entice prospective tourists: “Follow in the footsteps of Carrie & Co. as they conquer New York City! Drink where they drink, shop where they shop, and gossip where they gossip.” [5] The tour is both glorified, as in the Fodor’s travel guide (“New York City itself has been called the fifth major SATC character, alongside the fabulous foursome of Carrie, Charlotte, Samantha, and Miranda. Take a tour of Manhattan with this guide to the television series’ most memorable real-life locations”),[6] and ridiculed with the Telegraph report on the tour’s unexpected attractions: “There are some things you don’t expect to experience on a city bus tour. One is buying a pair of shoes. Another is viewing sex toys.”[7] Pointedly, the author of the Fodor’s Travel article observes that the city is indeed one of the show’s actors. However, when a city becomes intricately connected to an event, a story, or a movie, in turn the event or narrative imprints back onto the city.

This continuous feedback loop is visible in the case of The Da Vinci Code, both the book by Dan Brown and the subsequent movie, which popularized it even further. The book’s plot takes an advantage of iconic characters and places to situate its fictional narrative in a context readers can understand and relate to, a context that makes story more interesting by its association with these cultural icons. While this is a common strategy for commercial capitalization on third-party assets—success by association—an interesting outcome is that the success of a book in turn fed back into the original location. Taking advantage of the popular book and movie, the Louvre museum developed a tour that guides visitors along the Da Vinci Code thematic trail.[8] While this provides interesting narrative feedbacks and opportunistic commercial associations in addition to serving museum’s curious visitors, this feedback loop of (arte)facts and fictions further blurs the identity of the real and the imaginary, the physical and the virtual.

Even though architecture provides a memorable spatial iconography, today’s media redefine established mental maps with scattered memories and glimpses of visual excitement. While the media-defined image of the city may be temporal, it is also emotionally engaging and fulfilling to a large portion of the public. Other similar examples of media-infused landscapes are mental maps associated with Woody Allen’s and Martin Scorsese’s movies or with the Seinfeld TV show. In all of these instances, traditional topological and typological orientation points in the city are being replaced by cultural artefacts continuously redefined by media. The real and the imaginary are also blurred into a single form, with the general public merging factual and spatial with speculative and virtual into a single narrative. Media create a historical-fiction-novel equivalent of a physical world and extend what is firm (buildings, history) into the imaginary and the possible.

However, the endgame is not even what is present in the environment or projected outward by its form but what a recipient (the general public) perceives and registers. The collective memories, those discussed by Aldo Rossi in The Architecture of the City, [9] are not expressed and codified within buildings but by what consumers of culture walk out with from these experiences; what stays in their heads not on the building walls.

3. Contextualized Culture

From individual AR apps overlaying the physical world with context-aware information to broader repositories of geo-located data and Web sites/apps, such as the Museum without Walls [10], the established physical edifices such as museums and galleries start losing their value and relevance. While there is a significant appeal in seeing an original artwork in the controlled environment of a museum, there is also a sense of loss when the artwork is uprooted from its original cultural context and presented as a context-less object. Is it better to see an original painting in an isolated form or a high-resolution reproduction within a three-dimensional real-time rendered environment in the context of other work created by the same artist during a given time period? This question will certainly have various and often competing answers. However, with virtual high-resolution and high-fidelity recreations utilizing photogrammetry, virtual tourism becomes an attractive proposition, particularly as we consider that many masterpieces we encounter in museums are high-quality reproductions of original paintings vaulted away from the public for security and preservation purposes.

Situating artworks and other artifacts in their cultural or historical context may lead to a greater appreciation of their value as well as to forming new conceptual and semantical links fueling new interpretations. It could also refocus society’s focus from the areas that consume culture to places that create it.

The virtualization of physical artifacts discussed above, as a way to situate them more appropriately within a cultural and historical context, redefines the idea of identity. While it is usually considered that digital/virtual objects do not possess a unique identity due to their clone-like qualities (copies and originals are interchangeable), situating them within the original context may be a new form of defining the original.

4. Commonality of the Public Realm

This leads to a new understanding of what is the genus loci, or the shared perception and commonality of experiences of a place. Unlike the physical city, which by its shared nature is always “on” and WYSIWYG (what you see is what you get), the AR world can be turned off and can be either WYSIWYG or non-WYSIWYG, allowing for privacy within the public realm. This on-and-off transparency associated with WYSIWIG and non-WYSIWIG worlds is characteristic of electronic networks and online culture. It also puts in question a number of architectural and urban form constants, such as Rossi’s collective memory concept, or even Kevin Lynch’s five elements that form mental maps. Terms such as “district” and “edge,” or elements such as landmarks, become expressions created and shared by individuals operating in smaller groups without the need for broader compatibility or relevance. These mental maps become highly individualized, with strong references to non-physical objects that go beyond Lynch’s five elements.

5. Urban Games

Mobile phones have become powerful handheld computers that not only assist users in daily routines but also facilitate new forms of connectivity and affect the ways participants operate within our social structures.

Contemporary media emerge as a combination of gameplay (gamification), narratives, and open-ended virtual environments that mimic real-time life. Media not only mirror reality but increasingly become reality, or its emotionally inhabitable alternative. As stipulated by Marshall McLuhan in his seminal work The Medium is the Message, [11] the ways people experience the world ultimately merge with the content of what they experience. McLuhan observes: “All media are extensions of some human faculty—psychic or physical.” As with sandbox games, the future of media lies not only in storytelling but also in story-playing, a mapping of social behaviours within public spaces. This shift from consumption to authoring starts redefining the way people operate within cities and the sense of citizenship and ownership. Multiplayer gaming environments, electronic social networks, or mobile location-based games enable a diverse range of encounters without the need to personally engage with others face-to-face or reveal one’s identity. Digital media make it easier for many to engage with strangers, particularly for those who feel apprehension in interacting with strangers or just want to explore their inner self in a social context that is not predefined. They allow people to live multiple lives—if not sequential then at least parallel: concurrent incarnations that provide a continuous feedback loop and alternative perspectives on one’s life.

Figure 1. Augmented reality environment as social and design activism and urban games.

Figure 2. Mystery Spaces, a map with POIs arranged in the form of game play.

Now, through AR apps, these games are entering our physical surroundings, becoming context specific and a lot less virtual. By situating themselves in a defined physical context, they break away from the digital world’s uniformity and adapt to user needs. As Simon Games puts it: “Games are the new cinema, they are breaking free from the console and hitting the streets. These games are a new way of exploring ideas, meeting people and having fun. Hugely social, they are a new entertainment form.” [12]

Virtual environments allow for explorations of inaccessible or not-yet-materialized designs. They can be precursors of future physical urban spaces and potent drives in their realization. This is the case with AR and gaming environments (fig. 1) developed by the Tremont Underground Theater Space (TUTS) initiative [18]. The initiative is using AR gaming media not only to popularize ideas of the adaptive reuse of the abandoned public infrastructure but also to build social constituency and connect with general public. (fig. 2) The Mystery Spaces [13] approach relies on public events such as Common Boston 2012 to bridge the virtual experience with the physical activity in order to provide a critical mass of participation often necessary for social network-based activities.

6. Mapping Relationships

The City 2.0 paradigm, while deeply rooted in the traditional media relationship to the city, evolves towards a highly individualized and defragmented mental construct by connecting presence, past and all-possible-futures. These integrated media elements may no longer be universally recognizable or understood by the community nor contribute to universally shared collective memories of a place. The repositioning of urban networks with a focus toward users unavoidably shifts the metal maps of the public realm from objective “values” to subjective “feelings.” What constitutes a node or a landmark as one of five Kevin Lynch’s elements (from The Image of the City) becomes context and user dependent, with a strong time component.

City 2.0 returns to the phenomenological dimension advocated by Christian Norberg-Shulz. Discussed by Norberg-Shulz, the idea of genius loci (spirit of place), a combination of place and the phenomena associated with it, resurfaces in today’s media-enhanced cities as a relevant and potent concept. Location-aware functionalities present in ubiquitous mobile culture map directly onto the idea of genus loci as it relates to tangible and intangible human experiences. Media facades and mobile augmented reality extend the realm of the nonphysical setting of a place and the ways the “atmosphere” of the place affects the participant experience.

Figure 3. Corresponding parities between Web 2.0 and Architecture 2.0.

The key attributes of Web 2.0, such as interactivity, crowdsourcing, context-specific behaviour, collective knowledge, and collective authoring, directly link to similar categories within architecture and the public realm. In architecture, city, or the public realm, terms such as “participation,” “private and public,” or “collective memories” are familiar code words for user-centred design. They also reflect ways architecture and the city have evolved over time as a result of multiple and synchronous contributions: contributions that were not a result of the single creative act but a collaborative and an exquisite-corpse-like process.

Figure 3 shows corresponding parities between Web 2.0 and Architecture 2.0: “interactivity” and “participation,” “context specificity” and “private,” “ubiquity” and “public,” or “collective wisdom/crowdsourcing” and “collective memory” as defined by Aldo Rossi (1982). “Interface” is another shared concept. Architecture and design can be seen as a form of user interface (UI) focused on optimizing user experience (UX). The concept of a city as UI and UX to some extent is already present in A Pattern Language: Towns, Buildings, Construction, by Christopher Alexander. In his book, Alexander defines rules of spatial design based on observations of how people interact within and experience urban spaces. He argues that these behavioural patterns should inform the built environment. Interestingly, his patterns could inform not only the physical but also the virtual world. The creator of SimCity, The Sims, and Spore games, Will Wright, acknowledges the influence Alexander’s work had on his games:

[a] more appropriate source of inspiration we have found is things like architecture, and product design, because those are inherently more interactive design fields. SimCity was actually originally inspired by Chris Alexander, and going back and looking at design in general I’ve found a lot of inspiration from Charles and Ray Eames, Jay Forrester, Jane Jacobs, all the people who are sort of spanning the division between design, theorist, and a specific field – you know, urban design, architecture or whatever. I find that triangle really interesting to draw inspiration from. [14]

Wright is one of many who see architecture and the city as a creative framework for media-based environments.

The mappings between Web 2.0 and City 2.0 are possible because both environments, Web (network) and city (public realm), are spatial and social constructs. They go beyond linearity of experiences, providing a multiplicity of narratives with bifurcating possibilities. Their strength comes from the ability to interconnect individual nodes and create a system that supersedes its individual components. In many ways the Web and the city are two versions of the same interdependent social and cultural pattern. However, they differ in critical categories such as spatial continuity, identity, and perceived anonymity.

7. Final Thoughts

Cities are no longer purely physical artefacts—they are media, rooted in a graphical user interface (GUI), fine-tuned for the optimal UX, and accessed through ubiquitous networks and mobile apps. From cinematography designers have adopted discontinuity of time and space, with its asynchronicity of interactions and unexpected causality. At the same time, people expect to be constantly plugged into a larger, ubiquitous technological continuum of social networks and data flows. Co-location and direct interactions register differently today in the context of electronic networks. Urban environments become prime testing grounds for the physical-to-digital-and-back-to-physical metamorphosis cycle with an idea of digital physicality and physical digitality that forms a core theme of augmented urban lives today.

Media landscapes become interactive and reactive environments reflective of the human relationship with surroundings. They are not merely spaces that people inhabit, but also co-participants impacting and reformulating the roles people play within them. These new spatial and landscape attributes openly redefine the role architecture could play in the future, particularly its primary reading as a constant and permanent inscription into the landscape. Web 2.0, as one of the indicators of current media culture, not only redefines the way people interact online, but also sets new expectations toward daily activities and physical environments. Accustomed to dynamic and interactive media interfaces, users expect similar flexibility, adaptability, and intelligence from everyday physical spaces and objects as from digital constructs. Digital counterparts to the traditional, physical public realm may replace its particular elements or bring back elements that are already nonexistent, but most likely they will become an added layer of information inscribed onto the pre-existent space.

This paper unpacks the concept of a traditional city and reassembles it into a framework that considers the current and the near-future technological levels. It dissects the concept of Mediapolis in two dimensions: understanding (1) how existing conceptual frameworks that define our understanding of the city (e.g., Rossi, Lynch) map onto current media and electronic culture phenomena, and (2) what new opportunities are afforded by recent developments in electronic media culture.

The two critical dimensions/ecologies of the current situated and technologically driven environments involve the synergy between narrative and locative media. Narrative media merge architecture with other immersive and emotionally engaging arts such as motion graphics, movies, and video games, while locative/imbedded media provide situated experience with autonomous, agent-like live responses.

Not only are contemporary cities increasingly inhabited by cyborg citizens,^[i] but also an environment-context becomes a form of cyberneticorganism that should be considered as an animated and responsive entity that anticipates and adapts to changing circumstances.

Endnotes

1. Brecht, Bertolt The Radio as an Apparatus for Communication, 1926

2. Miya Knights, ‘Using Web 2.0 for business‘ComputerWeekly.com, 12 September 2007 <www.computerweekly.com/news/2240082941/Using-Web-20-for-business>, [accessed 10 Sept 2014]

3. Steve Hargadon, ‘Web 2.0 Is the Future of Education’ March 04, 2008, Steve Hargadon Blog, <www.stevehargadon.com/2008/03/web-20-is-future-of-education.html>, [accessed 10 Sept 2014]

4. Lynch, Kevin The Image of the City, (Cambridge: Technology Press, 1960)

5. Zerve.com, ‘On Location Tours: Sex and the City Hotspots’ Zerve.com, <www.zerve.com/OnLocation/SATC>, [accessed 10 Sept 2014]

6. Molly Moker, ‘Tour the Top 25 "Sex and the City" Locations’ Fodors Travel, May 27th, 2008
<www.fodors.com/news/story_3018.html>, [accessed 10 Sept 2014]

7. Francisca Kellett , ‘Sex and the City tour of New York’ Telegraph, 16 May 2008
<www.telegraph.co.uk/travel/destinations/northamerica/usa/newyork/1965817/Sex-and-the-City-tour-of-New-York.html>, [accessed 10 Jan 2015]

8. Visitor trails: The Da Vinci Code, Between Fiction and Fact <www.louvre.fr/en/routes/da-vinci-code>, [accessed 10 May 2014]

9. Rossi, Aldo, The Architecture of the City (Cambridge: MIT Press, 1982,)

10. http://www.culturenow.org/ [accessed 4 Jan 2015]

11. McLuhan, Marshall The Medium is the Message, (Corte Madera, CA: Ginko Press Inc), p.26

12. Pervasive Media Studio, <www.pmstudio.co.uk/collaborator/simon>, [accessed 10 Sept 2014]

14. William Wiles, ‘Will Wright interview‘ ICONEYE Online Magazine <www.iconeye.com/news/news/will-wright-interview> [accessed 10 Jan 2015]

15. http://bigbelly.com/ [accessed 4 Jan 2015]

16. http://www.nbcphiladelphia.com/news/green/Solar-Trash-Cans-Save-Philadelphia-Millions-200626771.html [accessed 4 Jan 2015]

17. http://www.theverge.com/2013/8/12/4614074/city-of-london-orders-renew-stop-tracking-citizens-with-trash-cans

http://www.takepart.com/article/2013/08/13/sharing-your-smartphone-data-trash-can [accessed 4 Jan 2015]

18. http://www.ohgizmo.com/2009/01/08/ces-2009-samsung-uvending-machines/

http://www.fahrenheit-212.com/u-vending-for-samsung/

19. http://www.smartsantander.eu/

20. http://www.smartsantander.eu/index.php/testbeds/item/132-santander-summary

21. http://blog.safecast.org/projects/

22. The Three Ecologies(Continuum, 2000, trans. Ian Pindar and Paul Simon), p. 24 (first published 1989).

Andrzej, Zarzycki

College of Architecture and Design

New Jersey Institute of Technology

Address:

Phone: +1 646.470.1070

Mail: andrzej.zarzycki@njit.edu