Introduction

 

The short exemplary analyses accompany the new online Meta-Thesaurus to demonstrate the potential of this tool. The represented keywords are selected for their potential to: build a bridge between “digital born” Media Art and traditional art forms (such as graphic prints, painting, sculpture, etc.) and their correspondences; identify developments as well as discontinuities with regards to aesthetics, genre, subject, and technology in image-media transformation; and support the systematic analysis of important critical innovations as well as the “afterlife” of historical themes in contemporary Media Art.

 

 

REPRESENTATION OF KNOWLEDGE

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Databases and the Representation of Knowledge

By Janina Hoth

 

The aesthetic and structural characteristics of knowledge systems have been a recurring subject in Art History and Digital Art. Every medium represents knowledge in its own distinguished way – from scrolls, books and libraries to card indices, keywords and search engines to contemporary inventions such as e-books, databases and Wikipedia (Keller 2014: 4-6). In order to gain knowledge from raw data, its classification and representation is a necessity. It needs to be organized, structured and presented in predetermined forms and functions if we want to be able to perceive, understand and develop it. One example is the engraving “Wisdom and Ignaz of Loyola in conversation in a library” by Johann Andreas Pfeffel from 1695 (fig. 1).

The figures of a scholar and a bishop point to the seemingly endless shelves of books and rooms behind them. Each figure holds an emblem in front of him. The left one is clearly marked as an intellectual personating wisdom and the other one is Ignatius of Loyola, the founder of the Jesuit fraternity. He personifies the schola affectus (school of hearts). The return to the school of hearts is the final educational process before becoming a Jesuit(Schindler 2014: 463). The central question, shown in the Latin phrase at the bottom of the image, leaves choice to the viewer which of the two men should be trusted in respect of education. Following the figurative logic, the answer is rather easy. The owl and the masks on the scholar's chair are signs of earthly wisdom as well as the occult. This indicates a possibly sinful life, should the knowledge not be taught rightfully. The Jesuit is decorated with angels and a halo. Clearly, the guidance of a Jesuit intellectual is needed in order to gain true education. The library does not show an actual place, but symbolizes encyclopedic knowledge. It seems as if the knowledge of all humanity lies behind them in their great collection. There is no idea of selection. As the engraving shows, the ideal of universal knowledge existed, but it was always structured and organized by ideological beliefs and political ambitions. Libraries were not just a space for wisdom, but symbols for representational purposes and control as well. They were supervised by educational control and accessibility (Friedrich 2013: 22).

This specific type of systemisation with ties to a specific place and the control of an organisation is a past example for controlling knowledge while representing it. Knowledge and its gathering have always been connected to a specific worldview as well as social and political ideas and strategies. At a first glance, the digitisation of knowledge seemed to have done away with this paternalism. The internet with its no space limit and global accessibility presented a radically new possibility for the old idea of the universal library.

At the beginning of the new digital age, as Lev Manovich for example wrote about in his book “The Language of New Media” (2001), theorists believed in the Internet to mark a possible end of the mediation of knowledge (Galloway 2011: 378). The digitisation has definitely changed the way knowledge is gathered, distributed, perceived and consumed. Today, the Internet has made it possible to liberate knowledge from bodies of institutions and distribute data unfiltered. But the digitisation is still a type of mediation generating new ways of representation. For example, the seemingly endless space for gathering knowledge challenges the limitations of the amounts, which we are able to process. Knowledge in its digital form is always in danger of becoming unperceivable and unusable.

Digital artists try to render the digitisation palpable and to give participants an idea of its processes. In general, collecting knowledge has become an abstract process that is based on data and computational calculations. Simon Biggs tried to portray this in his interactive work ‘Babel’ (fig.2). He created a website to visualize the Dewey Decimal system as a visual metaphor for the representation of knowledge on the internet. The user can navigate through a seemingly indefinite number of data collections on the net (such as museum collections, library sites etc.), which are shown as arrays and grids of Dewey Decimal numbers. While navigating with the mouse, the users transform the 3D graphic simultaneously. The more users navigate the denser and more complex the graphic becomes. When clicking on a number, one is redirected to the sources of information. Since the navigation is based on a numerical system, it is impossible to foresee the connected website. The feeling of disorder and absurdity of the specific classification system makes the representational system arbitrary and disconcerting. The more absolute knowledge on the internet becomes, the harder it is to comprehend as it transforms into an unreadable amount of data, it seems. As Biggs tried to show in his work, understanding data/knowledge has become a difficult task, which is hard to grasp for a single human being. The formal and mathematic logic of an algorithm collecting data is difficult to comprehend in how it gathers and processes data. The process itself seems out of reach, as it is too complex, too big and too abstract.

Although the library offers a controlled and limited, but clear system of classification, the digitisation appears to be more suitable for realizing the ancient idea of a universal knowledge in its appealing idea as well as overwhelming logic. It demands new definitions for the representation of knowledge. The difference lies not only in the general methods of collecting information, but in the perception of the reader.

While a book represents a static and definite textual form, electronic data and devices are disclosed in form and function, because they offer users constant references to other texts or media forms (Rowe 2013: 2-4). This demonstrates one of the key elements and abilities of digitisation, but can also constitute a total relativity of knowledge (Rowe 2013). Electronic texts are more similar to a scroll than a book in their ability as ‘collections of (verbal and visual) ideas that can arrange themselves into a kaleidoscope of hierarchical and associative patterns’(Bolter 2001: 77).

If one understands the scroll and the book as two different media, each creates a visual space for knowledge that has a distinct influence on how we perceive their content. The difference between the two media can also be seen in an undated engraving from the GSSG (fig. 3). The room is divided into a library with books and one with scrolls. While the books are arranged mostly in a clear manner with each in its own space, the scrolls are spread all over the room and hang one above the other in the shelves. The connectivity of the scrolls in a fluid transition from one to the other encourages an intertextual reading in comparison to the more separated books. The engraving portrays the development from one media to the next with the accumulation as well as classification of knowledge.

Intertextuality is a common element between the scroll and electronic texts, but the digitisation has created far more changes in the representation of knowledge. An analysis cannot be limited to electronic texts in PDF files or e-books. It introduced new and old issues concerning the disposability and usability of knowledge. As Biggs portrayed in his work, it has turned from letters and paper into data and codes. Processing data to information and then to knowledge online (and thereby digitising it), is a process far more complex than merely saving a file online. The transformation performs not only a change from one medium to another, but works on the semiotic level as well. In digital art, new views on the processes of data gathering have been developed. One example is Diego Caglioni's “Cloud murmur” (fig. 4).

Rather than recreating a space of knowledge like a library, Caglioni recorded noises in offices and data centers, on websites etc. His artwork consists of dozens of these sound recordings mixed together. In the background, it is sometimes possible to hear human murmuring, but the main component is a continuous flow of what we generally describe as ‘computer noises’. In this artwork, data is treated like a medium – something that is present around us every day and that structures our life and our perception of the world without us being aware of it. Caglioni presents a ubiquitous system of gathering, collecting and controlling data, where ideas of knowledge or education are secondary at best. Next to the written word, technical aspects and their representation are becoming a significant part in the reception of knowledge.

All the artworks represent a specific method of collecting knowledge and depict a way of representing an enormous amount of data to the viewer/participant. While the engravings connect data and knowledge gathering to a human being and a physical space, digital artists place this outside of a human sphere or represent it with an algorithmic formula. There is liberation in these new processes, but the idea of knowledge seeking may be lost. With the digitisation, the representation of knowledge has undergone significant changes concerning how and what we perceive as knowledge. The internet revived ideas of a universal library, but the large amounts of data today have limited this enthusiasm. As digital artists show, this fast development introduced new issues as well as advantages. The internet offers a previously unknown accessibility, but this craves for new methods in the representation of knowledge, that still need to be developed.

Links:

Fig. 1: Wisdom and Ignaz of Loyola in conversation in a library on GSSG

Fig. 2: Simon Biggs on ADA

Fig. 3: Engraving of Library on the GSSG

Fig. 4: Diego Caglioni on ADA

 

Bibliography:

Ackoff, R. L., "From Data to Wisdom". Journal of Applied Systems Analysis, Volume 16, 1989 p 3-9.

Bolter, David J. 2001. Writing space: Computers, hypertext, and the remediation of print. Second edition. Mahwah, N.J.: Lawrence Erlbaum Associates.

Brinker von der Heyde, Claudia, Jürg Wolf (ed.) 2011. Repräsentation Wissen Öffentlichkeit. Bibliotheken zwischen Barock und Aufklärung. Tagungsband zum interdisziplinären Workshop. Kassel university press Kassel.

Friedrich, Markus 2013. Die Geburt des Archivs. München: Oldenbourg Wissenschaftsverlag.

Galloway, Alexander R. Galloway, What is New Media? Ten years after The Language of New Media, Criticism summer 2011, Vol. 53/3, pp. 377-384.

Keller, Stefan Andreas (ed.) 2014. Wissensorganisation und -repräsentation mit digitalen Technologien. Berlin: de Gruyter saur.

Rowe, Christopher 2013. The new library of babel? Borges, digitisation and the myth of the new library. First Monday 18/ 2 – 4. http://journals.uic.edu/ojs/index.php/fm/article/view/3237/3416 doi:10.5210/fm.v18i2.3237

Schindler, Claudia 2014. Wissen ist Macht! Nicolò Partenio Giannettasio (1648–1715) und die neulateinische Gelehrtenkultur der Jesuiten in Neapel. Bibliothek Forschung und Praxis 38/ 3 : pp. 461–467.

List of figures:

Fig. 1: Andreas Pfeffel(engraver) 1697 font-weight: normal;">Weisheit und Ignatius v. Loyola im Gespräch in einer Bibliothek. Copper engraving.

Fig. 2: Simon Biggs. 2001. Babel. site specific work for a non-site.

Fig. 3: Artist and date unknown. Ansicht einer Bibliothek und einer Schriftrollensammlung, Copper engraving.

Fig. 4: Diego Caglioni. 2014. Cloud Murmurs. Sound Installation.

AUGMENTED REALITY

Fig. 1
Fig. 2
Fig. 3

The augmentation of the image space, or: unravelling the myth of ‘reality’ in augmented reality

By Sebastian Haller

Augmented reality is a deceptive term. Because its definition seems to be straightforward – it openly implies the augmentation of reality –, it somehow obscures its intrinsic features. This is due to a couple of reasons. Since its technological as well as discursive introduction in the early 1990s, augmented reality devices are commonly understood as imaging technologies that depict ‘reality’ and, at the same time, enrich this ‘reality’ by means of virtual information. This concept of AR was coined by its most prominent actors as the ‘reality-virtuality continuum’ (Milgram/Takemura/Utsumi/Kishino 1994, Azuma 1997). Surprisingly, even after twenty years after its introduction, the paradigm of the ‘continuum’ is still considered to be the central aspect of AR technologies; this is surprising, not because of its longevity – twenty years is an incredibly long time-span in the context of digital technologies –, but because of its intentional negation of possibly the only real consensus there is in the everyday as well as scientific discussion about mediated objects: That mediated content/media content is not reality – even though, one must add, every media requires a real material carrier. Neither media theorist nor ‘ordinary’ users of an imaging device, such as a digital camera, believe that any mediatised/mediated entity, i.e. something that is represented by the means of something else, is reality. But still, the discourse of augmented reality is still relying on this deception of ‘reality’ as its sine qua non. Unquestionably, contemporary digital devices offer a high degree of- to borrow from literary theory- of verisimilitude, or, with Roland Barthes, they convey an enhanced effet de reel

But if the conviction of ‘reality’ in AR is highly problematic, what further characteristic is intrinsic to AR? What happens if you detach the notion of ‘reality’, its sine qua non as implied by the Milgram and others, from AR? Does AR have any other characteristic features if you refute the notion of ‘reality’? If so, are those features strong enough to understand AR as a distinct imaging technology? Basically, this paper argues that, on the one hand, the supremacy of ‘reality’ can be abolished and, on the other hand, AR can still be defined as a distinct imaging technology. This objective will be achieved by focusing on the construction and augmentation of space. Instead of understanding the devices of AR as augmented reality, this paper will argue that according imaging technologies are (a) constructing a (unified) image space that reacts in real-time to a real physical environment and (b) they are augmenting this space with virtual information – i.e. AR devices establish an augmented space instead of an augmented reality. This augmented space necessitates four essential features: (1) a material carrier or medium, (2) a form that is perceptible through the medium that conveys a space, (3) the augmentation/distortion of this space through (whatever) information that is set outside the logic of the unified, homogenous space and (4) a spectator who (not only perceives) but decodes this complex relation between 1, 2, and 3 in the process of cognition.

Furthermore, I suggest that ‘augmented space’ is not just confined to the means of digital AR devices. Instead, augmented space is a visual phenomenon that emerges in all possible imaging technologies that are able to construct space and, at the same time, break with this (unified) space through additional (virtual) information. This means, augmented space is considered to be a phenomenon of visual culture that is not confined to new media technologies, but also to ‘old’ media, such as ‘traditional’ (painting, drawing, graphics etc.) and reproductive media (photography etc.). Paradoxically, this information (1) counteracts the space and, at the same time, (2) is related to information inside the space. Evidently, there are profound differences in the augmentation of space in regard of its technologies; e.g., digital devices that are able to react in real-time, augmentation of space is time-based and reacts ‘live’ to its environment and the movement of the user (see fig. 1). On the other hand, augmented space in ‘traditional’ technologies of image production, for instance baroque graphics, is a static phenomenon. Such graphics make use of an ‘augmented space’ to offer information that cannot (or hardly) be depicted ‘iconically’ (see fig. 2 and fig. 3).

 

Links:

 

Thamiko Thiel on ADA

 

Holy kinship in the temple on the GSSG

 

Triumph of Isaac on the GSSG

 

 

 

Bibliography:

 

Avram, Horea. 2013. “Augmented Reality.” In Theorizing Visual Studies: Writing Through the Discipline, edited by James Elkins, 88-92. New York: Routledge.

Azuma, Ronald T. 1997. “A Survey of Augmented Reality.” In Presence: Teleoperators and Virtual Environments 6, Vol. 4 (August), 355-385.

Barba, Evan. 2014. “Toward a Language of Mixed Reality in the Continuity Style.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 41-54.

Barthes, Roland. 1989. The Rustle of Language. Berkeley: University of California Press.

Bimber, Oliver and Ramesh Raskar. 2005. Spatial Augmented Reality: Merging Real and Virtual Worlds. Natick: A. K. Peters.

Bolter, Jay, Blair MacIntyre, Maribeth Gandy, and Petra Schweitzer. 2006. "New Media and Permanent Crisis of Aura." Convergence 1, Vol. 12, 21-39.[i]

Bourdieu, Pierre. 1990. Photography, A Middle-brow Art.Stanford: Stanford University Press.

Engberg, Maria and Jay David Bolter. 2014. “Cultural Expression in Augmented and Mixed Reality.”In Convergence: the International Journal of Research into New Media Technologies 1, Vol. 1, 3-9.

Fahle, Oliver. 2006. “Augmented Reality - Das partizipierende Auge.” In Das Spiel mit dem Medium: Partizipation, Immersion, Interaktion, edited by Britta Neitzel and Rolf F. Nohr, 91-103. Marburg: Schüren.

Fuchs, H. and J. Ackerman. 1999. “Displays for augmented reality: Historical re-marks and future prospects." In Mixed Reality. Merging Real and Virtual Worlds, edited by Y. Ohta and H. Tamura, 31-40. Ohmsha(Tokyo): Springer Verlag(Berlin).

Furht, Borko. 2011. Handbook of Augmented Reality. New York: Springer Science+Business Media, LLC.

Geroimenko, Vladimir. 2014. Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium. Cham : Springer International Publishing.

Gould, Amanda Starling. 2014. “Invisible Visualities: Augmented Reality Art and the contemporary Media Ecology.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 25-32.

Grimshaw, Mark. 2014. The Oxford Handbook of Virtuality. Oxford: Oxford Univ. Press.

Gurevitch, Leon. 2014. “Google Warming: Google Earth as eco-machinima.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 85-107.

Holloway-Attaway, Lissa. 2014. “Performing Materialities: Exploring Mixed Media Reality and Moby-Dick.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 55-68.

Hugues, Olivier and Philippe Fuchs and Olivier Nannipieri. 2011. “New Augmented Reality Taxonomy: Technologies and Features of Augmented Environment.” In Handbook of Augmented Reality, edited by Borko Furht, 47-64. New York: Springer Science+Business Media, LLC.

Immendörfer, Mirko. 2005. Der multisensuelle Raum : Prolegomena zum Bezugsverhältnis von Mensch, Raum und Technologie. Siegen, Univ., Diss.

Kipper, Greg and Joseph Rampolla. 2012. Augmented reality: An Emerging Technologies Guide to AR. Waltham: Syngress.

Lichty, Patrick. 2014. „The Aesthetics of Liminality: Augmentation as Artform.“ In Leonardo 4, vol. 47 (August), 325-336. 

Lichty, Patrick. 2009. „The Translation of Art in Virtual Worlds.“ In Leonardo Electronic Almanach 4/5, Vol. 16 (May), 325-336.

MacIntyre, Blair and Jay Bolter and Emmanuel Moreno. 2001. "Augmented Reality as a New Media Experience." In International Symposium on Augmented Reality, New York: IEEE and ACM International Symposium on Augmented Reality (ISAR'01), 197-206.

Mainzer, Klaus. 2014. Die Berechnung der Welt: Von der Weltformel zu Big Data. München: Beck. [Abs. ‘Von Virtual Reality zu Augmented Reality’, 170]

Manovich, Lev. 2003. “The Poetics of Augmented Space.” In New Media: Theories and Practices of Digitextuality, Edited by John T. Caldwell and Anna Everett, 75-92. London; New York: Routledge.[ii]

Mehler-Bicher, Anett and Michael Reiß and Lothar Steiger. 2011.Augmented Reality: Theorie und Praxis. München: Oldenbourg.

Milgram, Paul and Haruo Takemura and Akira Utsumi and Fumio Kishino. 1994. “Augmented reality: A class of displays on the reality-virtuality continuum.” In Telemanipulator and Telepresence Technologies – SPIE Vol. 2351, 282-292.

Milgram, Paul und Herman Colquhoun. 1994. “A Taxonomy of Real and Virtual World Display Integration.” In IEICE Transactions on Information Systems , No.12, 1-15.

Panofsky, Erwin 1991. Perspective as symbolic form. New York: Zone Books.

Papagiannis, Helen. 2014. “Working towards defining an aesthetics of augmented reality: A medium in transition.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 33-40.

Papagiannis, Helen. 2011. “The Role of the Artist in Evolving AR as a New Medium.” In Symposium On Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH), IEEE International, 61 – 65.

Rhodes, Geoffrey Alan. 2014. “Augmented Reality in Art: Aesthtics and material for Expression.” In. Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium, edited by Vladimir Geroimenko, 127-137. Cham: Springer International Publishing.

Ross, Christine. 2009. “Augmented Reality Art: A Matter of (non)Destination.” In Proceedings of the Digital Arts and Culture Conference. Retrieved from: escholarship.org/uc/item/6q71j0zh

Ross, Christine. 2005. “New Media Arts Hybridity: The Vases (Dis)communicants. Between Art, Affective Science and AR Technology.” In Convergence 4 Vol. 11, 32–42.

Schraffenberger, Hanna and Edwin van der Heide, E. 2014. “The Real in Augmented Reality.” In xCoAx 2014: Proceedings of the Second Conference on Computation, Communication, Aesthetics and X, edited by Carvalhais and M. Verdicchio, 64–74.

Schraffenberger Hanna and Edwin van der Heide. 2013. “Towards Novel Relationships between the Virtual and the Real in Augmented Reality.” In Arts and Technology. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, Volume 116, 73-80. New York: Springer.

Stapleton, Christopher and Jim Davies. 2011. “Imagination: The Third Reality to the Virtuality Continuum.” In 2011 IEEE International Symposium on Mixed and Augmented Reality. (ISMAR-2011), 53-60. Basel, Switzerland.

Summers, David. 2003. Real Spaces: World Art History and the Rise of Western Modernism. New York: Phaidon Press. 

Thiel, Tamiko. 2014. “Critical Interventions into Canonical Spaces: Augmented Reality at the 2011 Venice and Istanbul Biennials.“ In Augmented Reality Art: From an Emerging Technology to a Novel Creative Medium, edited by Vladimir Geroimenko, 31-60. Cham: Springer.

Tinnell, John. 2014. “Computing en plein air : Augmented reality and impressionist aesthetics.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 69-84.

Tönnis, Marcus. 2010. Augmented Reality: Einblicke in die erweiterte Realität. Berlin: Springer.

Veenhof, Sander. 2014. “Locally global.” In Convergence: The International Journal of Research into New Media Technologies 1, Vol. 20 (February), 10-13.

 

List of Figures:

Fig. 1: Tamiko Thiel, ‘Transformations’ (screenshot), 4-channel video installation and interactive online map, 2012.

Fig. 2: Joseph Friedrich Leopold, Heilige Sippe im Tempel, copper engraving, undated.

Fig. 3: Dirk Volkertsz, Triumph des Isaak, Sohn Abrahams, copper engraving (cropped), 1564.