Procedural visuality

Interdisciplinary perspectives on en/decoding the visuality of early Swiss digital games

Cumulative dissertation in the digital humanities

How were graphics programming and the visuality of 80s and 90s video games entangled? How did they influence and form each other? And, how does graphics programming relate to the designer, and the designing, of video game images?

• Adrian Demleitner
• adrian.demleitner@hkb.bfh.ch
• https://orcid.org/0000-0001-9918-7300

Table of Contents

• 1. Introduction and problem statement
• 2. Current state of research
  • 2.1 State of research
  • 2.2 Research gap
  • 2.3 Own research
• 3. Objectives and research questions
  • 3.1 Objectives
  • 3.2 Research interest
  • 3.3 Research question
• 4. Research design and methodology
  • 4.1 Research data
  • 4.2 Cumulative dissertation
  • 4.3 Methodology
  • 4.3 Theoretical position
  • 4.4 Possible problems
• 5. Timetable and pursued output
  • 5.1 Timetable Overview
  • 5.2 Projected tasks and output
• 6. Appendix
  • 6.1 Visuality
  • 6.2 (Historical) Visuality in digital games
    • 6.2.1 Ergodicity
    • 6.2.2 Vocabulary of analysis
    • 6.2.3 Techno-historic limits
• 7. Bibliography

1. Introduction and problem statement

Digital games are manifestations of designerly intentions, planned execution of a design into a final product. These games communicate much more than the intentioned designs. They also capture unspoken and subconscious assumptions, bias and ideologies, as well as historical contexts and designerly practices. In the following, this is also true for the graphics of these video games, on which this proposal is concentrating.

In my thesis, I assume a hitherto invisible rift between the research on the image of the digital game and its technological base. The image of digital games brings with its own specificities. The constituted image, visible on the screen, is being born from the hardware’s calculation, which in turn are triggered by program code. It is interactive and holds within itself a plurality of different futures, actualizing itself upon the player’s input. Research on the image of digital games more often than not focuses either on the constituted and visible image, or its technological origins.

Following my assumption is the thesis that there is a barely acknowledged interplay between the techno-historic limits of digital game development and the video games’ graphics (Blankenheim 2023), with implications on design practices. Working on this research question demands an interdisciplinary approach that can handle both domains, technology and graphics, halves of a proposed whole. The methods used in this project are lent from digital humanities, design research and video game studies and assembled into an interdisciplinary methodology of critical reading the visuality of digital games. Such an approach opens up our research objects to a plethora of interpretations and new insights.

Although the specifics of the digital game’s visuality have been anticipated and considered (Flusser 2011), as outlined in 2.1, little research has been conducted on the interplay and entanglement of a game’s technological structuring through source code and the image seen on screen. While the technological foundations of video game graphics are naturally not the only important aspect, they play a key role. Other aspects will be considered where necessary. Technology is means to an end but also entangled into socio-politic relations and part of power dynamics and interpersonal processes.

2. Current state of research

2.1 State of research

This dissertation is situated within image studies, while also being interested in specific aspects of technology and contextualized by video game research. This makes the digital humanities as well as design research fitting umbrella disciplines, combining critical approaches from the humanities with technological inquiries, as well as questioning the designers’ involvement and practice.

2.2 Research gap

When we are interested in an author’s work, we usually study her texts, their content, the context of the work, her life, and more. Analogously applied to my research, I am interested in the author’s writing tools and how these tools relate to the author, the writing and the work. The pen and the typewriter can both help to write texts. However, both are structurally formative for work and texts in their own way. This interplay of aesthetics and technology in video game graphics has rarely been researched through the aspect of code and programming yet.

Graphics programming frameworks¹ aimed to make programming accessible as a design tool. The designers who work with code primarily design the code or the resulting programme, which in turn generates the graphics (Willumsen 2016). Programming is often presented as a purely technical act, which has to do with the level of abstraction of code as well as the understanding of machines (Marino 2020). However, the act of programming is far more chaotic, riddled with frustration, accompanied by debugging of incomprehensible errors, and, most importantly, complexities that elude a single human being. The acquisition and virtuoso execution of creative coding takes a lot of time and a high tolerance for frustration.

I am interested in this act of translation from one mode (code) into another (graphics) as well as the translation as an event itself. Early video game graphics development is comparable to today’s practices of creative coding or digital information graphics. Designers use code to create reactive and dynamic systems that are participative and dynamic. There is only little research on these two foci, especially seen through design research and regarding early video game development.

Further reading is provided in 3. Objectives and research questions.

2.3 Own research

Recent case studies have helped me in formulating my research focus as well as questions. In the following I want to highlight three of these cases: working on video games corpora, analysing the visuality of games, and finally critically reading source code. All three case studies have been expanded upon in blog posts, articles or book contributions.

2.3.1 Corpora

To analyse the diverse digital-born research objects at hand, I have to create, maintain and analyse several corpora. More information on this dissertations material and the accompanying methods are outlined under 4.3.2. I was able to already start this process in the case of metadata regarding our list of digital games, as well as an image archive.

The work on metadata involved collaboratively working on a digital games list for the German-speaking countries², being involved in the maintenance and development of Swiss Games Garden³, and importing our games’ metadata into Wikidata⁴. The work on the metadata corpus was oriented towards pragmatically working with the data, and as such, is a work in progress. The metadata needs further refinements and has to be expanded. This case helped in getting an overview of what data is present and in which quality it can be acquired.

A lot of consideration has gone into the image archive⁵. The archive is focused on screenshots from our games corpus and some of their paratextual material. The considerations on how to structure and create such a corpus as well as the practical work on collecting the images has helped tremendously to get a basic idea of what image material will be available for the analysis through distant reading and design rhetoric. Part of the work in the image corpus was the development of an ontology to apply in Tropy, based on FAVR, which is an important aspect for creating proper metadata in the digital humanities.

2.3.2 Image research

I have been able to carry out two case studies regarding image research and design rhetoric. The first focused on analysing the visuality of a video game, and the second on a visual trope in digital games from the 1980ies.

In the first case study, I had a good look at the images from Ball Raider⁶, a game developed in Switzerland and published in 1987. The analysis found its way into an upcoming post on the Confoederatio Ludens blog⁷. The close reading of the visuality of the game helped form a hypothesis on the game, as well as its developers and designers. Part of the analysis was an interview with the graphic artist, as well as looking at formal and semantic aspects of the images present in the game. This case study initiated an inquiry into the visual trope of the bare chested barbarian.

This second inquiry focused on a handful of digital games, some of which were not developed in Switzerland, but have been formative for the design of later games. The output of this analysis⁸ will find its way into a post for the Confoederatio Ludens blog. The study is done in collaboration with Aurelia Brandenburg, who is also a PhD-student and researcher in the project I am taking part. While she is focusing on the games reviews found I video game magazines, I investigated why those games barbarian characters look like they do, as well as those looks effect on meaning-making. I developed a first hypothesis, was able to trace the historical development of the visual trope, and made first attempts in combining close and distant reading of visual material.

I have collected the material used in this case study in a repository.

Both studies helped in getting a better understanding of image research and design rhetoric regarding older video games.

2.3.3 Critical Code Analysis

The last case study was an attempt to critically read the source code of an Emulator⁹, VICE for the Commodore 64. The intent for this exploration was twofold – figuring out what kind of text source code is and using it as the epistemological vantage point to research a project and its contextuality. I tried again to combine close and distant reading. Hermeneutic distant reading of source code is not an established practice, and I considered this approach experimental.

The work on this inquiry helped me get a better view of the possibilities and shortcomings of critical code analysis. This method is key to my research and is outlined in 4.3. Especially experimenting with a distant reading approach was fruitful to see what kind of text source code is. The results of this study found their way into an upcoming book, published by a working group around the AG Spiele of the DHd.

Scripts and output of this study are collected in their own repository.

3. Objectives and research questions

3.1 Objectives

This dissertation has the following interrelated research objectives.

3.1.1 Historical design analysis of the visuality of Swiss video games

This objective connects the dissertation to the research project Confoederatio Ludens and its research goal of analysing the history of Swiss video game design and development. The focus is laid on the analysis of the image material. The goal is to identify flows and novelties, pop-cultural references and visual discourse regarding the visuality and how those relate to Switzerland as a historic unity. This dissertation adds to the interdisciplinary research project by sharing its findings. Part of this research objective is the creation of three corpora, which I outline and detail under 4.3.2, and their analysis through design rhetoric and research on the image.

3.1.2 Bridging the analysis of visuality and technology in video games

As outlined in the introduction, this dissertation attempts to bridge the analysis of two different domains, which forms the second research objective. Code and visuality are interrelated and part of the feedback loop between game, machine and player. This objective is distinguished by being researched through a methodological approach, the goal being testing and examining the possibilities of a joined analysis of the two halves of this proposed whole. Details on the joint-methodology of design research and critical code analysis can be found under 4.3.

The main discourse in video game research plays out on a spectrum between the game’s narrative, or its ludic qualities. Analyses geared either towards games being literature or a bundle of play mechanics. Only recently has the approach of games as images gained some traction again (Fizek 2022; Gerling, Möring, and De Mutiis 2022). A major takeaway from the disciplines’ origin story, the dispute between narratology and ludology, was the need for interdisciplinary approaches in order for a fuller analysis of digital games.

Next to furthering the studies on visuality in digital games, this research object attempts to bridge inquiries on the images seen on screen and the analysis of the images’ underlying structures, the code that generates them. The underlying thoughts were outlined in the introduction.

3.1.3 Discourse on multimodality of coding for visuality

It is hardly possible to see beyond the horizon of transformation from code to image. Looking at the visuality, one can only make assumptions about the code and vice versa. This has a profound impact on designing visuality through code.

The design theoretician Vilém Flusser worked extensively on the way we produce images. He started to differentiate between traditionally handmade images and technical images, which were produced through technical means, like an apparatus. A key aspect of technical images is the continued abstraction between designing and the designed. Instead of drawing an image manually, where the act of drawing and the result on paper are aesthetically intertwined, the designer of an image operates technical means, which in turn then produce the image. Flusser’s work included digitality, especially digital photography, but did not extend to the study of creative coding, or the visual design of digital games through programming.

The third objective wants to research this specific mode of visual designing through programming, as well as the consequences, impact and effects of this momentum of transformation from code to visuality. This means that I will tie into Flusser’s theory on the technical image and use that vantage point to inquiry this concept in the context of digital games visuality.

3.2 Research interest

My interests focus not only on how technical and visual aspects entangle in the image of video games, but also why designers pursue such an approach over other media.

I have followed a programming practice spanning at least two decades. My interest got sparked by the realization that the things visible on screen are produced by code. Next to being a professional software developer, I have extensively inquired coding as a creative practice. This vantage point led to a career with coding on one side and design studies on the other. This path culminates in this dissertation. I want to do research on visuality, bringing the analysis of underlying technologies and the visible image together. Within that framing, I am particularly interested in programming to create images, as a designerly act, and how working in one modality (abstract code) influences the other modality (the visible image). The image in video games has properties that can be found in dynamic data visualization or creative coding, and is a fitting study object to satisfy my interests.

My own interests correlate with missing research on the topic, and I have outlined more thoughts in 2.3.

3.3 Research question

Being a cumulative dissertation, this project feels more like a lesser charted territory that needs to be traversed and mapped out, than the definition of a precise question that can, akin to a scalpel, dissect research material. I will try nonetheless to outline an overall guiding question, with the added disclaimer, that it will develop over time and research. More precise inquiries will be worked out in the papers that lead up to the final thesis.

To what extent were the technological foundations of early video games and their visuality intertwined, influencing and shaping practices of graphics design? How do game developers leverage programming to shape the visual aesthetics of video games, and what implications do this approach hold for the intersection of technology and aesthetics in game design?

4. Research design and methodology

4.1 Research data

The research material consists of three corpora – the digital games list, and its metadata, the visual material regarding those games, and an archive of the source code of the games. Whereas I am actively working collaboratively with the research team on the former, I am solely responsible for the image corpus. From the games list corpus, I will choose a reasonable number of exemplary as well as exceptional games that I will work on in case studies throughout the different papers needed for my dissertation. Exemplary means that some of the games stand in for popular genres of the time, while some others can’t be put in such a schema. Further information on the corpora is outlined in 4.3.2.

4.2 Cumulative dissertation

This is a cumulative dissertation. The thesis takes form through four papers, which have been accepted for publishing in peer-reviewed journals. This means, that not all the outlined methods below are applied during the whole of the project, but only partially in their respective work packages. The final dissertation further needs a synopsis to frame the four papers. More information on when which methods are used towards which inquiry and output is outlined under 5.2.

4.3 Methodology

4.3.1 Literature Review

Relevant literature has been covered in point 2.1. The literature review was key to identifying the research gap outlined in point 2.2. Further literature review will be conducted towards visuality in video games from a media theoretical perspective as well as the notion of text regarding program code, as well as upcoming subjects where needed.

4.3.2 Corpora building and analysis

In this dissertation project, I will help build, maintain and analyse three different corpora.

1. A comprehensive list of Swiss digital games until the year 2000 and their accompanying metadata, such as factual information, computing systems, genre, and actors involved in the development and distribution
2. A catalogue of images containing visual material for the games in the first corpus
3. An archive of source code from the games in the first corpus

Metadata

The first consists of the digital games that are researched in the Confoederatio Ludens project. The project will maintain its own dataset, hosted on the Swiss Games Garden platform, which consists of a Drupal installation and a custom frontend. To uphold FAIR/CARE principles, I will maintain a duplicate dataset on Wikidata. Being linked-data and graph-based, Wikidata will enable some machine-assisted research approaches, such as visualizing networks of actors.

Images

The second corpus consists of visual material regarding the digital games from the games list. The images will mainly be taken directly from the games according to a predefined strategy, as well as collected from online archives. The strategy is to include as much material as is necessary to fully represent a games’ aesthetics and content. Next to formal elements, a list of the visual content of a game the Game FAVR (see 2.1) model of categorization of the visuality of a digital game can help to obtain the right material. Since the focus of the dissertation is on the interplay of code and image, paratextual material such as box arts or advertisements will only be collected for case studies.

• FAVR: Visual modes, ocularizations, framing mechanisms, construction of tangible space
• Formal Elements: Colours, textures, shapes, composition and layout, typography, etc.
• Semiotic Elements: Characters, objects, levels, interactive elements, etc.
• Relevant paratexts: Box art, manuals, advertisement, etc.

More information about the thoughts that went into the creation of the image corpus can found on my dissertation wiki under Image Corpus as well as 4.4.2.

Source code

As mentioned in 1. and 3.1.2, the source code of the researched games is of special interest to me. Critical code analysis treats code not only as functional instructions to a machine, but also as a text that can be hermeneutically approached and semiotically read (Marino 2020). Further, from the perspective of design theory, a digital game is a second order design (Willumsen 2016). The game developers don’t design the game, they design the structures that generate the game. Akin to the writing aid for an author, like pens or keyboards, source code has formal and structural influence on the what and how the game developers design. Taken together, these two approaches outline source code as a carrier of meaning, as well as the tools and material for designing games. This makes source code a core focus of my dissertation.

Working with source code can be difficult. Source code is text, but can reach levels of abstraction down to being a string of seemingly random alphanumeric strings, in the case of Assembler. A certain level of knowledge and experience is needed to fruitfully dive into source code with close readings. I have made first attempts in critical code analysis, as mentioned under 2.3.3. Distant readings, in which the digital humanities are especially strong, are yet to be established when approaching source code hermeneutically.

To read source code, one needs to obtain such, which is another difficulty. It is yet to be seen, if I will be able to acquire the source code of the games we research. Games written in Basic, which is not always compiled to machine code, as well as magazine listings have their source code directly accessible. But, these games only make a small portion of our corpus. In the case of games that were compiled, and where the source code is not directly accessible, we are dependent on the personal archives of the game developers. Decompiling the games is a possible approach, but the produced source code is an interpretation by the machine and not comparable to the original code written by the developers. Further thoughts on this problematic can be found in 4.4.1.

Archiving digital game source code could be done easily, for example, by setting up version control repositories. Working with source code academically is another problem. Legal issues, access rights, long-term archiving, referencing specific pieces of code in publications, and other issues arise. This dissertation will not be able to tackle all of them, but use the source code corpus pragmatically.

All three corpora are built and maintained with FAIR/CARE principles in mind. The metadata in the corpus on Swiss digital games will be regularly published as linked open data on Wikidata. The catalogue of images will be worked on in Tropy and disseminated in an Omeka archive, as well as on Zenodo. The archive of source code has no clear long-term path yet, but efforts are being made to create sustainable digital games archives in Switzerland. Cooperation with these archives is desirable.

4.3.3 Case studies

While the larger corpora can aid in identifying patterns in design and production, a set of case studies will help in qualitative and in-depth analysis of code and design practices. The case studies will be defined during corpora-building, at the beginning of the dissertation and will be worked on through the proposed interdisciplinary approach including design rhetoric, critical code analysis and oral history, as well as creating context through the HGP-method and discourse analysis. All of these are outlined in the following. The case studies should offer data and research material for all of these methods.

4.3.4 Horror Game Politics Method

The HGP method is a historical, source-critical approach to studying video games, which focuses on analysing production, product, and reception. Developed by Eugen Pfister and Arno Görgen, this method aids in reconstructing the original context through production-, media- and reception analysis.

Since I will study the visuality as well as the source code of digital games mainly from the 1980ies and 1990ies, it is important to situate the analysis properly in its historical context. The HGP method is building on Roland Barthes approach to myth-making, and was applied to reading into the political myth of horror games. This aspect of the HGP method will help me in the hermeneutic analysis of case studies.

4.3.5 Design rhetoric, Bernese model

Design rhetoric examines the rhetorical principles in designed artefacts regarding their intentions and effects. Rhetorical design analysis according to the Bernese model attempts to bridge the gap between a close study of designed objects and the discourse (social, political, psychological) that they interact with. This approach then tries to bring together intimate expert-knowledge on why things are designed as they are and how these designs relate to and shape the world at large.

Design rhetoric is at once interested in the designing, the designed, and the effects of both. It as an interdisciplinary approach that can profit from interactional expertise, the ability to converse expertly in more than one discipline (Collins and Evans 2009) and bridging formal analysis, hermeneutics and collaborative elicitation. There is a tension between the intentions of the developers and graphic artists, that are deposited into the source code, and the effects created in the image on screen. Design rhetoric’s interdisciplinary approach is in favour of bringing together these two separate domains and will accompany me throughout the dissertation.

An important aspect of the Bernese model is a focus on formal aspects of the designed as well as production processes. This favouring of a formal analysis helps in transcending approaches which mere state technical effects or capabilities, while not reducing the analysis to discourse alone. Design rhetoric after the Bernese model aids in capturing technical aspects but also reflecting on the influence of a reduced colour palette, i.e., on design practice and the designed, leading up to a discourse analysis.

4.3.6 Critical code analysis

Critical code analysis describes source code for software not only as functional instructions for machines, but also as cultural artefacts written by humans for humans. This means, that source code could potentially be read to acquire knowledge about socio-cultural and historical contexts. To what extent source code can be treated as a literary text, or leaning more on the Foucauldian notion of text, has been tested and described in 2.4.4. First preliminary results indicate that source code is rich in knowledge, but difficult to access through digital humanities distant readings approaches.

There are many different aspects, or aggregate states, of source code that could be analysed. Critical code analysis traditionally seems to lean towards code as literature, reading it rather literally, and trying to stay as close to the original code written than possible. This dissertation is interested in source code as the device for designing video game graphics. This implies that other aggregates could be of interest, as long as the designers’ intentions and practices are still legible.

The initial comparison between and author’s tools to write and source code doesn’t hold up completely. Source code is not merely the material means of writing. Source code contains aspects of the pen, the paper, and the words written on the latter. While being a means to an end, writing code to generate video game graphics, it is also a documentation about design practices and technical affordances, which is the second important focus of critical code analysis in this dissertation.

The third and last focus of this method’s application in this dissertation is on the relation between code and video game graphics as the designed output of a programming design practice. This interconnection was outline in 4.3.5 through design rhetoric.

4.3.7 Discourse Analysis

Discourse analysis is a method of analysing language and communication use in its social context. It goes beyond examining formal aspects to understand how language functions in communication and how it shapes and is shaped by social structures, power relations, and cultural norms. Discourse analysis will be applied in the case studies to draw out from formal analysis into the larger dynamics of effects and meaning-creation of the researched video games. This is in part to root this dissertation in the research project that it is part of, as well as to the overarching research inquiry into the early history of Swiss video game development.

4.3.8 Oral History

Oral history is a method of preserving and conveying historical information through spoken accounts, personal narratives, and first-hand experiences. This approach is part of the Confoederatio Ludens project, where we focus on interviews with key people in the video game industry of the 1980ies and 1990ies, including developers, publishers, or people that worked on electronics, among others. Unlike written history, which relies on written records and documents, oral history captures the memories and perspectives of individuals who have witnessed or participated in historical events. As such, they can add important perspectives and context to the video games that this dissertation is inquiring through case studies.

4.3 Theoretical position

The theoretical vantage point of this dissertation will be Vilém Flusser’s theory of the technical image. In his thinking, he started to differentiate between “traditional” and “technical” images (Flusser 2011). The latter do not necessarily represent reality, but are constructed and in turn construct reality. The “technical” image alienates the viewer from direct experience and creates its own world. Flusser’s work can be ascribed to critical theory, as he did not only ponder on the subject, but also warn of possible problems arising from the rise of the “technical” image.

Flusser’s theoretical approach will be expanded by Marino’s theories on code as a cultural artefact (Marino 2020). Critical code analysis is not only a methodological approach but also a theoretical perspective, in which code is not only textual instruction for machines. Code is also a socio-cultural artefact that can be approached through hermeneutics. Code is comprehended not just by programmers, but also by lawyers, artists, journalists, political activists, and literary scholars. Its utility extends beyond mere programming and finds application in political discourse, artistic creations, popular culture, and the documentation of history.

4.4 Possible problems

In the following I will reflect on possible weaknesses in the outlined research design as well as how to mitigate the risks.

4.4.1 Critical code analysis

Although researching context is key to this methodology, its main focus is source code. In the case of old digital games, such source code can be difficult to obtain. Problems can arise, for example if the original developers can’t be contacted, don’t want to share their code base or don’t have it any more. It is certain that some source code can be found, although maybe not of the originally targeted games. As a fallback, games from beyond the scope of the corpus could be included to find material for this approach. Being able to research the interplay of technology and visuality in our Swiss digital games corpus would be the best possible outcome.

It is one of these dissertations appeals to open data: To study digital games, we need not only their compiled executable versions, but also their raw source material, their game code (Konzack 2002; Willumsen 2016).

4.4.2 Video game image complexities

Video game images are amongst the most complex images to research. In comparison to a still image, video game images add the two dimensions of time and interactivity, leading to a large amount of possible image material. Studying video game graphics can quickly overstrain the available resources, making it necessary to reduce complexities and make choices regarding corpus saturation. Approaches in digital humanities, such as large corpus analysis and distant reading, can help to tackle some of these problems. Reducing the number of games analysed is applied to create a feasible workload. Finally, some compromises have to be made in choosing the image material, which are outlined in the considerations regarding the image corpus¹⁰

4.4.3 Image reception

As a historical analysis, research into the reception of the image, an important aspect of design rhetoric, is limited. This shortcoming can be balanced by the Horror-Game-Politics method, which is a source-critical approach that builds on research into the context.

5. Timetable and pursued output

These are estimates and projections which might change during the actual research and writing process.

5.1 Timetable Overview

5.2 Projected tasks and output

5.2.1 Ongoing tasks

• Maintaining corpora for dissertation as well as CH-Ludens
• Applying design rhetoric and visual analysis in CH-Ludens
• Making interviews for oral history project, relevant for dissertation and CH-Ludens

5.2.2 Output for dissertation thesis

WP3 and WP4 depend partially on work done in WP1 and WP2, mainly the gathering of research data, acquisition of knowledge and the definition of case studies. I will, where possible, with case studies and video games from the CH-Ludens corpus.

• WP0: Exposé as well as published material on the case studies
• WP1: Report on the analysis of metadata and corpora regarding the research question, including a reflection on the state of researching video game images in the digital humanities
• WP2:
  • Paper on the creative application and designerly practices around graphics modes¹¹ in early home computers
  • Paper on the development of video game graphics coding and programming languages in the 1980ies and 1990ies
• WP3: Research paper on design rhetoric in the context of ported games, which were perceived as the same game on different computing system, but differing massively in graphics.
• WP4
  • Paper on design rhetoric and critical code analysis of early video game graphics regarding the case studies
  • Report on proposed interdisciplinary approach to study video game graphics through design rhetoric and critical code analysis
• WP5: Synopsis

5.2.3 Additional output

• Video game (image) metadata modelling and ontologies (Wikidata, Tropy)
• Online publication for the exploration of the image corpus and/or the case studies
• Online publication for showcasing creative application of graphics modes and/or the history of video game graphics programming
• Possible co-authored articles
  • Source code as historical material (Daniel Gammenthaler)
  • Working with digital-born archives as source material (Lara Kothe)
  • Designerly implications of creative coding (Max Frischknecht)

6. Appendix

6.1 Visuality

In using the term visuality, I depend on the definitions brought forward by Gillian Rose. Here, visuality refers to the socio-cultural construction of how and what we see (Rose 2016). I’m building on these descriptions as I am interested in the image as a medium of communication, a semiotic device that attempts to transport meaning, and using rhetoric to transform intentions into affects. Coming from design rhetoric, I am interested in how designers and developers of games encode meaning and rhetoric into the images, through the application of code. Gillian Rose herself is building upon the work by Hal Foster, who defined visuality as the “how we see, how we are able, allowed, or made to see, and how we see this seeing and the unseeing therein”. This relates the term visuality to the slightly more complex scopic regime (Jay 2008).

6.2 (Historical) Visuality in digital games

As in every other medium, the visuality in digital games has its own specifics. Being of high relevancy to this dissertation, I’ll outline three of them in the following: ergodicity, vocabulary of analysis and techno-historic limits.

6.2.1 Ergodicity

Digital games are especially demanding of their consumers, as the medium relies on a high level of participation. The difference between reading a book, watching a film or playing a digital game was especially well captured by Espen Aarseth’s application of ergodicity in his book Cybertext. Coming from literature studies, Aarseth speaks of a “nontrivial effort [that] is required to allow the reader to traverse the text” (Aarseth 1997). This needed effort does not guarantee an exhaustive capture of the played game or a successful play-through. The player creates a path through the configuration of a textual machine, by participating in the game.

This ergodicity is an important aspect when considering the image visible on the screen. The image represents one possible state, defined by the elements that the designers of the game provided, configured by the player’s participation. At the same moment, the image on screen communicates choices for the player, who then can reconfigure it by acting within the game. The image holds its own future possibilities by being within a feedback loop between itself and the player. The above outlined process is paralleled by the communicative features of the image, which can also speak of information relevant to the game’s state, such as high scores or player lives, as well as transporting narratives that create context and experience for the player.

6.2.2 Vocabulary of analysis

As a rather young discipline, merely coming of age, video game studies draws from many other disciplines. This is through a lack of an established corpus of methods and approaches form within the discipline, as well as many researchers entering the field from other disciplines. This is as well true for the aspects that concentrating on researching video game studies.

Being intuitively close, a lot of vocabulary and approaches to study video game graphics have come from film studies. As both media rely on images played out over time, this makes somewhat sense. Nonetheless, can the film studies vocabulary only capture parts of video game graphics, while not being to describe others. Video game graphics are at once aesthetic and narrative devices, as well as informative user interfaces. The Framework for the Analysis of Visual Representation in Video Game (FAVR) (Arsenault, Côté, and Larochelle 2015) attempts to tackle this problem specifically and as such is a helpful tool in reflecting how video game graphics are analysed.

6.2.3 Techno-historic limits

The third aspect of relevance is the relationship between the image and its technological structures, from which it springs forth. Being digital born, the image in digital games depends on hardware and software to be seen. These can be considered the material aspect of visuality in digital games. Analogue to Aarseth’s coining of cybertext, Stefan Möring applies the term cyberimage to denote the images’ dependence on its underlying technological structures as well as the player’s interaction with the machine (Gerling, Möring, and De Mutiis 2022).

Today’s digital games playing devices are powerful computers, and a myriad of software and frameworks aid in the development and design. Game design in the early days of game development looked quite different. The capabilities of computers were fairly limited and the technical realization of the games had to be done in early programming languages, such as Basic, Assembly dialects or low-level languages like C. These languages do not offer the comfort and possibilities of today, and needed more abstraction towards the machine. These circumstances form the techno-historic limits of early digital game design and directly influenced formal and semiotic aspects of those games’ visuality. These limits as well as the intimate interplay between the technological foundation as a semiotic system in itself and the visible image will be considered in this dissertation through the application of critical code analysis (Marino 2020).

7. Bibliography