Synopse

• Setting the stage
• Relevance and broader impact
• Research design and methodology
• Agenda
• Appendix
• Bibliography

Up to this day, little advance has been made in studying the material conditions of what makes video game graphics - well - video games graphics. The difference to other visual media lies in the amalgamation of digital computing, and the expression of productive intent or creativity by people. The specifics of video game graphics are deeply rooted in how human ideas must be translated into instructions that a computing system understands, into computer code. Programming computers in return offered novel ways of expression.

Setting the stage

The 1980es marked the arrival of the home-computer. It was a time when computing systems became affordable and got marketed for consumers. Early 8-bit systems, such as the Atari, the Commodore 64 or the Apple II, quickly became popular and opened the door for digitality to enter the home. This period also marks a first step towards the democratization of digital creation. The computing systems were often equipped with software to write and edit text, and create computer graphics or music. All of these systems came as well with various features to write code and software programs. Whereas the former were adaptations of traditional approaches to creative expression into the digital realm, the latter represented a novel form of creation.

Writing code is the base to the development of software-based tools such as graphic and music programs. Programming also paved the way to a new form of media-amalgamation, video games. Starting out as interactive fiction, the 1980es computer systems were able to add graphics, animation and sound to the mix, all glued together by code. Up to this day, but even more so back then, programming and writing code for software or video game development can be a challenging act. It’s time-consuming and has a step learning curve. Acquiring the necessary knowledge has certainly become easier in the last forty years. Whereas learning to code is seen as acquiring a valuable skill set these days, this was not necessary the case in the 1980es.

It’s natural to ask, why did people expose themselves to these adversities in their creative expressions? Why did video game developers choose this path over, say, creating a comic, writing fan-fiction or developing a role-playing game campaign?

Research Interest

I’m interested about practices of video game graphics programming within the context of video game graphics in the 1980ies and 1990ies. My interests focus not only on how technical and visual aspects entangle in the image of video games, but also why designers pursue such an approach over other media.

I have followed a programming practice spanning at least two decades. My interest got sparked by the realization that the things visible on screen are produced by code. Next to being a professional software developer, I have extensively inquired coding as a creative practice. This vantage point led to a career with coding on one side and design studies on the other. This path culminates in this dissertation. I want to do research on visuality, bringing the analysis of underlying technologies and the visible image together. Within that framing, I am particularly interested in programming to create images, as a designerly act, and how working in one modality (abstract code) influences the other modality (the visible image). The image in video games has properties that can be found in dynamic data visualization or video game graphics programming, and is a fitting study object to satisfy my interests.

Research question

Being a cumulative dissertation, this project feels more like a lesser charted territory that needs to be traversed and mapped out, than the definition of a precise question that can, akin to a scalpel, dissect research material. I will try nonetheless to outline an overall guiding question, with the added disclaimer, that it will develop over time and research. More precise inquiries will be worked out in the papers that lead up to the final thesis.

To what extent were the technological foundations of early video games and their visuality intertwined, influencing and shaping practices of graphics design? How do game developers leverage programming to shape the visual aesthetics of video games, and what implications do this approach hold for the intersection of technology and aesthetics in game design?

Problem statement

In my thesis, I assume a hitherto invisible rift between the research on the image of the digital game and its technological base. The image of digital games brings with its own specificities. The constituted image, visible on the screen, is being born from the hardware’s calculation, which in turn are triggered by program code. It is interactive and holds within itself a plurality of different futures, actualizing itself upon the player’s input. Research on the image of digital games more often than not focuses either on the constituted and visible image, or its technological origins.

Following my assumption is the thesis that there is a barely acknowledged interplay between the techno-historic limits of digital game development and the video games’ graphics (Blankenheim 2023), with implications on design practices. Working on this research question demands an interdisciplinary approach that can handle both domains, technology and graphics, halves of a proposed whole. The methods used in this project are lent from digital humanities, design research and video game studies and assembled into an interdisciplinary methodology of critical reading the visuality of digital games. Such an approach opens up our research objects to a plethora of interpretations and new insights.

Although the specifics of the digital game’s visuality have been anticipated and considered (Flusser 2011), little research has been conducted on the interplay and entanglement of a game’s technological structuring through source code and the image seen on screen. While the technological foundations of video game graphics are naturally not the only important aspect, they play a key role. Other aspects will be considered where necessary. Technology is a means to an end but also entangled into socio-politic relations and part of power dynamics and interpersonal processes.

Research gap

When we are interested in an author’s work, we usually study her texts, their content, the context of the work, her life, and more. Analogously applied to my research, I am interested in the author’s writing tools and how these tools relate to the author, the writing and the work. The pen and the typewriter can both help to write texts. However, both are structurally formative for work and texts in their own way. This interplay of aesthetics and technology in video game graphics has rarely been researched through the aspect of code and programming yet.

Graphics programming frameworks, Such as Design By Numbers (and its successor Processing) or Graphics BASIC, aimed to make programming accessible as a design tool. The designers who work with code primarily design the code or the resulting programme, which in turn generates the graphics (Willumsen 2016). Programming is often presented as a purely technical act, which has to do with the level of abstraction of code as well as the understanding of machines (Marino 2020). However, the act of programming is far more chaotic, riddled with frustration, accompanied by debugging of incomprehensible errors, and, most importantly, complexities that elude a single human being. The acquisition and virtuoso execution of video game graphics programming takes a lot of time and a high tolerance for frustration.

I am interested in this act of translation from one mode (code) into another (graphics) as well as the translation as an event itself. Early video game graphics development is comparable to today’s practices of video game graphics programming or digital information graphics. Designers use code to create reactive and dynamic systems that are participative and dynamic. There is only little research on these two foci, especially seen through design research and regarding early video game development.

Relevance and broader impact

Original contribution to science

• multimodal inquiry of digital artefacts with a focus on programming practices, program code, and video game graphics design #todo

Historical design analysis of the visuality of Swiss video games

This objective connects the dissertation to the research project Confoederatio Ludens and its research goal of analysing the history of Swiss video game design and development. The focus is laid on the analysis of the visual material. The goal is to identify flows and novelties, pop-cultural references and visual discourse regarding the visuality and how those relate to Switzerland as a historic unity. This dissertation adds to the interdisciplinary research project by sharing its findings. Part of this research objective is the creation of three corpora, and their analysis through design rhetoric and research on the image.

Bridging the analysis of visuality and technology in video games

As outlined in the introduction, this dissertation attempts to bridge the analysis of two different domains, which forms the second research objective. Code and visuality are interrelated and part of the feedback loop between game, machine and player. This objective is distinguished by being researched through a methodological approach, the goal being testing and examining the possibilities of a joined analysis of the two halves of this proposed whole.

The main discourse in video game research plays out on a spectrum between the game’s narrative, or its ludic qualities. Analyses geared either towards games being literature or a bundle of play mechanics. Only recently has the approach of games as images gained some traction again (???; ???). A major takeaway from the disciplines’ origin story, the dispute between narratology and ludology, was the need for interdisciplinary approaches in order for a fuller analysis of digital games.

The design theoretician Vilém Flusser worked extensively on the way we produce images. He started to differentiate between traditionally handmade images and technical images, which were produced through technical means, like an apparatus. A key aspect of technical images is the continued abstraction between designing and the designed. Instead of drawing an image manually, where the act of drawing and the result on paper are aesthetically intertwined, the designer of an image operates technical means, which in turn then produce the image. Flusser’s work included digitality, especially digital photography, but did not extend to the study of video game graphics programming, or the visual design of digital games through programming.

Next to furthering the studies on visuality in digital games, this research object attempts to bridge inquiries on the images seen on screen and the analysis of the images’ underlying structures, the code that generates them. The underlying thoughts were outlined in the introduction.

Critical code studies on video game graphics programming in video game graphics in the 1980ies

It is hardly possible to see beyond the horizon of transformation from code to image. Looking at the visuality, one can only make assumptions about the code and vice versa. This has a profound impact on designing visuality through code. The third objective wants to research this specific mode of visual designing through programming, as well as the consequences, impact and effects of this momentum of transformation from code to visuality. This means that I will tie into Flusser’s theory on the technical image and use that vantage point to inquiry this concept in the context of digital games visuality.

The 1980s marked a pivotal era in bringing computing into the household. Video games serve as an ideal focal point for examining the amalgamation of programming and creative expression during this period. With programming skills not yet recognized as a well-established career path and lacking broader social recognition, code can now directly be viewed as human-produced artefact, fostering creative expression rather than solely serving functional purposes. This objective leans heavily towards studying coding for creative expression those beginnings.

Broader impact

Engaging in research on early video game graphics programming practices offers valuable insights into our contemporary relationship with digital technologies. By understanding the origins of coding as a form of creative expression, we can critically evaluate the current state of digital technology and its impact on society. In today’s world, technologies like machine learning and large-scale server infrastructures consume vast amounts of planetary resources. This consumption raises concerns about sustainability, especially when juxtaposed with seemingly frivolous activities such as spending excessive time on social video platforms. We must reconsider how we allocate resources and recognize the importance of preserving the planet for future generations.

Coding and software development in the 1980s involved working “closer to the metal,” meaning programmers had to grapple with the intricate constraints of hardware at a fundamental level. This hands-on approach forced developers to optimize their code and find innovative solutions to overcome limitations imposed by the available technology. It’s essential to determine where less complex technology suffices, as this can lead to more sustainable computing practices. By studying and preserving programming techniques that prioritize efficiency and minimal resource consumption, we can mitigate the environmental impact of technological advancements.

Despite the prevalence of modern gaming platforms, there exists a vibrant community dedicated to developing games for older systems such as the Game Boy Advance or the Commodore 64. These developers embrace the constraints of these legacy platforms, demonstrating that innovation can flourish within limitations and fostering a deeper appreciation for the heritage of gaming technology.

This broader perspective aligns with ongoing techno-social movements such as de-growth, post-collapse computing, and permacomputing. These movements advocate for rethinking computing to its essence, prioritizing sustainability and resilience in the face of environmental and societal challenges. By embracing these principles, we can foster a more sustainable and equitable future for computing and society as a whole. It is also related to recent succesful aspirations of the Swiss demoscene to register their practices as living tradition and intangible cultural heritage¹.

Research design and methodology

Research data

The research material consists of three corpora – the digital games list, and its metadata, the visual material regarding those games, and an archive of the source code of the games. Whereas I am actively working collaboratively with the research team on the former, I am solely responsible for the image corpus. From the games list corpus, I will choose a reasonable number of exemplary as well as exceptional games that I will work on in case studies throughout the different papers needed for my dissertation. Exemplary means that some of the games stand in for popular genres of the time, while some others can’t be put in such a schema. Further information on the corpora is outlined in 4.3.2.

Methodology

The selection of methods can be divided in main and contextual approaches. The main methods are focused on and the core of the papers, whereas the contextual methods aid in filling the gaps and shortcomings of the former.

Main Methods

Corpora building. maintenance and analysis

In this dissertation project, I will help build, maintain and analyse three different corpora.

1. A comprehensive list of Swiss digital games until the year 2000 and their accompanying metadata, such as factual information, computing systems, genre, and actors involved in the development and distribution
2. A catalogue of images containing visual material for the games in the first corpus
3. An archive of source code from the games in the first corpus

Metadata

The first consists of the digital games that are researched in the Confoederatio Ludens project. The project will maintain its own dataset, hosted on the Swiss Games Garden platform, which consists of a Drupal installation and a custom frontend. To uphold FAIR/CARE principles, I will maintain a duplicate dataset on Wikidata. Being linked-data and graph-based, Wikidata will enable some machine-assisted research approaches, such as visualizing networks of actors.

Images

The second corpus consists of visual material regarding the digital games from the games list. The images will mainly be taken directly from the games according to a predefined strategy, as well as collected from online archives. The strategy is to include as much material as is necessary to fully represent a games’ aesthetics and content. Next to formal elements, a list of the visual content of a game the Game FAVR (see 2.1) model of categorization of the visuality of a digital game can help to obtain the right material. Since the focus of the dissertation is on the interplay of code and image, paratextual material such as box arts or advertisements will only be collected for case studies.

• FAVR: Visual modes, ocularizations, framing mechanisms, construction of tangible space
• Formal Elements: Colours, textures, shapes, composition and layout, typography, etc.
• Semiotic Elements: Characters, objects, levels, interactive elements, etc.
• Relevant paratexts: Box art, manuals, advertisement, etc.

More information about the thoughts that went into the creation of the image corpus can found on my dissertation wiki under Image Corpus as well as 4.4.2.

Source code

As mentioned in 1. and 3.1.2, the source code of the researched games is of special interest to me. Critical code analysis treats code not only as functional instructions to a machine, but also as a text that can be hermeneutically approached and semiotically read (Marino 2020). Further, from the perspective of design theory, a digital game is a second order design (Willumsen 2016). The game developers don’t design the game, they design the structures that generate the game. Akin to the writing aid for an author, like pens or keyboards, source code has formal and structural influence on the what and how the game developers design. Taken together, these two approaches outline source code as a carrier of meaning, as well as the tools and material for designing games. This makes source code a core focus of my dissertation.

Working with source code can be difficult. Source code is text, but can reach levels of abstraction down to being a string of seemingly random alphanumeric strings, in the case of Assembler. A certain level of knowledge and experience is needed to fruitfully dive into source code with close readings. I have made first attempts in critical code analysis, as mentioned under 2.3.3. Distant readings, in which the digital humanities are especially strong, are yet to be established when approaching source code hermeneutically.

To read source code, one needs to obtain such, which is another difficulty. It is yet to be seen, if I will be able to acquire the source code of the games we research. Games written in Basic, which is not always compiled to machine code, as well as magazine listings have their source code directly accessible. But, these games only make a small portion of our corpus. In the case of games that were compiled, and where the source code is not directly accessible, we are dependent on the personal archives of the game developers. Decompiling the games is a possible approach, but the produced source code is an interpretation by the machine and not comparable to the original code written by the developers. Further thoughts on this problematic can be found in 4.4.1.

Archiving digital game source code could be done easily, for example, by setting up version control repositories. Working with source code academically is another problem. Legal issues, access rights, long-term archiving, referencing specific pieces of code in publications, and other issues arise. This dissertation will not be able to tackle all of them, but use the source code corpus pragmatically.

All three corpora are built and maintained with FAIR/CARE principles in mind. The metadata in the corpus on Swiss digital games will be regularly published as linked open data on Wikidata. The catalogue of images will be worked on in Tropy and disseminated in an Omeka archive, as well as on Zenodo. The archive of source code has no clear long-term path yet, but efforts are being made to create sustainable digital games archives in Switzerland. Cooperation with these archives is desirable.

Critical code analysis

Critical code analysis describes source code for software not only as functional instructions for machines, but also as cultural artefacts written by humans for humans. This means, that source code could potentially be read to acquire knowledge about socio-cultural and historical contexts. To what extent source code can be treated as a literary text, or leaning more on the Foucauldian notion of text, has been tested and described in 2.4.4. First preliminary results indicate that source code is rich in knowledge, but difficult to access through digital humanities distant readings approaches.

There are many different aspects, or aggregate states, of source code that could be analysed. Critical code analysis traditionally seems to lean towards code as literature, reading it rather literally, and trying to stay as close to the original code written than possible. This dissertation is interested in source code as the device for designing video game graphics. This implies that other aggregates could be of interest, as long as the designers’ intentions and practices are still legible.

The initial comparison between and author’s tools to write and source code doesn’t hold up completely. Source code is not merely the material means of writing. Source code contains aspects of the pen, the paper, and the words written on the latter. While being a means to an end, writing code to generate video game graphics, it is also a documentation about design practices and technical affordances, which is the second important focus of critical code analysis in this dissertation.

The third and last focus of this method’s application in this dissertation is on the relation between code and video game graphics as the designed output of a programming design practice. This interconnection was outline in 4.3.5 through design rhetoric.

Design research Design rhetoric examines the rhetorical principles in designed artefacts regarding their intentions and effects. Rhetorical design analysis according to the Bernese model (Scheuermann 2017) attempts to bridge the gap between a close study of designed objects and the discourse (social, political, psychological) that they interact with. This approach then tries to bring together intimate expert-knowledge on why things are designed as they are and how these designs relate to and shape the world at large.

Design rhetoric is at once interested in the designing, the designed, and the effects of both. It as an interdisciplinary approach that can profit from interactional expertise, the ability to converse expertly in more than one discipline (???) and bridging formal analysis, hermeneutics and collaborative elicitation. There is a tension between the intentions of the developers and graphic artists, that are deposited into the source code, and the effects created in the image on screen. Design rhetoric’s interdisciplinary approach is in favour of bringing together these two separate domains and will accompany me throughout the dissertation.

An important aspect of the Bernese model is a focus on formal aspects of the designed as well as production processes. This favouring of a formal analysis helps in transcending approaches which mere state technical effects or capabilities, while not reducing the analysis to discourse alone. Design rhetoric after the Bernese model aids in capturing technical aspects but also reflecting on the influence of a reduced colour palette, i.e., on design practice and the designed, leading up to a discourse analysis.

Contextual Methods

Literature Review

Relevant literature has been covered in point 2.1. The literature review was key to identifying the research gap outlined in point 2.2. Further literature review will be conducted towards visuality in video games from a media theoretical perspective as well as the notion of text regarding program code, as well as upcoming subjects where needed.

Case studies

While the larger corpora can aid in identifying patterns in design and production, a set of case studies will help in qualitative and in-depth analysis of code and design practices. The case studies will be defined during corpora-building, at the beginning of the dissertation and will be worked on through the proposed interdisciplinary approach including design rhetoric, critical code analysis and oral history, as well as creating context through the HGP-method and discourse analysis. All of these are outlined in the following. The case studies should offer data and research material for all of these methods.

Horror Game Politics Method

The HGP method is a historical, source-critical approach to studying video games, which focuses on analysing production, product, and reception. Developed by Eugen Pfister and Arno Görgen, this method aids in reconstructing the original context through production-, media- and reception analysis.

Since I will study the visuality as well as the source code of digital games mainly from the 1980ies and 1990ies, it is important to situate the analysis properly in its historical context. The HGP method is building on Roland Barthes approach to myth-making, and was applied to reading into the political myth of horror games. This aspect of the HGP method will help me in the hermeneutic analysis of case studies.

Discourse Analysis

Discourse analysis is a method of analysing language and communication use in its social context. It goes beyond examining formal aspects to understand how language functions in communication and how it shapes and is shaped by social structures, power relations, and cultural norms. Discourse analysis will be applied in the case studies to draw out from formal analysis into the larger dynamics of effects and meaning-creation of the researched video games. This is in part to root this dissertation in the research project that it is part of, as well as to the overarching research inquiry into the early history of Swiss video game development.

Oral History

Oral history is a method of preserving and conveying historical information through spoken accounts, personal narratives, and first-hand experiences. This approach is part of the Confoederatio Ludens project, where we focus on interviews with key people in the video game industry of the 1980ies and 1990ies, including developers, publishers, or people that worked on electronics, among others. Unlike written history, which relies on written records and documents, oral history captures the memories and perspectives of individuals who have witnessed or participated in historical events. As such, they can add important perspectives and context to the video games that this dissertation is inquiring through case studies.

Theoretical position

The theoretical vantage point of this dissertation will be Vilém Flusser’s theory of the technical image. In his thinking, he started to differentiate between “traditional” and “technical” images (Flusser 2011). The latter do not necessarily represent reality, but are constructed and in turn construct reality. The “technical” image alienates the viewer from direct experience and creates its own world. Flusser’s work can be ascribed to critical theory, as he did not only ponder on the subject, but also warn of possible problems arising from the rise of the “technical” image.

Flusser’s theoretical approach will be expanded by Marino’s theories on code as a cultural artefact (Marino 2020). Critical code analysis is not only a methodological approach but also a theoretical perspective, in which code is not only textual instruction for machines. Code is also a socio-cultural artefact that can be approached through hermeneutics. Code is comprehended not just by programmers, but also by lawyers, artists, journalists, political activists, and literary scholars. Its utility extends beyond mere programming and finds application in political discourse, artistic creations, popular culture, and the documentation of history.

Possible problems

In the following I will reflect on possible weaknesses in the outlined research design as well as how to mitigate the risks.

Critical code analysis

Although researching context is key to this methodology, its main focus is source code. In the case of old digital games, such source code can be difficult to obtain. Problems can arise, for example if the original developers can’t be contacted, don’t want to share their code base or don’t have it any more. It is certain that some source code can be found, although maybe not of the originally targeted games. As a fallback, games from beyond the scope of the corpus could be included to find material for this approach. Being able to research the interplay of technology and visuality in our Swiss digital games corpus would be the best possible outcome.

It is one of these dissertations appeals to open data: To study digital games, we need not only their compiled executable versions, but also their raw source material, their game code (???; Willumsen 2016).

The dynamics of the video game image

Video game images are among the most complex images to research. In comparison to a still image, video game images add the two dimensions of time and interactivity, leading to a large amount of possible image material. Studying video game graphics can quickly overstrain the available resources, making it necessary to reduce complexities and make choices regarding corpus saturation. Approaches in digital humanities, such as large corpus analysis and distant reading, can help to tackle some of these problems. Reducing the number of games analysed is applied to create a feasible workload. Finally, some compromises have to be made in choosing the image material, which are outlined in the considerations regarding the image corpus

Agenda

In the following I want to summarize my intentions and how they formulate the schedule and output. My research concentrates on structural aspects of video games as a combination of computing and creative expression, particularly focusing on the era of the 1980s and 1990s. I attempt to highlight the democratization of digital creation with the advent of affordable home computers like the Atari and Commodore 64. This means I will explore the role of programming in video game development, noting its challenges and its evolution as a valuable skill over the past decades and raise questions about why developers choose coding as a medium for creative expression in video game graphics, compared to other forms of media.

To pursue my research I will bring together two disciplines, the digital humanities and design research, as well as two methodological approaches, which are critical code studies and visual design analysis. The process leads through four working pages, stages or papers.

1. Creating and analysing video game metadata and image corpora to produce an overview of the problematic at hand and the basis of further research
2. Distant reading video game source code in order to deepen the technological means of production
3. Studying the specifics of ported games, which highlight the different technical needs for the same kind of output
4. And finally, bridging the methodological approaches of critical code studies and design research to shed light on video game graphics programming practices in the 1980ies and 1990ies

Timetable Projection

These are estimates and projections which might change during the actual research and writing process.

1. Paper - Metadata, Imagedata and Videogame Graphics

2. Paper - Distant Reading Source Code

• Paper on methodology of working on source code as a research object with a distant reading approach
• Development of video game graphics coding and programming languages in the 1980ies and 1990ies; ie creative application and designerly practices around graphics modes in early home computers
• Alliance of Digital Humanities Organizations – A Global Coalition of Digital Humanities Organizations

3. Paper - Case Study Ported Games

• Design research in the context of ported games, which were perceived as the same game on different computing system, but differing massively in graphics.

4. Paper - Bridging Design Research and Critical Code Analysis

• Bringing together case studies and my own source code study
• Paper on design research and critical code analysis of early video game graphics regarding the case studies
• Report on proposed interdisciplinary approach to study video game graphics through design rhetoric and critical code analysis

Appendix

• Visuality
• Visuality in early video games

Bibliography