Gesture as the Unit of Analysis for Videogames
Abstract
Videogame analysis inherits its vocabulary from other fields. We talk about mechanics, systems, rules, and verbs, or we borrow the shot, the scene, and the level from film and literature. Each of these names something real, and none of them names the thing a player actually has when playing. This paper proposes Gesture as a unit of analysis native to videogames. A Gesture is the surface area of contact between the player and the videogame, bounded for the purpose of investigation, and experienced as a whole at whatever scale the analyst chooses, from the split second of a jump to the full length of a role playing game. The paper defines the term, distinguishes it from the units it is often confused with, and specifies its internal composition through four components experienced as one thing during play, which are Action, Art, Arc, and Atmosphere. It situates the choices inside those components within a theory of Aesthetic Heritage, and it describes how the boundaries a videogame sets, its Permissions, are discovered through the same components rather than known in advance. Because the author is trained in interaction design, the paper is explicit at the outset about the established meaning of gesture in human-computer interaction, where the word names a mode of physical input, and it shows that the two senses are nested rather than rival. The term's heritage is traced through dance, painting, music, and film theory, and the framework is situated against MDA, Game Feel, procedural rhetoric, and the verb centered vocabulary of contemporary design writing.
Keywords: gesture; videogame criticism; unit of analysis; game feel; embodiment; aesthetic heritage; permissions; human-computer interaction
1The Problem of the Unit
Videogame criticism has never been short of vocabulary. It has been short of a vocabulary of its own. When we want to say something precise about a videogame we reach for words that were built to describe other things. From systems design we take the mechanic, the rule, the system. From film and literature we take the shot, the scene, the act, and the level as a kind of chapter. Each of these words is useful, and each of them describes something that is genuinely present in a videogame. The problem is not that they are wrong. The problem is that they are borrowed, and a borrowed unit carries the priorities of the field it came from.
The trouble runs deeper than borrowing. None of these words describes what a player experiences. A player does not experience a mechanic. A player experiences a moment of play in which a mechanic is one ingredient among many. A player does not experience a rule. A player experiences the consequence of a rule alongside the art, the timing, and the feeling of the moment the rule shaped. When we analyze a videogame using the mechanic as our unit, we are analyzing an input to the experience rather than the experience itself. We are describing the machine and calling it the play.
This is not a small confusion. It determines what a reading can see. A criticism built on mechanics is good at describing systems and poor at describing what it is like to be inside them. A criticism built on the borrowed units of film is good at describing framing and staging and poor at describing the fact that the person watching is also the person acting. The player is not an audience. The player is a participant whose participation changes the thing being read while it is being read. No unit taken from a non-interactive medium was built to hold that.
To read videogames as videogames, and not as film or literature or software, we need a unit that is the experience. We need to be able to draw a boundary around a piece of play and say that this, this here, is the thing under examination. That unit is Gesture. The rest of this paper defines it, defends it against the units it is most often confused with, specifies what it is made of, and locates it in the fields it touches. The claim is modest in one sense and large in another. Modest, because it asks only for a unit that matches the experience. Large, because almost nothing in the existing vocabulary was designed to provide one.
2A Note on Gesture in Human-Computer Interaction
Before the term can do any work it has to answer an objection that will occur immediately to anyone trained in interaction design, which is the tradition I come from. In human-computer interaction the word gesture already has a precise and established meaning. It names a bodily movement that a system interprets as input. A swipe, a pinch, a two finger rotate, a wave in front of a depth camera, a flick of a motion controller, all of these are gestures in the HCI sense. This is not a loose usage. It is a technical vocabulary with a history.
That history is worth stating plainly, because I am not proposing to overwrite it. Kurtenbach and Hulteen (1990) gave an early and careful definition of a gesture as a motion of the body that contains information, distinguishing the communicative movement from mere mechanical actuation. Saffer (2008) codified the design of gestural interfaces as a discipline in its own right, cataloguing patterns for touch and free space interaction. Wigdor and Wixon (2011) placed touch and gesture at the center of what they called the natural user interface, a design paradigm meant to succeed the graphical user interface. Norman and Nielsen (2010) supplied the necessary caution, arguing that gestural interfaces which abandon established conventions such as visible affordances and reliable undo can move usability backward rather than forward. Underneath all of this sits the older idea of the affordance, from Gibson (1979) and carried into design by Norman (2013), the property of a thing that suggests how it can be acted upon. The HCI gesture is, in this lineage, a learned or intuited action that an interface affords and interprets.
I take this tradition seriously and I am not renaming it. The word gesture is true in HCI. It describes something real and it describes it well. My claim is not that HCI has the word wrong. My claim is that HCI has narrowed the word to one of its meanings, the movement that signals a machine, and that videogames need the wider meaning the word had before HCI narrowed it.
The relationship between the two senses is nested, not rival. The HCI gesture is a kind of input. It answers the question of how the body signals to the system. The Gesture proposed in this paper is a unit of aesthetic experience. It answers a different question, which is what the shape of the whole encounter between player and videogame is across a bounded span of play. An input gesture is one of the many things that can occur inside an aesthetic Gesture. When a player flicks a motion controller to roll a virtual bowling ball, that flick is a gesture in the exact HCI sense, and it participates in a larger aesthetic Gesture that also contains the art of the lane, the timing of the release, the sound of the pins, and the feeling of the throw landing. The input gesture is a component of the aesthetic Gesture. It does not compete with it. It lives inside it.
Both senses descend from the same root idea, which is a shaped movement that means. In dance, in painting, in rhetoric, a gesture is a form that carries significance through its shape rather than through spelled out content. HCI inherited this root and specialized it toward the movement that a computer can read as a command. This paper inherits the same root and widens it back toward the aesthetic sense, while keeping the HCI meaning intact as a special case. To keep the two clearly separated in the text, I capitalize Gesture when I mean the aesthetic unit and I use input gesture or gestural interface when I mean the HCI sense. A reader from interaction design should find no theft here and no scary amount of overlap. The overlap is real, it is small, and it is exactly the overlap you would expect from two specializations of one older word.
3Gesture Defined
Gesture is the surface area of contact between the player and the videogame. It is the whole of what happens when a videogame is being played, bounded so that it can be investigated.
A Gesture is not a fixed size. It can be a micro moment, the split second of pressing jump and feeling the character leave the ground. It can be a longer moment, a single encounter or a single conversation. It can be a period of play across an evening. It can be the entire videogame, the full arc of plot and mechanics across a hundred hours. Naming a Gesture is choosing a boundary box around play. The boundary is drawn by the analyst, for the purpose of the reading, and it can be drawn at any scale that is informative. This freedom of scale is a feature and not a vagueness. The unit is defined by its role, which is to be the experienced whole under examination, and not by any particular duration.
The phrase surface area of contact is doing real work in the definition. Contact means the meeting of the player and the system. It includes what the system presents and what the player does within and in response to what is presented. The player is not outside the videogame looking in. Play is a meeting of a body and a system, and the Gesture is the shape of that meeting over the span we have chosen to look at. This is the sense in which Gesture is close to the account of play that Keogh (2018) develops, in which the player's body and the videogame's systems are not cleanly separable during play but form a single embodied circuit. Keogh draws this account from the phenomenology of Merleau-Ponty (2012), for whom perception is not a matter of a mind receiving data from a body but of a body already involved in a world. Gibson's ecological account of perception (1979), which grounds the affordance, belongs to the same broad commitment, that perceiving and acting are aspects of one process rather than two.
Word choice here matters, and surface area is not a casual metaphor. A surface has extent. It can be small or large, simple or complex, smooth or textured. It is the place where two things touch, which is precisely what a Gesture is, the place where a player and a videogame touch. The metaphor also carries the right sense of two sidedness. A surface belongs to both things that meet at it. The Gesture belongs neither to the player alone nor to the videogame alone. It is what exists between them while they are in contact, and it stops existing when the contact stops.
4What Gesture Is Not
The fastest way to sharpen the definition is to say what Gesture is not, because the confusions are predictable and each one, once named, tightens the concept.
Gesture is not a mechanic. A mechanic is a rule the system runs. A Gesture contains mechanics but is not reducible to them. The same mechanic produces different Gestures depending on the art, the timing, and the situation around it. A double jump in a bright forgiving platformer and a double jump in a punishing horror platformer are the same mechanic and different Gestures, because everything around the mechanic differs and the player feels the whole and not the rule. Analyzing the mechanic tells you about a part. Analyzing the Gesture tells you about the whole the player actually had.
Gesture is not an action or an input. A button press is an input. A discrete in fiction event, a swing of a sword, is an action. The Gesture is the experienced moment that the input and the action participate in. This includes the case where the player does nothing. Watching a cutscene is a Gesture. Holding still and waiting is a Gesture. The absence of direct interaction is still contact, because the player is still in the meeting with the system, still receiving, still anticipating, still held. Anthropy and Clark (2014) give us verbs, the things a player can do, and verbs are indispensable, but a verb names a capacity and a Gesture names the lived whole in which that capacity is enacted or withheld. The distinction from the HCI input gesture belongs here too. The input gesture is one available action. It is not the experienced whole.
Gesture is not designer facing. It is defined from the side of the player. It is not the set of things the designer built. It is the experience those things produced when a player met them. This orientation is the entire point, and it is what separates Gesture from the units that describe the machine rather than the play. MDA (Hunicke, LeBlanc, and Zubek 2004) moves from the designer outward, from mechanics through dynamics to aesthetics. Gesture starts where MDA ends, at the experienced result, and it treats that result as the primary object rather than as a downstream effect to be inferred.
These three negations point in one direction. A Gesture is not any of the parts that produce it and it is not the machine that runs those parts. It is the experienced whole. Everything the rest of the paper adds is an account of what that whole is made of and where its materials come from.
5The Composition of a Gesture: The Four A's
A Gesture is not a simple atom. It has internal structure. When you play, you experience a Gesture as one thing, undivided, immediate. When you analyze, you can pull that one thing apart into four components. They are Action, Art, Arc, and Atmosphere. I call them the four A's. They are experienced as inseparable during play and they become separable only in analysis, in the same way that a chord is heard as one sound and written as several notes.
The order of the account is a convenience. In the lived Gesture there is no order. The four are simultaneous.
Action is what the videogame has the player do, including the absence of direct or immediate interaction. This last clause is not a hedge. It is central. A cutscene is an Action, and the Action is watching. A forced pause is an Action, and the Action is not acting. Action covers the full range of what the player is doing within the contact, from the most demanding input to the most complete stillness. It absorbs the verb, in the sense of Anthropy and Clark (2014), and it absorbs the input gesture, in the HCI sense, but it is larger than either, because it also holds the doings that are not verbs and not inputs, the waiting, the watching, the deciding, the refusing. Action is the doing side of the surface of contact.
Art is the art direction, the lighting, the staging, the camera grammar, the composition, and the sound design, together with the heritage of these choices. Art is not graphics quality. A pixel art videogame makes art choices as fully as a photoreal one, and often more deliberately, because every pixel is placed. The word grammar is exact here. A camera has a grammar in the sense film theory means, a set of conventions for how framing, movement, and cutting produce meaning, and a videogame either uses that grammar, refuses it, or invents against it. Bazin's attention to what happens inside the shot rather than only in the cut between shots (1967) is a direct source for treating staging as meaning, and Bordwell and Thompson's account of mise-en-scène (2017) gives the working vocabulary. Sound belongs to Art on equal terms with image. The point of naming this component Art rather than aesthetics is to reserve the word aesthetic for the heritage discussion in the next section, where it does different work.
Arc is the amount of time and the shape that time takes. It could be a micro moment, a long moment, or an entire videogame. Arc is not the same as pacing, though pacing is part of it. Arc answers two questions at once, which are how long this lasts and what shape it has across that length. A jump has an arc measured in fractions of a second, a rising edge and a falling one, a moment of hang at the apex. A boss fight has an arc measured in minutes, with phases and turns. A role playing game has an arc measured in dozens of hours, with acts and reversals and a final descent. Arc is the temporal extent of the Gesture and the internal form of that extent. Because a Gesture can be drawn at any scale, Arc is where the chosen scale becomes concrete.
Atmosphere is the feeling and the vibe of the moment. It is what emerges when Action, Art, and Arc combine. This is the one component that cannot be pointed at directly in the machine, because it is not a thing that was placed. It is the result of the placement. Atmosphere is not added on top of the other three. It is what the player feels because of them. A designer does not build atmosphere the way they build a mechanic or place a light. They build the Action, the Art, and the Arc, and the Atmosphere is what those produce in a player who meets them. This makes Atmosphere the most player side of the four, the component that most fully belongs to the experience rather than to the object.
The reason to hold all four together under one unit is that a player never receives them apart. The dread of a corridor is not the lighting plus the slow walk speed plus the low drone plus the length of the walk, presented as a sum. It is one dread, produced by all of them at once, and felt as a single quality of the moment. The four A's are an analytic instrument, a way of asking of any Gesture what the player was doing, what the videogame presented, how long it lasted and in what shape, and what all of that produced as feeling. Used well, the instrument explains the whole by naming its contributions. Used poorly, by mistaking the four separated components for four separate experiences, it loses the very thing it was built to study.
6Aesthetic Heritage
Once a Gesture is decomposed into its four components, a further question opens under each one. Where did this choice come from. Every choice in every A has a lineage. The lighting of a scene descends from somewhere. The particular feel of a jump descends from somewhere. The shape of a difficulty curve descends from somewhere. The name for this lineage is Aesthetic Heritage, and the word aesthetic here is meant in its full sense, that a videogame is an aesthetic object and that all four of its A's are aesthetic choices with histories, not only its visible surfaces.
This is the reason the visual component was named Art and not aesthetic. Aesthetic is reserved for heritage, and heritage covers all four A's. A jump mechanic has an aesthetic heritage. A control scheme has an aesthetic heritage. A pacing decision has an aesthetic heritage. To restrict the aesthetic to the visual would be to repeat the error the whole framework is trying to escape, the error of treating only the surface as art and the systems as mere engineering.
There are three ways a videogame can stand in relation to its heritage, and naming them turns heritage from a vague sense of influence into a usable analytic move.
The first is inheritance. A videogame uses a solution because it works and because it has been proven to work. The double jump exists in a thousand videogames because it solved a real problem of air control and it kept solving it. When a designer reaches for the double jump they are inheriting, taking a settled solution without necessarily commenting on it. Inheritance is the ordinary condition of a medium that has a history. Most choices in most videogames are inherited, and there is nothing lesser about this.
The second is quotation. A videogame deliberately references a source, and the reference is meant to be legible as a reference. Camera work that quotes a specific film, a level that quotes a famous level from another videogame, a musical cue that quotes a known theme, all of these are quotations. Quotation differs from inheritance by intention and by visibility. The designer wants the source to be recognized, or at least to be recognizable to the reader who knows the source. Quotation is how a videogame speaks to the history it comes from rather than merely using it.
The third is transformation. A videogame takes something and makes it new, so that the source is present but changed into something the source did not contain. The bonfire in Dark Souls transforms the checkpoint. A checkpoint saves progress. The bonfire saves progress and also refills healing and also respawns enemies and also anchors a fragile, hostile world with a small circle of safety, so that the same act that preserves you also resets the danger. The checkpoint is the heritage. The bonfire is the transformation. Transformation is the most creatively charged of the three relations, because it is where a medium adds to its own inheritance rather than only drawing on it.
The framework for these three relations owes an obvious debt to the study of influence in the other arts, and the sharpest statement of that study is Bloom's account of the anxiety of influence (1973), in which strong work is produced through a charged and often combative relation to what came before, a deliberate misreading that clears space for the new. Videogames are young enough that their anxiety of influence is still forming, and much of what they do is still inheritance rather than transformation, but the categories apply. Heritage comes from everywhere, from other videogames, from cinema, from music, from visual art, from theatre, from dance, from literature, and from architecture. The analyst who reads a Gesture through its heritage is asking, of each of the four A's, not only what it is but where it learned to be that.
7Permissions
A videogame is not only what it presents and what it lets the player feel. It is also a set of boundaries. What the videogame allows, what it requires, and what it forbids together define the space the player moves in. This set of boundaries is the videogame's Permissions.
There are three kinds. The allowed is the possibility space, everything the videogame lets the player do. The required is the set of demands, everything the videogame makes the player do in order to progress. The forbidden is the set of limits, everything the videogame prevents the player from doing. A wall you cannot pass is a forbidden. A tutorial you must complete is a required. A hidden route you may find is an allowed. Together they draw the shape of the possible, and that shape is meaning as much as any image or sound, because what a videogame allows, demands, and refuses communicates what it values.
The claim that matters most about Permissions is a claim about how they are known. Permissions are not a container that sits around the Gesture, specified in advance and handed to the player as a rulebook. They are discovered through the four A's as the player plays, the way a person feels the extent of a dark room by moving through it and meeting its edges. A player learns a wall is solid by walking into it. A player learns an action is required by being stopped until they perform it. A player learns a route is allowed by trying it and not being refused. Permissions are felt out through the Gesture. This is why Permissions belong to the reading of the Gesture and not to a separate structural diagram of the videogame. They are experienced, and they are experienced through the same four components as everything else.
There is an important qualification. Sometimes a player does know Permissions in advance, and when they do, the knowledge comes from outside the immediate contact. It comes from genre convention, from marketing, from other players, from reviews, from prior exposure to the same videogame or others like it. A player who has played many first person shooters knows a great deal about a new one's Permissions before touching it, and that prior knowledge shapes the early Gestures. This does not contradict the claim that Permissions are discovered through play. It says that discovery can be accelerated by prior heritage in the player, which is itself a matter of the player's own history with the medium.
Permissions also divide along the line of intention. Some Permissions are designed on purpose, placed deliberately as boundaries meant to shape the experience. Others are accidental, boundaries that no one intended but that the player meets all the same, the unintended wall, the exploit, the sequence break, the emergent limit. Both kinds are real and both shape the experience, because the player who meets an accidental boundary meets it as fully as a designed one. Reading Permissions means reading both, the intended shape and the shape that actually materialized, and it connects directly to the next point.
8Scale and Nesting
Because a Gesture can be drawn at any size, the analyst chooses the scale, and the right scale is the one that makes the reading informative. A study of how a jump feels wants a small Gesture, bounded to the fraction of a second where the jump lives. A study of how a videogame handles loss across its whole length wants a large Gesture, bounded to the entire work. Neither scale is more correct. They answer different questions, and the boundary follows the question.
Gestures can be nested. A Gesture can sit inside a larger Gesture, and the relationships between the nested and the containing Gesture are part of reading a videogame. The jump lives inside the platforming challenge, which lives inside the level, which lives inside the videogame, and each of these is a Gesture at its own scale. The way a small Gesture inflects the large one it sits inside, and the way the large one conditions the small, is often where a reading finds its most useful observations.
This power should be used with restraint. Perceiving Gestures inside of Gestures without limit is usually counterproductive, and the analyst who multiplies the unit without a reason tends to dissolve the reading rather than deepen it. The nesting is a tool for showing how a particular small moment carries a particular large meaning, or how a large structure shapes a particular small feel. It is not an invitation to decompose a videogame into an endless tree of sub Gestures for its own sake. Draw the boundary where it helps, note the nesting where it explains something, and stop.
9The Heritage of the Term
The word Gesture is borrowed on purpose, and the borrowing is itself an instance of Aesthetic Heritage, a transformation of a term rather than an invention from nothing. In dance, painting, music, acting, and rhetoric, gesture already names a meaningful unit, one that communicates through form rather than through explicit content alone. A gesture in dance is not a message spelled out in words. It is a shaped movement that means through its form, its speed, its weight, its direction. Carrying the word into videogames carries that inheritance with it, and the inheritance is the point, because it tells us what kind of unit Gesture is.
In film theory the relevant tradition is mise-en-scène, the attention to what is staged and enacted within the frame. Bazin's writing on what happens inside the shot, and his preference for the long take and deep focus that let meaning develop within a continuous frame rather than being assembled through cutting (1967), is a direct inspiration. Videogame Gesture extends this. It is not only what is in the frame. It is what the player does within and against the frame. There are even readings in which occurrences from outside the videogame's rendered space affect the Gesture, a sound in the player's room, a second player on the couch, a timer running in the real world, because the contact between player and system is not confined to what is drawn on the screen. The frame of a videogame is more porous than the frame of a film, and Gesture is built to accommodate that porousness.
The heritage of the term also explains its relation to the HCI sense discussed at the start. Both the aesthetic gesture of dance and the input gesture of interaction design descend from the same root, the shaped movement that means. HCI specialized the root toward the movement a machine can read. This paper specializes it toward the whole aesthetic encounter. The two specializations are cousins, and naming their common ancestor is the cleanest way to hold them apart without pretending they are unrelated.
The heritage matters because it locates the kind of unit Gesture is. It is an aesthetic unit and not an engineering one. It belongs with the units that other aesthetic media use to talk about form and experience, and it asks the same kind of question those media ask, which is what this does to the person receiving it, and how. A videogame receives its player as a dance receives its viewer and also, unlike a dance, as a dance would receive a viewer who was also dancing. Gesture is the unit built for that double condition.
10Position Among Existing Frameworks
Gesture does not arrive on empty ground. Several frameworks already describe parts of what it covers, and locating Gesture against them clarifies it and pays the debts it owes.
MDA (Hunicke, LeBlanc, and Zubek 2004) moves from the designer's side to the player's side, from mechanics through dynamics to aesthetics. It is built to help designers reason from the rules they write toward the experience those rules produce. Gesture inverts the direction. It starts from the experience and works backward to understand what produced it. The two are complementary rather than opposed. MDA is a way to build toward an experience. Gesture is a way to read an experience that has already happened. A designer might use MDA to construct and Gesture to evaluate, and the aesthetics that sit at the far end of MDA are close kin to the Atmosphere that sits inside a Gesture, arrived at from the other direction.
Where Swink's object is the sensation of control, Gesture's object is the full contact, of which the sensation of control is one important region.
Game Feel (Swink 2009) is a major inspiration and in some ways the nearest neighbor. Swink builds a careful bridge between the mechanical and the aesthetic at the scale of moment to moment control, analyzing how input, response, and simulated physicality combine into the felt sensation of controlling a virtual body. That bridge is exactly the kind of thing Gesture is meant to hold, and Game Feel can be read as a detailed study of a particular class of small scale Gestures, the ones made of real time control. Gesture keeps Swink's bridge and extends past feel. Feel lives at the small scale of real time control. Gesture also reaches the longer arcs of play, the atmospheric qualities of a whole videogame, and its narrative experience, both implied and explicit. Where Swink's object is the sensation of control, Gesture's object is the full contact, of which the sensation of control is one important region.
A Game Design Vocabulary (Anthropy and Clark 2014) gives us verbs, the things a player can do, and the language of verbs has done real good by pushing design talk toward the player's actions and away from abstract systems. Verbs are indispensable and they are not the same as Gesture. A verb names a capacity. A Gesture names the experienced whole in which that capacity is enacted, together with everything else present in the moment, the art, the timing, the feeling, and the absence of action as much as its presence. Gesture holds verbs inside its Action component and adds the three components a verb by itself cannot supply.
Procedural rhetoric (Bogost 2007) argues that videogames make arguments through their processes, through the rules they enforce and the behaviors those rules encourage, and it is a powerful account of how systems mean. Gesture is compatible with it and differs in emphasis. Procedural rhetoric locates meaning in the process, in the machine's rules considered as a claim. Gesture locates meaning in the contact, in what the process produces when a particular player meets it. Sicart's critique of proceduralism (2011) presses exactly this point from within game studies, arguing that meaning is not delivered by process alone but is completed in play, and Gesture agrees, because the Gesture is by definition the thing that exists only when a player is in contact with the process. Sicart's later work on the ethics of gameplay (2013) treats the played situation, rather than the designed system, as the site of moral weight, which is again a move toward the experienced whole.
Behind these design oriented frameworks sits the older debate in game studies over whether videogames are best understood as rule systems or as fictions, the debate that ran between the ludological emphasis of Aarseth (1997) and the narrative emphasis of writers such as Murray (1997), and that Juul (2005) sought to reconcile by describing videogames as half real, as real rules that produce fictional worlds. Gesture does not take a side in this debate so much as it dissolves the need to. The rule and the fiction are both present in the Gesture, the rule as part of the Action and the Permissions, the fiction as part of the Art and the Atmosphere, and the player experiences them as one contact rather than as a rule layer and a story layer to be weighed against each other. The framework of player psychology that surrounds all of this, the account of flow as the balance of challenge and skill (Csikszentmihalyi 1990), the study of fun as the pleasure of learning (Koster 2013), and the practical craft of iterative playtesting (Fullerton 2024), all describe forces that shape the Atmosphere of a Gesture, and all can be brought inside the reading rather than left beside it.
None of these frameworks is a competitor to be defeated. Each names something real. Gesture names the thing they are all circling, which is the player's experience taken as a unit, and it gives that unit a boundary the analyst can draw and a composition the analyst can examine.
11A Worked Reading
An instrument is best shown in use, so consider a single Gesture read through the whole method. Take the moment in Dark Souls (FromSoftware 2011) when a player, low on healing and deep in hostile territory, finally reaches a bonfire, kneels, and rests. Bound the Gesture tightly, to the few seconds from the approach to the rest taking hold, and read it.
The Action is a small one on its face. The player walks the last steps, presses a single button, and watches an animation of the character kneeling. There is almost no input, and the near absence of input is itself part of the Action, because the videogame takes control for the length of the animation and the player's doing becomes waiting and watching. The Art is doing heavy work. The bonfire is a warm point of orange light in a palette that has been cold and gray for a long stretch, the camera settles, the ambient hostility of the soundscape drops away, and a soft tone marks the rest. The Arc is short in duration and pointed in shape, a release of tension that has been building across the much larger Gesture of the journey to reach this point, so that the few seconds carry the weight of the minutes before them. The Atmosphere that emerges from these three is relief, and a specific relief, fragile and provisional rather than triumphant, because the player already knows what the rest costs.
That knowledge is the Aesthetic Heritage and the Permissions arriving together. The bonfire is a transformation of the checkpoint, and reading it as a transformation explains the fragility in the Atmosphere. A checkpoint inherited plainly would give pure relief, safety banked. The bonfire refuses the plain inheritance. Its Permissions are felt in the same moment as the rest, because resting is required to refill healing and the required rest also respawns every ordinary enemy the player just fought through. The allowed, refilled flasks and a fixed point to return to, and the required, the respawning of the world, and the forbidden, the inability to keep both progress and a cleared path, are discovered in the act of resting rather than read from a menu. The player learns the shape of the bargain by making it. The relief is real and it is shadowed, and the shadow is the transformed heritage of the checkpoint operating as a permission the player feels rather than reads.
This is a small reading and it demonstrates the whole method. Four components named and shown contributing to one felt result, the source of a key choice located as a transformation of an inherited form, and the videogame's boundaries surfaced as something the player discovers through the same components. The reading does not require the mechanic as its unit, though the mechanic is present inside the Action and the Permissions. It takes the experienced whole as the unit and explains that whole by its parts and their heritage.
12The Intended and the Materialized
A Gesture can be unintentional and still be a Gesture. The experience a player has is the only experience they have at that moment, and the shape of the contact between the system and that particular player is the Gesture, whether or not anyone designed it to be that. The bug that produces a moment of unexpected beauty, the emergent tactic no one planned, the strange calm of a loading screen at the end of a long night, all of these are Gestures, because all of them are contact between the player and the videogame.
There is always a gap between the affect a designer intends and the affect that materializes in a player. This is true in every creative form, and it is not a defect to be explained away. A film intends and a viewer receives, and the two never coincide exactly. In videogames the gap is wider, because the player is not only receiving but acting, and the acting introduces a variability that no design can fully anticipate. The same corridor produces terror in one player and boredom in another, and both experiences are real Gestures. Reading Gesture takes the materialized experience seriously and does not privilege the intended one as the true meaning against which the actual experience is measured as error.
This is where Gesture reconnects with Permissions and with the distinction between designed and accidental boundaries. The materialized Gesture includes the accidental as fully as the designed, because the player does not receive a label telling them which boundaries were meant. They meet the boundary, and the meeting is the Gesture. A criticism that reads only intention reads only half of what happened. A criticism built on Gesture reads what happened, and treats intention as one input to the reading rather than as its verdict.
13Conclusion: A Complete Reading, Not a Preface
Gesture is where reading begins and it is also where a full reading can be carried out, because the unit comes with its own internal structure and its own account of where its materials come from. Once a Gesture is bounded, three moves are available and they can all be made within a single reading. The Gesture can be decomposed into its four components, the Action, the Art, the Arc, and the Atmosphere, experienced as one thing during play and separated only in analysis. The choices inside each component can be situated in their Aesthetic Heritage, in the inheritances, quotations, and transformations that each choice descends from. And the boundaries the videogame sets, its Permissions, can be read as they are discovered through the same four components, the allowed, the required, and the forbidden felt out through play rather than known in advance.
I have deliberately kept the whole framework in one paper rather than splitting the unit from its composition. The composition is not a separate subject to be deferred. A unit without an account of what it is made of is a name without a method, and the four A's, Aesthetic Heritage, and Permissions are what turn Gesture from a slogan into an instrument. Where later work is needed, it should add depth to these parts rather than remove them from the definition of the unit. There is a full paper to be written on Atmosphere alone, on how feeling emerges from the other three components, and a full paper on the aesthetic heritage of specific mechanical lineages, and a full paper on Permissions as a theory of meaning in constraint. Those papers would extend this framework. They would not be needed to complete it.
The claim to establish first remains the basic one. Videogames deserve a unit of analysis that is the experience of playing them, drawn at whatever scale the experience occurs, composed of the four things every played moment contains, sourced from a heritage that reaches across every art, and bounded by permissions the player feels out rather than reads off. That unit is Gesture. It is native to the medium, it holds what the borrowed units drop, and it is compatible with the neighboring vocabulary of interaction design, systems design, and film theory that it draws from and stands beside.
§References
Aarseth, Espen J. Cybertext: Perspectives on Ergodic Literature. Baltimore: Johns Hopkins University Press, 1997.
Anthropy, Anna, and Naomi Clark. A Game Design Vocabulary: Exploring the Foundational Principles Behind Good Game Design. Upper Saddle River, NJ: Addison-Wesley, 2014.
Bazin, André. What Is Cinema? Vol. 1. Translated by Hugh Gray. Berkeley: University of California Press, 1967.
Bloom, Harold. The Anxiety of Influence: A Theory of Poetry. New York: Oxford University Press, 1973.
Bogost, Ian. Persuasive Games: The Expressive Power of Videogames. Cambridge, MA: MIT Press, 2007.
Bordwell, David, and Kristin Thompson. Film Art: An Introduction. 11th ed. New York: McGraw-Hill Education, 2017.
Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York: Harper and Row, 1990.
Fullerton, Tracy. A Playful Production Process: For Game Designers (and Everyone). Cambridge, MA: MIT Press, 2024.
Gibson, James J. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin, 1979.
Hunicke, Robin, Marc LeBlanc, and Robert Zubek. "MDA: A Formal Approach to Game Design and Game Research." In Proceedings of the AAAI Workshop on Challenges in Game AI, 1-5. Menlo Park, CA: AAAI Press, 2004.
Juul, Jesper. Half-Real: Video Games between Real Rules and Fictional Worlds. Cambridge, MA: MIT Press, 2005.
Keogh, Brendan. A Play of Bodies: How We Perceive Videogames. Cambridge, MA: MIT Press, 2018.
Koster, Raph. A Theory of Fun for Game Design. 2nd ed. Sebastopol, CA: O'Reilly Media, 2013.
Kurtenbach, Gordon, and Eric A. Hulteen. "Gestures in Human-Computer Communication." In The Art of Human-Computer Interface Design, edited by Brenda Laurel, 309-317. Reading, MA: Addison-Wesley, 1990.
Merleau-Ponty, Maurice. Phenomenology of Perception. Translated by Donald A. Landes. London: Routledge, 2012. Originally published 1945.
Murray, Janet H. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. New York: Free Press, 1997.
Norman, Donald A. The Design of Everyday Things. Rev. and expanded ed. New York: Basic Books, 2013.
Norman, Donald A., and Jakob Nielsen. "Gestural Interfaces: A Step Backward in Usability." Interactions 17, no. 5 (2010): 46-49.
Saffer, Dan. Designing Gestural Interfaces: Touchscreens and Interactive Devices. Sebastopol, CA: O'Reilly Media, 2008.
Sicart, Miguel. "Against Procedurality." Game Studies 11, no. 3 (2011).
Sicart, Miguel. Beyond Choices: The Design of Ethical Gameplay. Cambridge, MA: MIT Press, 2013.
Swink, Steve. Game Feel: A Game Designer's Guide to Virtual Sensation. Burlington, MA: Morgan Kaufmann, 2009.
Wigdor, Daniel, and Dennis Wixon. Brave NUI World: Designing Natural User Interfaces for Touch and Gesture. Burlington, MA: Morgan Kaufmann, 2011.
§Ludography
FromSoftware. Dark Souls. Tokyo: Bandai Namco Entertainment, 2011.