Friday, April 20, 2012

A simple model for measuring skill ceilings in video games

If I was not already doing a PhD in AI and Games Design, I think I would most likely gravitate towards something along the lines of player skill in games. It is a topic that for some reason I am quite fond of, not because I consider myself to be all that skillful in games (really only a select few), but more because I feel it is a very important topic for game designers to understand. In my opinion, making a game that tailors towards skillful play can only be done if you have at some point been a skillful player in something. I am not talking about beating your friends a few times in a random game at a LAN. I mean actually being good, and interested in being good, at a game, even if it is just from a theory-crafting or spectator point of view. When you realise that skill in a game is often more than just mastering the basic mechanics of the genre, you can begin to see that skill is often much more complicated and harder to define than previously thought. High skill in one game does not automatically mean high skill in another, and games that, to the casual observer, may seem simple, mindless or even primitive may in fact have more skill than any of the other games in the genre.

Lately, and usually the cause of most of my rants on this blog, I have had conversations with people I did not wholeheartedly agree with. To simplify, I felt they were somewhat wrong. Or perhaps just ignorant. It is hard to tell. My impression was that they had never been at a stage where they could comfortably describe themselves as skillful at or even knowledgeable about a game, even from a meta/external level. To me, this has skewed their perspective of what skill actually is in games. Particularly, it had deformed their idea of what a 'skill ceiling' should be in a game. I got the impression that they thought the actions you perform in one game take the same amount of skill and are used in the exact same way in another game of the same genre. This is an incredible misunderstanding and a terrible oversimplification of determining the skill ceiling of a game. Even the simple act of pressing a button in response to an event can be drastically different conditionally from one game to another. To describe something like 'aiming' in an FPS game as a generic action, unvarying in skill in all FPS games, is like saying that boxers all just throw the same weighted punches at each other. I hate to burst your bubble guys, but depending on the weight/style of boxing, certain boxers will punch harder and faster than others. The same can be said of skill in video games.

I have beaten around the bush concerning skill in video games before. Once I mentioned skill having a physical/mechanical execution process to it. That's one part of it. In this post, once and for all, and for the benefit of 'grading' possible games in the future from a 'skill ceiling' perspective, I will introduce to you the rest of my model. Academically, this model does not incorporate much, if any of the minor research I have found around this topic. There really is very little. Instead, it is more of my own opinion on the components that define skill in video games. It is therefore subject to change and open to suggestion.

The skill level capable in video games can not simply be described as any one thing. It can more correctly be viewed as a combination of interrelated, similar but separate actions/processes that players are capable of executing while playing a game. In my opinion, these action/processes are the following:

Tactical thinking - having or developing a strategy to counter your opponent
Example: saving a powerup, unit or cooldown for use at a more opportune or unexpected moment

Multi-tasking - a player's ability to perform seperate but useful actions or observations that are of benefit to their current game state
Example: queueing up unit production, observing your resources and moving soldiers to the front line

Mechanical Dexterity - the physical speed, precision, timing and variety of input allowed to and required by players to be sucessful
Example: pulling off a tricky combination of moves in a fighting game

Threat Assessment - a player's ability to quickly assess a situation in game and respond to it in a useful or beneficial way
Example: realising your opponent will defeat you for X reasons during your next encounter in an FPS, and fleeing from the encounter temporarily

Prior knowledge, memory or experience - a player's long term and short term memory regarding specifics of both the game and the ongoing events in it.
Example: Details about enemy specifics or recollection of opponent traits, patterns and weaknesses all fall into this category

A skillful game may not necessarily have all 5 of these components. It is entirely possible that a game can have only a few of the components, but in a high enough concentration for it to still be viewed as a skillful game as a whole. An excellent example of this would be chess, requiring high amounts of Tactical Thinking and Prior Knowledge from players, but almost nil of anything else. What you may have noticed, and what was previously mentioned, is that many of these skill components have areas where they will undoubtedly overlap. Your Prior Knowledge will greatly affect your Tactical Thinking, as will your current assessment of threat. Your prior Knowledge will also determine your ability to perform actions, contributing to your level of Mechanical Dexterity. Both Mechanical Dexterity and Multi-tasking go hand in hand, the ability to do one assisting the ability to do another, but they are still inherently different. Maintaining a balanced economy in a strategy game (multi-tasking) is not the same as your ability to pull off a combo in a fighting game (mechanical dexterity), but the result from one will affect the other. In that regard, your ability to multi-task does affect your ability to assess threat. The faster you take into account all the variables which define your safety in a situation, the more efficiently you can make a helpful threat assessment. The relationships between these components can be seen in the figure below.


There is something missing though, and its a big one. A constant, unstoppable variable that without consideration, skill in video games would be non-existant. That variable is, quite simply, time. Unless you are playing a game with potentially unlimited time, such as a turn based game like chess, time can drastically affect how significant each of these components become, especially if left unchecked. Time defines how intense each of these components actually are, and is the most crucial element for truly understanding how high a skill ceiling a game can possess. Allow me to explain.

Three seconds. Just three seconds of game play. Imagine, for three seconds, you are playing Starcraft 2. In those 3 seconds you performed the following: queued up another worker, checked to see if your new unit was almost done, checked the mini-map for enemy dots and started selecting a group of units. In this time you applied 4/5 of the components of the prior model to a semi-intense degree. The only thing you did not do was change your existing strategy via Tactical Thinking, whatever that strategy was. Now imagine, for three seconds, you are playing Modern Warfare 3. In those 3 seconds, you reloaded your gun while crouching behind a chest high wall. In this case you probably applied at most two of the components, though to a much less degree. Both of these scenarios can be described as passive, non-action filled scenarios where you simply performed actions typically required of a player prior to engaging in combat. One, however, had a lot more skill to it than the other.

I know what you are thinking. How about when they are in combat? What happens to skill levels then? Obviously both would spike, possibly even firing components that were inactive in the described 3 seconds. It may even be possible that elements of MW3 would even exceed that of SC2, for a brief moment in time. Because of this, it is difficult, if not impossible, to measure the skill ceiling provided by games using random 3 second intervals. The only way to approach this topic is if you were to average the amount of skill that is possible by players over a period of sufficient game play time. Assigning arbitrary values in the form of a scale is also ... difficult, as a scale of numeric 'skill' is non-existant. Any scale you use may, unit for unit, not be equally 'skillful' in comparison to other skill components. But for the sake of getting this done, lets just make this simple...

Assuming a most skill based map, environment, level, song, track, weapons, gear etc, over a game play period of approximately 10 mins of continuous play with each individual skill component having a maximum value of 100 'skill' I hereby present to you what I think the skill ceilings of certain games are. They are generated from my observations of the most highly skilled play available to each game, from various sources:

Of course this is all quite subjective. Until a more in-depth analysis can be done employing methods capable of actually gathering the true values these components should represent, these ratings should be considered ... a little off. Not to mention biased. But it's a start. An attempted theory, if you will, quite subject to my own personal criticism. First of all, it is hard to define arbitrary values in the form of a scale for each component, for many reasons. One reason is that each individual component can not really be compared to the others numerically, so a 'skill value' is somewhat inappropriate to use. Really, what this skill measurement system is attempting to describe is a purely visual way of observing how much skill can be injected into the sorts of games described, and how certain games, by the very nature of their design, offer higher or lower skill ceilings than others that are similar. In this case, I would like to think that Quake 3: Arena has a higher skill ceiling that say Modern Warfare 2, even though they are (apparently) in the same genre. This is because a game like Quake 3: Arena offers higher levels of many, if not all of the 5 components in the model. In a 10 min gameplay session, a Quake 3 player will have had more interactions, more observations and assessments, more inputs and a longer history of changes to an ongoing strategy. Basically, time defines the skill ceiling of a game, and a game that is inherently faster, more diverse and offers more player freedom will inevitably have more room for skill.

Don't believe me? Watch this video of a professional Quake Live player, describing his 1v1 match against another opponent.



If you can show me a video where a professional modern day FPS gamer (i.e. Halo, MW etc) describing his/her gameplay in this detail with the same number of variables, items, controls, tactics and decisions being made, I will give you a $100. Seriously. When players are capable of doing actions similar to that in the previous video, when they have the ability to, even if the majority of people who play it do not, then a skill ceiling for a game can be drawn. Only through the analysis of the high-end players, what they do and what they are capable of in certain games, both hardcore and the professional, can a true understanding of and appreciation for a skill ceiling in a game be realised.

By no means am I saying that the people who play a more skillful game are inherently more skillful than the people who play a less skillful game. The skill ceiling of a game is simply just that. A player can play a skillful game and be nowhere near the cap, just as a player can be playing a slightly less skillful game and perceive themselves to be near the top of the possible skill ceiling. For the most part, they would probably be right. This model only focuses on the maximum skill allowed by a game and by no means suggests that it is the skill of the players who play them. Unless, of course, you are talking about the very best players of that game in the world. If I use the game Unreal Tournament from 1999, the game I feel I had the best grip on out of any other game, I can describe what I mean by this, and demonstrate another way in which this model can be used: personal mappings of one's own skill.


When it boils down to it though, the maximum skill capable in a game is demonstrated by the players. The maximum skill is, however, defined by the game. This, at least for prospective game designers, should be a very important realisation.