Playtesting with Purpose

Playtester Processing

A core tenant of game development is playtesting. You must playtest a lot in order to refine ideas, improve the flow of play, gain insight and new perspectives, and to simply make sure what you have is fun.

As fun as it is to play something you’ve created, playtesting is hard. First off, it needs other people to work well; solo playtesting can only take you so far. It also requires you to be observant, to watch, listen, and take notes on how your other players interact with the game – both overtly and indirectly. And then you also have to listen to your player’s thoughts about their experience with the game, what they liked, what they didn’t like, their praises, criticisms, and suggestions for improvement.

I will leave the specific things I watch for, the questions I ask, and the feedback I try to elicit from my playtesters for another article – the topic I want to focus on here is how to take that feedback and use it to effectively improve your game. This article is about optimizing your playtesting process, tracking and evaluating the changes you make as your game goes through its iterations, identifying core problems and the relative merits of possible solutions, and limiting the frequency that you see the same comments or complaints coming up over and over again.

But first, lets talk about:

Taking Notes

ALWAYS bring your notepad to a playtest, and take notes while the game is happening. If you leave it until the end, or only take notes on the post-game comments, or leave it until later, you will forget. You will lose context, you will forget some of the important things that happened, and you will not get as much value out of the playtest as you could have. Playtesting involves a significant investment of time, often more than actual development. Make sure you get as much value out of it as you can.

Unless your game is very small and plays quickly, start by tracking each playtest on a fresh sheet of paper by logging the following information:

• Game Name & Version Number
• Date & Time
• List of Participants (along with their Contact Info if you don’t have it elsewhere, their experience with playtesting, how often they’ve played this game before, and where they are sitting relative to the game setup)
• Purpose (if you are trying something specific, or features/components you are focused on analyzing)

The notes that you take are going to be influenced by the purpose of your playtest, and everyone has their own style. I pay special attention to any Questions, Comments, and Ideas raised by the players while annotating each as well as my own Observations to connect them to the player and the type of note (Q, C, I, or O). Tallies can also come in handy if the same question gets asked repeatedly – knowing how many times it was asked and by which player can be important.

For example, in a recent test of my F5 game the question “What is the current speed / intensity?” was asked 14 times (that I caught!) – 11 of them by the same player who was sitting opposite the State Board that displayed the information. This piece of information demonstrated a significant need for improving the visibility of the Speed and Intensity value.

Translating Notes into Problems

The most important step in playtesting is taking the results of a playtest (your notes) and turning them into action items. That requires first reading your notes, and then interpreting them to understand the underlying problem that produced it. Every Question that is asked is something that was unclear to the player, every critical Comment is a reaction to something that seemed odd or wrong to the player, every Idea presented is an attempt to solve a problem that the player observed.

Take each note and record the underlying problem or problems that prompted it. Regardless of how trivial, or misguided, or inane a problem may be – seek to find the underlying cause even if you don’t agree that it is a problem, or that the failure lies with the player. The best case scenario is that players of all types, attitudes, perspectives, and skill levels will be playing your game – and every challenge that a player experiences during playtesting will be discovered a dozen times over in the real world. Understanding what these problems are and why they happen is critical to solving them for your players before they run into them.

In the case of specific ideas that a player suggests, sometimes these are being made in an attempt to fix something, or they can be presented as a solution in search of a problem. If a suggestion doesn’t appear to be in response to anything, it helps to follow up with that player to try and understand what prompted their suggestion, where it come from, and why they think it might be an improvement. Some playertesters just happen to have a favourite mechanic, or their own game ideas that they want to see that may not have any connection to what you are doing. Any suggestion is worth noting and even disconnected suggestions may be worth considering, but understanding the playertesters motivation for presenting their idea can help you evaluate whether or not it is useful to you.

The list of problem statements that you create will guide you as you are making revisions and keep you focused on the reason behind the revisions you make, and how well those revisions succeed at accomplishing their purpose.

Once you have your list, you may find it useful to prioritize them. A simple “High, Medium, Low” can be enough to be a reminder to spend your time trying to fix the big problems rather than the little ones. Your priorities will also shift over time, a cosmetic issue during initial playtesting is probably a low priority to fix – but the same issue when you are trying to finalize your production assets can become critical.

Your problem list will be a running log of issues to resolve that will grow and shrink over time as new problems are added and old ones get resolved. It is also a good idea to periodically review the list, re-examine your priorities, and make adjustments.

Problems into Solutions

When you have your list of problems and have figured out which ones you want to address during your next revision, now is the time to start collecting all those changes you have been brainstorming, gathering, and writing down all this time. Take each change and break it down into its self-contained components. A change may easily require a dozen different adjustments to your game, list out what those adjustments are and separate out any that make sense and will work on their own without the others. This is now your list of possible solutions.

Your goal is to connect solutions to the problems that they try to solve, and the select a set of these solutions to implement in your next revision.

Now, everyone and every project will have its own set of problems, solutions, limitations, and ways of working. I prefer to emphasize simplicity and keep the scope of each revision small. I will make a handful of small changes, test, and then go back and do it again – a very incremental approach. This may or may not work best for you, but the following describe what I look for when selecting solutions to include in my next revision:

1. Prefer simple, less complex solutions over more complex ones.
2. Select solutions that solve more than 1 problem at a time.
3. Only include 1 solution per problem per revision.
4. Select solutions that complement each other or can use combined assets/pieces.
5. Avoid multiple solutions that impact the same rule in different ways.
6. Prefer incremental adjustments over drastic ones.
7. Prefer solutions that reduce rules or exceptions over ones that create them.

Once I have a plan in place for my revision including problems I want to address and solutions that I am going to try to solve them, I create a checklist of tasks that need to be completed for that revision. Each solution is given a row with checkboxes for Assets (modifications to physical objects) and Rules (modifications to the master rules document).

During the process of making the required changes, oftentimes I will change my mind, think of a different or better solution, decide one approach isn’t going to work, or that I simply don’t have enough time to finish work on one solution before the next time I want to run a playtest. My checklist of tasks gets modified and adjusted regularly until I decide a revision is done.

The Final Result

So why do all this work? Can’t you just go with your gut, make a few changes, and keep going until you’re happy?

Well… yes. Of course you can. There’s nothing wrong with doing it that way. And sometimes I end up doing it that way too.

But what I have found from taking consistent and deliberate notes, identifying problems, and being meticulous about my solutions is that I end up saving time.

• I don’t spend time rediscovering issues I’ve already discovered and forgot to write down.
• I don’t try the same thing more than once unless I try something new and it ends up making things worse.
• I have a reasonably good idea if I try something and it does make it worse.
• I seldom have to worry about which change I made was responsible for improving/worsening a problem.
• I have to spend less overall time playtesting multiple combinations of sets of rules to find which works the best.
• In the end I have a solid roadmap of my development process and a reliable record of things that worked and didn’t work that I can use for future projects without relying on recollection and experience.

And sometimes, just writing things down will help you unlock a stubborn problem for good.