"How should good playtest be done?" Topic

What methodology should by used? Who should do it? How Long should it take? What should be considered.

In my opinion

1. The game should be play tested by people who have no vested interest in the game and should be best done by people who have no particular emotion towards yo the author.

2.It should be played at least 30 times and played not to make it work successfully but to cause it to crash and burn, to find out where the rules work and where they don't where they are bad and where good, and what is fun and what is not.

3. It should take at least 1 year per page of the rules and 3 battles per page of the rules.

4. Detailed reports of notes of each playtest game should be kept as a record of progress.

(3) is ridiculous, but I'm with you on the others

Not sure that point 3 is necessary or realistic. On what basis did you come up with those numbers?

Point 1 is an ideal. But by the time they have played several games and provided detailed notes then even independent playtesters will have a vested interest in the game. Nor am I sure how you find these paragons in the first place.

Initial playtests don't need to be independent, and probably shouldn't be in the interests of speed of development. What might be called 'beta testing' needs to be independent, but can only be properly done once the basic mechanisms have been resolved and the rules essentially written (because you are testing the writing as much as the rules-as-understood-by-the-author). At that point, if it's going to take ten years for a ten page set of rules then they aren't going to get published.

I think it would also be good to find inexperienced gamers or gamers who don't know anything about the period, because you never know what they will want to try to do. I don't know what the magic number of playtests is, but 30 seems light.

Point #3 is just silly.

I have play-tested many rules and I am going to disagree with you.
Bearing in mind what you are trying to achieve (fun, historical, addictive..) test each part of what you are trying to do. Does the command and control work, does combat work as expected. Are max/min die rolls having too much influence.
Get some experienced gamers in and see if they can break the game, with tactics or unbalanced troop selections.
Once the game is working in the way that you are happy/expected it should, then get some new players who have no experience of the project, give them a set of the rules and see what they say.

BTW I should mention an experience I had with a group of gamers who had been testing my WW2 rules. We had started in the early war, not too powerful AFV but then started to go later and the Germans were getting stuff like Panthers. So the German player clamoured to fight with a decent number of Panthers. I set them a Kursk scenario and they were slaughtered by the Russians (took them about 30 minutes to lose).
So they all thought it was impossible. So I suggested that they try again, this time using smoke to isolate the Russian positions. It worked, they started to grind through the Russian positions, losing the odd Panther here and there.
Then the Russian armoured reserve was released. The SU-152s ripped the Panthers to shreds and the rest of the German forces were destroyed by T34s.
Lessons learned, the way to win was already in the rules, the German players had just not known how to play it. The SU-152s were massively under-pointed and so all self-propelled artillery costs increased (their increased and unexpected effectiveness, was due to the ability to fire both as on-board artillery and direct fire weapons).

Methinks Otto might have been slightly tongue-in-cheek with #3 to make a point. I got a chuckle out of it.

1, 2 and 4 are requirements for playtesting. 3 is just an observation about how it's worked so far.

I would add 5: at some point, it should be playtested by people who do not have the author present and have no guidance other than the rules. There should be a very detailed AAR, if not a video, to make sure that whatever the rules effect may be, there is no disagreement about what the rules say.

#3, OMG, 24 years before I can publish !

Are you publishing and charging for it?

Yes I should have been more specific. That time limit is to get to the point where you can drop tinkering and watching it and relax. Until you can get it to it works is abut a year per five pages. After that it should work Ok, but if it doesn't drop it. What I meant was until you can consider it more or less idiot free.

Winston. Are you asking me or PJ.

If you're asking me it makes no difference. I publish them myself and I give them away for free.

Interesting discussion. Here is my methodology:

The approach I've taken is to reverse engineer a tank video game into a table top simulation. Why? Video games are intuitive, easy to understand and fun to play. I'm not a fan of tank video games but with about 150 million(?) users/players or registered you can't argue with success.

Rather than using previous game designs, I started with the military manuals and use that nomenclature and terminology in the game. Former tank crewman pick up right away, others need an explanation, that's where the challenge comes in.

I've mostly been testing the game at conventions with new players. They need to understand the basic game concept very quickly and game play needs to be intuitive and ideally something they've experienced before. Since no one is going to read and understand multiple pages of rules at a convention players must be ready to play with a 10-12 minute explanation and example of play. The video game concept fills that requirement.

Actually, I prefer testing with new players as they bring a perspective to the game that I lack. They find holes in the system that I took for granted. They'll try tactics I had not thought of. At one convention the top player was a 14 year old that had never played a war game before. He understood the frame-by-frame and reaction concept better than experienced war gamers. He was a natural.

When play testing I observe players expectations and how they try to adapt to the game play. Rather than forcing them into something that does not fit their expectations, I'll make changes to the game. If players do not pick up on a concept or mechanic after a few tries I need to change it.

I developed the game with all of the detail, chrome, bells and whistles (including infantry and artillery). Everything a player could think of. The next step is to develop an abstracted free intro version. Then a more detailed version and then have an option for all of the chrome. I feel trying to develop a simple system from the start can close the door to increased details, modifiers and special rules.

I finally got the basic game flow down but had a hard time explaining how to use real life tactics in the game that players never heard of. To help solve that problem I developed play aids to eliminate calculations and tactic/strategy cards. The cards show the pre and post reqs, options and effects. The play aids, combined with tactics cards and QRS flow chart it's been pretty successful at enabling new players to come up to speed quickly and have fun with little interference from me.

My recommendation is to watch the players and when you see them having difficulties ask them what they are expecting or what is missing to get them to move along.

Wolfhag