"Who playtests your published rules?" Topic

If you're going to charge any amount of money for them, may I suggest that it not be done "in house"?
This applies if it's printed or an online PDF.

If it's just you and your buddies, it's just a bunch of dudes who already think they're perfect.
You definitely NEED outside perspective.

Having a YouTube video showing "the gang" playing the rules they write is hardly instructive. They already, maybe subconsciously (?), know the rough edges and gloss over them.

A "famous" 🙄 English company, who I shan't name, regularly issued new revisions of their rules, bragging about "the most heavily playtested set of rules ever". Then a month later, out came the full page "amendment sheet".

Totally agree. Your group needs outside perspective.

I did the majority of my play testing at conventions

<appreciative laughter emoji>

Being the least experienced with the period play tester of some rules helped find some holes. Holes that had been missed because everybody knew "formed pike don't operate in heavy woods" so the rules did not penalize them for it.

As Steve Jackson once said "playtest the dumb strategies."

We don't charge for our rules, QILS. We do charge, minimally (like 1970's prices), for scenarios.

Prior to publishing, the rules were playtested by "our group", which is a core cadre of 5-8 who meet (mostly) every month, supplemented by ~100 people who show up 2-3 at a time. We added running games at conventions and running after-hours events for people at military training facilities. During development, we tracked both variety and repeat players (above talks about getting different people in, but there is also a value in repeat players) and a rough estimate experience. Over three years, we had 53 runs of 17 different scenarios with a total of 89 players (21 novice, 27 seasoned, 50 veteran).

Scenarios get a minimum of three runs with at least (optimal player number) x 2, and at least 3 novice players.

"Most heavily playtested rules ever" is puffery.

Classical advertising speak. It's unfalsifiable. Nobody knows how many wargame rules are out there, let alone has data on the amount of playtesting. Also, "playtesting" is a vague concept. I mentioned that I track playtesting (it is a development activity, you track that, right?). So, I also have a guidebook on what "playtesting" means (that has evolved over the years), and occasionally specific playbooks for specific scenarios to be playtested.

That said, "most playtested' is not incompatible with fast and frequent updates.

The top-ranked anything is not necessecarily good. The best global warming data and models we have are not good enough for the type of predicitions and statements that frequent the press. The best space travel humanity has right now is not good enough to get people to Mars.

If you thought 53 runs (>200 hours of play time) was a lot, try this. 300 players x 50 games = 15,000 playtests. That is a phenomenal amount of effort. Last year over 20,000 people registered for and played at least one game in a WH40K tournament system. link From that single community, one game during the month after release outstrips the playtest base. A year later, forget about it. All those events aren't "playtests", but per above, maybe they are in someone's definition.

It is feasible to do something like compare the rule content of the last (finished, no further updates) version of 40K (1000 pages across 6-7 volumes, but not all of it is rules), to the rule change content of published errata, FAQ, etc. (again, a 10-page errata may have two minor rule changes in it) to juge the volatility in a reasonable manner. If someone has ~$500K lying around, I can get some operations reasearchers on it.

Play testing is very important I agree.
I suspect no one would disagree?
The amount can never be enough but practicality matters too.

martin

You definitely need some play tests without people present who are familiar with the rules.

Playtesting NEVER stops. We are still making minor changes to our rules 16 years on (see the 5' 2") thread. It helps to have outsiders to do the work but its hard. Certainly UK conventions are no help to rules applicable only to keen addict's of the period, not casual placers of an occasional game.

However we did spend an inordinate time testing specific simple scenarios to check the basics. Playing a full game does not always allow sufficient repetition of a situation to ensure you understand what an outlier result is.

After 15 years we produced issue 2 updated with better written work and some improvements but the basic underlying mechanisms have remained robust. Rule writing never ends it is continuous improvement in action.

I helped playtest Midgard (in so far as I played a pre-publication version), then bought a copy, if that counts.

What I'm actually looking for is "hard core" playtesting.
Where the rules are given to someone who was not involved in the writing, and allowed to "figure it out", and then submit analysis AND criticism.

Not easy to find fully independent readers with the required understanding of history ans enthusiasm. We have sold a few hundred copies world wideover the years but that is not a big circulation. So finding large numbers of playtesters is was hard. We did get a few of our club members to do a cold read of the final run which did find issues. One comment went down in the anals of our history. The reviewer said "its not very well written". His reply was classic, in response to our comments that compareed to other rules (he plays lots of commecial rules) was oures "were better than a lot of wargames rules, but that was a low bar" That seems to be a poor reflection on the industry as a whole and to be fair does justify your original post,

It is interesting that in play testing some of our playtesters made suggestions we addopted that made play faster with no loss of fidelity.

Again we struggle more than some, as our rules are not aimed at the mass market so real playtesting at a convention is not precticable. We did test the odd mechanism at conventions like movement but to some, very ingrained by conventional gamers and not very history or tactics inclined struggled to adopt a diffrent approach. Those who were more historically and tacticaly savey picked it up and use them effectively. That was the best we could do.

Playtesting is a huge issue. Most of the 2000 hr of writing was aimed at identifieing contradications/incosistancies in the integrated package and getting the english as good as our limited capabilities allowed. Writing technical reports in a specialised industry, is not the same as writing for a broader audience. We have got better at it over the years hence the release of Issue 2.

UC, agree with you. It is hard to find playtester who are not your friends. To find people fully independent that you would trust their opinion for a small publisher would be great. I haven't found any. John I would be happy to sign you up though. Many people volunteer but just want to see your product. You never hear back.
Also as Martin said, " practicality matters too." At some point you just release it because a game is never truly done.
On the flip side, I donate all my earnings to charity and offer a money back guarantee. I make games I want to play and then share. I am fortunate that this is just a hobby for me.

Question: You have some new guys test your rules you've spent a lot of time on and are almost finished. Their feedback after a few sessions is that the game does not give the right "feel" for them and does not "seem" to be realistic. This play tester has never been in the military or combat and knows very little about real tactics of the period.

Do you consider his feedback valid? Would you accept any recommendations or changes from them? What specifics do you ask for?

I developed my rules and action timing from the manuals, AARs, real combat footage, documented weapons performance, driver/mechanics at tank museums in the US, Europe and Russia and real former tank commanders and crewman, some who had experience on a variety of WWII tanks. That part is done and validated. My play testing is on the action sequence and navigating the play aids.

There is no orders phase. As soon as an action or order is executed, like shooting, the player, seeing the result, immediately issues his next order which is normally, reload to shoot at the same target again, reload and engage a new target, move/halt, or go into over watch (default order).

Young players with no war gaming experience but have played online tanks video games catch on quickly because the game is really modeled on intuitive actions in combat, not IGYG.

Many old timers take awhile to catch on because IGYG does not allow you to intuitively do what a real crew does but computer simulations force you to.

I have had some great suggestions from new player on how to organize the rules and play aids I had not thought of.

My play testing centers around developing intuitive play aids and action sequences players find easy to navigate. I mainly observe them to see if they are able to follow the process. If they are having a problem I first look at what I can change because I'm sure someone else will run into the same issue.

If a new player with minimal knowledge of AFV combat, tactics and weapons performance says the game does not have the right "feel" I'm not going to change anything unless he gives me something historical I can work with.

A good play tester should try to "break" the system and find unrealistic loop holes rather than just playing the game and saying whether they like it or not.

If you have an amazing set up and crappy rules most novice players will say they had a great experience. I've had a good time with crappy rules and a great setup.

After a game I do a quick debriefing to ask players what they learned, liked, didn't like and any issues with the play aids or action sequence. I try to get specifics.

Wolfhag

I agree with OFM, but its hard to find playtesters. People occasionally post free rule sets they would like people to take a look at right here on TMP and I don't recall ever reading a report where someone actually played them. If you have an active gaming group, that's a plus, but its not people outside as proposed. Gaming conventions are the only places where I've seen much playtesting with new players actually occur.

Forced Feedback has its own problems:

1. Required comments reduce feedback quality – If every playtester must write comments after every game, many will provide filler ("seemed fine," "no issues") simply to complete the form. The volume of feedback increases, but the signal-to-noise ratio decreases. Anseel & Sherf (2025), A 25-Year Review of Research on Feedback in Organizations.

2. It encourages testers to invent problems – When people are told they must identify issues, they often go looking for something—anything—to criticize. This can overrepresent minor preferences while underrepresenting the overall play experience. Watling et al. (2019), Challenging Feedback Myths.

3. It changes how people play – Instead of experiencing the game naturally, testers begin "auditing" the rules. While analytical testing has its place, requiring it from everyone can reduce the value of observing how players actually interact with the design. Gray (2017), Game Testing: All in One.

4. It increases volunteer burden and reduces participation – Every additional required field or essay question adds friction. Some volunteers will submit rushed responses; others will simply stop volunteering, leaving fewer playtest reports overall. Nielsen (1993), Usability Engineering, discussing participant burden in user testing.

5. It discourages positive observations – A form focused on "What was wrong?" biases reports toward defects and misses equally valuable information about what was engaging, intuitive, or memorable. Knowing what works is often as useful as knowing what doesn't. Anseel & Sherf (2025), emphasizing balanced, improvement-oriented feedback over criticism quotas.

For a wargame/scenario there is also the problem of your new people have played it ONCE. How many times have you played something right the first time?

I always ask for voluntary feedback. I have a lot of experience how to parse many different types of feedback you get from operators. I also always take my own notes, and if possible, have someone else taking notes. "Everybody stopped asking how things work on turn two." is gold for assessing usability. "Everybody kept asking how X works through the last turn" is also gold for assessing usability.

wolfhag comments about the bona fides of a playtester are very valid. If they know nothing serious about the period then thay are not a good source of hiastoric feedback. They may help in feed back on quirks in the system, but not on the historic detauil. Watching Band of Brothers does not make you any sort of historian. Having feed back from someone whohas read lots of WW2 fact books (not fiction) gives the feedback historic weight and is usefull.

I read manuals extensively for our rules but that is not feedback its just basic research. It does highlight some "myths" that seen to persit in "standard rules".

We did have one individual moan that the rules wre not standard format and should be changed back as "that was how wargames were played",. Needless to say that feedback was worthless to us.

Folk over impressed by the visual side are often no use for feedback for us. It can be heavily biased by the number of models on the table as it "looks good" even is massively unhistoric, many rules encourage visual density at the expence of reality, great for some rules perhaps but such folk as feedback are of no use to us, trying to relate to the real world not the standard of medolling. We are trying for thr "!empty battlefield" amd they hate hidden troops it detracts form the visual appeas despit it being almost ludicrous technically.

Gettin good feed back is very difficult.

Our published rules were play tested for at least 6 years at all of the HMGS East conventions, plus a couple of playtests with groups out of the country. When professionals, amateurs and interested fans of the subject all enjoyed the game, we felt it was ready to publish. The rules don't capture the nitty gritty but certainly capture the feel. There was one rule set that did capture all the small stuff but took over 9 hours and felt stagnant.

I agree about the forced feedback.

If the best you can do is attempt to get the right "feel" you'll end up getting a lot of different opinions.

Before writing the rules, I did all of the research for the timing of actions, tactics, etc., with the help of people more knowledgeable than me and tank crew members. Then I compared it to AARs, combat footage, and training standards for validation.

Example: The Germans gave accounts of early in the war of Panzer IIIs which surprised and flanked a T-34/76 (buttoned up when firing), could get off up to 3 shots before the T-34 could react, engage, and fire once. That can happen in my game with no additional special rules or die rolls. It's all about historical timing.

My play testing is mostly to see how quickly players come up to speed and start playing on their own and navigating the play aids. Most of the testing was done at conventions, so a max 10-minute explanation of moves and navigating the play aids, and then we start. I don't use a rule book because the customized data cards and play aids, along with QR codes to bring up a short video, eliminate the need to look up rules.

The feedback from many new players was about balance; the Shermans were firing more often than the Panthers, and it was not fair. I explained the historical mechanics, and they were fine with it. Other feedback was about wrong historical data and myths.

I originally used the real military nomenclature and terminology but had to simplify it. New players are mostly at a disadvantage to experienced players because they know the rules better. My solution was to make a customized data card for each vehicle, which gives the player everything they need to know and how to do it. This includes tactics.

As soon as an order is executed, the player observes the results and issues his next order.

Example: Play the (OODA) Loop: Units that fired immediately loop back to Observe the results and issue their next overwatch, move, or shoot order, and record their new Act Time.

Limited Intelligence: Must fire (Reload Action Timing) the main gun at the same target at least one more time if not on Fire or Brewed Up. Common Sense exceptions (more immediate danger).

Engage a new target: Situational Awareness Check. After any Delay, if the new target is within 100m of the last target, use the Reload Action Timing and 2+ Shots row on the gun chart. No Snap Shot.

Shoot at the same target: No Situational Awareness Check is needed. Determine the 2+ Shot Action Timing (Reload) and Act Time to shoot and record it. No Snap Shot allowed.

Shoot & Scoot: No Situational Awareness Check is needed. Immediately after shooting, place a speed Movement Marker with the unit on the segment matching the game time & Reload.

Overwatch: Default order.

Wolfhag

Wolfhags comment about not using technical terms as the audience is not aware of them indicates a different target audience to ours which is primarily period tecatical enthusiasts who want to use the correct terms where possible. Again rules havwe to have a specific uidience,. A lowest common denominator set is not ever going to plaese everybody.

Wolfhag – wash your mouth out; D6 is a swearword in our circles we only use D20 . Seriously for us D20 is better. It allows the equivalent of 1 roll ve 2 in some cases speeding the game up. Some say an opposed roles is a better option, pertsonally we disagree, fine for die rolling obsessed gamers but not to those who just want quick answer without the excessive preocupation with die rolls certain audiences like and demand. Same as our pathological dislike of "event cards" random, not well matched to the situations gambling, for no sound reason. Again taget audience issues.

UshCha,
A goal in the game was to get new players up to speed as quickly as possible with game mechanics that they most likely never saw before, using a max 10-minute intro.

Introducing and explaining terms such as Battle Sight, Burst on Target, Ranging Shot, Reverse Slope Defense, and Bounding Overwatch at the start of the game was not worthwhile. I could introduce these actions and tactics as an example without the technical jargon. As the game progressed, I'd give explanations in detail.

In my last post, I gave an example of what is on the data card for the player to select and how to execute.

The important thing is that the players are able to understand and use the tactics without needing to know the exact terminology.

In my videos, I explain the jargon.

I'm a D20 kind of guy too. I used D6 as an example because many people use it and the issue of die roll modifications.

Wolfhag

A few years ago, I took a mostly finished version of my game to a convention, went to the kids' room, and got two volunteers to test/play the game. Neither one had played a war game before. The one on the left is 16; the one on the right is 14 with a slight learning disability but had played tank video games.

There were no rules to read and no video shown. Most of the help I gave them was navigating the data card, which was laid out just like the turn sequence, so all they had to do was follow it step by step. The 14-year-old controlled 10x T-34/76, the 16-year-old 4x Tiger Is.

I considered the session a success.

Wolfhag

Any time you kept two teenagers off screens for a while i would also consider that a success! : )

Seriously for us D20 is better. It allows the equivalent of 1 roll ve 2 in some cases speeding the game up.

In which case does two people rolling a die at the same time take longer than one person rolling a die by themselves?

I find it hard to find playtesters. People here on the TMP are not interested for the most part. I for example put up a request awhile back and got 6 peeps interested. Sent them copy after copy as revisiins were done but only 1 answered me back. Hey I understand for what ever reason no interest. I am really forced to do as many convention games as I can.

Any other ideas are most welcome.

etotheipi – we do not optimise for two player games. Its not in our specification. In the uK one on ne games tend to dominate at ;east in our area.

Your preferred game has nothing to do with how the mechanism described works. Also, I'm pretty sure one-on-one is, in fact, two players, as described in the question.

greenknight,
I and others have run into the same thing. My opinion is that it depends on your approach.

The worst approach seems to be just asking for people, sending the info and waiting for feedback. Personally, it takes a lot of effort to go over a new rule set, understand and play it and give feedback.

TMP has a lot of talented and knowledgeable people and I think that's a problem in finding someone because many people on TMP are dedicated players and have developed their own rules and not really looking for something new. Some are confirmed know-it-alls too <grin>.

My approach is going to be doing a Zoom call with interested testers. Then if interested, send the info and do the testing in real time session with them at their place like Face Time or Zoom.

I'm also putting a QR code on important parts of the rules to give a short video example of the rule. Some designers charge the potential testers a nominal fee for the material to see how interested they really are.

Good luck.

Wolfhag

I associate bounding overwatch more with modern infantry than ww2. I think it'd be called fire and movement in ww2. I seem to recall some veterans and Dad saying that. My father taught infantry school.
Or maybe this is just Uk Vs Us terminology.

He also called platoon movement the three legged beast. 2 legs down, one moves. Then another moves.. Not sure that's out the manual like, but seems a fair description.

Hmmm three.
Tell them three times, show them three times was another one.

@Wolfhag

Good points, you have set me to pondering :)

Thank you

Chris

greenknight,
Thanks for asking, it gave me some ideas too.

Wolfhag

Andy,
When I was in the infantry we never used bounding overwatch as a term but I have seen that today it is and it sounds like regular fire & movement which in essence is bounding overwatch.

Manual: link

BOUNDING OVERWATCH
14-36. The commander uses the bounding overwatch movement technique when he expects to make contact with enemy forces. There are two variations of this technique: alternate bounds and successive bounds. In both cases, the overwatching elements cover the bounding elements from covered, concealed positions with good observation and fields of fire against possible enemy positions. They can immediately support the bounding elements with maneuver or fires alone if the bounding elements make contact. Unless they make contact en route, the bounding elements move via covered and concealed routes into the next set of support-by-fire positions. The length of the bound is based on the terrain and the range of overwatching weapons. The commander can use the uncommitted part of the force whenever he feels it is needed as part of an immediate and controlled reaction to any threat to the bounding force. In bounding overwatch, all movement keys on the next support-by-fire position, which should offer at least some of the following advantages:

Cover and concealment.

Good observation and fields of fire.

Protection for stationary weapon platforms.

14-37. If the unit uses alternate bounds, the lead element moves forward, halts, and occupies a support-by-fire position that is covered at all times by the rear overwatching element. That former rear overwatching element advances past the former lead element and takes an overwatch position. The initial lead element then advances past the initial trail element and occupies a new support-by-fire position. One element moves at a time. This method is usually more rapid than successive bounds. (See Figure 14-10.)

Wolfhag

They couldn't just call it "leapfrog"?

So, to people in general, do you have special rules for Bounding Overwatch, or as Wolfhag says, is it just fire and maneuver?

we don't have "rules" for bounding overwatch its just a form of fire and maneouvre you can use if you deem it neccessary. Like it says how long the bounds are depends entirely on the terrain and commanderes decide as neccessary.

This is the same way we deal with formations. We define radio nets and the limitations of visibility for vehicles. You fined you adopt formations to optimise arcs of visibility/fire arcs positioning to rapidly expoloit information.. Having vehicle to far spread out maens that the information they obtain may not be as useful as other elements on the same net may not be able to explot it as quickly.

In effect you design in as much as you can, the causes for tactics not the tactis themselves.