"Exploring History in the Wargame" Topic

Richard,

I was not accusing you, personally – or anyone in particular – of hijacking the thread; if it read like that, please accept my apologies.

After reading a large number of posts on other threads concerning the Game/Simulation debate, I simply felt that this has proved to be a rather pointless, sterile argument.

I used the analogy of historical novels because I believe that historical hobby (as opposed to military training)wargaming is essentially the same: an artistic, imaginative, entertainment in which there will inevitably be a great variety of interpretations of the past, and of methods of creating playable games from them. Just as historical novels range in quality from Barbara Cartland's 'bodice rippers' to MacDonald Fraser's 'Flashman' series and Patrick O'Brian's novels of naval warfare, so do sets of wargame rules and game structures.

As you rightly say, a wargamer can 'choose which interpretation of history he feels most comfortable with and find a set of rules that conform to that.'

I also agree that 'one can find points in between [closed and open games]where you combine aspects of both; where commanders in a tabletop game are still not provided with all of the information that they would ideally like to have.'

But such compromises – as I believe they are – between the reality of battle and the perfectly legitimate desire for hobby wargames to be social/sociable events rather than CPX's have sometimes resulted in some strange, counter-intuitive mechanisms! I'd better not give examples, for fear of starting another debate…

You say: 'the game can serve as a legitimate model of historical reality that we can use to gain some appreciation of reality.' Personally, I feel my own appreciation of, say, Napoleonic warfare has been gained far more from reading memoirs and historical analysis, taking part in reenactment, wearing a replica uniform and firing a black-powder musket, than by subsequently playing wargames; indeed, for the reasons stated in my previous post, the rules I began wargaming with gave me a false impression of the significance of musketry, as it is now understood, in infantry combat. That was rectified by reading, not by playing wargames.

Which is not to say that persuading someone with much less prior knowledge of the period to play a game might not prove very informative for them.

Arthur

No, the thread hasn't been hi-jacked … and, for me, at least, has been an interesting read (a good number of positive contributions). I share Richard's ambitions for what the wargame can be.

I take Arthur's point about novels and have used the movie analogy myself over the years: the ability to create or explore narratives and, particularly, to vary the intensity of focus rather than simulate just one perspective. I don't think that implies a game which is entirely open (although when using the format to tell a story – which is what the presentation at the gallery was doing – it usually will be).

I take Arthur's recent comments entirely on board. The wargame is only part of the exploration of the history that inspires it. To the other parts Arthur cites I would add visiting the battlefields – especially with a well-informed guide. I am grateful to the Battlefields Trust in this respect and have always found people only too willing to help.

I must say, though, that the wargames and the models do continue to inspire and to help me organise my thinking. There is a disipline in designing a scenario or putting a game together that makes you dig that bit deeper because you actually do need to know … a hole you need to plug, a riddle you need to incorporate.

And I'm sure that's a good thing.

Phil
ecwbattles.wordpress.com

Phil wrote:
"I must say, though, that the wargames and the models do continue to inspire and to help me organise my thinking. There is a disipline in designing a scenario or putting a game together that makes you dig that bit deeper because you actually do need to know … a hole you need to plug, a riddle you need to incorporate.

"And I'm sure that's a good thing."

Absolutely! Researching and designing the wargame stimulates and informs you; then, playing your game may do the same for others. That's why the 'DIY' model of writing one's own rules, scenarios &c., is preferable to the 'consumerist' model of buying pre-packaged rules, army lists, scenarios &c.

Having read Phils reply I wonder if the real debate is not just games vs simulation, but also between narrative games where each side has a specific aim in the action and the line up equal points and see who wins style.

There is a danger in Historical bodice rippers like those of Cartland and Bernard Cornwall in that they promote certain myths and untruths. Cornwalls most famous creation typifies this. Sharpe is portrayed as unusual as a Commissioned Sergeant, but as both Houlding in 'Fit for Service' and Holmes in 'Redcoats' makes the point that notonly was it not unusual it was in fact quite common, indeed a third of the British Armies Battalion Officers were commissioned NCO's in the Napoleonic Wars. Indeed there is even records of one who was commissioned twice…

John the OFM's arguemnt wouldnt hold water if every rules design decision was based on an opinion, but they are not

artillary outrange muskets and muskets outrange swords….(irony on…) that's just of course my prejudice speaking, nothing to do with technical research or evidence.

effectiveness at a specific range is subjective, but isnt totally disconnected from reality.

Making sequences of firing where the 'reaction ' is in front of the 'ambush' is a design decision that is a disconnect from reality for some percieved gain in gameplay and will always seen to be intuitively flawed….However, the justifying catcall of 'its fun'….implying that it has some sort of monopoly on fun sounds so uninformed that it sounds irrational. they could have as well stated 'its got dice'

taking the last of one of these situations and trying to justify it by hoovering up every other aspect of rules design and research to tar everything with the same 'only a game so none of it is valid' brush…well!

This is from the designer's notes (p. 25) old hex-and-paper wargame 1776, from Avalon Hill.

The idea that "a game is a good simulation, therefore it is a good game" is not necessarily valid for all purposes. Certainly no one will argue the fact that no game available to the wargamer to pursue his hobby is a perfect simulation of real life events, or even an attempt at a pure simulation of events. Rather, the game is unto itself. The simulation game is a game that TENDS to operate as a simulation but is still a game as far as the uses to which it is to be put. The value of the simulation aspect of the game is in the number and kind of decision making interactions that the subject simulated generates. If we are simulating something that does not directly of indirectly create a device for competitive decision making in the game sense, then the usefulness of the effort to simulate is brought into question. Selective simulation must be employed to distill the essence of the conflict being re-created while keeping the design in manageable proportions.

There's more; I can repost if anyone is interested.

This was written 36 years ago, and I think it is very "on point." Apparently the arguments haven't changed much in the meantime…

Scott S:

Yes, it is 'on point.' And yes, sadly the arguments haven't changed much within the wargame hobby, but they have in the larger simulation game industry. There has been four decades of simulation and game development since 1776 by thousands of simulation and game designers. This means that the statement has been developed and passed by in significant ways:

Selective simulation must be employed to distill the essence of the conflict being re-created while keeping the design in manageable proportions.

All simulations, no matter how detailed are 'selective simulations.' All simuations have to be in managable proportions in relation to their purposes. It isn't something unique to wargames.

There are now concrete methods for 'distilling that essence' in practical, concrete design methodology--how to's.

Now there are methods for objectively establishing that a simulation game does indeed 're-create' what it was designed to, regardless of its topic, proportions or medium, game, computer, role-play or pencil and paper.

The hobby 'discussions' regarding game design and simulations always center around 'how' more than what. There are practical, proven methods detailing that 'how.'

Bill H.

This means that the statement has been developed and passed by in significant ways

Not on here, apparently. It's the same argument over and over and over.

"If we are simulating something that does not directly or indirectly create a device for competitive decision making in the game sense, then the usefulness of the effort to simulate is brought into question. Selective simulation must be employed to distill the essence of the conflict being re-created while keeping the design in manageable proportions."

This should be tattooed on the inside of the eyelids of some game designers I can think of.

"Not on here, apparently. It's the same argument over and over and over."

No, It's the same propoganda

…which when hit with statements like this…..

"Now there are methods for objectively establishing that a simulation game does indeed 're-create' what it was designed to, regardless of its topic, proportions or medium, game, computer, role-play or pencil and paper."

say…its only a game and you are fooling yourself believing otherwise.

Thousands of man years of research in mathmatics, computing, game design theory and its offshoots/offroots used in civilian commercial use succssfully on a daily basis,

Now that I think of it, We have now reached the stage that there is one combatant who cannot tell the difference between a simulation and his real combat……. The UAV drone operator. He is of course fooling himself if he thinks it accuratly simulates flight.

Scott S. wrote:

Not on here, apparently. It's the same argument over and over and over.

Yep, but it seems to be reserved for the wargame hobby. It doesn't have to be, not anymore.

Bill H.

On a lighter note, isn't Avalon out of business? Shows you what they know!

Its like the joke with some merit, your on an airliner and all the flight crew get ill at once, they no longer would ask is there a pilot on board, but has anyone played the flight sim for this plane…

Let me put it this way:

Can a tabletop wargame really simulate war? No. There is too much involved in war for any game to fully represent it. This sort of thought process is the same as the "Total War" question in history. There has never been a truly total war, because it is actually an impossibility. You cannot really mobilise everything and everybody for warfare.

However, that is decidedly not the same as saying that a game cannot simulate aspects of war. We certainly do that, as evidenced by many a "historical result" and not a few times when historical tactics work well in the game.

Now, for my money, a game needs two things at once, because for me they are interrelated. First, the history of the time must come through in the game. Historical tactics work, historical organizations and fighting styles happen because they make sense, and the like. I play an English Civil War game to get a certain amount of feel for the period and what happened. Second, it must be fun to play. As a historian and (obviously) a history buff, the history is a big part of it. However, play mechanics, rules complexity, and other factors unrelated to the history are also part of that mix.

Is this all a very clear and easily articulated thing? No. But, like the US Supreme Court on pornography, "I know it when I see it."

Bob Bailey

Well, I enjoy toy soldiers. So I play with them in war games. Some of these games are, more or less, "informed" by history. I am not convinced what any of us do, on the table top, are simulations of any real combat.

If I want to learn about history, I'll read history.

Nice to see I'm branded a "Nazi" for having those opinions. Sheesh.

Lots of people use simulations on a day by day basis, learn from them constantly and have a better understanding of the real thing than the people who are actively involved with the real thing.

This happens in lots of fields of life….why not in wargaming? What makes it so special?

If I want to learn about history, I'll read history.

Waste of time. All you will get from books is the bias of the authour. John the OFM has said so, so it must be right! :-)

Nice to see I'm branded a "Nazi" for having those opinions.

I think the nazi comment relates to people who feel the need to ram such opinions down others throats and derided anyone with a contrary view as deluded.

Lots of people use simulations on a day by day basis, learn from them constantly and have a better understanding of the real thing than the people who are actively involved with the real thing.

This happens in lots of fields of life….why not in wargaming? What makes it so special?

If you think what happens on your table top is an accurate simulation of actual warfare, the same way operating a 747 simulator simulates flying a plane, then more power to you.

My war gaming is informed by history. Therefore, my IJA troops in Saipan have Type 97 Chi-Ha tanks, and not Type 89 Chi-Ro tanks. Does that mean I'm reproducing an accurate simulation of Japanese tank warfare?

I think the nazi comment relates to people who feel the need to ram such opinions down others throats and derided anyone with a contrary view as deluded.

That's not what was written.

Bowman, take a deep breath and read the thread again. Especially this bit:

"John (the OFM) says to me, (and I theoretically quote) "Richard, I want to play a game that is just that; a game. It's historical content is minimal as it merely represents the prejudices of the rule writers." I say, "Good on you, old chap, I really hope you enjoy it". And I mean that sincerely. Folks.

I say to a chap, be that John (the OFM) or another who has decided to adopt the stance of an "It's only a game" Nazi, that I am looking to play a game that I hope will allow me to gain some historical perspective about a particular engagement. What does he say to me? He tells me that I am a complete blooming idiot for attempting any such thing as it can't work. It's "just a game"."

Then read what badger22, who has had his ass in the sand, wrote. No-one is saying that it is "an accurate simulation of actual warfare" or that we are learning history. What we're saying is that you CAN have a game that teaches you about certain aspects of combat and that gives you a greater understanding of history, as it helps you understand the conditions under which certain decisions were made, for example.

Bowman wrote….

"If you think what happens on your table top is an accurate simulation of actual warfare, the same way operating a 747 simulator simulates flying a plane, then more power to you."

That same flight simulator with added encryption is what operates real life drones!…If you were standing on the drone pilot's shoulder and he pressed the trigger button on his joystick to launch a 2000kg guided bomb at some insurgents, would you tell him at that point you are 'unconvinced' it represents real combat?

But that is only one type of simulation.

I use simulations to design buildings….I understand and have learned to use them so I understand how real buildings will behave in use…to the point that I understand what aspect of the construction and its erection sequence will be the most critical for that specific structure, something the builder himself will never have adequate knowledge of ….even though he is building the real thing. He is more often than not, told how to build it.

This is standard practice in construction all round the world.

So that I can clarify this ….

Real builders building real buildings are utterly dependant on non-builders operating simulations and these are not all by computor….they are done to a rule book, normally known as the ACE or British standard or more recently eurocode. The computor systems are in fact a mixture of mathematical analysis and the rulebooks application. The professional bodies of these non-builders generally demand compulsary logging of their professional development ie, the learning from the simulations.

But apparently, people remain unconvinced that there is in fact any relationship between simulation and real life….

According to some, this is impossible, a delusion held by the niave and the arrogant, and for you, you would be 'unconvinced'

If you accept the fact that there is some relationship, between simulation and reality, then every set of rules has some level of realism and it is quantifyable..Anyone can use mechanisms to build a better model and a better simulation of combat.

I do agree with KevanG. Simulation design is being done all the time, even on the tabletop. I think a great part of the controversy regarding what our wargames can and can't do in representing history is a lack of any fundamental agreement about what simulations are and can be. This isn't some mystery, or some question that only our hobby has faced. The answers are specific, functional and used on a daily bases, except by our wargame designers.

When Bowman wrote:

"If you think what happens on your table top is an accurate simulation of actual warfare, the same way operating a 747 simulator simulates flying a plane, then more power to you."

The belief here is that a tabletop simulation is somehow fundamentally different than a 747 simulation. One uses game rules and the other computer program rules. The computer can handle more information, but neither are ‘accurate simulations of anything actual' in the sense Bowman seems to hold as the standard. Both use procedural systems, to represent something else by artificial means.

The computer can handle more information, but more information doesn't make it a more accurate simulation, just a simulation of more. The accuracy is in how well it mimics the selected parts of reality. Both the tabletop wargame and computer simulations use the very same methodology and techniques to represent reality, just different mediums, wargame rules and a computer. Both face the same limitations and both are artificial, not ‘actual.' No one wants to play at ‘actual warfare' anymore than they want to put an untrained person in the pilot's seat of an actual 747. Simulations can and do represent ‘selected aspects' of reality. The question is what the game designer or 747 simulation designer have selected to represent in their systems and how well they have done it. Only the uses are different, one is meant to be an entertaining game and the other training for real flight.

Bob B. makes the point about the selective aspect of simulation/wargame design. No simulation can do it all. The problem is that wargame designers aren't specific enough about what history has been selected, is being represented, which is VERY different from the 747 simulation designer. Because of this information vacuum regarding our wargames, Bob says:

"Is this all a very clear and easily articulated thing? No. But, like the US Supreme Court on pornography, "I know it when I see it."

This is the problem. It should be a clear and easily articulated thing. It is simulation 101 that the participants should be educated in what exactly is being simulated. To use Bob's analogy, the only time the question of what constitutes ‘pornography' comes up is when the creator of that entertainment denies that it is pornography. It wouldn't be a question at all if the author of stated upfront that he was producing pornography and how.

Wargame designers are upfront about what they are doing: they are simulating, say they are recreating, representing, mimicking, etc. etc. history and battle. They are doing this with very concrete game mechanics, so any answers should be very concrete too. The question is what history is being represented where?

A game designer recently published a set of rules that he stated were a "balance between playability and historical accuracy." However, what history was included [remember all simulations involve selection] or where the history is represented by the rules is never explained—not even what he meant by ‘historical accuracy'. This information vacuum includes what history was sacrificed for playability.

The designer of the 747 simulation includes specific descriptions of what part of the simulation represents what part of reality, what has been left out and what hasn't. It is clear and easily accessible, and necessary for the simulation to work well as a simulation for the participants. The participant knows exactly what design relationships exist between the simulation system and what it represents. In fact, without that knowledge, the actual simulation experience becomes a guessing game of what artificial simulation aspect represents what part of real flying.

That is where we are: Guessing what history is represented and how. So we endlessly debate unknowns rather than having what should be included in our wargames: Clear and easily accessible information about what specifically is being represented. That is what simulation designers provide.

Bill H.

> The participant knows exactly what design relationships exist between the simulation system and what it represents. In fact, without that knowledge, the actual simulation experience becomes a guessing game of what artificial simulation aspect represents what part of real flying.

That is where we are: Guessing what history is represented and how. So we endlessly debate unknowns rather than having what should be included in our wargames: Clear and easily accessible information about what specifically is being represented. That is what simulation designers provide.
(McLaddie)

Thanks.

But what makes you say (in relation to the historical wargame) that we are left guessing about this? Most game designers (historical, ancients, at least …).. explain their rationales, subject their data to scrutiny and explain their parameters to those parties that ask. Generally.

Those who are arguing that there can be meaningful historical content in a wargame (and note I never claim simulation or realism – 'trigger' words you will not find in any of my assertions) are generally playing straight. And are usually more than willing to explain where we think the value is and why we have constructed things the way we have.

It is the 'it's only a game' brigade who are ducking the issue.

Plainly it is more than just a game to the majority of the peole who engage with it – even if that is only in the detailing of the models or the research of orbats or in the attempts to portray convincingly aspects of historical battlefields. So saying 'it's only a game' is just a cheap way of failing to answer exactly your 'what is represented' question.

A patronising way of saying 'I don't care'.

It has been refreshing to follow this thread and see that a reassuring number do care about the content and respect the work that goes into providing it.

Phil Steele
ecwbattles.wordpress.com

But what makes you say (in relation to the historical wargame) that we are left guessing about this? Most game designers (historical, ancients, at least …).. explain their rationales, subject their data to scrutiny and explain their parameters to those parties that ask. Generally.

Phil:

Well, I am making a generalization, so there is a continuum here, to be sure. There are designer's notes that do provide rationales and explanations, some do it better than others. However, they nearly universally remain inadequate for two important reasons: [even when claiming years of research and playtesting]

1. The specific history and/or sources that they are representing with their game mechanics are not identified--at all. The notes provide generalizations rationales with no historical references. It is left up to the player to attempt to make specific connections between play and history, combat and battles. I can provide examples, if you'd like.

You read players guessing at and insisting on those connections constantly in all TMP threads and wargame lists.

2. What specific game mechanics represent--and don't represent--are not identified. There are lots of game mechanics that don't represent anything, but simply support the game process. Other mechanics are meant to represent specific battlefield dynamics. When a gamer plays the game, what specific connections are being made with play, what activities aren't?

Such things are explained in detail in all simulations,that 747 fight simulator, for instance, and wargames are far more abstract than a flight simulator, thus needing far more clarity identifying connections between the history mimicked and game play.

I can give examples of this failure in our wargames, but I hesitate to be specific because I have found that this is seen as a criticism of a particular game instead an example of a general design problem, and the discussion goes south.

Those who are arguing that there can be meaningful historical content in a wargame (and note I never claim simulation or realism – 'trigger' words you will not find in any of my assertions) are generally playing straight. And are usually more than willing to explain where we think the value is and why we have constructed things the way we have.

I agree. That has been my experience. Unfortunately, 'simulation' is one of those words with a lot of baggage in our hobby, suffering from a lot of misunderstandings. The word 'realism' on the other hand, is so vague as to be meaningless in wargame design terms. It devolves into "I like it" or "I don't like it" because it doesn't meet some gamer's expectations… which is easy to do when designers aren't specific about what to expect in play visa vie history anyway.

Or "realism" is mistakenly taken to mean 'more detail', such detail having no inherent ability to add 'realism' to a game system or mechanic, and can just as easily detract from any impression of 'realism.'

Wargames are very technical creations, designed to do specific things, to represent very specific activities, very specific history. The methods and technical skills for doing all that with clarity are being by practiced by simulation designers in many fields every day, including and especially the game industry outside out hobby.

As long as there isn't any clarity, the 'it's just a game' folks will continue to find it far easier to deny that still vague content and representation in play, than to accept it.

It is refreshing to see folks defend what nearly 100% of all wargame designers claim for their designs today, that historical content and representation in historical wargames, even those who insist 'it's only a game'…

Bill H.

It is the 'it's only a game' brigade who are ducking the issue.

Plainly it is more than just a game to the majority of the peole who engage with it – even if that is only in the detailing of the models or the research of orbats or in the attempts to portray convincingly aspects of historical battlefields. So saying 'it's only a game' is just a cheap way of failing to answer exactly your 'what is represented' question.

A patronising way of saying 'I don't care'.

Phil, you are using a very wide brush here. I don't know anyone who says, "I don't care". You are trying to invoke a "straw man". If having the correct detailing of your models, or having the correct orbats, or like my example states, not deploying Type 89 Chi-Ro tanks for Saipan convinces you that you are playing aa accurate representation of an actual battle then good on you. Who am I to say how to enjoy your hobby.

As to your "cheap answer". I am playing a game. For me to enjoy this game (which is the only reason I work hard at this hobby- for my enjoyment) I try to get as many historical details correct as I have control over. Therefore, the Romans fighting my Sassanids are not Imperial Legionaries, my Saipan Japanese drive Type 97 Chi-Ha tanks, and my 1914 French do not storm trenches with Chauchats. Just because I try to get the details right, I'm suddenly simulating warfare?

KevanG and McLaddie go off on tangents how simulations apply to other things, and I agree with them. I just feel that what we do on the table top leans for more to the extreme of "a game played with toys", than the other extreme of "accurate simulation modeling real life". Getting some of the details right, is not enough.

Tangent?…oh no sir!

Every wargame is a simulation.
Every simulation is a model of a real system,

My point is that despite simulations being accepted as accurate ways of modelling the real world accurately enough to actually construct it to the outside world, some wargaming people beleive that it is beyond the capability of man to use the simulation as any model of combat, let alone an accurate one.

"……..and my 1914 French do not storm trenches with Chauchats. Just because I try to get the details right, I'm suddenly simulating warfare?"

the warfare you are simulating is defined by the parameters you put into it. You will not have changed the realism of the wargame by including or excluding the tanks. You have altered it's authenticity in a historical context. The simulation will not have changed a bit if you use the same simulation….i.e the rules.

Where it lies between 'game played with toys' and 'modeling real life' will be determined by the rules..ie. the simulation, but do not be fooled into thinking that just because it is a 'game played with toys' it can never be anything more.
monopoly is a game played with toy money…but it can sure teach you that income needs to be higher than expenditure to avoid bankrupty

Any historical game played with toys is starting to model real life as soon as the rule writer put finger to keyboard….thats what he is trying to do.

"Getting some of the details right, is not enough."

I go back to what I said before for simulations outside of wargaming…..Getting some of the details right is more often than not more than enough.

What makes wargaming people think it is so special it is beyond our capability to model combat?

I think some of the confusion in discussing simulations is is over the idea that "a game played with toys" is somehow at one end of a continuum with accurate simulation game at the other. Nope.

Many games are simulations, they accurately represent parts of reality, the dynamics of the game mimicking the dynamics of some portion of real life. Monopoly is a good example, and as the designer meant the game to model how capitalism works, it does the job in some respects as a simulation.

Our toys are simply game markers. They, like all game markers, record the "where" and "what" in the wargame processes. In our case the markers represent military units and their traits and conditions, just as a military symbol on a cardboard chit or on a military map would.

We certainly play with toy soldiers. No doubt about it. It's what they represent in the wargame that matters in our hobby. Those who insist we are just playing with toys are discounting the game rules entirely.

Bowman wrote:

KevanG and McLaddie go off on tangents how simulations apply to other things, and I agree with them. I just feel that what we do on the table top leans for more to the extreme of "a game played with toys", than the other extreme of "accurate simulation modeling real life". Getting some of the details right, is not enough.

The two aren't mutually exclusive.

1. A game played with toys can accurately model something of real life or history. It's all in the rules and game play, and that is true whether we simulate using toy soldiers, cardboard chits or a computer program.

2. Kevan and I were simply pointing out that simulating with games and doing the things that wargame designers are already claiming to do with their designs is not some extreme between toys and 'accurate simulations', or outside the ability of current tabletop game rules to achieve. Other disciplines, businesses, entertainment industries are already doing it.

3. And yes, 'getting the details right isn't enough'. That too is simulations 101. A simulation and a game are procedural systems, dynamic interactions of several processes. All the details can be right and the system can still fail to simulate anything at all. That's why, like games need playtesting to be sure they play well, simulations also need to be tested to see that they actually model what they were designed to model. [And yes, there are proven methods for doing just that.]

It's not the number of details, or just whether they are 'right', but whether the details work together to effectively to simulate what they were collected together to simulate with the game system.

The very old myth that 'more detail creates a more accurate simulation' still haunts our hobby. It is a very counter-productive belief when it comes to either game or simulation design, one not found outside our hobby. [not to say that you, Bowman, adhere to it.. just saying.]

Bill H.

Bill,

In DBM and DBR Phil Barker explains how the individual troop types 'work' and how the rules intend to simulate their interactions, and then the army lists explain why the individual troop types are rated the way they are in a specific historical context, often with reference to a particular historical source about why these people were Pike (Inferior) and why those crossbowmen are Bow (Superior). Do they hit the mark as successful simulations for you?

Regards

"My prejudices are better than your prejudices!"

I'm starting to see that!

"My prejudices are better than your prejudices!"

Its always better to use predjuces…opinions need back up

Whirlwind wrote:

In DBM and DBR Phil Barker explains how the individual troop types 'work' and how the rules intend to simulate their interactions, and then the army lists explain why the individual troop types are rated the way they are in a specific historical context, often with reference to a particular historical source about why these people were Pike (Inferior) and why those crossbowmen are Bow (Superior). Do they hit the mark as successful simulations for you?

Whirlwind:
I did own DBM and DBR at one time, but sold them many moons ago. I remember the explanations as being a lot of unsubstantiated generalizations, but I've seen a lot of rules in the meantime, so I don't trust that memory. Give me an example from the Designer's notes and I might be able to make a more informed response.

DBM or any other game designed as a simulation should provide three parts for the player, so they know what they are and aren't recreating in game play:

1. What specific history was used in designing the game system. [One specific source/example is enough]

2. How the game mechanics are meant to model those aspects of history and battle identified.

3. Once the system has been designed, was it tested as a system to see if it actually did succeed in simulating the history noted in #1.

The simulation target is identified [1], the method for hitting the target is created [2] and whether the effort was 'accurate', and actually did 'hit' the target is proven [3]

The depth and meaning of the play experience is dependent on the players knowing what is being simulated, where and how, with enough specificity that the players don't have to guess at what is being modeled in play.

If Barker provided those three explanations in his designer's notes, then you don't need me to tell you whether the game is a simulation or not, and anyone else would come to your conclusion. That would be my standard for simulation design--whether some one would enjoy playing it is another question, but knowing what the game play offers in the way of history and battlefield dynamics could only add to that enjoyment.

Just for clarity concerning generalizations vs specifics:

Many game designers generalize about why many rules have units moving much slower on the tabletop than they appear to have in actual battles. They say it is because of the many command errors, terrain, and general confusion that slowed them. That is a generalization.

The question is what history did they read to come to that conclusion? Specifics would provide an historical example illustrating what that generalization was based on at the level of the game. Then the designer, once the game has been designed, would test to see if that game generalization allows the system to mimic actual battlefield movement.

Anyway, that three part process is necessary to establish whether a simulation design works as a simulation.

Best Regards,

Bill H.

Wow..I have a headache after ploughing through the above.

I have one extra aspect to add to this argument. Actually attempting to write rules for a period creates a learning situation. The author has to confront their predjudices (or at least some of them), they have to recognise that they cannot hope to create a fully detailed simulation without loosing playability, and so they have to decide which aspect they want to cover in detail and which they want to generalise. The background research often tears down what they thought they knew and replaces it with (hopefully) fact based assumptions….and I use the word assumptions deliberately. We can only aspire to recreating some aspects of warefare at a particular place and time, not at all places and all times.

Now some details can be found that give a very close match to what we expect to see, such as musketry hit rates at various ranges from military test firings. We then have to make an educated guess at how closely these figures match the hit rate of unit under return fire, who are scared, confused, and not as well trained as they might like. BUT it gives us pointers as to some of the critical issues that should be included. As noted above these critical issues change over time as our interests and understanding change.

Overtime I have moved away from rules that concentrate on which weapon system your legionarii are using at each stage of the fight, and how many of the enemy they are taking out of the fight, to rules that look at the decision points and the decisions made. That reflects my experience as a re-enactor. Both types of rules cast light on a different aspect of a battle. Both tell something about the mechanics of battle and both totally miss something….the fear, panic, confusion, pain, and risk the combatants had to deal with.

So yes it is only a game…no one gets killed…but a well designed game should cast a little light on at least the mechanics of battle if not the psychological impact.

But hey thats just my two penneth.

Ah don't worry about it Bill – DBM/R woud fail your test #1.

Regards

A glib comment, Whirlwind, and good for a laugh …

Of course, laugh over, Phil Barker has discussed all his sources at very great length for 40 years – the index to Slingshot will tell you that. He is a fine example of a rules writer all of whose output for the ancients period passes test #1 with flying colours.

Because of his honesty about sources, we might be in a position to discuss test #2 and test #3 and be critical …

Given the succinct format of the DB series of books, a comprehensive list of what made up the sources archive is not included … see Slingshot for the first 3 decades or so, then I believe the discussion of those sources continues on the DBMM list (which is also quite extensive).

A feature of the proposed format for DBA V3 will see the main source(s) engaged given in the notes to each army list (so you can check and/or explore further) …

You may notice from my work on the Naseby project that I used a modified version of Armati for reconstructing this battle (so I have no vested interest in validating Barker/WRG – indeed, it is more the derivative systems like Armati, FoG and historical spins from the WAB stables that might struggle … as so much is built on Barker's work).

Phil Barker would be one of the writers I had in mind as being quite happy to explain their methodology and discuss the sources for their point of view. But you can add in Neil Thomas, Rick Priestley, Jervis Johnson, Richard Bodley Scott etc. as well. I've never found any of them other than happy to explain themselves and actually quite pleased when players take as much interest in the research side of designing the mechanisms as they do.

Phil Steele
ecwbattles.wordpress.com

Copied over from previous Sim/Game thread as I am wondering if I'm on the right track, if there is one….

So as I gather from the above,

A simulation of complex things is never perfect.
All historical wargames are simulations.
No historical wargame is a perfect simulation.
Nobody has tried to validate the results of historical wargames by reference to historical outcomes.
Therefore its not possible to say to what extent they are good or bad simulations.

I think that all wargames attempt at least to some extent to simulate aspects of historical reality. For example, cavalry move faster than infantry, artillery shoot further etc.

Some aspects are reasonably easy to check – how far can a musket shoot? What thickness of RHA can a 6pdr sabot penetrate? etc. Some I think are very difficult if not impossible, eg How many casualties can my battalion take before it runs away?

Maybe then the approach could be to think of evaluating wargames rules on how well, on average, they replicate the results of whole battles. Now we know that many battles hinged on fluke occurences, were horribly unbalanced to begin with, have insufficient detail recorded to wargame them faithfully etc. However if one took a large-ish number, then this ought to even out a bit.

So how about this plan to validate a set of, say, Napoleonics rules?

1- choose 100 reasonably-sized battles with a decent amount of information recorded about them

2- play out each battle with the exact orbats and starting positions and orders using your set of rules (maybe repeat a number of times!)

3- The percentage that agree with the historical result could give at least a rough indication of whether the said rules are a good simulation.

I wouldn't care to wager whether the more complicated rulesets would be 'better' by this measure than the simpler 'fudgier' rulesets. Maybe the fudge-factors represent the experience, reading and historical understanding of the authors and do actually improve the simulation, perhaps unconciously, or maybe not, but I wouldn't bet against it!

Bowman … Fred C is right about the sense in which Richard used the term Nazi. This thread began a few days after those of us who attempt to include useful historical content in our historical games were branded, not for the first time, 'naive and arrogant' for thinking other than it being only a game. Period.

As for your games, which, as far as I can see, nobody has criticised (certainly not me), it sounds from your description as if you have included some interesting historical detail where that is fascinating to you. So I think you are making good use of the game.

I am the last person to suggest that that makes it a simulation of any sort (see my comments to Bill, I avoid the word simulation and any of the baggage and/or implications it brings with it) of all the words you are trying to put into my mouth, simulation and accuracy don't fit. Nowhere in this thread do I use them.

What I have drawn attention to is plausibility, historical input and period detail. And the way those elements can enable the game to spark enthusiasm, they can engage people's imagination in the historical narrative, they can help explain and explore the events of the past.

Using a reconstruction of the battle of Naseby to do just those things in a public (non-wargamer) gathering (as I have done before) reaffirmed my confidence that there is something worthwhile in doing so. Compared to some gamers, the public are actually quite open-minded about this.

You said …
> I just feel that what we do on the table top leans for more to the extreme of "a game played with toys", than the other extreme of "accurate simulation modeling real life". Getting some of the details right, is not enough.
…

That is fine for you ('your table, your focus') … I sort of agree, but put the greyness more in the useful middle of the spectrum.

But the point here is that although many TMPers assert the 'game played with toys' extreme end, nobody here has for a moment asserted the 'accurate simulation modeling real life' position. Perhaps you need to read the posts more clearly.

You seem to rewrite what people have said in much more extreme and dogmatic terms solely in order to be able disagree. Hmmm.

I am more than happy to let the gamers have their fun games with no challenging content as they please. I am also more than happy to discuss, promote and enhance the historical game amongst those who appreciate its added value.

Thankfully, though, it does seem that when the more snide members of the former camp interupt a worthwhile discussion of the more serious sort with the rudeness we often see, there are plenty of us who will stick up for what we believe in.

This thread touched upon issues of authenticity and historical meaning/content, yes. But mostly it was about using the wargame and its formulae to explain and explore history – both amongst wargamers (and historians) and with members of the public. To bring the events of the past to life.

It can do that very well.

Phil Steele
ecwbattles.wordpress.com
soawargamesteam.blogspot.com

Overtime I have moved away from rules that concentrate on which weapon system your legionarii are using at each stage of the fight, and how many of the enemy they are taking out of the fight, to rules that look at the decision points and the decisions made. That reflects my experience as a re-enactor. Both types of rules cast light on a different aspect of a battle. Both tell something about the mechanics of battle and both totally miss something….the fear, panic, confusion, pain, and risk the combatants had to deal with.

Elenderil:

I think what you are describing is moving from the false notion that a simulation is a collection of data bits, the more bits the more accurate the simulation to a view far more in line with actual simuulation design today.

Your preference for "rules that look at the decision points and the decisions made" is about the play, not the factiods or numbers. That what a wargame simulation is doing, providing players with similar options, decisions and consequences in a game system to those faced by battlefield commanders. Not all the decisions etc., but selected historical dynamics. All the data bits can be right and the system still unable to simulate anything. To rephrase what you said, It's the play that counts, not what is counted…

Bill H.

Some aspects are reasonably easy to check – how far can a musket shoot? What thickness of RHA can a 6pdr sabot penetrate? etc. Some I think are very difficult if not impossible, eg How many casualties can my battalion take before it runs away?

Timbo:

I think that some facts are easy to check, but the actual dynamics of battle can render such statistics meaningless. Often design approaches other than technical weapons tests are required.

For instance, what does it matter how far a musket can shoot if the military doctrine of the day never had troops shooting at that range? Or what do you do when one battalion takes 40% casualties in one battle and never runs but another a very similar battlaion… or the same one, runs with only a few wounded?

Other simulation designers face the same problems, not enough information, or information that involve 'soft' dynamics such as when and why a battalion would run, and other other human behaviors. How do you simulate that?

One such method is not to count casualties, but to review all the instances where battalions ran and find the similarities in circumstances, building a generic model without regard to casualties at all.

Another is to do what simulators call a "WAG", a 'wild-assed guess.' The designer sticks in some variables and runs the system to see if it works along historical parameters, changing the variables, testing, correcting, until the system actually models historical events without actually knowing much about 'the right statistics'. And that is just two of many methods.

My response to your notes:

So as I gather from the above,

A simulation of complex things is never perfect.
No simulation is perfect, compex or otherwise. No game is perfect, complex or otherwise. Complexity has little to do with whether a simulation is perfect, but complexity certainly makes perfection a more complex goal…

All historical wargames are simulations.
That depends on the designer's goals, but in most cases, I can't see how that conclusion can be avoided. Most all historical wargames are designed to mimic, represent, model, portray, recreate something of military history, which would be the exact same goal of a historical simulation of war. In most dictionaries, simulation and wargame are interchangeable terms.

No historical wargame is a perfect simulation.
What is a perfect simulation? A perfect simulation would be one that modeled what it was designed to simulate without error, no more, no less--regardless of how much history it was designed to simulate.

Nobody has tried to validate the results of historical wargames by reference to historical outcomes.

For the most part, that is obvious simply by picking a historical battle and attempting to run through it using the game rules or reading any designer's notes, or checking out provided scenarios.

Certainly some designers have done that testing to some extent, but not in any methodical way, to judge from what they've said, haven't said, and the results. Of course, part of the problem is we generally don't know what parts of history were jetisoned in the name of 'playability'…meaning that the game wasn't even designed to simulate certain aspects of the battle.

Therefore its not possible to say to what extent they are good or bad simulations.

Yep, it isn't. Not enough information about what was done has been provided to make that judgement… just guesses and our results in 'testing' the design… which are also guesses as to what was designed intentially and what wasn't.

Maybe then the approach could be to think of evaluating wargames rules on how well, on average, they replicate the results of whole battles. Now we know that many battles hinged on fluke occurences, were horribly unbalanced to begin with, have insufficient detail recorded to wargame them faithfully etc. However if one took a large-ish number, then this ought to even out a bit.

Currently there are eight basic ways to test a simulation to see that it actually works as one. Most simulation designers use around four most easily done based on the medium. Your approach is one of them.

However, most all simulation designers have to deal with 'insufficient detail' and are still able to create functioning simulations. There are a number of methods for getting around that common problem.

My two kopeks to add to yours.

Bill H.

Hi McLaddie,

great, at least I have a similar hymn book open, if not exactly the right page yet ;-).

I'd be curious about the other three commonly used methods, at least on a VERY simple level. Are they top-down assessments as well?

I'd be curious about the other three commonly used methods, at least on a VERY simple level. Are they top-down assessments as well?

Timbo:

I am not sure what "other three commonly used methods" you are referring to, design methods, methods for taking care of insufficient data, or methods for playtesting a simulation to see if it works? In each case, there are more than just three…

Bill H.

Wargames are not simulations though they use a significant amount of simulation in their mechanisms.

Having historical value in a wargame is not dependent upon it being a simulation.

The criteria given above for validating a simulation and proposed for validating the claims to historical content in wargames remain the opinion of the various authors (we can accept them, reject them or modify them as we choose*).

For all the deference given to military simulations here, in the real world helicopters still crash.

Whilst I don't believe history books are just an accumulation of the authors' prejudices, I do think history changes according to who writes it and what questions they intend to address. This is right: history is a living human art, not a finite science, an exploration not a compendium. The historical wargame will be a similar exploration, dependent upon what the designer has put into it, natural variation, and what the players bring to it (the human factor).

The complexity of the wargame will be dependent upon a large number of external factors including the age and experience of the players, the amount of information accessible**, and the time available to play.

A large number of players have always enjoyed (even primarily played because of) the game content and the opportunity to try their skill and luck against their friends – not particularly in order to 'simulate' real warfare. Recognising the added value the historical game can include in no way demeans the philosophy or intent of those players who choose not to be interested in it.

Because … although it may be more than just a game to many of us, it is a game for everyone – and what you get out of it can depend heavily on what you open yourself up to.

Phil
ecwbattles.wordpress.com

*though, as ever, there should be some place where we explain the methodology. There is no reason why the methodology should be explained in the rulebook – the rulebook should contain what the authors and publishers believe the reader/players want.

**thus it is not impossible to play an 'ancient Sumer' game – however, most of the exploration will be built on analogy with later periods of warfare rather than solid fact we know from Sumerian accounts. In many ways, of course, the in-built latitude of the game lends itself quite well to this sort of exploration.

Phil wrote:

Wargames are not simulations though they use a significant amount of simulation in their mechanisms. Having historical value in a wargame is not dependent upon it being a simulation.

Phil:
I fail to see the difference. If the purpose of the wargame is to represent/mimic something of history or battle in their mechanisms, how can they have a "significant amount of simulation" in them, but the wargame not be a simulation?

How does a wargame have 'historical value' if it isn't simulating that history? Where is that value, in simply calling a stand of figures a battalion?

What is the dividing line here? In practical wargame design, in what most all designers claim their games do, with concrete game mechanics, the system is designed to simulate *something*, in this case something from history.

The criteria given above for validating a simulation and proposed for validating the claims to historical content in wargames remain the opinion of the various authors (we can accept them, reject them or modify them as we choose*).

Actually, I am simply restating what simulation designers are all doing right now. It isn't somebody's opinion, but a simple techinical methodology, tested and proven, rigorously and over decades. The same methods are used across disciplines and industries, including the game industry, when creating simulations for a variety of purposes. To simulate is a technical statement, not an opinion.

If I said I designed a wargame, would that be just an opinion, or a technical statement about what my design is supposed to do?

If we are talking about what is being simulated, that history is open to opinion, but whatever the interpretation, it still is based on something, and that something is what is being simulated. The question is whether the game system models that 'what' accurately.

For all the deference given to military simulations here, in the real world helicopters still crash.

Ah, duh. That is why simulations are so valuable. No one gets hurt if you crash a simulation Apache while learning to fly one. I am not sure what point you are making in stating the obvious, or what deference we are giving 'military simulations.'

Whilst I don't believe history books are just an accumulation of the authors' prejudices, I do think history changes according to who writes it and what questions they intend to address. This is right: history is a living human art, not a finite science, an exploration not a compendium.

We could get into a debate over the nuances, but this discussion is not about art vs science. The bottom line is this:

1. There is some historical source[s] that leads the game designer to believe "X", unless he dreamed it up one night. We can argue about whether his belief is opinion, finite, a prejudice etc., but it still remains that his conclusion has to be based on something in the historical record. That's all we know of history, so if the game is going to be a representation of history, it damn well better have some concrete reference to it--whether we think that is a established fact or just someone's conclusion/opinion/prejudice about that historical source.

He wants to represent that 'X'--simulate it in his wargame. For the simulation to work, for the participants to know what is being simulated, the designer has identify that 'X', what mechanics are meant to model it, and then test the system to see if it does succeed.

That is a technical relationship, not some science vs art issue. Both science and art can be and usually are involved, depending on how you define each.

The historical wargame will be a similar exploration, dependent upon what the designer has put into it, natural variation, and what the players bring to it (the human factor).

I can agree with that, but it doesn't negate or change anything regarding simulations…many scientists could say the very same thing about their 'explorations' using simulations…and what they personally bring to that simulation design.

And the players are a major factor in a wargame actually simulating, so the human factor is a necessary component, not only in what is being simulated, but how.

The complexity of the wargame will be dependent upon a large number of external factors including the age and experience of the players, the amount of information accessible**, and the time available to play.

Actually, I would think that the complexity of the wargame would be wholly dependent on the designer and what he wanted to accomplish. Complexity itself has little to do with whether game system is a successful simulation or not.

A large number of players have always enjoyed (even primarily played because of) the game content and the opportunity to try their skill and luck against their friends – not particularly in order to 'simulate' real warfare. Recognising the added value the historical game can include in no way demeans the philosophy or intent of those players who choose not to be interested in it.

I agree. And? The reasons gamers come to the table doesn't change why a wargame was designed, or what the mechanics do. Those elements are finite and established long before a particular gamer plays the game for whatever purposes he brings to the table. I play for a variety of reasons, depending on my mood, and support those gamers that are not interested in history and just play a game. More power to them.

That isn't the issue here. The issue is what the designer has chosen as his design goals, what he claims his wargame rules offer, what his mechanics are designed to provide in the way of history.

Because … although it may be more than just a game to many of us, it is a game for everyone – and what you get out of it can depend heavily on what you open yourself up to.

I certainly agree with that.

Bill H.

@ YesthatPhil:

My comment really wasn't intended as a laugh. DBR as written would fail Bill's #1 because Phil B doesn't justify, using a historical source, the way 'Lancers' operate within the rules. I have absolutely no doubt that he could so, but Bill isn't on about that.

Regards

> Phil:
I fail to see the difference.

Indeed, that has been evident for some time.

And at some length, again, you insist we must apply your methodology to how we discuss the topic. A series of hard lines of definition. And I have explained a different approach largely replacing those hard definitions with layers of subtlety.

Not because I am being argumentative, but because I believe it is in the layers of content that the wargame builds its added value.

I have just deleted a lengthy paragraph illustrating that – because I think I've made the point already and this thread is starting to degenerate into a series of lengthy posts restating positions already taken.

Although many wargame designers do a lot of what you imply they often feel their way towards the result and for many it is how you the players responds to the journey the game takes you on and the narrative it allows you to participate in that counts (not 'how many rounds per minute can they fire' … though that may play a part in the design in its proper place).

I do not disagree about the backbone of the game's mechanics and the tests we might apply to them – but not all of the content is to be found there.

Phil

As a simple example of non-crunchable content, the DB genus of games classifies chariots either as _Cavalry or _Knights according to battlefield purpose (over which there is considerable debate, but, hey, who are we ….)… Despite the fact that a chariot is clearly neither of the above, the author decided not to use the word 'chariot'. Language is a layer in the historical wargame, as is the composition and depiction of the miniature armies.

Phil

Bowman … Fred C is right about the sense in which Richard used the term Nazi. This thread began a few days after those of us who attempt to include useful historical content in our historical games were branded, not for the first time, 'naive and arrogant' for thinking other than it being only a game. Period.

Hi Phil,

Don't you think it a bit ironic using the term "Nazi" in an argument extolling the virtues of accurate historical content, to describe people you disagree with? I wonder if the terms "naive and arrogant" are not too far off.

Godwin's Axiom and all that.

And at some length, again, you insist we must apply your methodology to how we discuss the topic. A series of hard lines of definition. And I have explained a different approach largely replacing those hard definitions with layers of subtlety.

Phil:

I am all for subtlies, and my insistence, if it is that, is an attempt to deal with the discussion of game design as a highly technical activity with desired outcomes that are very specific in both the concrete mechanics and content.

Your example of Barker classifying chariots as knights or cavalry is a good example of that subtle layer. In the end, he chose that 'subtlety' to simplify and still represent chariots. No problem. The question still remains: do the renamed chariots, now with the same game traits as cavalry or knights 'act like' the historical chariots in the simulated battle using the game rules?

Because that question deals with a concrete, finite set of game mechanics representing what we know of ancient battle, it can be tested to see if it actually does that representation using proven methods. If that is some 'hard line', it is no harder or finite than the very specific rules Barker creates, or the very specific things that the game design does in play.

Not because I am being argumentative, but because I believe it is in the layers of content that the wargame builds its added value.

I am not trying to be argumentative either. I believe that those layers of content or the added value is lost with what I am suggesting. I know that because I have created many simulations games and found that the subtlety is not lost, but the opportunities are increased. "Hard lines" and subtlety are Ying and Yang in most cases. You can't have one without the other. At the moment it is all subtlety without reference to anything concrete.

Although many wargame designers do a lot of what you imply they often feel their way towards the result and for many it is how you the players responds to the journey the game takes you on and the narrative it allows you to participate in that counts (not 'how many rounds per minute can they fire' … though that may play a part in the design in its proper place).

I would be interested to know how you see my position as denying or countering or ignoring what you've said above. Those 'hard line' simulation game designers still have to feel their way to results, the only question is whether the designer can be sure of his results. And absolutely, it is how the player responses to the game play, the journey, that counts. Simulation game designers are always talking about 'narrative' in their work.

There has been a great deal of study in how a finite simulation game system can produce the game experience, that narrative, one that does correspond to the history it was designed to represent.

I do not disagree about the backbone of the game's mechanics and the tests we might apply to them – but not all of the content is to be found there.

Okay. What content are you thinking of outside of the game system, the set of procedural rules that make up the game? Whatever the players experience through playing the game will be with the game system.

Where else is it to be found? Certainly there are a great many things outside of the actual game play that add to the game experience, from the figures and terrain to the friends you play with, but that is outside the actual rules, which is what we are talking about here.

If anything, I feel that you are assuming that somehow the simulation tests and methodology I am advocating somehow ignores or diminishes the content and subtlety you are speaking of. From what I understand of your points, it doesn't.

Bill H.

Oh, ah, sorry Bill H, I wasn't entirely crystal clear -

"Currently there are eight basic ways to test a simulation to see that it actually works as one. Most simulation designers use around four most easily done based on the medium. Your approach is one of them."

I was wondering what the other three 'easy' tests of simulations were.

> Hi Phil,

Don't you think it a bit ironic using the term "Nazi" in an argument extolling the virtues of accurate historical content, to describe people you disagree with? I wonder if the terms "naive and arrogant" are not too far off.

Godwin's Axiom and all that.

Bowman, all I'm doing is trying to help you understand why other people (not you, not I) use the language they use.

But if you are just here to snipe, I will, of course, let you come to any conclusions you choose.

Phil

Phil:

Egad, my years as a rules writer have stuck again. One thing I have learned to do is avoid negative instructions, like 'you can't do…' in rules. However, I have found that after years of doing it, I drop the negatives when I don't mean to:

What I wrote was this: "I am not trying to be argumentative either. I believe that those layers of content or the added value is lost with what I am suggesting."

What I meant to say is:

"I am not trying to be argumentative either. I believe that those layers of content or the added value is NOT lost with what I am suggesting."

Bill

Timbo:

About what I said: Currently there are eight basic ways to test a simulation to see that it actually works as one. Most simulation designers use around four most easily done based on the medium. Your approach is one of them

I meant that because of the variety of types of simulations, using everything from computers to paper and pencil, designers will find some of the eight tests easier to apply than others, and that generally to make sure that a simulation does work without any surprises, four of the eight tests are used. There aren't really four 'easier' than the others. Here are the eight system tests for a simulation, to see if it does indeed mimic the reality/history it was designed to represent. In all the tests, it is a comparison between reality/historical performances and the simulation system performance.

1. Face Validation: The simulation is critiqued by experts--that is, folks who have lived through the experiences simulated by the simulation. The model must appear reasonable when reviewed by folks experienced with the real-world system. These experts do not include somebody that has read books on the game topic, but those who have experienced the real thing at the same level of the game decision-making.

For instance, designers of some flight simulators like Red Baron I, II and 3D actually had WWI pilots and those that had flown WWI aircraft examine the credibility of the computer game. A few board game designers have done this for modern combat games. This isn't possible with most historical wargames.

2.Sensitivity Analysis: This is a system check: Does it simulate the way it was designed to? The designer changes the data in the game sytem and mechanics and then predicts what will happen, probably something very unhistorical. For instance, if you increase the range of musket fire to 400 yards, does the Napoleonic game play differently, does it play like the ACW? What should happen is that the simulation results will demonstrate less relationship to the ‘real-world events' the more extreme the data changes are.

The affects of the changes also should be predictable. IF a simulation doesn't show those changes in game dynamics or doesn't behave predictably, there is something wrong with the simulation--the data changes had an incorrect impact on the design performance. This may sound like not much would be revealed, but no matter how good the components are, when synthesized into single system, it is often hard to predict how it will work. Strange outcomes can occur. This is the test that reveals such things.

3. Extreme-Condition Tests: This is taking the game for a ride on the wild side to see if the system holds up. Some designers report doing this. By changing the numbers of troops, terrain, officers, and other variables to extremes, does the simulation behave in a predictable fashion, reflecting those changes. This is different than ‘B' above. The system mechanics and rules aren't changed to extremes, just the player tools and game environment like troop strengths, terrain etc.

Sometimes you can change elements of a design drastically, having ten times the troops on one side or the tabletop all woods and there is little difference in play. For instance, give the British in the Peninsula twice as many cannon in various battles. Does that radically change the predicted outcomes of the game play? It should, but there are some designs [even currently published] where it makes absolutely no difference with game play or outcomes. These kinds of simulation weaknesses often appear when gamers are making up new scenarios for the rules--because the designer didn't do it first.

4. Validation of Model Assumptions: This is comparing the simulation results against current reality to see if they match results. Does actual results of a factory process match the simulation of it? For instance, I could test the performance of participants before and after my training programs as they perform in the real-life situations which were simulated.

Historical simulators can't do this directly, because they can't test their simulation directly against reality, only indirectly. There are a number of ways to do this and you provided one, though it doesn't have to be 100 battles. Statistically, if you have 15-30 for comparison, that is more than enough.

5. Consistency Checks: Game designers report doing this at times, but from what has been said, rarely in a methodical way. The designer checks the play of the game over time and many different games to see if the simulated historical dynamics continue to always function within historical parameters—sort of a longitudinal study. This is often confused with simply play testing the game. Does the system produce historical results [within play, not at the end of the game] for Waterloo, Eylau, Leipzig, Talavera and Marengo? Or do the rules succeed with some and fail with other battles.

What is checked is the consistency of the history portrayed, not the effective flow of the game mechanics.
Do the units always move within reasonable parameters compared to the historical events etc.

6. Turing Tests: With this test, the movement, combat, battle dynamics are recorded and compared against the historical movement, combat, battle dynamics. These snapshots of the action are presented in identical ways--the games against the historical events, are compared.

Usually this is tested further by handing the data to a knowledgeable third party or several different experts in the history of the war as a blind test. The issue here is whether the actions in the simulated battles are so outside the norm that the differences are obvious when the source of the maps or statistics aren't identified. They are asked if they can pick out which results are the game and which are the actual history. If they have difficulty doing this, that supports the validity of the simulation dynamics.

7. Validating Input-Output Transformations: I love the engineer speak. For a wargame, the designer would look at input-output variables like casualties in comparison to the real event[s], not only the final tally, but at different points in the simulation play. For instance, American divisions invading Iwo Jima recorded 30% casualties. Does the game allow for the same kind of results from the same kind of invasion? Again, there are several ways of doing this test, statistically and without statistics, depending on the input-output comparisons desired.

Wargame designers often do this in designing a game, but not in any methodical fashion—just here and there, rather than for the entire design and every variable—which is the necessary scope of this validation.

8.Validation Using Historical Input Data: This one I think would be a must for wargame designs. However, I rarely hear of designers even considering this one, even though it would seem one of the more obvious approaches.

This test takes the simulation and plays it with all the historical decisions and chance events being made at the corresponding historical points in time. The question here is: Does the design allow for the historical decisions, and as a consequence, do they produce similar results? That is, if I used a Napoleonic set of rules, could I recreate the Battle of Waterloo in a reasonable fashion, assuming the all the right cards, die rolls were made using the historic decisions? Could the rules allow the player to recreate the Union Brigade's charge, or the French cavalry attacks, or the fight for the Hougumont? If it is difficult to match the history with the game play, then how much of a simulation is it?

A couple of examples with just movement: One set of current rules provides a Pickett's Charge scenario, but the rules require the Confederates to advance 3 times as long to reach the Union lines compared to the historical event, nearly an hour and a half compared to twenty minutes. Or another set includes an Austerlitz scenario where the rules keep the opposing forces moving about 1.5 hours behind the historical movements and subsequent events.

I have given a very brief description here, so don't be fooled by that simplicity. These eight tests have enjoy a lot of variations in application and all require some serious record-keeping for them to be useful in determining whether the design passes or fails the tests.

The tests were originally based on simple system tests, particularly electrical, engineering and computer systems, but because a simulation is a procedural system, particularly a simulation game, the tests, with modifications were found to validate the effectiveness of simulations.

What has been done over decades to prove the tests work is this:

A simulation of say a manufacturing process or communication net is tested and passes four of the major types of validation tests. THEN it is tested against the real manufacturing process or communication net. What they found was that passing the validation tests were predictors of whether the simulation actually could match real world results.

That gave designers confidence that if the simulations passed four of the tests, it was a valid simulation, even if it modeled something it couldn't be compared to, or didn't exist yet. That has proven to be the case over and over again.

That ability to effectively simulate the unknown or untestable has made simulations very useful in a variety of industries, disciplines and education. Simulating the 'untestable' is what historical wargames do. It can now be done with a reasonable amount of confidence.

I'd be glad to give wargame examples of each test, some of the variations…

Bill H.

Bill …

Thanks for the clarification. I'll be honest, I think I do get you argument :) I read it exactly the way you meant it and didn't even miss the absent 'not'.

I see Timbo has started a Historical Results thread that might take up some of this …

Phil

Hi Bill,

it was really nice of you to post that information, thanks very much!

Lots to think about here, and as you say, to do these validations properly would require a fair bit of effort.

yeah, this gave me the idea for the Historical Results thread, but I don't expect this to give anything more than a very fuzzy impression, due to the uncontrolled nature of the 'tests' ie games. but it might be interesting, I live in hope!