"Simulation Game Design" Topic

Since I am criticizing people's definitions, perhaps I should list a couple of my own. Whether or not they support Bill's position, I don't know yet, but offer them here in case they contribute to the discussion, or in case someone can use them against one of my own previous posts.

To show you how out-of-date I am, mine are from a book I own written in 1970 "A Primer on Simulation and Gaming" by Richard F. Barton, then at Texas Tech.

The Preface states "This book is an introduction to simulation and gaming for the administrative professions, for the behavioral sciences, and for education."

From the Introduction:

[Pg 1]"To simulate means to give the impression of something else. A simulation, although it is a 'thing' in itself, is meaningful to its creators and to its users only in terms of other things. To simulate also means to give the effect of something else – so that the meaning and usefulness of a simulation is not merely in a visual or sensory likeness but in a similarity of ideas or a conceptual likeness as well…"

[pg 4]"… At this point we must introduce a special term: object system. The object system is the system we want to study; it is the 'object' or subject matter of an investigation or learning experience. If we could study the object system directly, we would not need a simulated system to experience or to experiment with. The object system is sometimes called the real world… this object system may not now, or may never, exist, but such a condition does not obviate the usefulness of our example, or even of simulation in general."

[pg 5] Models of Object Systems

A model is a representation of something else, as, for example, the model aircraft flown in a wind tunnel is a representation of the actual flight of an 'object system' aircraft.

In order to construct a model, we need to know something about the object system we are interested in. The knowledge we use to construct a model may be generally accepted laws or principles about object systems like the one we want to study. In the absence of such well-founded knowledge, we may make tentative assertions about the object system and then proceed to build a model that reflects these hypothesized characteristics. Usually we try to make the structure of the model correspond in some degree to the structure of the object system.

We may model only a part of the object system, or we may model it all…"

[pg 6] "…We may use any convenient and useful device to construct our model: unrelated physical material, as in the case of the cisterns, tubes, and water used to simulate a market; scaled-down physical analogues as in the case of the model aircraft; an exact duplicate of part of the object system itself, as in the case of the space capsules…; combinations of actual parts and hypothesized behavior, as in the case of war games; word descriptions…; or mathematical formulae, as in the case of computer programs that simulate the economy.

Simulation Defined

Simulation is simply the dynamic execution or manipulation of a model of an object system for some purpose.

A simulation of an object system never attempts to become part of the object itself…As a practical matter, for simulation work to proceed, the user must focus on a particular object system that remains within intended conceptual bounds during the period of the study."

-Ned

Rich, "our industry" has NO consistent definitions.

So you think gamers don't have a good understanding of these: 1. Turn sequence

2. Unit sizes

3. Rate of march
 4. Fire

5. Melee. Perhaps you are new to our hobby.

To simplify something is to take something that exists, remove some complexity, and end up with the same thing in a simpler form.

This is a definition of a model. What you need is a better definition. Here's one from my dictionary: "make (something) simpler or easier to do or understand : an overhaul of court procedure to simplify litigation."

"simplify battle"

Oh I see, your problem is a linguistic one. What I should have said was that we simplify ‘aspects of' battle in order to create gaming components. I agree. That's much better.

we WANT ONLY SUBJECTIVE DEFINITIONS (and discussions) and are not interested in anything else because we take it as an article of faith that using objective definitions are impossible and unhelpful.

An article of faith that things can't be defined? That's ludicrous. [ludicrous: so foolish, unreasonable, or out of place as to be amusing; ridiculous. See absurd.] Words mean things and it is incumbent on us to know what they mean. Otherwise not only is wargaming impossible But communication is impossible.

Karsta: Are you trying to say that if model is based on a bunch of data (or another models) and not somehow directly to reality, it shouldn't be called modeling, but 'transposing'?

Either you can't read or I can't write. The transposition of real ranges into wargaming ranges is not called modeling.

Really, take a step back and try to see the broader picture. Distinctions you are making are completely artificial. They are not even useful.

Of course not. They are not my distinctions.

Rich

Good stuff Ned, thanks for sharing.

Ned: To simplify something is to take something that exists, remove some complexity, and end up with the same thing in a simpler form.

Rich: This is a definition of a model. What you need is a better definition. Here's one from my dictionary: "make (something) simpler or easier to do or understand : an overhaul of court procedure to simplify litigation."

No, it is not a definition of a model! When you model something you don't end up with the same thing, not even in a simpler form. The whole point of modelling is to represent something with something else.

Again, I can't see the difference here: "make (something) simpler (that is more plain, basic or uncomplicated in form)" vs. "…remove some complexity, and end up with the same thing in a simpler form".

Rich: Either you can't read or I can't write. The transposition of real ranges into wargaming ranges is not called modeling.

I'm probably to blame, but I still don't understand where you see the difference. It would be easier to accept division between modelling and transposing if it would somehow help us. Now that you say that these distinctions are not useful, I'm even more puzzled.

3. Simulation accuracy and the 'How and Whys' of the 8 validation tests.

Okay, I want to simulate historical information from the U.S. Tactics manual.

My simulation game interpretation of that will be modeled with one inch representing 50 yards. Now I have to accurately simulate it.

For the Union Army: [Note that is all the history I can claim to simulate—because that is all the history I have—just the Union army's SOPs.]

So, in my game, the mechanics will portray:

Canister as having the highest effect at 200 yards, limit 350 yards. Grape ranges will be between 350 yards and 600 yards, with 400 yards the most effective range.

Horse artillery, as opposed to mounted artillery can move within 300 to 400 yards of infantry and do serious damage to enemy infantry before being completely wiped out, if the player wants to take the hurt in return.

Effective range of smoothbore 12 pounders is 1100 yards. For rifled 12 pounders the effective range is further, but maximum range is 4000 yards to be conservative.

However, based on the Army Officer's Pocket Companion, I'm going to have smoothbore artillery fire is not beyond 1200 yards.

But wait, I have to figure out what that 'highest effect' consists of. There are a few ordinance tests, but we are talking about battlefield results, which include morale, disorder as well as casualties, time and distance, training, experience etc. etc.. How can I find that information, let alone simulate it all? And what happens if I can't find the information at all?

Simulation designers handle this problem in a number of ways. They can:

1. Deep Analysis: Do what Rocky did and take a wide variety of historical situations [artillery firing and the effects] and number crunch averages or constants for one or all of the variables.

2. Representative Event: They can take one event that a lot is known about and use it as representative of all events

3. Average Events: Designers can take two or more events where a lot is known and average them.

4. Controlled Events: Take ordinance tests, concrete physical aspects like sighting and trajectory that were tested and known and extrapolate an average.

5. Black Box it: This basically means 'guessing'. The designer puts a value in the place of the unknown information with his best estimate and moves on.

Now, at this point, I am sure you are surprised by #5. In our hobby it is called Black Box mechanics, or just plain 'fudging.' Simulation designers don't fudge, but they do guess. The difference it that their guesses are tested to see if they actually work.

Let me tell you, there isn't a simulation designer alive that has designed a simulation with all the information and data he or she felt was neede--unless they only simulated what they had information for—which is a rare luxury. Whether it is that simulation of a galaxy millions of light years away, or the Weather, or the traffic patterns, or a flight simulator, or a wargame, most all simulators face the problem of not enough data.

Wargame designers often act as though this problem, particularly simulating historical evidence, was an unique problem, something only they have to face. It is a very common problem.

So how can we possibly simulate information we don't have?
Often you will read gamers or designers talking about 'reasonable game results', or it 'feels' right, or 'historical results' as the test for a wargame/simulation. The problem with each of those conclusions is that they don't mean much. Some gamer had a feeling, or the designer didn't see final game results too far out from the expected. However, what those parameters for 'reasonable' or 'historical' consist of are never articulated.

It's all subjective and widely variable, even though the assumption behind the statements is that an objective or 'reasonable' conclusion is being made.

Certainly game designers are free to design that way, but for a simulation designer those 'reasonable' results—to be reasonable--have to meet some specific criteria.

That's where the eight tests come in.

First, these tests were developed over many years as checks for systems, particularly computer programs. As computer programs have dominated simulation design, and as all simulations regardless of the medium, are functioning systems, the crossover and parallel development made sense—and worked.

The idea was to design systems that could accurately simulate aspects of the real world. Real world systems, environments and machines could be tested, analyzed, and modified without ever having the expense of doing the same things physically. In training, participants could do this same testing and modification in learning skills and tactics. Thousands and millions of dollars could rest on a simulation 'working' correctly.

However, along with the common problem of a lack of sufficient data was the that many simulation designers had no real world system to compare and check the simulation against. Again, a similar problem faced by wargame designers trying to design simulations of past events… there is no way to test the simulation as there was no way to fight the real battle over again and again to check the simulation

One major value of simulations is the ability to test systems and environments that don't exist or are too big to test—like galaxies and the weather--or even the new design for an assembly line that hasn't been built yet. Simulations were a way to test systems that had a huge number of variables without the expense of physically testing each one and then in unison—if simulations 'worked'. In attempting to do these things, the designers faced particular issues in developing functional simulations. They were:

1. Never enough information on which to build the simulation.

2. No way to tell whether the simulation accurately mimicked reality.

3. And because of this, there was no way to 'fine tune' or correct a simulation that wasn't working correctly.

Does that sound familiar? Most all wargame designers have identified one or more of those issues as the reasons wargames can't be anything other than someone's 'opinion.' While the games are supposed to model history, how do you tell? That question leads to some rather odd backpedaling on the part of game designers. Here is something Frank Chadwick wrote in his Designer's Notes for the newest edition of Volley & Bayonet:

The scientific notion of testability provided an additional motivation [to design an 'army level' game]– not that I make any claims as to the game being particularly precise, objective, or scientific. All rules sets portray the capabilities of different armies as – unavoidably – the author sees them. But without being able to set up and re-fight entire historical battles, it is nearly impossible to find any sort of objective standard against which to measure the author's judgments. With Volley & Bayonet, for better or for worse, you can.

Here the designer is saying that while his design isn't particularly objective, the game's scale allows some kind of objective standard to test against that other games [of smaller scales?] don't provide. And in the end, the gamer is the one who applies this 'objective standard' to the author's design decisions. But Frank never identifies those objective standards the gamer is to use…

A simulation designer has to have objective standards if his design is meant to mimic objective reality. And he needs those objective standards to test the simulation. In all cases, this testing is done before the customer is given the tool to work with. Designers don't hand the simulation to the customer and say, "I don't claim that this is a particularly objective design, but you test it and judge if I did it right."

To be fair, what could Frank say? He could make no claims that the game was particularly objective or 'tested', nor did he. The 'scientific' part is a curious addition. While notion of testability was an extra motivation, the game remained not particularly scientific or objective, and the testing was left to the gamer.

However, we aren't talking science here, but simply functioning simulations. Science does use simulations, as well as business trainers and the game and entertainment industries. Some simulations are scientific, many aren't. However, they all have to simulate accurately.

So, methods had to be developed that would establish that a simulation worked without having anyway to test it against reality. And they had to have some confidence that the results would be 'accurate'. Often millions of dollars rested on it. That is one reason the eight tests were created.

The way these eight tests or methods were 'vetted' occurred over many years and many different industries and disciplines. This is how designers validated the tests themselves: a simulation was built that could be tested against reality. First it was tested with the eight methods, and when it appeared to be working 'accurately', having passed the tests, it was then tested again against reality.

Over time, these tests were developed into methods that have achieved a high level of confidence. Enough confidence to bet millions on the results when they can be tested directly. They are used in research, computers, training designs… etc.

And before this gets too scary or 'scientific'. A note—there is nothing magical about them or particularly 'scientific'. While these methods are used in very technical ways by computer experts and science researchers, mechanical engineers and aircraft designers, I've used them on very unscientific and non-computer training simulation games. And they work there. The methods have a lot of different names and forms, some extremely technical, and some not at all, their basic functions are the same.

As a game designer, wanting to recreate some history, wouldn't you want to know what you could do about:

1. Not having enough historical information on which to build the simulation?

2. How to know your wargame succeeds in mimicking a past reality you can't test it against?

3. How to find where it doesn't succeed in simulating the past you've chosen, and 'fine tune' or correct the system until it does?

I am talking making strides in doing the simulating better, with more confidence and specificity, not some perfection or ultimate anything. What would it feel like if you as a player knew specifically what history you were simulating in a wargame and how the game did it? No questions, no wondering, no bull?

Next, those 8 tests.

Best Regards,

Bill H

Rich wrote:

…h[is] problem is, nothing is being modeled. Bill's definition of simulation is that a simulation is a model of reality, something that exists in time and space. The critical aspect of this description is something I asked Bill to define: what is a model. I went to a website called System Thinking and used their definition of model. "A model is a simplified representation of a system at some particular point in time or space intended to promote understanding of the real system."

For our purposes a model is a simplified representation of an object, event or process.

When a game designer goes to the historical record and uses the data on canister fire he bases his range estimates and effects estimates on this data. Is this modeling? No. He does not simplify this data. He simply converts the range figures and effect estimates into gaming data. This is called transposing. You are taking data in one form and converting it into another form that can be used by the game. You have simplified nothing.

Rich:
The definition of a model you found is just fine. And if you would go back over our previous discussion, you will find that I did define a model--you just didn't accept it. You felt there were stark difference between a model and a simulation that I didn't, if I recall, particularly in reference to a simulation being a process/system. And I don't, particularly with the definition you provided.

And as for my creating a model of artillery actions in the Civil War, I only listed as an example the kind of information I would use to develop that model.

1. I wasn't listing all the information to be used or

2. How it would be used [transposing or not],

3. Let alone how I would use it in creating a 'model' of ACW artillery actions.

That wasn't the point of my discussion at all.

So to criticize me for not defining a model to your satisfaction, or to fault me for not actually modeling the information I provided is really missing the entire point I was trying to make, besides injecting issues I didn't address at all.

Wargaming and simulation design as a technical art is tough enough without all the bird-walking and critiques of issues
outside the focus of the discussion.

I will be glad to talk about modeling when we get to it.

Best Regards,

Bill H.

Bob wrote:

Bill H. -- I'm taking from your explanations that a big part of simulation is an explicit understanding by the participants that they are, indeed, playing a "simulation".
Much of the distinctions seem based on the players knowing -- and caring -- about at least some of the significant details. Without that, much of the effort that you describe required by the designer is for naught.

Bob:
Yes, and no. [How is that for a direct answer? ;-j]
Katie Salen and Eric Zimmerman, in their excellent book, Rules of Play: Game Design Fundamentals say that "The rules are the game." This is true for a Simulation game, but the rules are also 'representational'. If the players don't accept or abide by that rule, then like any game where the players don't abide by the rules, it doesn't work. However, if the players don't follow the rules, the simulation or game doesn't work for those players. It is a player choice, not some fault of the simulation/game. Certainly a simulation game should make it as easy as possible to do, but if the players refuse to 'play along' there is nothing the designer can do, and it isn't the fault of the design.

This would tread into discussions of player motivation and willingness to put their experience in the hand of the designer more so than with a more casual effort, I would think.

Absolutely. How can it be avoided? When simulation games are supposed to provide an 'experience', a 'feel' of a historical event, 'recreating' it, how can that be avoided? How 'casual' a gamer's effort has to be is entirely up to the designer, not some finite level required by all simulations. It isn't a unique issue with simulation designers. Game designers ask the very same questions. For instance, in the Rules of Play, the authors note:

Pong is still alive and well today…

Which all brings us back to the question: Why? Why do people love Pong?

Although this book is not about Pong, or about computer and video games, it is a book about game design. It is crucial for game designers to understand why people play games and why some games are so well-loved. Why do people play Pong? We can think of many reasons:

Game designers have to understand player motivation etc. in an effort to design 'effective', successful games.

That said, some subset of the player community might be trusted to do the vetting, and the rest "just play", but that seems to be a precarious situation for a designer to put themselves in. A formal play testing process would fulfill much the same purpose, would it not?

No. It could help, but unless it is consciously/methodically done, all you've accomplished is tested the play of a game--not established the success of a simulation. Actually, simulation game designers [including the authors of Rules of Play] advocate designing the game, play testing all through the design process, and once the game system works well, THEN determine if the system simulates--as only a design properly functioning would be the one you'd want to test.

The depth of detail you describe might be used internally for validation, yet not provided to the general public -- the old "proprietary intellectual property" approach, no?

I hope I laid this concern to rest. A designer doesn't have to lay out the depth of detail they used. They have to establish that what they are representing as history actually exists somewhere as historical evidence.

The part I think I'd find damnably frustrating is to go through such a formal validation process, only to find that the result stinks, and some tweaking is needed to make the experience enjoyable. That would then break the entire basis of "simulation", would it not?

Well, yeah, that would be frustrating. But then, not any more frustrating than a game designer who works hard to design an enjoyable game only to find no one enjoys it…;-j

Actually, while the frustrations you note are dangers of any design effort, there are ways to mitigate or avoid the problem you see--mostly because those two issues, an enjoyable game experience and a functioning simulation are not mutually exclusive issues, or even a problem of choosing one over the other. Simulation Game designers have long faced those issues, and have come up with effective design processes to avoid them. I'll get into that.

Simulation design ain't easy, but it sure doesn't have to be as complicated and frustrating as our hobby often makes it…

Bill H.

Bob wrote:

"I'm taking from your explanations that a big part of simulation is an explicit understanding by the participants that they are, indeed, playing a "simulation". Much of the distinctions seem based on the players knowing -- and caring -- about at least some of the significant details."

On page 57 of Barton's book, the the section "Man-Model Simulation", he mentions the following:

"THE MODEL… The model is the main source of stimuli in man-model simulation. It represents that part of the object system that is not represented by live participants.

VERISIMILITUDE… Since the model represents to participants aspects of object system reality, it must give to them the appearance of or the effect of reality. Simulation does not require reproduction in every detail, but it does require capturing the relevant aspects as the participants see them. This effect is called verisimilitude, which means that, while the simulation is obviously an artificial representation, it has the quality of being true to life or to human experience.

The model must be complex enough to provide verisimilitude, yet be simple enough to process in the available time.Computers,by speeding up processing, enable designers to build more complex models."

4. The Eight Tests for Simulation Accuracy

There are a number of ways to accomplish these tests, and depending on the discipline or purpose of the simulation, a variety of methods under each test. I will give a few examples of what this can entail, but my intent is to provide a basic explanation.

In testing a simulation, the subsystems and processes can be tested individually, but the entire system as a whole also needs to be tested. And most importantly, simulation designers have found that if a simulation can pass four of these tests [any four] the odds that the simulation works are something like 4:1 or 80% probability of working. That has been proven over the years in a number of arenas so that it is considered a given in simulation design.

As Rocky noted, when his flight simulator was done and tested, the feedback wasn't that the pilots liked Rocky as designer, or his particular take on piloting a plane. No, the satisfaction for Rocky, the goal for a simulation, was that the pilots reported that his design worked and felt like the 'real thing'--that it worked as a simulation of something else, not Rocky's ideas or his 'likes', but the common experience of reality for pilots.

Game designers who claim to be simulating are intentionally or unintentionally shooting for that same goal. These 8 tests can provide a way for wargame designers to achieve that goal.

A. Face Validation: Here the simulation is critiqued by 'experts.' By experts, we are speaking of those who have actually lived what the simulation is modeling. The model must appear reasonable to those who know the real-world 'system.'

These experts do not include somebody that has read books on air combat or flew a B-17 when it is fighter combat being simulated. These experts are those who have experienced the real thing. Of course, this eliminates this option for most historic simulations. And as Rocky has said, they can provide invaluable feedback, both objective and about the 'feel' of the simulation.

Designers of some flight simulators like Red Baron actually had WWI pilots and those that had flown WWI aircraft examine the credibility of their computer game. A few board game designers have done this for modern combat games such as Fire Fight. For game designers who can't speak to such 'experts', the only other option is to do research not previously done to find combat narratives that match simulation game events and dynamics. This is done after the simulation has been created and is functioning as designed, not before.

B. Sensitivity Analysis: Here the designer changes the data in the game and then predicts what will happen, probably something very unhistorical. For instance, if the fire power of the muskets are changed, say doubled or halved, does the game still function as before? What should happen is that the simulation results should demonstrate less relationship to the ‘real-world event' the more extreme are the data changes. Also the effects of the changes should be predictable.

IF a simulation doesn't show any changes in game dynamics or acts in an unpredictable manner, there is something wrong with the design--the changes in the data had no impact on the design performance or created situations the designer didn't or couldn't predict. Obviously, something has to be changed. This sensitivity analysis then might look at several sub-systems individually.

C. Extreme-Condition Tests: Some designers report doing this. Instead of changing the system mechanics or data, the designer introduces new scenarios, changing the numbers of troops, terrain, officers, and other variables to extremes. Give one army three times the artillery, or twice the number of cavalry, etc, or lots of defensive terrain and few troops. Does the simulation behave in a predictable fashion, reflecting those changes? This is different than ‘B' above. The system isn't changed, just the player tools—such as troop strengths etc.

Sometimes you can introduce drastically different scenarios, and there is little difference in game play or outcomes without serious changes in the rules. This weakness in the simulation often becomes obvious in designing new scenarios for a particular set of rules. Change the scenario, and the system starts to break down as a simulation. It is an indication of the lack of validity in the simulation system.

D. Validation of Model Assumptions: This is comparing the simulation results against current reality to see if they match results. Does actual results of a factory process match a simulation of it incorporating the same conditions? Historical simulators can't do this directly, because they can't test their simulation directly against reality, not like Rocky could with flight simulators. Only the one result reported from the past is available. For instance, I could test my training designs against participant performances before and after in real-life situations.

Professional military simulation trainers like David Bartlett can test whether their combat designs work. They follow individual soldiers in Iraq after being trained on computer ‘first-person shooters' as see if the simulation training made a difference, and where. Bartlett found that "When the time came for him—the trained soldier—to fire his weapon, he was ready—capable of doing that. His experience leading up to that time through on-the-ground training and playing simulations, enabled him to execute." [This was part of a Washington Post article by Jose Vargas on combat simulation training. This kind of validation is something historical game designers can't do.

However, what they can carry out is a validation of Model assumptions by running historical scenarios that haven't been used to create the simulation game as a test. Can the system recreate a new historical event. And by can, we mean if we run the units through their historical paces, can the simulation mechanics account for them? Again, this can also be done on a sub-system basis.

E. Consistency Checks: Game designers report doing this at times, but from what has been said, rarely in anything approaching a methodical manner. The designer checks the play of the simulation game over time and many games to see if the simulated historical dynamics continue to always function correctly. This is often confused with simply play testing the game. What is checked here is the history portrayed, not the effective flow of the game mechanics or the 'fun' factor.

And this is an important issue. Often times when wargame designers play test their games, they mix the historical simulation analysis with how the game plays as a system.
Simulations game designers never combine those two kinds of tests. Either the players are there to test the play of the game, or they are there to test the simulation processes of the game—never both at the same time.

It is far too complicated to do both effectively at the same time. To focus on just one at a time takes some discipline on the part of the players and planning on the part of the designer, but it really pays off in a smooth play test process.

F. Turing Tests: With this test, the movement, combat, battle dynamics are recorded and compared against the historical movement, combat, battle dynamics. For example, several games of Waterloo are recorded at the end of each turn to the end, and then they are compared with the actual historical battle mapped out at the same ‘turn' intervals.

Then the designer compares these snapshots of the action, the games against the historical events. The issue here is whether the actions in the simulated battles are so outside the norm that the differences are obvious when the actual battle isn't identified. Usually this is tested by handing the data to a knowledgeable third party—say several different experts in the history of Napoleonic warfare. They are asked if they can pick out which actions are the game and which are the actual history. If they have difficulty doing this, that supports the validity of the simulation dynamics. [And no, they aren't told which battles, but of course they will recognize the general situations of several. }

For non-computer simulations, this is most effective when snapshots of parts of the event, say a battle are used, rather than the entire event. This is done at the lowest 'grain' of the simulation, say the brigade level for AOE or battalion level for SHAKO.

G. Validating Input-Output Transformations: I love the engineer speak. For a wargame, the designer would look at input-output variables like casualties in comparison to the real event, not only the final tally, but at different points in the battle. For instance, can the Napoleonic rules actually produce the historic casualties on all sides when Waterloo is recreated event by event with the rules? That is, is it possible within the rules, not a requirement of all play. Does the game allow for the same kind of results from the same events in a battle?

At times, game designers get kind of confused over this issue. I know of one who felt that because a game of Bautzen played with his rules created close to the same numbers of casualties each hour as the real battle, it was doing a good job simulating—even though the game events were quite different from the historical events. All he really proved was that his rules caused the number of casualties he designed it to cause—which isn't bad, but that isn't the same as comparing the exact same events and having the game allow for the exact same losses.

H. Validation Using Historical Input Data: I rarely hear of designers even considering this one, even though it would seem one of the more obvious approaches—and one already mentioned by somebody in this thread. This test takes the simulation and plays it with all the historical decisions and chance events being recreated at the corresponding historical points in time. The question here is: Does the design allow for the historical decisions, and as a consequence, produce historical results? That is, if I used the SHAKO or NAPOLEON game mechanics, could I recreate the events of say Pakenham's attack at Salamanca with the same events and results? Obviously, when two players are at the helm, the game will most likely be very different because of different decisions.

The question here is whether historical events can be simulated? Of course, this means assuming the right cards, die rolls, and movement etc. etc., and if the game decisions were identical to the historical decisions. If it is difficult to match the two, then how much of a simulation is it?

I. Experimental Design: [Maxim: Test, test, test.] This is play testing with a vengeance—it's like test driving a car over a wide variety of terrain in all kinds of weather. What can the system take before falling apart? This is done three ways:

1. Test the extremes of chance: what happens when the British only roll ones? What happens if they get only bad cards? How wide a range of chance can be experienced before the game not a game anymore? How often can that kind of thing happen?

2. Test the solidity and performance parameters of the design. Of course, gamers do this all the time after they get a design. For instance, many games have ‘variants' and added scenarios which test the structure of the original design. The designer should be testing that first, as much as possible. The designer does it to know the specific limits and range of his design—and its simulation flexibility and fidelity under all conditions. Does the system work at different scales, with different combat arms, in different theaters of war.

What happens now is that wargamers trip over hidden design abnormalities in this kind of stretching the limits. Play Volley & Bayonet at battalion scale and watch what happens to the casualty rates compared to the Brigade scale. Play Empire V with the skirmish rules from Empire III or Age of Eagles and watch what happens. That kind of distortion is addressed in this step. Here the designer is asking ‘what can the players do with the system and how far can it be stretched before it breaks?'

3. Test extremes in play: How wild can strategies get and still have the design function as a game? This includes ‘gaming' the system. Do the rules allow you to succeed with undesirable or unhistorical play? Some of this ‘gaming' the rules will come out in blind testing and is solved there as play issues, but here it is pushed on purpose as hard as it can be pushed for simulation analysis.

It is easy to see that wargame designers 'sort of' do some of these things 'some of the time' in designing a wargame, but not in any methodical way that would actually establish that the design does indeed simulate the information it was designed to simulate. It is more involved that just producing a game, but then, if the claim is that their design is a simulation game, it would be nice to know it actually has been tested and does work as a simulation.

The hobby is filled with a lot of vague, feel good tests for simulation validity that don't establish a thing other than one gamer likes the game, another doesn't. That gamers feel good about a game isn't bad, but those 'tests' sure don't mean what most folks think they mean.

How many times have you heard a designer or gamer speak of the ‘historical result' of the game as the proof of simulation or historical accuracy? The idea behind this conclusion is the game results fall within some preconceived idea of what the possible, historical variations could have been. At best, this is a very limited and generally un-provable conclusion, and at worst means nothing much at all in terms of simulation game design.

No concrete or technical meaning at all.

Because the designer has already decided what a 'historical result' means for in his design, seeing a game produce such a 'historical result' only means one of two things:

1. IF the designer's definition is known, then a 'historical result' simply means the game does what the designer designed it to do, not that such a result actually simulates anything—that is a different set of tests.
or
2. Regardless of what the designer aimed to do, the game meets some personal notion of what that particular gamers feels is a 'historical result'. As the designer rarely provides any technical/historical definition of what constitutes a ‘historical result', this is just another "It feels right" comment on the part of the gamer. That is fine for that gamer, but it isn't simulation design.

The problem with this test is:
1. There is only one historical result for a battle, and gamers are not saying that the design always plays out the exact same way when they see results as 'historical'. To talk about ‘historical results' means the speaker has established some range of game results they believe fall within what were possible outcomes during a historical event. To design for such a 'range' without then testing that range as a simulation doesn't prove a thing.

2. Only the designer could determine what those parameters are for his game, and he would have already determined that his game produces ‘historical results' according to his lights. The gamer might disagree with the criteria, but the designer is the one determining what "historical results" means for his design.

That is not simulation validation, only design goals.When the design is working correctly, the designer would still have to establish that his design actually does simulate.

Game designers could produce functional simulations. They could let gamers know what simulation tests their design passed. They don't have to. But if they are going to claim that their games do simulate some portion of history, it sure would be nice to know that they actually have tested that claim and have some functional basis for it.

Best Regards,

Bill

Bill,
The 8 tests are used to find out if the game works as a simulation, and before that the mechanics should have already been tested if they work as a game, so what is the objective of experimental design? The layout of the post is perhaps a bit confusing as it makes experimental design look like a part of the 8 tests.

Also, is Turing test widely used name for test F? Sure it resembles the actual Turing test, but the name might be confusing sometimes.

Karsta:
Good questions. "Experimental Design" is part of the eight tests--I should have mentioned that it is part of C.'Extreme Tests'-- I was copying it off something else and did get confused--it shouldn't be listed as 'I' or the 9th test--It was a more detailed description of some of the approaches to Extreme Tests that can be used and simply misplaced…

Is "Turing Test" a widely used term? Yes, depending on the discipline. As the test, in all it's variations and applications developed from the actual 'Turing Test' and has the same basic purpose in testing the simulation system. It is often called a Turing Test though it is not the "Turing test" used with computer programs or systems.[ You wouldn't believe the variety of terms used to describe what is essentially the eight tests I gave.]

The question is 'How do you know a simulation works?' It has to be tested somehow. The eight tests were created and vetted to answer that question for a variety of mediums, purposes and subjects.

I hope that answers your questions. And no, I have never used the above tests to validate a computer simulation or mechanical system. All I know of those are what simulation designers have told me. I have used the eight tests in validating training simulations that contain no computer programs or aids. I have always been quietly astonished that I could sit down with computer simulators and talk about those eight tests as a common methodology, and that they worked for both, when our simulation mediums were soooo different.

Bill H.

Thanks Bill, makes much more sense now.

I have encountered some of these tests in my mechanical engineering studies, but never as a complete list like this. I guess it would be quite hard to apply some of these, like face validation, for things we do. Sensitivity analysis, testing for extreme conditions and checking the transfer functions however, sound very familiar.

Definitions

Ned: Simulation is simply the dynamic execution or manipulation of a model of an object system for some purpose.

Bill: Simulation is a model of an aspect of reality, something that exists in time and space.

System Thinking: "A model is a simplified representation of a system at some particular point in time or space intended to promote understanding of the real system."

Data Acquisition methods:

1. Deep Analysis:
2. Representative Event
3. Average Events:
4. Controlled Events:
5. Black Box it:

Note: none of this involves creating models (necessary if there is to be a simulation). They are simply data acquisition methods.

The Eight Tests for Simulation Accuracy

A. Face Validation:
B. Sensitivity Analysis:
C. Extreme-Condition Tests:
D. Validation of Model Assumptions:
E. Consistency Checks:
F. Turing Tests:
G. Validating Input-Output Transformations:
H. Validation Using Historical Input Data:
I. Experimental Design:
1. Test the extremes of chance: what happens when the British only roll ones?
2. Test the solidity and performance parameters of the design.

All this is interesting but irrelevant at this point. It assumes facts no yet placed into evidence. That is, Bill has yet to show how a model can be created. This must first be shown before we go through the efforts of testing it.

Rich

Rich,

I am not clear what level of detail of how to construct a model you are looking for a description of.

Am I correct you have been taking your material from Gene Bellinger's website? I ask because I didn't see a detailed description there of how to build a model.

Bellinger wrote in one section:

"Model

Model, like so many words in the English language, has a multitude of meanings depending on the context in which it is used. In a systems context I have come to understand model to mean:

A simplification of reality intended to promote understanding.

We often deal with things, later I'll call them systems, that are so complex as to be beyond the limits or our intuitive comprehension. As such, we construct models, simplifications of the real thing, which allow us to study that which we seek to understand.

Whether a model is right or wrong is simply a value judgment, whether it is correct or incorrect is something that will be evident in time. The most important question to ask should relate to the extent to which the models we develop promote the intentioned development of our understanding. The extent to which a model aids in the development of our understanding is the basis for deciding how good the model is.

In developing models there is always a trade off. A model is a simplification of reality, and as such, certain details are excluded from it. The question is always what to include and what to exclude. If relevant components are excluded there is a chance that the model will be too simple in nature and will not support the development of the understanding desired. On the other hand, if too much detail is included, the model may become so complicated that, again, it fails to promote the development of the deeper levels of understanding one seeks. One cannot develop every model in the context of the entire universe, unless of course your name is Carl Sagan.

Personally I find I learn a lot from models developed by others. In an attempt to return the favor many of the models I have developed over the years are available for downloading on various web pages on this site.

When I first began developing models what seemed the greatest hindrance to progress was a blank sheet of paper. I used to spend endless amounts of time trying to figure out where to start because I wanted to make sure I got it right. As a result of my insistence on getting it right I got it wrong, simply because the model wasn't progressing. Sound like a typical Catch-22?

What I finally realized was that it simply doesn't matter where one starts! Any place you start is the beginning. You can build the model from the top down, from the bottom up, from the inside out, or from the outside in. If one pursues continued elaboration of the model sooner or later one arrives at an equivalent well defined understanding.

Now that I have stopped worrying about getting it right, admitting that each stage of model refinement or elaboration is just another approximation of some more elaborate system, I have begun to make more progress, and hopefully am developing better models.

What I have found to be absolutely essential is that if one builds a model with the intent of simulating it, each step of the elaboration must be relatively small and manageable, and must represent an operational simulation. That is, each elaboration must run as a simulation. Every time I have developed multiple levels of model refinement or elaboration without testing it I have set myself up for numerous headaches."

In an earlier post I provided:

"[pg 5] Models of Object Systems

A model is a representation of something else, as, for example, the model aircraft flown in a wind tunnel is a representation of the actual flight of an 'object system' aircraft.

In order to construct a model, we need to know something about the object system we are interested in. The knowledge we use to construct a model may be generally accepted laws or principles about object systems like the one we want to study. In the absence of such well-founded knowledge, we may make tentative assertions about the object system and then proceed to build a model that reflects these hypothesized characteristics. Usually we try to make the structure of the model correspond in some degree to the structure of the object system.

We may model only a part of the object system, or we may model it all…"

[pg 6] "…We may use any convenient and useful device to construct our model: unrelated physical material, as in the case of the cisterns, tubes, and water used to simulate a market; scaled-down physical analogues as in the case of the model aircraft; an exact duplicate of part of the object system itself, as in the case of the space capsules…; combinations of actual parts and hypothesized behavior, as in the case of war games; word descriptions…; or mathematical formulae, as in the case of computer programs that simulate the economy."

Rich wrote:

Data Acquisition methods:

1. Deep Analysis:
2. Representative Event
3. Average Events:
4. Controlled Events:
5. Black Box it:

Note: none of this involves creating models (necessary if there is to be a simulation). They are simply data acquisition methods.

Rich:
Yes, for the most part that is true, but not entirely. A model built on the 'black box' approach to the information to be modeled will be different from one where the designer has done 'deep analysis'--at least, we would hope so.

And no, it isn't irrelevant--but to the point. We were talking about games [existing systems], and how they could also be simulations, and how those existing models [the games] could be 'tested' or validated as simulations. Discussing how to create the model itself [the game and subsystem] wasn't necessary for the issues being discussed. How the model could be 'accurate history' and how the model could be established as a functional simulation was the focus.

As to how a model can be created: That depends entirely on the medium and the goals for it including the model being a game. I can address that, but that wasn't my focus.

Best Regards,

Bill H.

Bob et al:

5. Simulation Design: So What?

I imagine that the five 'tests' that would be easiest and most applicable to wargame design are:

B. Sensitivity Analysis:
C. Extreme-Condition Tests
D. Validation of Model Assumptions
G. Validating Input-Output Transformations
H. Validation Using Historical Input Data

As I said before, designing a functional simulation isn't about some designer's unique vision of history, or their opinions about it, or even the kind of rules that most gamers consider 'fun.' A designer is free to design a game anyway he pleases. However, if he is designing a system that simulates something other than his personal flights of imagination, he has to prove there is a connection between the 'reality' identified and the game which simulates it. That is a technical distinction, not opinion, or science or outside the boundaries of art.

For instance, Le Feu Sacré III has just come out. The designer says this:

Le Feu Sacré puts the player in the position of the commander and focuses on realistic decision making and chains of command.

That is what simulations can do: provide 'realistic decision making', but there is no way to know if LFS does it, unless we know the history simulated. That 'decision making' has been proven to actually match the reality that makes it 'realistic.' Otherwise, the statement is meaningless in technical and gameplay terms.

To actually know this information, have it 'proven'? What would this do for the hobby, for gamers? For one, it would make gaming much clearer, and simpler. Sooner or later, the player is going to wonder what exactly is being simulated, what is realistic?

Here' an example from the SHAKO list just a day ago about a particular rule—the rules have formed troops firing at a range of six inches, skirmishers nine inches. The question was obviously:

Why on earth would the muskets fired by skirmishers have a greater range than close order muskets, unless they are rifles of course?

This information isn't written down anywhere else, certainly not in the rules. Thank goodness for email lists, huh?

Two gamers respond on the list. They aren't the designer or developer or test groups for the game. So they either heard this from the designer, another interested party, or they are stating what they feels is a decent rationale. Here is what they said, one with the post below and the other saying he was in agreement:

Ranges in Shako don't represent the actual ranges of the weapons -they reflect tactical doctrine. Skirmishers are assumed to be moving about, and also to be firing their weapons at longer ranges, while their close order counterparts will be firing volleys, but only at very close range. So the greater range of skirmishers reflects the fact that the base is the centre of a cloud of skirmishers, and also that skirmishers did indeed tend to fire at longer ranges, or to put it another way, close order troops were not allowed by their officers to waste ammunition on long-range fire, instead firing only at effective range.

I asked the posters if this was the designer's rationale for the difference or theirs. One said it was his and hadn't seen anything about if from the designer. The other posted this:

… I put down a possible rationale to the rule. But even if the explanation would have been a paraphrase from Conliffe himself, it would still be only a "possible rationale for the rule".

We play games. whatever rules we agree to follow, they are and always will be only "possible rationales to the events" in themselves. It goes with the whole concept of playing wargames and not warrring for real. :-)

At the moment, most gamers don't know why the rules are the way they are, and thus believe any rationale is as valid as another for a particular design, and that's what wargames are about. The only other option is 'warring for real.'

It's like two gamers debating why chess rules are the way they are. Why do knights move over pieces in an 'L' shape, or pawns only attack left or right, or why castles move in straight lines. Or why the queen is the most powerful piece on the board, particularly when the game was fashioned in the Middle Ages went women weren't politically powerful, let alone militarily.

Now you and I can think up all sorts of 'possible rationales' and have fun doing it, but the only one that counts in explaining the rules is the designer's--THAT is what explains 'why the rules are the way they are.'

It would be much better for hobby discussions of history and game design technically, if the gamers knew what history was actually being offered and that they actually did simulate something, rather than batting about 'possible rationales' because all that is left is 'warring for real'.

It would also enhance design. If I know what history is being simulated with a particular rule or mechanic, I have far better tools to simulate the same thing effectively in another design--or apply the same idea in another area.

And when gamers change rules, they would actually know what part of history they are monkeying with, and what the rules were supposed to do originally. Too often game rules are changed because the gamer can't explain the 6" vs 9" firing ranges, so changes the rule to match something he CAN explain, his 'rationales'.

A lot of ink and effort is wasted on discussions that are simply attempts to fill in the vacuum left by wargames that don't explain how they simulate history--history that remains unidentified.

Best Regards,

Bill H.

Other views of Simulation Game design.

I thought I'd post this, as it gives the same ideas from a different vantage point--in the computer game arena. It also begins to address Rich's concern about Model design. All italics are mine…

The entire article can be found at:

link

Designing User Interfaces to Simulation Games.
A summary of Will Wright's talk, by Don Hopkins.

Will Wright, the designer of SimCity, SimEarth, SimAnt, and other popular games from Maxis, gave a talk at Terry Winnograd's user interface class at Stanford.
He reflected on the design of simulators and user interfaces in SimCity, SimEarth, and SimAnt. He demonstrated several of his games, including his current project, Dollhouse.

Here are some important points he's made, at this and other talks. I've elaborated on some of his ideas with my own comments, based on my experiences playing lots of SimCity, talking with Will, studying the source code and porting it to Unix, reworking the user interface, and adding multi- player support.

The anatomy of a simulation game:
There are several tightly coupled parts of a simulation game that must be designed closely together: the simulation model, the game play, the user interface, and the user's model.

In order for a game to be realizable, all of those different parts must be tractable. There are games that might have a great user interface, be fun to play, easy to understand, but involve processes that are currently impossible to simulate on a computer. There are also games that are possible to simulate, fun to play, easy to understand, but that don't afford a useable interface: Will has designed a great game called "Sim Thunder Storm", but he hasn't been able to think of a user interface that would make any sense.

In other words, user interface are the points where the gamer interacts with the game. There is a lot of study that has gone into this area of game AND simulation design.

On the user model:

The digital models running on a computer are only compilers for the mental models users construct in their heads. The actual end product of SimCity is not the shallow model of the city running in the computer. More importantly, it's the deeper model of the real world, and the intuitive understanding of complex dynamic systems, that people learn from playing it, in the context of everything else about a city that they already know.

This means that a simulation is a melding of what the design does and what the player does with it. This includes the knowledge they bring with them about the subject of the simulation. The more knowledge the player has, the more information they need about what is being simulated for that 'intuitive understanding' to function well.

In that sense, SimCity, SimEarth, and SimAnt are quite educational, since they implant useful models in their users minds.

NOTE that the designer is stating what his simulations can do, not what it was designed to do… It was designed to be entertainment.

On the simulation model:
Many geeks have spent their time trying to reverse engineer the simulator by performing experiments to determine how it works, just for fun. This would be a great exercise for a programming class. When I first started playing SimCity, I constructed elaborate fantasies about how it was implemented, which turned out to be quite inaccurate. But the exercise of coming up with elaborate fantasies about how to simulate a city was very educational, because it's a hard problem!

The actual simulation is much less idealistically general purpose that I would have thought, epitomizing the Nike "just do it" slogan. In SimCity classic, the representation of the city is low level and distilled down compactly enough that a small home computer can push it around. The city is represented by tiles, indexed by numbers that are literally scattered throughout the code, which is hardly general purpose or modular, but runs fast. It sacrifices expandability and modularity for speed and size, just the right trade-off for the wonderful game that it is.

Some educators have asked Maxis to make SimCity expose more about the actual simulation itself, instead of hiding its inner workings from the user. They want to see how it works and what it depends on, so it is less of a game, and more educational.

But what's really going on inside is not as realistic as they would want to believe: because of its nature as a game, and the constraint that it must run on low end home computers, it tries to fool people into thinking it's doing more than it really is, by taking advantage of the knowledge and expectations people already have about how a city is supposed to work. Implication is more efficient than simulation.

A great game design distinction if used consciously…

People naturally attribute cause and effect relationships to events in SimCity that Will as the programmer knows are not actually related. Perhaps it is more educational for SimCity players to integrate what they already know to fill in the gaps, than letting them in on the secret of how simple and discrete it really is.

In other words, a simulation can operate better as an educational tool if the players don't know what the game design actually has modeled as cause and effect "lessons"--which are finite and comparatively 'simple.'

As an educational game, SimCity stimulates students to learn more about the real world, without revealing the internals of its artificial simulation. The implementation details of SimCity are quite interesting for a programmer or game designer to study, but not your average high school social studies class.

In other words, not everyone is interested in how the design simulates.

Educators who want to expose the internals of SimCity to students may not realize how brittle and shallow it really is. I don't mean that as criticism of Will, SimCity, or the educators who are seeking open, realistic, general purpose simulators for use in teaching. SimCity does what it was designed to and much more, but it's not that. Their goals are noble, but the software's not there yet. Once kids master SimCity, they could learn Logo, or some high level visual programming language like KidSim, and write their own simulations and games!

Other people wanted to use SimCity for the less noble goal of teaching people what to think, instead of just teaching them to think.

Everyone notices the obvious built-in political bias, whatever that is. But everyone sees it from a different perspective, so nobody agrees what its real political agenda actually is.

So the designer's biases are obvious, but why is not clear to the player.

I don't think it's all that important, since SimCity's political agenda pales in comparison to the political agenda in the eye of the beholder.

;-7 In other words, the bias the gamer brings to the game is far more powerful than any instilled by the designer.

Some muckety-muck architecture magazine was interviewing Will Wright about SimCity, and they asked him a question something like "which ontological urban paridigm most influenced your design of the simulator, the Exo-Hamiltonian Pattern Language Movement, or the Intra-Urban Deconstructionist Sub-Culture Hypothesis?" He replied, "I just kind of optimized for game play."

Then there was the oil company who wanted "Sim Refinery", so you could use it to lay out oil tanker ports and petrolium storage and piping systems, because they thought that it would give their employees useful experience in toxic waste disaster management, in the same way SimCity gives kids useful experience in being the mayor of a city.

They didn't realize that the real lessons of SimCity are much more subtle than teaching people how to be good mayors. But the oil company hoped they could use it to teach any other lessons on their agenda just by plugging in a new set of graphics, a few rules, and a bunch of disasters.

And there was the X-Terminal vendor who wanted to adapt the simulator in SimCity into a game called "Sim MIS", that they would distribute for free to Managers of Information Systems, whose job it is to decide what hardware to buy! The idea was that the poor overworked MIS would have fun playing this game in which they could build networks with PCs, X-Terminals, and servers (instead of roads with residential, commercial, and industrial buildings), that had disasters like "viruses" infecting the network of PC's, and "upgrades" forcing you to reinstall Windows on every PC, and business charts that would graphically highlight the high maintenance cost of PCs versus X-Terminals.

Their idea was to use a fun game to subtly influence people into buying their product, by making them lose if they didn't. Unlike the oil company, they certainly realized the potential to exploit the indirect ways in which a game like SimCity can influence the user's mind, but they had no grip on the concept of subtlety or game design.

On the game play:

Usually the game is separate from the simulation. Games can be based on conflicts and goals, that are external to the simulation itself. The simulation goes on doing its thing, and the user can play different games with their own sets of goals. The simulation does not consider fires spreading between buildings to be an error condition or a source of conflict -- that's just the way the simulator's supposed to behave. But the user might, unless the game they're playing is pyromaniacal.

Here Will is pointing out that the simulation, the environment, whether fires or buildings, is what the player operates with and in, but doesn't dictate the player's goals within the game.

The design of the game play has a lot to do with the user's model of the system, and SimCity elegently supports a number of different user models, games, and toys in one program. You can use the terraforming tools and natural features to play with it like a sandbox or landscaping toy, without even starting the city simulation phase of the game. You can even use it as a painting tool, drawing colorful designs and cartoons with land, water, roads and buildings. SimCity comes with several scenarios with different conflicts and goals, and has a menu of disasters you can invoke to destroy your city, or challenge yourself to recover. You can start your own city from scratch, and develop it in any direction you want. A satisfying feature of SimCity 2000 is the ability to put signs in your city, to name roads and buildings and parts of town. How else could you personalize a simulated city?

In other words, the player is given tools for designing the simulation contents before playing the simulation. the player's model. Gamers do this all the time with wargames, but most often without any knowledge of what each point actually is meant to simulate. Designers talk about their design providing 'a tool box' for the players, but doesn't mean anything is explained as to what is being represented.

On the user interface:

Will demonstrated the close up and overall views in SimEarth, and showed how SimCity 2000 integrated these with zooming in one window. He talked about information density and screen size.

Post Morta:
After designing SimCity Classic, then SimEarth, then SimAnt, then SimCity 2000, here's one way Will compares them: With SimCity Classic as the standard against which to measure, SimEarth was too complex, SimAnt was too simple, and SimCity 2000 was just right.
SimEarth:
SimEarth and SimAnt did not support the same level of creativity and personal imprinting that SimCity does. With SimEarth, anything you do is quickly wiped out by continental drift, erosion, and evolution; you can walk away from it for a while, come back later, and it will have evolved life or shriveled up and died without you, looking pretty much the same as if you had slaved over it for hours. It was too complex a simulation for people to grasp or effect in a satisfying way.

The time scale slows down as the game progresses, from geological time, to when life appears, to when intelligence appears, to when technology is developed. There was some trouble conveying this to the users. One thing that supported the notion of time scale is how the view controls along the bottom of the global map were ordered in a temporal progression, in the order you'd need to use them, from the continental drift display, to the technology display.

Bill H.

Just a note on 'User Interface':

All simulations have a Point of View, and from that point of view, control aspects of the simulated reality in common with the a 'real person' in the real world. Even "god-like" player POVs still have this restraint.

For wargames, the POV is at some level of command. One problem often laid on miniature wargames is that players are making decisions at several levels of command. That shouldn't be a problem if the 'user interface' was better defined. That is why Will's Game simulating a thunderstorm didn't work. He couldn't find a POV for the player…

Bill H.

Play testing methods

Here is a discussion of play testing a game from a professional game designer, Ian Schreiber, It is part of a new on-line class in game design—pretty conventional information on game design. The processes have been long studied and refined. There are personal preferences of course, but this is a great overview of playtesting by someone outside our wargame culture—just to establish that other folks [lots of them] are grappling with the very same issues. It covers procedures that are applicable to any testing, including the eight tests I just mentioned for simulation validation. This is the kind of testing that is done before establishing whether the functioning game is a functioning simulation.

The entire article can be found at Game Design Concepts:

link

Again, italics are mine.

Different Kinds of Playtesting

The word "playtesting," like the word "game," is overused and can mean different things to different people. In general, the term covers any activity where you are playing a game in progress for the purpose of improving it. But different playtests may have different goals, and it is important to know what your goals are before you do anything. I'll be playing a bit fast and loose with terminology here, so in this case the concepts are more important than the labels I'm giving them.

Playtesting can and should have several phases, with different goals—not just one single approach to the entire game design.

Bug Testing (or Quality Assurance)

The purpose of QA is to find errors in the game's behavior relative to its design. "Fun" does not enter the equation. If the designer says that the game should do one thing and it actually does another (even if what the game is doing may be superior), that is a bug that needs to be identified.

Obviously the designer of a simulation game has two purposes for the game: fun and to represent something else. If the game isn't doing what it was designed to do in the first place—as a system—then it can't achieve either goal.

Normally, we think of bug testing as specific to video games. Board games do have a corresponding kind of testing, where the purpose is to find holes in the rules and dead ends in gameplay – gaps in the game that the designer did not cover.

Focus Testing

In a focus test, you bring together players that are part of the target audience's demographic in order to determine how well a game serves their needs. This is normally done for marketing purposes, but if game designers are involved it can also help to make the game more enjoyable for that particular demographic.

For wargame designers, the demographic may or may not go beyond their particular club. Even with large number of developers or playtesters, there you aren't necessarily getting a good cross-section of the potential customers. Even so, this kind of playtesting, where players just say what they think after playing it, rather than act as game developers can be powerful.

Usability Testing

In a usability test, players are given specific tasks to accomplish in an attempt to see whether they understand how to control the game. This is done frequently in the greater software industry to make sure that a piece of software is easy to learn and easy to use. Video games can take advantage of this as well, and results from a usability test can be used to either change the controls or modify the early levels to teach those controls more effectively.

In board games, usability is doubly important, because there is no computer to respond to player input for you. If you misunderstand how houses work in Monopoly and place them on Community Chest spaces, the game will not stop you. By observing players who are trying to play your game, you can learn a lot about how to design the various game bits so that they are easy and intuitive to use.

Obviously, this is true for miniature games too. I remember playing V&B wrong in regards to retreats for several years because there was statement regarding whether units were 'squared up' or not, other than a diagram, which wasn't self-explanatory. That kind of thing can be avoided with this kind of test. And just as obvious is the need for the players to be unfamiliar with the rules at the start of the testing.

Balance Testing

A fun game can quickly become boring if some kind of play exploit exists that lets a player bypass most of the interesting choices in the game. If only one strategy can win and it is just a matter of which player follows that strategy the best, it is not as interesting as if there are multiple paths to victory. Likewise, if one player has a clear advantage over the others, it is important to identify that so that players do not feel the game is being unfair. The purpose of this kind of test is to identify imbalances in the game so that the designer can fix them.

This is also where the 'game lawyers' should be brought out in force. If they can't find the glitches that allow for unbalanced play, no one can.

Fun Testing

A game can be usable, balanced and functional and still be uninteresting. That elusive "fun factor" may be hard to design intentionally, but when people are playing the game it is pretty obvious whether they are having fun or not. Certain aspects of the game may be more fun than others, so it is also important to figure out what parts of the game need to stay the same… not just what to change.

All of these forms of testing have some elements in common. Best practices are similar if not identical. All are important to the success of a project. So why make a distinction?
The reason is that each is appropriate at different stages of completion in a project. Each kind of testing has different goals, and you need to know what your goal is before you can achieve it.

There is a great book out by another game designer, Raph Koster, called A Theory of Fun for Game Design that is a primer on how games are 'fun.'

Order of Effects

When should you do which kind of playtesting? What order do you do them in? A lot depends on your particular project, so some of this will be up to your judgment as the designer. However, there are some rules of thumb.

• Very early on in the project, you need to make sure your project will meet its design goals (usually the "design goal" is to make a game that's fun to play). Testing for fun is necessary to make sure you do not spend a lot of time building on the wrong foundation. If you are making a game for a specific market, focus testing may be involved at an early stage as well, simply to ask the target audience if a game with a particular concept sounds interesting to them at all.

And again, this is before testing to see if it actually is a simulation.

•Once you know that you have something, you need to solidify the mechanics. Design the whole game, making sure that all the details are taken care of. Test for "bugs." (Note that bug testing in software projects is often done continually throughout the project, increasing in intensity toward the end. Non-digital games are easier to "debug" though, and a "bug" can stop a playtest in its tracks, so it is important for us to have a complete set of rules early in the process.)

•Once the game is fun and the design is complete, gradually shift from testing for fun to testing for game balance. Make sure that all the numeric values and player abilities are where you want them to be.

Here is where I would test the game as a simulation. As I designed it to be a simulation, the tests simply see if I succeeded. If I haven't, then adjustments need to be made. And yes, depending on the severity of the changes, this might mean repeating the above tests to some extent. That's simulation game design…

•When the game is working and balanced, towards the end, you'll want to think more about the usability of the game. When you change usability you are not changing any mechanics, merely the way those mechanics are presented visually to the players.

This is an important step that is often neglected.

If you've ever encountered a game that you could only learn by being taught by another player (as opposed to reading the rules yourself), that is the kind of usability failure you want to avoid in your own projects. You may also do additional focus testing at this time, to make sure that the theme and visual elements of the game appeal to the target audience.

As I said, these are just guidelines. If it is incredibly important that your game be well received by a particular demographic, for example, you may be doing focus testing throughout the project at all stages. Do not let this order of things be your master.

From all that I have read of wargame designers discussing playtesting, particularly on TMP, most all to a little bit of everything above in any playtesting they do, never really targeting any one thing during the process. Everything is on the table in every playtest, including the history involved.

Different Kinds of Playtesters
As there are different kinds of testing, there are also different kinds of testers. Each kind of tester has their own strengths and weaknesses, and some are more important for some kinds of testing than others.

• Yourself. You are your own most valuable playtester. Do not forget your ability to play your game on your own. You know your game better than anyone.

• Other game designers. If you are lucky enough to personally know some other skilled game designers, you can get some very useful testing done through them. They are able to critically analyze your game and propose design solutions. (If you do not know any professional designers, perhaps you can at least make contact with other participants of this course.)

• Close friends, family, and confidantes. People close to you who are willing to provide their time to test your game are very useful. They are approachable and can make themselves available as a favor to you. Take good care of them, and do not abuse their kindness. Note that these people may not fall into any of the other categories, so while they are good for early tests, they may not be appropriate in more focused testing for bugs or balance since they may not know what to look for.

• Experienced gamers. Skilled game players are great at finding exploits and dominant strategies in a game, and are appropriate for balance testing.

• Complete strangers. People in your target audience are appropriate for focus testing and usability testing, and they are absolutely critical when testing for fun. Finding them can be tricky, though, because it is not in most of our natures to just walk up to someone we've never met and ask them to play a game. We will talk more about this in the coming weeks.

We all know this, but rarely is there a conscious effort to use the different gamers according to their own strengths. I've included the following to give an idea of how deeply this aspect of play testing has been thought of in the professional arena and is now being taught.

Order of Familiarity

In general, you will want to go through testers in order from more to less familiar. Test with yourself first, then with close friends, then with acquaintances that are useful (because they are designers, gamers, or part of the target market), and then with strangers.

If you show your work to other people too early, it will likely be in such a rough state with multiple design flaws and holes in the rules that it will waste their time and frustrate them, and you want to treat your playtesters better than that. Also, if you start playtesting with strangers too early in the process, you may not get useful feedback – if your game prototype is in a rough state with only crude art and components, for example, the playtesters may be so busy commenting on the poor quality of the pieces that they will not be able to concentrate on the gameplay.

At this point you might be tempted to just do all of the playtesting by yourself, so that you don't need to rely on other people or keep track of them. In practice, the designer eventually gets too close to their own project and is so familiar with the game's systems that they can miss some really obvious flaws. If you keep the same set of playtesters for long enough, they will suffer from this problem as well. You need to bring in fresh sets of eyes to look at your game on a continuing basis throughout the project.

Playing By Yourself

In the early part of playtesting, when you are playing the game on your own, here are some things you should be looking for:

Does the game meet your design goals?

Is it fun, at least for you? While you are not the ideal playtester to judge effectiveness most of the time, if you are not having fun then most other people will probably not either.

Are there any holes in the rules?

A "hole" is a situation where the rules simply do not say how to proceed. For example, perhaps one of your rules is that a player's army can attack another player's army, but you don't yet have rules for resolving the attack. What happens in this case? In practice, what happens is that the players sit around and wait while the designer figures out what to do!

As an example, consider these rules for Tic-Tac-Toe played on a 4×4 grid:
• Players: 2
• Objective: Get a straight line of symbols.
• Setup: Draw a 4×4 square grid.
• Progression of play: On your turn, place your symbol ("X" or "O") on an empty square.
• Resolution: If either player on their turn has a set of four of their symbol in a straight
line (across, down, or diagonally), they win.

If you try to play this game just following the rules, you'll quickly realize that you can't even start – nowhere does it say which player is X or O, or who takes the first turn! To fix this, you would add a situation to handle this. For example:
Setup: Draw a 4×4 square grid. Choose a player to go first, who is assigned the symbol "X". The other player is given the symbol "O".

Are there any dead ends?

A "dead end" is a game state where there is no way to proceed further, but the game is not resolved. Consider our revised 4×4 Tic-Tac-Toe rules above. Suppose that both players fill up all squares on the board without anyone winning. At this point the game cannot proceed, because the rules say a player must place their symbol on an empty square. There is no empty square, so the player cannot take a turn. But there is also no resolution, because neither player has won. In this case, a new rule would have to be added (such as: in the resolution, if neither player can make a legal move and no one has won, then the game ends in a tie).

Are any of the rules unclear?

It is natural for us to assume things that are in our head, to the point that we often forget to write them down in our rules. Try to look at your rules and see if there is anything you are assuming that your players might not.

Are there any really obvious rules exploits?

Is there a single strategy that wins the game easily? Try to find it. It's much less embarrassing if you find and fix it yourself, as opposed to having it discovered by your playtesters (or worse, your players after you release the game). Clarity and exploits are often hard to find in your own game; you tried to design this game to not have any problems, after all. Still, make an honest effort, and sometimes you will be rewarded by finding and fixing errors early (which saves a lot of time in the long run, leaving you more time to iterate on other parts of your design).

You might think that looking for exploits is something to do later in the project when balancing the game. Sometimes it is. It is a matter of degree. If an exploit is so powerful and so obvious that it prevents your play tests from giving you real information about your game, fix it now.

Solo Test Difficulties

There are a few things that are hard to test alone:
• Realtime multiplayer games, such as games where you must slap a card or say an
answer faster than your opponent.
• Hidden information games, where each player has information that only they know
and that is important to keep secret from the opponent.
• Trading, negotiation, and auction games, where each player must place a value on an
item, and different players may value things differently (and especially when players
can artificially extort high prices or drive up the cost of an item at auction just to make
their opponent pay more).

For the latter two, it is possible to play anyway, by simply limiting your actions to what you think you would do if you were in each player's situation, knowing only what they would reasonably know. Some people find this more difficult than others.
The simplest answer here is, for the purposes of this project, to not use mechanics that you can't test yourself. The alternative is to bring in another player or two early on in this case only, after you take things as far as you can on your own.

Let It Grow

Experienced designers often talk of a game "making itself" – as if the game has a life of its own, and the designer is merely guiding it rather than creating it. On the surface, this seems strange because really, the game is just sitting there and doing nothing unless the designer is playing it or changing it. What's going on here?

I think that what is really happening is that the creation of a game is a learning process. You may have some idea of where you want your game to end up, but the final version may be very different from what you originally envisioned. The reason why it changes is that at the beginning, you don't know very much about your game. You have some basic ideas, but you don't actually know how the mechanics will interact, or what the actual dynamics and aesthetics will be. As you playtest, you learn more about how your game's systems are working. As you learn, you become more able to predict the effects that changes will have on the system.

Right now, though, you don't have that experience… at least not with this game. Playtesting on your own is your first act of discovery. As you discover, it may seem as though your game wants to grow in a new direction, as if it has a life of its own. If you feel that, go ahead and listen to your game. See where the process of discovery takes you.

Homeplay

As the nature of the work at this stage is solo, you do not have to post anything on the forums. Work alone until Monday, at which point we will start involving others.
Before Monday, August 10, noon GMT, create a playable prototype of your game. You may find it useful to review the section of Level 4 on prototyping. Remember that you can make a prototype in about 15 minutes – it doesn't have to be pretty and it doesn't even have to be complete (yet), but it does need to be at the point where you can sit down and play it by yourself.

Then, play the game by yourself, at least once. In your idea notebook, write down any problems you encountered or questions that you ran into. Trust me, no matter how obvious the things are that you need to fix, you will forget them if you don't write them down.

Finally, write down the rules once your game is at least somewhat playable. This is also for your own reference, so that you do not forget.

Best Regards,

Bill H.

Ned, I agree with everything you wrote. And, I have been using Gene Bellinger's website. I even agree with Bellinger's statement: "combinations of actual parts and hypothesized behavior, as in the case of war games." However, there is a world of difference between military designed war games and what we do with rules, figures, and terrain. Not all wargames are created equal. My objection is when we try to describe the designer of our rules as simulation designers. To my mind they are game designers. The two are not the same.

Let's take Bellingers statement, "A model is a simplification of [an aspect of] reality." In our type of wargaming what is the aspect of reality the game designer is modeling and how does he do that? He cannot say he is modeling "battle" because "battle" is not an object system nor is it an aspect of reality. "Battle" is a collective name we give to a group of activities that at different times existed as an aspect of reality sometime in the past. Waterloo existed as an aspect of reality; Wagram existed as an aspect of reality, and Augsburg existed as an aspect of reality. However, all three battles never existed as a single unified aspect of reality to be modeled. So you can model Waterloo but you can't model battle.

What would it take to model the battle of Waterloo? Obviously you need to model the most salient aspects of the battle. Let me make it perfectly clear that I am not a Napoleonic gamer. I'm working off Wikipedia. And, I will list only a few of the salient characteristics. One would be the position of the British. Another would be the fight for Hougomont, Ney's charge, and the taking of Placenoit by the Prussians. Any one who wants to model Waterloo needs to reflect at least these salient characteristics. However, the game designer does not reflect any of these salient characteristics. The reason why? He is not creating a simulation. He is creating a game. As the game designers Roger Callois, Ian Schreiber (who teaches non-academic game design and who Bill refers to) and our own beloved ‘what's his name' said, games are make believe. They are not an aspect of reality.

Let's turn to consumer wargames (what we play with rules, figures, and terrain). It seems to me all pre-modern wargames share at least 5 components:

1. Turn sequence

2. Unit sizes

3. Rates of march

4. Fire

5. Melee

You cannot engage in a wargame with out at least these five components. They are a must. On the other hand, you can model Waterloo without using any of these components. It's not like Wellington turned to Napoleon and said "your turn." Unit sizes refers to the ration of real men to figures. Not needed. Rates of march are rates per turn. No turns means this is useless. In a simulation you want to make sure that all activities are occurring at the proper moment in battle. But you don't need rates of march to figure out when the French attacked the British line. As for fire and melee, they are also not needed. A simulation doesn't need to know how a decision came about. It only needs to know the results. In other words, the French attacked the British position and after a furious fight was thrown back. A simulation designer may want to include some of this material but he doesn't need to. He can design a simulation without any of this information. A wargame designer MUST include this information in order for the players to be able to play the game.

Rich

Rich: A simulation designer may want to include some of this material but he doesn't need to. He can design a simulation without any of this information. A wargame designer MUST include this information in order for the players to be able to play the game.

Yes, some kind of simulation could be designed without using any of that information, but it can also be designed with that information. Wargames are a subset of simulations. I don't think anyone has tried to say anything more. There are often differences between military designed games and consumer wargames, though in some cases the differences are quite small. Gulf strike is a good example of a game that has found use in both fields. This Strategy Page article names some others too: link

I'm still not sure where are you actually trying to get with all this. The point of this thread (at least how I understand it) is that methods used in design of simulations and other kind of games can be used to help our hobby too. Are you trying to say that this is not possible?

Karsta: Wargames are a subset of simulations. I don't think anyone has tried to say anything more.

That's my point. Wargames are not a subset of simulations. They are a completely different genre. They are games. And, games are not a subsets of simulations.

I'm still not sure where are you actually trying to get with all this. The point of this thread (at least how I understand it) is that methods used in design of simulations and other kind of games can be used to help our hobby too.

What I'm getting at is if you are going to design wargames you should know the nature of your project. You are designing a game not a simulation. I do agree with you there is much we can learn from outside our hobby. Just because we use some of these ideas does not mean we are simulations. Simulation creation takes one approach. Game design takes another approach. If you want to create a game and are taking the simulation approach you're taking the wrong approach.

Rich

Bill: We were talking about games [existing systems], and how they could also be simulations, and how those existing models [the games] could be 'tested' or validated as simulations.

This one sentence shows the conceptual errors of your approach. How do you go from games are existing systems to games are existing models? To paraphrase Ted's definition a simulation is a model of an object system. You've got the game as the system and the game as the model of that system. It cannot be both. You are trying to make connections that cannot connect. I don't think you have really come to grips with the concept of what is a wargame, the kind of games we play with rules, figures, and terrain.

Now I have advanced the idea that a wargame consists of at least 5 components needed for play. These five components are required for a pre-modern wargame. These components are not necessary for a simulation. Also, you cannot simulate ‘battle' because battle is a collective name and does not exist as an aspect of reality. Therefore, if you are going to model ‘battle' you must model a specific battle. If you wish to simulate another battle you have to create a new model for that battle.

The game designer, on the other hand, is not creating a simulation and can therefore use the 5 components to allow gamers to try to replay any of these battles. This can be done because the tabletop battle is make believe and not an aspect of reality.

Bill, this is an example of where you assert 2+2=5.

[Author of] Le Feu Sacré puts the player in the position of the commander and focuses on realistic decision making and chains of command.

Bill: That is what simulations can do: provide 'realistic decision making', but there is no way to know if LFS does it, unless we know the history simulated. That 'decision making' has been proven to actually match the reality that makes it 'realistic.

Just because simulations provide ‘realistic decision making' does not mean LFS is a simulation. Ian Schreiber states games also involve decision-making. Nor does ‘realist' decision-making means it is an aspect of reality. When to throw in your reserves is a realistic decision challenge. It doesn't matter a wit that the reserves are orcs.

I thought I'd post this, as it gives the same ideas from a different vantage point--in the computer game arena. It also begins to address Rich's concern about Model design.

Actually Bill, this is an example of going outside our hobby to prove something about our hobby. I'm not saying we can't learn from this. What I was asking for was an example of model design within consumer wargaming. I agree that computer game designers create models. What I am challenging is that rule writers create models. If rule writers do not create models then they don't create simulations.

Rich

Rich wrote:

Wargames are not a subset of simulations. They are a completely different genre. They are games. And, games are not a subsets of simulations.

Rich:
I didn't say games were a subset of simulations. I said wargames were. If you insist that they are 'completely different' regardless of what game designers are doing, then there is no point of discussing further. They aren't completely different 'genres.' They can't be. Designers in and out of the hobby state categorically that the are the same, the goals for their designs are explicitly those of both wargames and simulations . Military men, commercial game designers, the majority of hobby wargame designers, and even dictionaries say they are.

When two systems, wargames and simulation games have the same design goals, employ the same mechanics modeling the same subjects, how can you say they are 'totally different genres?' I'm not defining their design goals, wargame designers in and out of the hobby are defining them.

This one sentence shows the conceptual errors of your approach. How do you go from games are existing systems to games are existing models? To paraphrase Ted's definition a simulation is a model of an object system. You've got the game as the system and the game as the model of that system. It cannot be both.

The game system can also be a model of an object system—at the same time, sport. It's done all the time. That is why universities like MIT and little colleges like Wayne University offer majors in "Simulation and Game Design." You can say "It cannot be both", but from the designer of SimCity to MIT says it can… And that is counting our hobby designers such as Art Conliffe, Bob Jones, and Bill Gray. I designed simulations for more than a decade in education and business. Designed simulations that were also games, or games that were also simulations. It all depends on what the game system is designed to do.

What I'm getting at is if you are going to design wargames you should know the nature of your project. You are designing a game not a simulation.

You keep insisting on that, but it doesn't match the reality in our hobby, commercial game design, research and training simulations, let alone military wargames. Some one doesn't know the nature of what game systems and designers are capable of—the nature of current game design, let alone simulation design. You can insist it is me, but I'm not going to change my view because you insist I 'don't know.' Please, how about some game designers that say you can't design a game and a simulation together? For every hobby designer that you might find who says that, I can quote one that doesn't agree. Further more, I know you won't find anyone saying that out side of the hobby. It's a given. That's why scores of universities offer a BA in Simulation and Game Design, and not just those two separately.

The game designer, on the other hand, is not creating a simulation…

Even if the game designers say they are? Even if they claim their designs are both? IF commercial game designers in and out of our hobby say they are creating both, if everyonein the simulation and game design fields say they can and do, and have been for decades now, where do you get this idea?

…and can therefore use the 5 components to allow gamers to try to replay any of these battles. This can be done because the tabletop battle is make believe and not an aspect of reality.

I am not sure what you are saying here.

A tabletop simulation game can capture some aspects of reality—you insist they can't, but lots of people are claiming to do just that in and out of the hobby, and even more are proving it. What else can I say. If you insist it can't be done, then there is nothing to talk about. I've given you a lot of examples in an out of the hobby.

Some game designers claim to be doing just that—capturing some aspects of reality—offering 'realistic command' for instance; some claim they are offering only make-believe, and a few blightly claim to be doing both at the same time.

Just because simulations provide ‘realistic decision making' does not mean LFS is a simulation.

Quite true. That is why those tests were created: to establish whether the game design is also a functional simulation. However, if LFS claims to provide exactly the same thing that simulations provide—in fact only what a simulation could provide—that LFS simulates 'realistic' Napoleonic command decisions in some fashion, so he is claiming his game is a simulation, ipso facto.

Ian Schreiber states games also involve decision-making. Nor does ‘realist' decision-making means it is an aspect of reality.

So you are saying that Ian is stating the decision-making his simulation game offers can be 'realistic' without having any connection to any aspect of reality? Interesting.

When to throw in your reserves is a realistic decision challenge. It doesn't matter a wit that the reserves are orcs.

Are we talking about the little models acting as place holders in a game system, or the game system itself? Don't confuse the two. If it is the game system we are talking about—then it does matter how it is done, and where, when and what is done within the rules--the game system. Again, I could use poker chips instead of orcs figures. The question is whether the rules do what they claim to do, how they are 'realistic.' What the figures look like isn't best the point.

Actually Bill, this is an example of going outside our hobby to prove something about our hobby.

No, to prove something about simulation games, the ability to design them etc. You act as if our little hobby is the only ones designing wargames/simulation games for entertainment or that because our markers are lead figures instead of cardboard chits or a computer screen, the design issues are 'completely different'. Different mediums, each with their own challenges, but sharing the very same design issues and needs. That is why I provided that excerpt from the SimCity discussion. Same issues, same design challenges, with the same design solutions.

I have given you a lot of examples/quotes from hobby designers and that have been apparently ignored by you.

One of my points has been that our hobby designers claim to be doing the very same thing other commercial designers are, but they aren't because they 1. don't provide enough information about the models they claim to have created, and 2. don't test their designs to see if they actually do simulate history/past reality.

I'm not saying we can't learn from this. What I was asking for was an example of model design within consumer wargaming.

Right, we can learn from this when the genres are 'completely different.' You have been working very hard insisting that there is no connection between our games, other hobby simulation games and simulations altogether. You feel our hobby games have no relation to simulations or even wargames outside out hobby. And I said I'd be more than happy to provide examples of model design within consumer wargaming. [By this you mean miniatures, because all the examples of consumer computer and board wargames and simulation games has been rejected by you as not pertinent.

So, I fail to see where there is enough connect for 'learning from this.'

As for the example of a model design within consumer wargaming. How can I provide one, when you say it is impossible? At some point you would have to entertain the possibility for me to actually be able to do that.

I agree that computer game designers create models. What I am challenging is that rule writers create models. If rule writers do not create models then they don't create simulations.

Okay, What is a computer model? It is simply a program, a set of processes: If this happens, then that is the result, if that is chosen, then this will occur. It is called a program because that is what it is, a set sequence of processes and events that create the model. That is all a set of rules are: A program.

A computer program can be more complex, it can carry out many of the administrative 'die rolls' itself etc. that gamers have to in our wargames [for the most part], but in the end, a computer program is still just a set of rules. They create a model of an object system, just as the rules to a hobby wargame create a model—with rules and a sequence of processes.

The medium, whether board, table, role playing, or computer, does dictate what and how much can be done with the model [As Will says, "Computers have simulation limits.] But all those different simulation games create models using 'rules' to make a functioning system—a model of something else, the object system being simulated.

I mean, really, what do you think the designer of LSF had to do to provide 'realistic command decisions' with the game system? Using game rules, he had to attempt to build a model of Napoleonic command systems, and to believe it is "realistic", had to have some relation to that reality. It had to model the object system--what we know of the historical command systems during the Napoleonic wars.

Bill H.

Rich wrote:

However, there is a world of difference between military designed war games and what we do with rules, figures, and terrain. Not all wargames are created equal. My objection is when we try to describe the designer of our rules as simulation designers. To my mind they are game designers. The two are not the same.

Without repeating what you might say Ned, my response is:

I am not "trying to describe the designer of our rules as simulation designers." Never have been.

And you know what Rich? For this discussion, it really doesn't matter whether they, to your mind, are just game designers and that the two, simulation and game designers, are not the same.

Why? Because the hobby wargame designers themselves say they are simulation game designers, either directly by saying their designs are simulations, or by implying it when they claim to be "simulating battles" etc.

The only questions I've been addressing here are:

1. Can they design simulation games? and
2. How can we tell if they've been successful?

If you have already rejected their claims as impossible from the get-go, that game designers can't be simulation designers too, then there is no point in further discussion. You've answered those questions for yourself.

I have never been trying to prove that hobby designers are simulation designers.

Most fail to do much of anything that simulation designers have to do to create a functioning simulation. I haven't been insisting that they must be simulation designers. I wouldn't be discussing the topic at all if the designers hadn't already claimed to be designing simulations or offering only what simulation games can provide.

I have been trying to outline how they might be both and how we gamers could know if they've been successful.

Hope that clarifies that.

Bill H.

Rich wrote:

"The reason why? He is not creating a simulation. He is creating a game. As the game designers Roger Callois, Ian Schreiber (who teaches non-academic game design and who Bill refers to) and our own beloved ‘what's his name' said, games are make believe. They are not an aspect of reality."

Rich also wrote:

"Words mean things and it is incumbent on us to know what they mean. Otherwise not only is wargaming impossible But communication is impossible."

From the website of our own beloved 'what's his name' on his latest product:
link

"If you have a club, or a larger collection and gaming area, Lasalle can also be used to simulate historical battles of the Napoleonic Wars, such as Quatre Bras, Albuera, Saalfeld, or Eggmühl. In these cases, multiple players with multiple forces and a larger table will allow your group to simulate the excitement of a significant battle."

FWIW, I've been following this, and don't for the life of me understand where Rich is coming from.

It seems obvious to me: game designers (even amateur, no-talent schmucks like myself) often claim that their rules simulate some aspect of the genre they are written for. I've been nothing but intrigued, enlightened, and challenged by the proposal that such stipulations should be backed up with some sort of justification/explanation --and more importantly: that I have a chance to examine my own meager efforts in the light of the tests laid out.

Good stuff, as far as I'm concerned. Where's the harm/foul that requires admonition/opposition?

Rich: Wargames are not a subset of simulations. They are a completely different genre. They are games. And, games are not a subsets of simulations.

Huh.
Okay, here's how I see it. I thought that drawing it as Venn diagram might help to solve some possible misunderstandings: link

No one is trying to say that games (yellow area) and simulations (blue area) are the same thing, or that one is a subset of other. Some things, like historical wargames, just seem to have characteristics of both (wargames would be in the grayish area and other simulation games in that somewhat green area).

I'm not really sure actually if wargames should be seen as subset of simulations (like I said) or only as an intersecting set (like how I draw it). Are all wargames simulations? Does that orange area exist? Which games should we condemn there? Does it even matter? (I wrote more about that in the first page of this thread)

This is not about whether wargames are simulations or games, but that they can be both at the same time! That means we don't take one single approach to the matter, but we design them to be fun and to model whatever we choose them to represent. Both aspects need their own methods of validation.

Karsta wrote:

I'm not really sure actually if wargames should be seen as subset of simulations (like I said) or only as an intersecting set (like how I draw it). Are all wargames simulations? Does that orange area exist? Which games should we condemn there? Does it even matter? (I wrote more about that in the first page of this thread)

Karsta:

I think the answer depends on what the designer claims his wargame is providing. Most wargames are designed to replicate some aspects of war, real combat dynamics to some degree. Most wargames identify specific historical or current combat environments as being represented. By doing so, the conclusion can be avoided that they are attempting to simulate, model some object system with their wargame.

Now there are games that I would characterize as wargames that don't simulate much of anything, mostly because I know they weren't intended to model anything.

Richard Borg, in creating his games BattleCry, Command and Colors Ancients and Memoir'44, was quite clear about his design goals. He states that his games are 'Stylized History.' At best, with period names and figures, they provide some illustrations of the principles of war, like concentration of force, etc. but not an actual historical environment.

The board game Axis & Allies is an other example of a wargame that isn't a simulation. The original designer Larry Harris, speaking of his original 1972 design 1942:

"The goal was not to detail WWII, but to provide the players with a sense of the basic strategies and core decisions--and their results."

I enjoy playing both Borg's and Harris's games, and I don't worry about any simulating.

Again, the Designer decides where the game fits on that Venn Diagram with his chosen goals, not me or you. He is the one that claims his design 'simulates historical battles.' Our question is 'how can a designer do that, and how can you tell he has succeeded?"

You might add the design purpose to your Venn diagram… games, simulations and simulations games, and THEN decide where wargames fit depending on what the designers have for design objectives…

And yes, I do agree, both game and simulation need to be tested, validated, proven. It is a sad state of affairs when the argument is that simulations can't be fun or designed for fun. Read any wargame promotion. Implicit in each and every one is that recreating historical challenges in game form IS fun.

Best Regards,
Bill H.

Karsta:

Rats, missed this:

"By doing so, the conclusion CAN'T [not can] be avoided that they are attempting to simulate, model some object system with their wargame."

Bill:
Didn't even notice the error, but read it as intended.
Granted, those are pretty good examples of 'orange area' games. Still, they all have some small aspects that hint towards simulations. Perhaps designers intent is indeed what should count.

You might add the design purpose to your Venn diagram… games, simulations and simulations games, and THEN decide where wargames fit depending on what the designers have for design objectives…

That diagram was created pretty fast with the objective of explaining to Rich where we are. It needs work if it's going to be used to explain anything more. As I briefly mentioned in the first page, we need to take a closer look into design objectives if we really want to classify games; just deciding if something is a simulation or not won't get us far, though it might be a good start.

One of the core issues I see here is that designers often fail to communicate to players what their rules are supposed to do and what kind of players might like the game. There's never going to be a game that is perfect for everyone. Using a Venn diagram here didn't occur to me before, put it might be helpful to visualise possible combinations of different design objectives.

I'll be looking into this and possibly in the future I have something constructive to post. One thing that has interested me is whether we could apply some of the theories developed by an indie rpg project called The Forge. For example, they have stuff explaining how different players have different reasons for playing games and how these could be taken into account when designing (and selling) games.

Karsta

here is the Venn diagram that expresses how I view the issue. I think game designers can learn from simulation designers but the two are not the same, nor can they become the same.

link

Sorry about the slockiness of the diagram. I have a number of drawing programs none of which could handle a simple venn diagram. I finally had to copy a two circle diagram and convert it into a three circle diagram. But I think it put across my idea.

Rich

gweirda: FWIW, I've been following this, and don't for the life of me understand where Rich is coming from.

It's very easy. If you want to create a set of wargame rules then study game design. Study what makes a good game. Bill already referred to a website for creating games. Game design does not include making models of aspects of reality.

Rich

Bill: I didn't say games were a subset of simulations.

I was addressing Karsta. And I did not say games were not a subset of simulation. I said wargames of the type we play with rules, figures and terrain.

When two systems, wargames and simulation games have the same design goals,

They don't have the same design goal. The rule writer's design goal is to create a game. The simulation designer is to create a model.

employ the same mechanics modeling the same subjects

They don't employ the same modeling mechanics. Rule writers use gaming mechanics and modelers use modeling mechanics.

The game system can also be a model of an object system—at the same time, sport

That's not what you said. You said games were existing system and games were models of existing systems. "We were talking about games [existing systems], and how they could also be simulations, and how those existing models [the games]", sport

Quite true. That is why those tests were created: to establish whether the game design is also a functional simulation. … Are we talking about the little models acting as place holders in a game system, or the game system itself?

Sorry but those test cannot turn a game into a simulation, And Bill, all good games require decision making as part of its system.

You act as if our little hobby is the only ones designing wargames/simulation games for entertainment

Nope, I act as though our hobby is not involved with simulation gaming.

I have given you a lot of examples/quotes from hobby designers and that have been apparently ignored by you.

That's because they are designers who are not from our hobby.

[By this you mean miniatures, because all the examples of consumer computer and board wargames and simulation games has been rejected by you as not pertinent.]

They are not our hobby [rules figures and scenery] thus not pertinent because the subject is our hobby.

They create a model of an object system, just as the rules to a hobby wargame create a model—with rules and a sequence of processes.

Wargames do have rules and processes but this doesn't make them simulations. This is unless you think games like Candyland, Chutes & Ladders, and Parcheesi are simulations. They also have rules and processes.

So you think rules to a wargame makes a model. What model do they create?

I mean, really, what do you think the designer of LSF had to do to provide 'realistic command decisions' with the game system? Using game rules, he had to attempt to build a model of Napoleonic command systems, and to believe it is "realistic", had to have some relation to that reality. It had to model the object system--what we know of the historical command systems during the Napoleonic wars.

I don't know the command system in LSF so I can't answer your question.

Why? Because the hobby wargame designers themselves say they are simulation game designers, either directly by saying their designs are simulations, or by implying it when they claim to be "simulating battles" etc.

So if they claim to be chickens do we treat them as chickens? It's not what they claim, it is what they actually do that counts.

The only questions I've been addressing here are:
1. Can they design simulation games? and

2. How can we tell if they've been successful?

Consumer simulation games are called Sims, i.e. Sim City. Included are the rules for playing the game, programming code to develop the characters and computer code to generate landscape on which the game is played. Go on google and type in "simulation games" and it will refer you to these computer-generated games. In this respect your questions is irrelevant. Who cares if they can program simulated games.

However, I suspect that what you meant to say is can they design games which are simulations. In order to answer this, you first have to establish that wargames can be simulations before you can answer your question. You must address this question before you can go to your question. Rule writers do not create simulations therefore you question is answered.

Most fail to do much of anything that simulation designers have to do to create a functioning simulation.

That's because they are not creating functional simulations.

I have been trying to outline how they might be both and how we gamers could know if they've been successful.

They cannot be both. Game designers do not create models. Simulations designers do create models. How can you design a game that is both a model and not a model?

Rich

NedZed let us turn to the words of our beloved "what's his name."

"If you have a club, or a larger collection and gaming area, Lasalle can also be used to simulate historical battles of the Napoleonic Wars, such as Quatre Bras, Albuera, Saalfeld, or Eggmühl. In these cases, multiple players with multiple forces and a larger table will allow your group to simulate the excitement of a significant battle."

Notice that "he who must not be named" states you must first have your own figures and a table for terrain before his rules are of worth. Now, if you use his rules (with your figures and terrain) you can simulate historical battles.

Simulate: "imitate the appearance or character of."

With your figures and terrain and his rules you can create the appearance of Napoleonic battles. He is not claiming you can model history.

Rich

Karsta wrote:

"Granted, those are pretty good examples of 'orange area' games. Still, they all have some small aspects that hint towards simulations. Perhaps designers intent is indeed what should count."

Karsta:
Yeah. How can a game deliver on something that it wasn't designed to provide? By accident, which is just that, a design accident. ;-j And yes, I know you did it pretty fast. I was a good visual. I just thought of adding the objectives of each.

Best Regards,

Bill H.

Rich wrote:

They don't have the same design goal. The rule writer's design goal is to create a game. The simulation designer is to create a model.

Rich:
You keep saying that, even though I have provided inumerable examples of designers who state they are doing both with one design. You seem to be the only one that can't see both goals achieved with one design--and refuse to accept the scores of designers in commercial, training and military arenas doing just that--and basically suggest that all the designer in our hobby haven't and can't regardless of what they say.

I have just been reviewing ways that they can prove it.

Stating that games can't be simulations or be done together in one design doesn't mean a thing in the face of designers claiming to do it and proving they are doing it all the same.

They don't employ the same modeling mechanics. Rule writers use gaming mechanics and modelers use modeling mechanics.

Again, game designers say they are--including those in our hobby. Making the statement doesn't prove much. I have yet to see anything from you to suggest why that these folks are delusional, and being paid good money for it.

Here's a question for you: von Riesswitz's Kriegsspiel, game mechanics or modeling mechanics?

If you say game mechanics, it would be a reasonable answer as most all of our hobby wargames use the mechanics created by von Riesswitz. However, von Riesswitz designed it only to model battlefield dynamics--period. In other words, he was only attempting to create a model.

However, If you say the game rules are only modeling mechanics, then why are we using so many of the same mechanics today in game design and for entertainment? Now the very same mechanics unchanged for von Riesswitz's wargame are played for entertainment as a game.

It just ain't so, Rich. The same mechanics have been and are being used in both simulations and games, and then simulation games. Want more examples?

Sorry but those test cannot turn a game into a simulation, And Bill, all good games require decision making as part of its system.

;-7 No, they can only help prove the game does simulate. The designer can use them to help turn the game into a simulation. And Rich, all good simulations require decision-making as part of its system.

Again Rich, you can say it in red letters and there will still be a legion of examples to the contrary in our hobby and outside of it, commercial entertainment and otherwise.

Wargames do have rules and processes but this doesn't make them simulations. This is unless you think games like Candyland, Chutes & Ladders, and Parcheesi are simulations. They also have rules and processes.

So you think rules to a wargame makes a model. What model do they create?>

I think what continues to amaze me is your insistence that games have rules and processes that can only do one thing completely divorced from the designer's intentions and abilities. They are just tools, Rich. So when you say, "Wargames do have rules and processes but this doesn't make them simulations."

I say no, the rules and processes do make games simulations if that is what the designer created them to be, and the designer is successful in his efforts.

So what models do they create? What models do the designers claim to be creating? Models of command, combat, Napoleonic battle? That is the choice of the designer. It is then just a question of whether they have succeeded.

So if they claim to be chickens do we treat them as chickens? It's not what they claim, it is what they actually do that counts.

No, I agree with you there. It is what they do that counts. That means we check for feathers. You are basically claiming that no game designer can design both a game and a simulation, in or out of the hobby. So, are all these hundreds, probably thousands of designers delusional? In all their efforts to define simulations and games, somehow they have failed to realize they can't do them both? Quite a position, to categorically state, without any test, that they can't ever be chickens… regardless of what they claim or offer as evidence.

Go on google and type in "simulation games" and it will refer you to these computer-generated games. In this respect your questions is irrelevant. Who cares if they can program simulated games.

Don't get confused with the industry dominance or popularity of computer simulations--there are thousands of pages under that heading. You can find a host of sites on those thousands of pages of 'simulation games' that discuss the designs outside of computer games.

Perhaps you'd want to start with an MIT publication: "Rules of Play" that I mentioned. In it, the authors cover all games and their commonalities, including simulation games! ..not just computer games. It is required reading for any students at MIT working towards a Simulation and Game Design BA.

However, I suspect that what you meant to say is can they design games which are simulations. In order to answer this, you first have to establish that wargames can be simulations before you can answer your question. You must address this question before you can go to your question. Rule writers do not create simulations therefore you question is answered.

To your satisfaction at least. I think you should let those 'Rules Writers' speak for themselves there, Rich, rather than telling them what they are doing or can do. I certainly have answered that question several times in several ways. It has made no impression…

They cannot be both. Game designers do not create models. Simulations designers do create models. How can you design a game that is both a model and not a model?

Rich, I keep hoping that you'll at least acknowledge where I have explained where they can with examples… but obviously you believe that if you repeat that statement enough times, somehow I'll agree, that all those 'rules writers' and simulation game designers who don't agree with your pronouncement and state they are doing the exact opposite will disappear.

I know what you believe, I just don't have a clue why. Why simulations and games must be in their separate circles, regardless of what designers and rules writers say, or the evidence to the contrary. Why you insist that all those designers I have mentioned are wrong when they state they design simulation games.

Best Regards,

Bill H.

Rich wrote:

"Actually Bill, this is an example of going outside our hobby to prove something about our hobby. I'm not saying we can't learn from this. What I was asking for was an example of model design within consumer wargaming. I agree that computer game designers create models. What I am challenging is that rule writers create models. If rule writers do not create models then they don't create simulations."

and:

"Bobstro is absolutely correct. What is needed is a concensus as to what the term simulation means. I would add that we need is a consensus as to what simulation means with regards to consumer wargame design"

and:

"Notice that "he who must not be named" states you must first have your own figures and a table for terrain before his rules are of worth. Now, if you use his rules (with your figures and terrain) you can simulate historical battles.

Simulate: "imitate the appearance or character of."

With your figures and terrain and his rules you can create the appearance of Napoleonic battles. He is not claiming you can model history."

Rich,

I am still trying to figure out your linguistic distinctions so our words mean things and we can have effective communication. I am not trying to put words in your mouth, only to understand what you mean.

Am I correct that you are saying:

a) there is a difference between the verb "to simulate", and the term "simulation"?

b) that one can "simulate something" without automatically having the "simulated thing" be "a simulation"?

c) that all the members of the "consumer wargame industry" commonly accept, share, and use the same definition of "to simulate"?

d) that anything that is "a simulation" cannot be "a consumer industry wargame"?

e) that the reason behind d) above, is because "to simulate" in the consumer wargame industry "gives an appearance" but CANNOT create a model, and that "a simulation" MUST include a model?

f) that any "consumer industry wargame" that claims to create a model would be claiming a logical impossibility?

g) that there is a difference between the verb "to model" and the noun "a model" analogous to the difference between "to simulate"and"a simulation"?

h) that a "consumer industry wargame" could "model" something, but could not create "a model"?

I am also confused by this statement:
"Game design does not include making models of aspects of reality."

I assume that the definitions intended for "game" and "model" are key to the meaning of the sentence -and those definitions must not be the ones I find in the dictionary (and, for model at least, have been posted previously, ie: simplified -usu. mathematical- description of a phenomenon), because using them I come to the conclusion that I am one of those designers Bill speaks of who claim to be simulating (if, again, the dictionary meaning of simulate -ie: pretend, immitate- is used).

Though I am certainly doing so poorly and without the use of the formal tools/tests listed/referenced above (though in my defense I at least don't ask anyone to pay for what I do…), I find it odd to read that not only am I not doing what I thought (and claim) to be doing, but that doing so is impossible!

It seems to me that models/simulations can be elements of a game. What is a table/chart for, say, melee, if not a model? And if it's part of a game, then game design (this one at least) does indeed include making models of aspects of reality, doesn't it? It may be a poor one based on no more than ideas plucked from the creator's nether regions --but if the designer says it simulates (immitates/models) a real-life/historical phenomenon, then what is being discussed is how that claim can be examined/tested. The conclusion may very well be that it is not a good simulation --but I don't see the basis for dismissing the claim at the outset as an impossibility.

Dunno…maybe not enough coffee yet… ; )

Rich:
Actually you drew an Euler diagram, but… yeah. As you said, simulations and games are not the same. I agree. That was the whole point I tried to make with that diagram. Things can be games and simulations at the same time without games and simulations becoming the same. We know from examples mentioned in this thread, even if they are outside of our hobby, that those two sets can't be completely disjointed. Whether something is part of both or only other is probably best to leave for designer to decide. It seems most wargame designers want their games to model something. That's where simulation methods can help.

Actually you drew an Euler diagram

Thanks.

The point is, if you want to make a better wargame learn how to make better games. War simulators are doing something similar to what we are doing. Along the way we can use some of the insights they have gained to improve our games. We can do this without being simulations.

I understand the desire to model something. But building a game is not the same as creating a model. We need to have a good foundation in what it takes to build a good game. Then, bring that understanding into the realm of wargaming. That the only way we are going to get better wargames.

Rich

Designing a Model and A game system

One of the things that I have found very helpful is the common understandings that have developed concerning games and simulations.

One, but certainly not the only one, is the idea that all simulation and game systems have only four component parts:

TIME progression in the system

EVENTS caused by the system and player decisions

ACTIVITIES the processes that move events along

DECISIONS the control the player or designer has through the activities.

Both games and simulations use the same four components in various combinations--often the same combinations--to create a system. The simulation game uses the components to not only create a game system, but also have that same system model something else too.

These four components can be used to regulate the other three in a system, but the backbone of any game or simulation is how TIME or the progression of 'play' is handled. In Chess, it is one turn per side, in Life, it is both turns and squares, as a 'life span' is being represented.

In most wargames this progression of time, what happens first, second, etc. is the game turn, whether exact times are indicated [like ten minute turns], or simply units of time [five turns represents a day in the game, or like chess with just turns.] Wargames like Armati, Fire and Fury, Command Decision, use this division of Time as the way to organize and control the progress of Activities,Decisions, and Events in "phases".

James Machin actually has a game system Valor and Discipline that runs Time like a clock, with each minute being turned over. Activities, Events and Decisions are all moderated by a flipping of minute cards…

However, it's just as easy to have Activities control the progression of Time. In some games like Grande Armee or The Sword and the Flame, have command pips or cards, representing a set number of activities. These control the passage of time, even though exact 'times' are not portrayed. The board games such as For the People use cards to control the passage of time. Each side has a set number of 'activities' with each card. When the cards, and thus the activities are all used, the turn, a three month period of time, is completed.

It is just as easy to have Events control the passage of time. In Piquet 'turns' end when a game 'event' occurs, in this case when the various number of 'pips' are used up. George Jefferies' "Variable Bound" system was based on the idea of moving from event to event, what he called 'a change of situation.' All decisions, activities, and the passage of time were controlled by these Events.

While there are games outside of the hobby where player decisions actually control the progression of Time, Events and Activities, there are no such games in miniature wargaming. An example of how player decisions could control the progress of a wargame is if the player chose a set number of cards and arrange them in a particular order. The cards are game or 'command' activities, each requiring certain amounts of time to complete. Both players would turn over a card at a time simultaneously. The card with the least amount of time required, happens first. If both cards had the same activity and thus required the same time, both would complete the activity at the same 'time', then on to the next cards turned over.

Thus the player's decisions [choosing and ordering the cards] not only controlled the progression of time, but also what Activities and Events occurred--as well as how long the turn lasted. The turn being The cumulative number of minutes for the run of cards depending on the cards chosen [the result of player decisions]

I'll expand on this much more. These design concepts not only help clarify many game mechanics, but also provide a conceptual structure for design innovation and trouble-shooting. They are proven design tools used throughout the game design and simulation communities.

Rich wrote:

The point is, if you want to make a better wargame learn how to make better games. War simulators are doing something similar to what we are doing. Along the way we can use some of the insights they have gained to improve our games. We can do this without being simulations.

Something similar?

Some are, some aren't. Some simulation designers say they are doing the very same thing, and prove it. You can't use any of insights from Simulation Design in Game Design if you insist they aren't or can't be the same thing.

I understand the desire to model something. But building a game is not the same as creating a model. We need to have a good foundation in what it takes to build a good game. Then, bring that understanding into the realm of wargaming. That the only way we are going to get better wargames.

Building a game is exactly the same as creating a model; i.e., a system with its own logic and processes. Monopoly, Chess, Chutes and Ladders all are systems, a set of rules built from Activities, Events, Decisions, and the progression of Time. Simulations are systems based on rules built from the very same components. The only difference is the Simulation game system also models some aspects of a chosen reality instead simply representing the game designer's imagination.

I am all for knowing how to build better games--and a game's purpose can certainly be divorced from simulation design if so desired. And yes, not all of the needs of a good game design are identical to building a better simulation…But a significant number are the same.

What's more, the same system can be both a good game and a good simulation--because games and simulations have design structures, well as some goals, in common. And until you recognize that, you are limited in your ability to build "a better game." That is a major problem with wargame design in our hobby today. To insist that games and simulations are completely different is to be blind to important parts of game design itself.

And I have just provided an example above, the four components, recognized by both game and simulations designers as shared design elements

Best Regards,

Bill H.

Rich: War simulators are doing something similar to what we are doing. Along the way we can use some of the insights they have gained to improve our games.

Well, at least we can agree on that. That's the most important issue for me in this thread, so I don't really feel the need to press on further.

It's ok if you still want to disagree about whether games can be simulations or not. It's just that you have stated and explained your arguments and it seems no one can still understand how you reached your conclusions. If you want to convince others, you have to find new arguments or new ways to explain them. It looks like we are now just going around in circles.

I feel that this thread has fulfilled its purpose: Bill could post his articles, answer questions and defend his views in a discussion. I'm a bit surprised the whole thing was kept so civil . Perhaps it would be best to leave this thread as a useful reference for the future discussions and not turn it into another 10+ page monster. If someone finds a new way of approaching the matter he can always start a new thread.

Ned,

a) there is a difference between the verb "to simulate", and the term "simulation"?

Simulate: "imitate the appearance or character of" [New Oxford American Dictionary]

Simulation: "A model of some aspect of reality" [Bill}

There are other definitions of simulation than just Bill's. For example, one could say games are representation or simulation of something real, but the game itself is make believe. Or to put it in a wargaming perspective, a wargame battle is a make believe representation (simulation) of a real battle. Here, simulation simply means a representation. No model is involved. This is why it was important for Bill to define what he meant by simulation. Because of this, Bill's definition is our working definition.

b) that one can "simulate something" without automatically having the "simulated thing" be "a simulation"?

Correct. One can create thye appearance of something without it being a model of some aspect of reality.

c) that all the members of the "consumer wargame industry" commonly accept, share, and use the same definition of "to simulate"?

I wish. The term is very frequently misused.

d) that anything that is "a simulation" cannot be "a consumer industry wargame"?

Correct. Wargames do not models of some aspect of reality. They are make beleave.

e) that the reason behind d) above, is because "to simulate" in the consumer wargame industry "gives an appearance" but CANNOT create a model, and that "a simulation" MUST include a model?

No. Our wargames (rules, figures, scenery) do not use models while simulations, by definition, must use models.

f) that any "consumer industry wargame" that claims to create a model would be claiming a logical impossibilitHow many claim to create models?

Wargame rule designers write gaming rules, not models.

g) that there is a difference between the verb "to model" and the noun "a model" analogous to the difference between "to simulate"and"a simulation"?

No. A model is a simplified representation of a system at some particular point in time or space intended to promote understanding of the real system. [Gene Bellinger, Systems
Thinking]

h) that a "consumer industry wargame" could "model" something, but could not create "a model"?

No when you model something you create a model.

Rich

gweirda: I am also confused by this statement:
"Game design does not include making models of aspects of reality."

Let's put that into closer perspective. Wargame rule [that's the only part of a wargame the designer has control of] designers do not create models of aspects of reality. I also made a distinction that I was addressing pre-modern wargames. I do that because I know nothing about modern warfare.

I am one of those designers Bill speaks of who claim to be simulating (if, again, the dictionary meaning of simulate -ie: pretend, immitate- is used).

That is not the definition being used. We are following Bill's definition: simulation is a model of some aspect of reality.

It seems to me that models/simulations can be elements of a game. What is a table/chart for, say, melee, if not a model?

I look up 8 definitons four for table and four for chart. In none of them is the term model used. Table/charts for melee are not a simplified version of real melee. What they are are gaming mechanisms for resolving melee between two opposing stands of figures.

but if the designer says it simulates (immitates/models) a real-life/historical phenomenon

This is the problem I had with Bill in earlier discussions. There are different meanings for the word ‘simulation'. Simulation designers when they design their simulations build models of aspects of reality. As I wrote earlier, simulation can also mean simply a representation of something real. For example, wargames are make believe battles but they represent real battles. This does not mean that the rules are models of reality because the rules are make believe. Make believe is the antithesis of a model of an aspect of reality.

The conclusion may very well be that it is not a good simulation --but I don't see the basis for dismissing the claim at the outset as an impossibility.

It may well be a representation of something but it is not a simulation as Bill defined simulation. Your game is a make believe representation of something real.

Rich

I look up 8 definitons four for table and four for chart. In none of them is the term model used. Table/charts for melee are not a simplified version of real melee. What they are are gaming mechanisms for resolving melee between two opposing stands of figures.

Rich:
You can continue to look up definitions, but until you start thinking in terms of working definitions, all you will have is eight definitions by somebody—not what a chart does in a simulation or a game. That is defined by how it is used.

"But if the designer says it simulates (immitates/models) a real-life/historical phenomenon." This is the problem I had with Bill in earlier discussions. There are different meanings for the word ‘simulation'. Simulation designers when they design their simulations build models of aspects of reality. As I wrote earlier, simulation can also mean simply a representation of something real.

This is a perfect example, Rich. First of all, ALL simulations are representation of something real, not the thing itself. That is why it is called a model. But the real issue is practical design issues.

If you attempted to build a simulation that is "simply a representation of something real" and then you try to "build a model of aspects of reality", you will end up attempting to do the same thing in both cases—that is the practical reality of designing games and simulations.

Define away Rich--they don't control what designers do or attempt to do. The act of designing a simulation leads to the same goals, same technical challenges, and the same solutions, regardless of how you define a simulation or decide what that definition should allow designers to do or not do.

And if you don't believe me, try building a historical simulation based on both the above definitions. Use the same medium in both cases. See if you actually can employ different methods in building the two if you actually try to 'represent' and 'model'. Then, when you are done, see if anyone can tell the difference between the two designs…

For example, wargames are make believe battles but they represent real battles. This does not mean that the rules are models of reality because the rules are make believe. Make believe is the antithesis of a model of an aspect of reality.

A design can represent real battles, but doesn't model reality? Really Rich, that distinction is nonsense, particularly if you are actually building a simulation, a game, or a simulation game of a real battle. You can't represent a real battle without building a model of that reality.

You know, Model as in "replica", "mock-up", "reproduction", "copy", a "representation.

So a game can represent real battles but it doesn't represent real battles? Meaningless semantics, Rich. They don't enlighten, or direct, or support game or simulation design, particularly when the discussion is about the technical realities of both, and what designers are doing with their time.

Nonsense. Such distinctions don't help clarify what designers are doing Rich, or claiming they are doing.

It may well be a representation of something but it is not a simulation as Bill defined simulation. Your game is a make believe representation of something real.

That distinction is absurd, as in "Illogical", "meaningless", "bizarre", and "incongruous"…particularly if we are talking about actually designing such things.

The only thing that is make-believe is the idea that those distinctions mean anything or help anyone when a person sits down to build a design that 'represents real battles.'

It may well be a representation of something but it is not a simulation as Bill defined simulation.

You don't seem to grasp 'my' definition of a simulation.

Your game is a make-believe representation of something real.

That says it all.

Bill H.

The bottom line is:

Game designers didn't create their technical definitions and once all done, began designing games. Simulation definitions were created before designers could create them.

They were created while games and simulations were designed.

The definitions help them design simulations and games.

These 'working definitions' apply to designing, supporting them in the act, not deciding what designers can and can't do, or what can or can't be combined…particularly when those combinations have been done and are being done.

Bill H.

To get an overview of where game design is today--both conceptually, practically, commercially, the primer, now a text book published by MIT for their Simulation and Game bachelors is:

Rules of Play: Game Design Fundamentals by
Katie Salen and Eric Zimmerman.

The authors are both experienced game designers. The book is about game design in ALL their forms, from computers to pencil and paper Tic-Tac-Toe. What is examined is the present state of the art--what we know about game design.

In other words, what makes for a good game.

Yet, in the examination the authors include many pages on:

1. Wargames, including quotes from James Dunnigan and David Chandler.

2. Simulation games with numerous examples, many not computer games at all.

The discussion is inclusive--how games simulate, how games simulate war.

Whatever diagrams you are drawing Rich, the game design community isn't placing games in one circle and simulations in the other, with no overlap.

Just in the newspaper today, the University of California at Irvine that it has established the Center for Comuter Games & Virtual Worlds around a "Cyber-Interaction Observatory" for research… Next fall UC Irvine will debut a four-year undergraduate program "Game Science." The National Science Foundation has awarded UC Irvine researchers a $100,000 grant to study the differences in how US and Chinese gamers play "The World of Warcraft."

The newspaper [Sacramento Bee, front page, 12/3/09 reported:

While support such as this has been held up by Congressmen as an example of fiscal responsibility, UC Irvine has persisted, courting industry partners and raising millions of dollars in corporate funding and federal grants to study computer games and "virtual worlds" or simulations that can be used for everything from stroke rehabilitation to courtroom re-enactments."

"A lot of people, when they hear 'games' and 'university', they see nothing in common", said Walt Scacchi, a senior research scientist who helped found UCI's Game Culture and Technology Lab in 2001. "They think games are mindless and not something to be studied seriously. Like many new disciplines, there's often a high degree of skepticism."

Our hobby struggles over such things as 'simulation games' and 'playability' while the larger game design community has answered those questions in practical, functional ways that work--to the tune of Billions of dollars.

Our hobby certainly doesn't have to get anywhere as serious about game and simulation game design as the University of California at Irvine, but on the other hand we don't have to go re-inventing the wheel either, or worse, never finding the answers to game design issues that others have solved decades ago and moved on.

Best Regards,

Bill H.

Bill: This is a perfect example, Rich. First of all, ALL simulations are representation of something real, not the thing itself. That is why it is called a model. But the real issue is practical design issues.

This is why I use definitions. You have the habit of making unsubstantiated assertions. Picasso's Guenica is a depiction, a representation, a simulation of the German and Italian bombing of the city of Guernic in the Spanish Civil War. If you have ever seen the painting there is no way you can call it a model. Remember Gene Bellinger,s definition of model: "A model is a simplified representation of a system." Guenica is not a simplified representation of anything.

Remember Ian Schreiber's statement that a game is a make believe representation or simulation of something real. Make believe cannot be a model of some aspect of reality! A wargame design does not create a simplified representation of anything! You are simply wrong.

Define away Rich--they don't control what designers do or attempt to do. The act of designing a simulation leads to the same goals, same technical challenges, and the same solutions, regardless of how you define a simulation or decide what that definition should allow designers to do or not do.

Ah but definitions help us to understand what it is the designers do. Or at least helps some of us.

And if you don't believe me, try building a historical simulation based on both the above definitions.

After all this time Bill you still don't get it. We are not discussing the definition of simulations. You defined it. The challenge is to design a set of rules using the definition of simulation. So you think simulation designers do the same thing as wargame rule designers?

Lets take the battle of Waterloo. A simulation designer, in order to create a model of the battle, needs a set of procedures to provide a precise organizational breakdown of the armies involved. He needs a set of proceedures to describe the geographical layout of the battle. He then needs write a set of procedures to develop a time line to connect individual events such as the attack on the British on the French left wing. He will need some sort of algorithm for determining success or failure of an attack. He then needs to determine the major events of the battle and how they fit into the overall timeline of the battle. He probably needs to do other things but this is enough. If he were to create a simulation of another battle, whole new procedures must be created: new armies, new terrain, new timeline and new events.

Now the rule designer, what does he do to build a model of Waterloo. Nothing! He does not create a simulation of Waterloo. Look at LaSalle. I don't believe there is even a mention of Waterloo in LeSalle. He writes a set of general procedures by which figures can be moved, [The procedure for movement is a general procedure not based on any specific battle.] He writes a set of procedures by which combat is replicated. He write a general procedure by which firing and it's results are determined, and so forth. His procedures are general procedures and not based on any one battle. Unlike the simulation designer, he does not write procedures to determine the terrain of the battle. He does not design the procedures for determining the forces involved. While he does determine how certain events such as artillery fire is conducted, there is no effort to tie these events to a timeline of the battle. In other words, the rule writer provides the general procedures by which different actions on the table can be conducted. The terrain, the force, the timeline of events is not his to control. The gamers themselves control these. The game designer simply provides the procedures by which certain events can be performed on the tabletop. There is no way he creates a simplified representation of an aspect of reality; because what he creates are the tools (procedures)for make believe battles.

So I guess the bottom line is simulation designers create models of aspects of reality. Wargame rule designers create tools by which the gamers can use their figures and terrain to play their make believe battles.

Rich

Rich wrote:

After all this time Bill you still don't get it. We are not discussing the definition of simulations. You defined it. The challenge is to design a set of rules using the definition of simulation. So you think simulation designers do the same thing as wargame rule designers?

Lets take the battle of Waterloo. A simulation designer, in order to create a model of the battle, needs a set of procedures to provide a precise organizational breakdown of the armies involved. He needs a set of proceedures to describe the geographical layout of the battle. He then needs write a set of procedures to develop a time line to connect individual events such as the attack on the British on the French left wing. He will need some sort of algorithm for determining success or failure of an attack. He then needs to determine the major events of the battle and how they fit into the overall timeline of the battle. He probably needs to do other things but this is enough. If he were to create a simulation of another battle, whole new procedures must be created: new armies, new terrain, new timeline and new events.

Now the rule designer, what does he do to build a model of Waterloo. Nothing! He does not create a simulation of Waterloo. Look at LaSalle. I don't believe there is even a mention of Waterloo in LeSalle. He writes a set of general procedures by which figures can be moved, [The procedure for movement is a general procedure not based on any specific battle.]

Rich:
I don't know where you got your ideas of what a simulation designer 'has to do', but you are way off base in what they do and can do to simulate either Waterloo or Napoleonic battle in general. They certainly can and do build 'general procedures', based on your description.

I didn't say La Salle can simulate the battle of Waterloo, the designer says his rules can. He claims those 'general procedures' of his can be used to 'simulate Napoleonic battles.'

So of course, we are back to what 'simulating' means…

So I guess the bottom line is simulation designers create models of aspects of reality. Wargame rule designers create tools by which the gamers can use their figures and terrain to play their make-believe battles.

That's not my bottom line. You have yet to make clear why those 'tools' have to be divided without option between models of aspects of reality on one side and make-believe on the other. You really don't understand what simulations are and can do. And obviously I am not going to change your mind on that, even after all the examples and books I have provided to the contrary. You insist on make-believe being the onlyoption--why, I still don't know.

Stay with your make-believe battles. It sure is a whole lot easier, and you certainly don't have to pay attention to what wargame designers actually say they are designing if you play them.

Bill H.

Fitting a round simulation into the square hole of a game system makes it more difficult. What helps is there is no absolute level of modelling that makes the game a simulation.

The real truth of game/simulation or a game that simulates something is to just be a wee bit better than other games that are said to simulate that thing. That's all it takes.