"AI & Historical Scenario Design" Topic

Do you use artificial intelligence when researching to create a historical miniature-wargaming scenario?

That would be a "Hell no!"

Would not want to give up the joy of actual research, and introduce ~10% chance of hallucinations to boot.

+1 miniMo
I despise AI.

AI has its uses. I tried creating some Historical scenarios. I spent a lot o time working on it. I found AI was making things up. I caught it multiple times. Finally, I just quit. If they were made up scenarios it could do a decent job.

Thanks.

John

Sorry you guys feel that way, but my experience has been very positive. There really is a learning curve for the user AND for the AI. The more you work together the better the results get. I use both ChatGPT and GROK, both on the upgraded ($20/month for one. $30 USD for the other), and I assure you my budget is such that $50 USD/month matters. I keep both because they check each other, and also may suggest different good ideas. Both of them have now read hundreds of pages of my stuff and know me as historian and teacher and gamer. Either will occasionally refer back to something I said six months before, on a different topic, but relate it appropriately to the new one. I have yet to catch any important error. Omissions are more liklely but still rare for topics I know.

I fear you guys are playing King Canute.

I will add that I probably spend an hour a day, minimum, using one or both. $50 USD/month for 30+ hours of entertainment is a good deal, quite apart from instant access to information.

When I was a little boy we had the WORLD BOOK encyclopedia and I would pull down a random volume and read whatever caught my eye. I am 70 years older now and well educated, but the thrill is the same.

Have any of you taken a look at the Buffalo Wallow post I did a few days ago? I knew about that event only that it had happened, no details at all, and had, I think, a very useful and informative exchange with GROK about it. Give it a look.

TMP link

I agree, John, AI probably has its uses. But I do think if you've written a number of scenarios, you know all the dodges & AI has little, if anything, to offer.

And the actual creative act is very important. I don't think I'm surprising anyone on this forum to say we are involved in a creative hobby & anything that limits that creativity is probably to be avoided.

Creative, yes, but why would having access to knowledge or tools limit creativity ? I would think the reverse is the case.

I used an airbrush once. The result was horrible. It must be because the tool is fundamentally flawed. So I stopped.

Recently I've been using AI to generate my scenarios. I input basic parameters and what type of action I'm looking to recreate, here are a few AI-designed scenarios:

link

link

PA, yes. Do you then give the AI feedback?

@doc mcb,

I agree with you. Used properly – i.e., knowing its strengths and weaknesses, AI is a useful tool that gets me there quicker. It's best to think of it as a not very informed research assistant that you can send to the library and come back with a useful synthesis of the literature search. Don't expect it to be a highly experienced researcher. So, yes, the first results will be a learning curve for you and the AI. Eventually, well honed it becomes a type of mirror to your own self and an extension. Therein lies the danger that it comes an echo chamber of your own thoughts.

One final thing – what's the difference between a woodworking hand tool and power tool? Answer = a hand tool allows you to ruin a piece of wood slower than a power tool.

Might comment on your Dances with Wolves…discussed this with Co-Pilot and you might appreciate some of that.

I use LLMs every day for work. As I find more ways to use LLMs to save time on time-consuming tasks with no reward, I'll use them more, but I can't see how it's less work to use an LLM to generate an historical wargaming scenario than to just write the thing myself.

I try asking LLMs once in a while to summarize an historical topic. Sometimes that works, sometimes that's a disaster of confabulation. I spend a lot of time narrowing down citations, double-checking facts, asking followup questions, asking for references, etc. Unless I know what sources the LLM had in its training data and what sources it can currently search, I can't know where its answers came from. When I find made-up citations and bad references in the data, my trust goes out the window.

What LLMs are really good at is summarizing and searching human language writings. If you can hand over electronically formatted research materials the LLM can read, it will be able to very quickly write summaries, find passages, answer questions, etc. If you've ever tried to write an historical scenario from the works of several authors who each briefly touched on an event, or writings in an oral storytelling style, you'll understand why this might be useful. Instead of spending hours turning pages and tabulating all the facts sprinkled through reams of text, you can make an LLM do it faster for you.

It should also be useful to have an LLM research and translate from documents written in a foreign language. Of course, the problem here is that double-checking is impossible.

I have a new dream to use an LLM to write me smartphone software that will allow a group of players to walk up to a table for the first time and step through a fast-play, double-blind hidden movement map campaign that leads up to the tabletop battle we all came to play. All that programming and IT support used to be a huge cliff to climb, now it should be a much shorter step.

- Ix

The machine has no sense of humor nor imagination. But John McBride has both and the machine can mimic mine. A story:

The topic was eagles scavenging (for a scenario of Comanche families defending their butchering after a buffalo hunt.) I mentioned seeing eagles around the Racoon Mountain TVA reservoir, which is high above the Tennessee River. TVA has excess power -- which is hard to save -- during the night, and pumps a million gallons of river up into the big lake. Then next day when Chattanooga has all its light on they bring the water back DOWN to generate electricity. (Evidently a pump and a generator are the same thing, depending on which way the power and the water is flowing.) I mentioned seeing the eagles living on the fish in the lake, but wondered how that worked given that the lake emptied every day? And then I wondered how a fish might feel getting sucked up during the night and then dropped back down in the the river the next day, over and over again? ChatGPT recognized the whimsey, assured me that the intakes had screen to keep out the big fish, and said that I need not be concerned about the fate of a "Sisyphus carp."

Of course "Sisyphus carp" is hilarious, and when I inquired, the AI said no, I don't do humor, but I recognize patterns, including the patterns of humor. And it proceeded to analyze that, with a side trip to Chuck Norris jokes proving it could generate them endlessly on any topic I asked. Abd it noted that of course it knew all the Greek myths, and the idea of endless up and down was right there ready to be applied.

So maybe the AI does not have humor nor imagination, but it surely understands that I do, and helps with that. It may not create but it helps ME to create.

No. The AI also doesn't roll dice for me, or choose which armies I build. Research is one of the pleasures of historical miniatures gaming. Build an AI which will fill in computer-generated forms for me, and I'm all in. For that matter, I'd pay for bespoke fiction, certain story types having been dropped from the human repertoire. But I don't outsource research to a piece of software programmed by people I neither like nor trust.

doc, I know you're pushing the concept for all it's worth, but I also know that AI gets as confused on the whole concept of reality as some people I could name. How much time are you spending spot-checking your outsourced research? Have I any reason to take as fact any of your long posts in which you quote ChatGPT and Grok as though they were eye-witnesses or methodical historians?

But I don't outsource research to a piece of software programmed by people I neither like nor trust.

That's not really how the type of application we're talking about works. (I detest the word AI for a stupidly broad range of tools the same way I detest how wargamers use the word period to mean "a specific stretch of time." "oh, and a specific location, not everywhere during that time." "um … and only specific actions of specific people during that time." "you should guess which ones I mean.")

What you should be concerned about is the corpora … the data that has been collected. Unless your research includes bespoke volumes, not available in libraries (I have a lovely bio of Marie Curie written by one of her daughters that the popular GPTs have heard of, but do not have the text of.), the current crop probably has the info you are using.

You could ask it. And you should. Never ask for processed responses without citations.

Don't think of it as doing research for you. Think of it as giving you a guided, focused tour through more references than you could have possibly read.

Along those lines, with the citations, also ask for a precis of the relevant parts of the reference and for it to explain why that one was chosen (I usually ask for 3-4 sentence explanation).

How much time are you spending spot-checking your outsourced research?

Less time than reading the entire reference and personally selecting the 5% you need?

No, apart from the automatic AI summaries of search results.

I'm sure AI models have shamelessly plundered all my IP though, which is why I've stopped uploading rules and scenarios on my blog. Musk can get stuffed.

The issue of trust remains. ChatGPT and Grok are products of the same culture--maybe even the same programmers--who were cheerfully generating images of Black Confederate colonels and female East Asian SS troopers until it got too obvious even for Californians. I'm not sure any of the firms involved does "honest" let alone "neutral."

Without at least spot-checking--well, I believe Ambrose Bierce once defined a translator as someone who tells you what it would have been to his advantage for the speaker of a foreign language to have said. So far, I have yet to see an AI advocate come any closer to checking a result than running one piece of software against another produced by the same culture. For me, that's like watching the same movie three times and being impressed because it had the same ending each time.

Never ask for processed responses without citations.

Ask the lawyers how well that has been working for them.

Point, miniMo. And remember how nicely footnoted "Waterloo: New Perspectives" was--right up until it became obvious the footnoting was to documents which didn't exist or which didn't support the author's statment. I don't trust other human beings' research without spot-checking. I see no reason to exempt programmers from big tech firms.

I trust the carbonized scrolls dug up from the Pompeii library that are undergoing X-ray scans more than I trust AI.

When I was a little boy we had the WORLD BOOK encyclopedia and I would pull down a random volume and read whatever caught my eye. I am 70 years older now and well educated, but the thrill is the same.

Wow, me too although I don't know if I should claim being well educated. But I can't claim to be educated since I did not go to college.

Wolfhag

The main advantage of college is being told which books to read.

And a lot of us ignored that as far as we were able to get away with, and educated ourselves in the library.

Today the internet makes all of human knowledge available == but we are trying to drink from a million firehoses.

AI can help.

John, yes, you should NOT trust AI any more than you trust anything else. Verify.

But without AI you get x done. WITH an AI you get 100x done. If there is a 10% error rate, you still have 90x good stuff and 10x unreliable -- and don't know which is which -- versus the single x you were able to do alone. (And that also might contain errors.)

Doc:
link
Judges do not agree.

Judges do not agree.

There is no sanction for "using AI". What you are looking at is:

Federal Rule of Civil Procedure 11 (Rule 11(b))

By signing and submitting a filing, an attorney certifies that:

11(b)(2): legal arguments are "warranted by existing law" or a good-faith argument for changing it

11(b)(3): factual contentions (including cited cases/records) have evidentiary support or will likely have support after investigation

So, if you used AI, didn't use AI, had a junior who didn't write the citation right or remembered the wrong thing and didn't look it up, make a cut and paste error (people actually blamed cut and paste instead of the operator when that was a new capability), used a textbook instead of an authoritative source (what … books can't be wrong? wargamers read books!), or plain old making up.

The judge doesn't care why you made the same error that lawyers have been making for centuries. While they didn't have rule 11 (or American federal courts) in the 1600's, citations for contempt based on the same type of error trace back to English Common Law.

I know for some people who don't like something, facts are not really relevant to their argument.

Ask the lawyers how well that has been working for them.

It's working about as well as shoddy work has been in the past. This argument is about equal to saying chainsaws are poor tools because of tens of thousands of chainsaw injuries in emergency rooms every year (including single digit percentages of those categorized as "amputation").

Yes I do sometimes. AI is a useful tool for all sorts of things. It can research thousands of places, archives, museums, articles, war colleges etc., in just a few minutes. Then it puts all of that together into whatever your working on. Why wouldn't you use it?

But it isn't perfect. You need some knowledge of your subject because AI doesn't get everything right. Challenge it when something doesn't make sense.

I'm working on a solution with some friends to eliminate all of the problems by researching only verifiable data.

For now you can:
Inside ChatGPT you can create a Project.

Upload documents, videos, etc. you want to search

Then tell ChatGPT: Use only uploaded documents unless I specifically ask for internet research.

When searching tell ChatGPT to use only specific websites (give URL) and not to use sites like Wikipedia.

Normally it will show the source of the info so you can check or eliminate it.

This works surprisingly well.

Wolfhag

Wolfhag,

That will mitigate some of the issues, but you still have a probabilistic LLM in the core of the GPT.

The GPT generates "new" content … that is, content that follows the sense of the corpora (collected input data) by probabalisticaly linking different material together.

"Hallucination" is a crappy term because an LLM doesn't understand material, let alone think (nor does any AI). While "linking together of multiple low-probability associations in a way that violates the semantic sense of each of them" isn't as catchy as "hallucination", the generator in the GPT still has a chance of doing that.

For mechanized artillery, the words "fire", "track", and "battery" have multiple meanings depending on context. A GPT could still crosswire those contexts and create a "hallucination".

To your last point, limiting the corpora of an application does limit this any many of the other problems with GPT outputs. What you are doing is actually a leading edge trend in AI engineering – bespoke corpora for specific applications.

Or, put another way, while I liked K-Pop Demon Hunters, the lyrics to the song "Soda Pop" only contribute noise to my use of a GPT to analyze aspects of the Crimean War.

(I use this example in classes. KPDH is about a year old now, so I have to pick a newer pop reference soon. But it is a fun song.)

No

To your last point, limiting the corpora of an application does limit this any many of the other problems with GPT outputs. What you are doing is actually a leading edge trend in AI engineering – bespoke corpora for specific applications.

Thanks. It appears to be the only way to ensure you have the correct data. My wife's company is doing the same thing and has employees taking AI courses, she's an IT Project Manager. There are ways to get ChatGPT to go back over the prompt again stating it missed material and had made some mistakes. Some AI are better than others.

I'm working with some commercial game designers and professional historians to collect and categorize accurate and verified historical data to put into a repository. Our first trial will be using NotebookLM. It looks very promising and it has some really cool features that can help you generate a video and podcast.

My son is a Field Service Engineer (non-degree hardware) for a US company that sells hi-tech comm equipment to the US and foreign military. They were having a software problem which was not being addressed for his clients. He went into Claude and had it write the code to solve the problem and he tested it out. Then he had Claude give him a through explanation as if he were a 5 year old so he could explain it to the electronic engineers.

Now his boss has him meeting with the engineers for new product design because he knows the HW and SW side and works closely in the field and trains clients. He leveraged this into a raise and more time working off site. Fortunately, he's much smarter than me.

I went down to my small volunteer fire department in TN to volunteer. I'm too old fire firefighting but the Chief said he needed help with their system and writing grants. I told them I'd see what I can do. Using ChatGPT I came up with a plan to gather all of their info for boiler plates when submitting grants, found out how they are scored, main points to emphasize and had it fill out a grant proposal as a pilot project.

We'll be putting together a 10 year plan for expansion and plot out what new equipment we'll need and what needs to be replaced and it will be tied in with the rapid residential development going on in our area.

We are also looking at a bar coding inventory system to track what we have and the date equipment needs to be replaced to target grants in the future to replace it. I'm learning as I go along.

Wolfhag

he had Claude give him a through explanation as if he were a 5 year old so he could explain it to the electronic engineers.

While I wouldn't specify 5 year olds (poetic license, no doubt), reveiw and restate is a great use of AI for a lot of things. For longer than AI has been a buzz word I have used LLM tech to do that for rules.

Since there are half a dozen free, online options nowadays, I have taken to having them several read scenarios, and summarize them. The I collect the summaries as a set, feed them back into the tools for compare and contrast. It's a great technique for finding subtle problems as well as identifying things that are unclear.

I use a similar approach to writing fiction. It doesn't write the material. Occasionally, I take a rewording recommendation. Often I don't because I am trying to take the story in a different direction than the mass of the corpora. But when the tool says "you didn't resolve the tension in this issue … let me help" it's good feedback because it's highly likely I didn't want the tension resolved. (If you do, don't read my fiction.)

link

They're also good for checking the consistency of the material against other material from the period. And great for managing jargon and slang.