Emergent goal seeking AI for RPGs

Computer RPGs I’ve learned to play rely heavily on combat as the way to gain experience. Couple of previous post discussed about alternatives to combat, and Matt’s comment made me thinking about a bit different type of way to perhaps create emergent play in RPGs. No idea if this would work, but here goes.

Imagine there’s bunch of goblins guarding a treasure cave. Each goblin would have motivations, such as these:

  • Need for food (avoiding hunger)
  • Shelter (avoid rain for example)
  • Being safe (for example, sticking with a bigger group or perhaps running away)

And so on…

Now, our hero arrives near the treasure cave and thinks for a moment about the situation. In our typical RPG, you’d usually draw your sword and hack’n'slash em.

But what if we could instead have AI that reacts to different stuff. For example, hero could leave food near and make noises that attract goblins to check out the noises. When goblins go check noises (that would be programmed in a common “guarding behavior” AI), they would see food… and hungry goblins would remain there to eat the food. Others might instead return. If there wasn’t any food, all goblins would return back.

Or what if hero uses a rain spell: it starts rain and it causes goblins to scatter: some go find shelter at nearby trees… some perhaps would go back to cave.

These type of motivational goals, combined with behaviors such as guarding might create interesting or different options for players, without need for custom scripting for every single event. Instead of focusing scripting events, you would focus on creating new goals and behaviors.

Here’s some more motivational goals for creatures:

  • Ensure safety of your child (works for bears as well as humans)
  • Greed (wants gold)
  • Item collector (wants to collect for example different swords or beast claws or whatever)

Et cetera. By adding more and more of these motivational layers, and then with behaviors/intelligence on top of that, we might be onto something.

You could add there some scripting or random events: perhaps somebody else approaches the cave and goblins immediately attack that guy… leaving our hero to sneak past the goblins. Or perhaps hero decided to hire somebody to “ride near this cave the next midnight”.

Experience might be rewarded from trying different options, if there would be some sensible algorithm for that. Or, experience might be based on finding items or visiting locations, not killing creatures between you and the treasure.

Thoughts?

10 thoughts on “Emergent goal seeking AI for RPGs

  1. Matt

    Hi Thomas,

    Nice points. I especially love the image of an army of Tamagotchi. I mean, running the supply chain for an army is really no different to Diner Dash or probably Sim City because the desires are often amalgamated into large blocs. Reality or pseudo reality isn’t always like that.

    I think it’s possible to have a possessed NPC learn from the possessions and provide a greater number of responses than would normally be programmed. For the decisions about villages and wolves, there would have to be some way to have the intelligence controlling the villagers (artificial or otherwise) well-informed. I still think there’s a possibility in a game where players like me – who have decent free time in small bursts- can contribute to a game like WOW in an asynchronous manner.

    Maybe easier to visualise in a game like Resistance. Where your game might depend on the outcomes of my game. But your game is an FPS. And mine is a supply chain management game. That’s the sweet spot.

    Reply
  2. Matt

    Nice conversation :) about stuff I love talking about!

    Programming the base needs for every creation that has motivations is a lengthy task. Unless you provide some sort of internal tagging system. As in “Barliman Butterburr loves gold more than being dry more than beer more than food more than sex more than conflict”. In the game then, you have to label things with these standardised tags. Gold, beer, comfort, sex, food, conflict and work out your algorithm from there. I imagine there’s something to be learned from Tamagotchi here.

    That said: I mentioned in the previous comment about bringing the casual gamer population to bear.

    Is there a way to have the wolves (mentioned above) played by a casual player? A way to have complex decisions handled by players (in a Quora type situation) where important decisions are pushed to other players? Flip the situation? We have these incredible devices which are permanently connected to the Internet? Why not?

    Or can the system cache the ‘success decisions’ from other people playing through? As in, if the antagonists n the game had a better chance of beating the player by following a certain path, why not record that path? Think of it as a leader board for NPCs. Why not score antagonists in games the same way? You’ll get better decisions in the long run.

    Reply
    1. Thomas Morgner

      The good part about games is, that they have a rather simple world. Drinks, for example, can all basically be the same thing. Add slight variations alcohol levels or health options (each one variable for common game systems), and you are ready to go. You probably already tag objects in the game anyway, to allow your character to drink every drink with the same animation.

      The Tamagotchi is a good example, actually. It has a very limited set of variables, yet still allows you feel that it is alive. I love that idea – when we think of NPCs as a army of Tamagotchis, then the picture becomes much clearer.

      However, be careful when you allow multi-player scenarios. Human player tend to game the system. In single player mode your user may happily immerse in the world, but in multi-player things get nasty fast. Cheater gaming the weakness of the system, groups of people deliberately manipulating the market (as seen in Ultima Online) – your common game AI stands no chance. To win against a band of human players you need at least human intelligence in your AI. And when you are at that point, why produce games – take over the world for real :)

      Letting players decide is a interesting idea. Most MMO-RPGs have the problem that players disappear when they log out. It’s hard to produce more than a hack-n-slash (WOW!) game with that. A player can’t open a shop if that shop is closed most of time (and shops would be useless for customers as well).

      But borrow from Matrix (the movie) and you get NPCs that exist all the time, but sometimes they get possessed by players. The player then can play his character as he like (with penalty/reward for staying ‘in character’). When he logs off, the character continues on auto-pilot.

      When you let the character learn from its player’s actions, then your character adapts over time and appears smarter and more realistic.

      That sort of learning of what is good and bad should be possible with a small neural-net – assuming that you can prepare clean high-level inputs from the player’s actions and result of these actions. This is what “Black and White” used for their critters. Combine that with normal scripting with heavy use of manually coded rules and tagged objects, and you have something quite powerful.

      FYI: There’s a paper describing the “Black and White” approach:
      http://192.5.53.208/u/brown/242/assts/termprojs/games.pdf

      That sort of approach would not make creatures intelligent or think for themselves, but it would present players a “living” world where emergent behaviour can happen.

      A final word: Like with all statistical approaches – if you let good and evil learn from the same source all you get is an average mix of good and evil – a goovil so to speak. It probably would not be much fun to play such a game. Games – like movies and good books – need a distinction between good guys and bad guys to be enjoyable. So have separate AIs for good and evil (with as many sub-types as you can add – the more the better, I would say!).

  3. Thomas Morgner

    I don’t believe that you have to create a full scale AI to make a believable world. Its more about covering the basics, and working with the emergent behaviour you get from generic systems.

    Instead of thinking really big, think small. The state of a system is whatever the system reports it to be. At first glance, no one will be able to tell a scripted system from a intelligent AI.

    Start simple: Add a schedule-system ala Ultima 6 (sleep at night, get up in the morning, work, pub in the evening, back to sleep) as basic routine. That alone makes your NPCs stand out from 90% of the games.

    Next add the basic motivations: Hunger, Fear, Attraction, Greed. Let them override the basic schedule and provide scripts to solve these motivations and you have a lightweight human behaviour.

    Implementing that is rather trivial, and scaled out over a larger number of NPCs gives a good approximation of a working village.

    If you need to decide what is good or bad – well, there is no global good or bad in the world we live in. People love you when you improve their situation, people hate you when you mess with their happiness.

    In the wolf-vs-fence example: Track what the result of what the player is doing after he announces to help you. Don’t worry about the wolf or the villagers feelings. Just measure!

    Lets do that in the real world: I, smartie-bear, come into your town and promise you to make your sheeps safe (so no more lost sheeps).

    * Choice A: Now I go into the woods, and kill the wolf. I come back two days later and you have not lost a sheep.
    * Choice B: Now I go into the woods and do nothing. Without me doing something, the wolf has a heart attack and dies. I come back two days later and you have not lost a sheep.
    * Choice C: I chop some wood and build a fence. The wolf comes, finds no entry and leaves. I come back two days later and you have not lost a sheep.
    * Choice D: I kill loads of cows from other villages and the wolf is busy eating them for the next two months. I come back two days later and *you* have not lost a sheep.

    Now, as a villager, all you care about is “I come back two days later and you have not lost a sheep.” Thus you love me and hopefully pay me.

    How to program that?

    Assign each sheep a value. Sum up each villager’s sheeps to calculate the villager’s score/equity. When the wolf kills a sheep, their score goes down and they are unhappy (delta to last week is negative). If you promise them to sort out the problem [=in-game trigger], forcast next week’s sheep-loss and let them wait. If the score-loss is less than forecast, your villager will be considered happy.

    Conclusion:

    Don’t think about concrete examples, thinks about general clauses. It makes your life a lot easier.

    Reply
    1. Juuso

      Excellent points, that’s exactly how it needs to go. You can apply the same stuff to very many things, and there can be consequences too.

      Let’s say you chose to kill cows of that other town… Somebody might have seen it and is not happy for what you did. Itmight trigger hunt against you. Or if they know you are too powerful, they might refuse to trade with you or help you when you show up.

  4. Fili

    Well, you can’t have something like “violence is bad, diplomacy is good”. Let’s say we have these situations:
    - a village is constantly attacked by a pack of wolves. You have 2 options: kill the wolves (violent option) or bring 2 sheeps from the village each week to feed and calm down the wolves (diplomacy option). Which one would the villagers prefer? I think they want the wolves killed…
    - a village is disturbed each night by some beggar. You have 2 options: kill the beggar (violent option) or give some money to the beggar and make him go away (diplomacy option). In this case, killing the beggar would make the villagers hate you.
    So you would end up having to think about all possible outcomes and decide for each of them if it’s good or bad. You can’t say violence = bad, diplomacy = good.

    Reply
    1. Juuso

      I dont think game would need to say which is good and which is bad – let the player decide that :)

      P.s. You could build fence to keep wolves away… and then it would require heckofa AI to link “wolves are away because hero built fence, therefore villagers like hero”… without scripting ;)

  5. hermitC

    Well, a little research including try&error would be necessary. But that could be done with information gathering from other players or NPCs which already know enemy weaknesses. Espionage would be an interesting new dimension in this regard.

    Reply
  6. Rob Basler

    It sounds neat, but I could see it easily degenerating into the problem that old adventure games had, where oftentimes you’d end up using every item in your inventory on every puzzle to see what one thing the designer intended you to use in that spot in the game.

    If your game features combat, would you really expect players to spend all day using their rain spell – not knowing whether or not each creature disliked rain, or would they just use their fire spell and burn ‘em all up?

    Reply
  7. hermitC

    In this regard I think non-violent approaches have to have positive side effects while violence should have more bad side effects.

    For example slaying everybody crossing your dungeon path will damage your reputation and tavern guests won’t tell bad people any secrets. When goblins, orcs and other humanlike-nonetheless-seen-as-evil beings have reasonable intentions WHY they act as they do gamers won’t see ‘em just as foes any more. Maybe there’s a way to make friends with orcs and prevent wars upon diplomacy. In terms of “the pen is mightier than the sword”.

    Reply

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Pro-Human Quiz: