[WIP] Improving the AI

[Neon_Knight]

AI is a vital to most games as a key mechanic for providing challenging and interesting situations and choices for players. Sometimes AI implementations unnecessarily try to address features that don't improve the experience, and often AI implementations fail to make the game as interesting and challenging for the player as they should have. By challenging, I don't mean hard, but rather engaging and though provoking, and not easily exploitable.

In terms of implementation, rather than provide a laundry list of features like path finding, obstacle avoidance, etc., I think it's more important to consider how the player will perceive the AI. The number one requirement for good AI is to never (or at least almost never) look stupid. Players will much more readily notice (and judge AI based on) recurrent examples of brain dead behavior, rather than occurrences of brilliant tactics. Certainly, players appreciate smart, unexpected, and emergent behavior, but only if it's reasonably obvious what's happening. To this end, it's vital that your NPCs provide a lot of feedback about what they are thinking and doing through speech and animation.

The AI needs to understand the rules, strategy, and tactics of the game as well as a player of similar skill level would. It needs to have a knowledge model about the game world and game state that is similar in limitations to what a player would know. For FPS games, it's also very important for the AI to have a human-like aiming model, with the same kinds of strengths and weaknesses in hitting targets under various conditions that would affect a human player.

[The challenge is] successfully implementing all the features and capabilities I've described. One important factor is prioritizing AI sufficiently, and early enough in the project, in the face of many other issues and features that are more immediate needs during the development of the game. It's also important to consider the AI implementation for the design of any game features with which the AI will interact. Many games seem to have AI whose development was rushed or tacked on late in development.

Some areas of the bot AI in OA needed to be polished and perfected. I've been working on several areas of the code for quite some time, but there's still a lot of work to be done. These two days that have passed, I've been working almost exclusively on the AI. Here's a test pk3, up to commit d0cc77c.

Things that need to be tested:

• Domination (general behavior)
• Possession (general behavior)
• Rocket jumping behavior
• Grappling hook behavior
• Just in case there are problems, 1FCTF behavior

To do:

• The problem reported by Sub-Zero in Discord, regarding the bots not using MG in long distances. I think I've found the culprit, but I need to be sure.

[andrewj]

One thing I've noticed is how the bots don't like to simply jump down to a lower level, they always walk to the edge, look down and slowly walk off the edge.  It is very unhuman like, and makes the vulnerable to attack while doing this.

The other main thing that bugs me is that the bots will engage you in battle while their health is high, but get a rocket hit on them (or whatever) and suddenly they are running away to get some health.  I get the logic behind it, but I think humans are more likely to try to finish off their opponent before going to heal up.

[fromhell]

That last bit corresponds to a bot's defined self preservation value IIRC.


My curiosity about the bot code is about that interbreed/script generating stuff that was gutted out of the bot library altogether.  There's probably some of that in gladiator but i didn't check that either.  Unfortunately any questions I might have will never be answered as Mr. Elusive passed away some years ago.


If I go to edit the AI though it'd be for 'cosmetic' reasons, like giving more personality potential (i.e. specific character reactions, love/hate relationships) to flesh out the cast more...

[Gig]

First bit of feedback.
Scenery:
- I tried on OACMPDM10, that I know has the aas compiled with grapplereach option.
- I set bot_grapple 1
- I was often using g_grapple to give the grapple to all users (note: after you enable/disable it, the grapple is given/removed to the player on respawn).
- I was using bots at skill 4, because at skill 3 and lower in their personality files ([botname]_c.c) CHARACTERISTIC_GRAPPLE_USER is set to 0.5 instead of 1.0, I suppose meaning they are less interested into using the grapple. Exception being Liz that for lower skill levels has it set to 0.1!
- I often played in TDM mode, ordering them to "follow me", to see if they used the grapple to reach me.

Things I noticed:
- It looks like they do not try anymore to use it if they do not have it. Good.
- It looks like they do not shoot at the same spot over and over again aborting it before starting to be pulled. But I would need more tests, as for the moment I only found one place where they would actually use it to follow me.
- It looks like if I go in places they cannot reach also by standard means, they will no go there anyway not even with the hook (on the roof of the BFG building).
- The only place of that map for now I found they use the grapple to follow me is inside the warehouse, going from the lower floor to the stairs near to the crates. Sometimes they succeed, but unfortunately more often they bump against the stairs and fall down, then try again (and they may fail again or succeed). I should mention that those specific stairs give some issue to bots in general, at times they jump down from the stairs instead of reaching the Dom point a few units in front of them. So it would be good to find some more places where to try the grapple.
- I'm not sure if bots do have any interest into picking grappling hook up, even in case they don't have it already. I brought five bots following me very near to the grapple spawn point, then ordered them to "roam" and they scattered without anybody of them taking the hook.
- I had to do the tests in TDM mode because I noticed that in Domination and Double Domination (maybe also other modes, but I haven't tried others today) bots do not reply correctly to some team orders. They do "report" correctly, and accept correctly the "dominate A/B" commands in DD, but they do ignore completely the "follow me" and "camp here" commands, they do not even reply to the chat in Dom, or reply with "I'll dominate point A/B" to a "follow me" order in DD! (Reminder, bots listen to team orders only using "team chat", not public chat).
- Bots do not seem to move when they have the "flight" item (which sometimes spawns in the warehouse)!

PS: It would be useful, for testing purposes, to at least temporarily have a way to "pimp up" the "will" of bots to actually use the grapple. It looks like they use it very rarely, even if they are at skill 4.... so it's hard to observe their behavior...

[Neon_Knight]

I still have to work on the DD/DOM team orders, they're not properly implemented. (And some special changes have to be done for those modes, not sure how I would handle the menu in DOM but UT2004 had "Attack Point A" and "Attack Point B" instead of "Attack" and "Defend").

[Gig]

I still have to work on the DD/DOM team orders, they're not properly implemented. (And some special changes have to be done for those modes, not sure how I would handle the menu in DOM but UT2004 had "Attack Point A" and "Attack Point B" instead of "Attack" and "Defend").

I'm at the phone right now... IIRC in DD "roam" was interpreted by bots as something like "choose yourself the A/B point you wish to attack", which may fit. Maybe adding a more descriptive alias for the command, such as "attack/do at will"?
Maybe in Dom "roam" may do the same ("choose yourself a dom point to attack")?
In case you also plan to add the ability to give specific orders about which exact Dom point to attack, there is the problem that their number and their names may vary... so one should make the orders menu "dynamic" to read the point names from the map... or should name them "first one, second one, third one..." following the order they are listed in upper right corner, which may nor be optimal.
Maybe one could instead just add "attack" (attack neutral and enemy points) and "defend" (roam between the points of our team, or only around one of them), in addition to "roam" (choose yourself)....

JUST BRAINSTORMING...

[Neon_Knight]

One thing I've noticed is how the bots don't like to simply jump down to a lower level, they always walk to the edge, look down and slowly walk off the edge.  It is very unhuman like, and makes the vulnerable to attack while doing this.

The other main thing that bugs me is that the bots will engage you in battle while their health is high, but get a rocket hit on them (or whatever) and suddenly they are running away to get some health.  I get the logic behind it, but I think humans are more likely to try to finish off their opponent before going to heal up.

I've noticed that behavior too, should be fixable because it does look unnatural.

As for the second paragraph, this piece of code from ai_dmq3.c shows that behavior:

Code:

/*
==================
BotFeelingBad
==================
 */
float BotFeelingBad(bot_state_t *bs) {
	if (bs->weaponnum == WP_GAUNTLET) {
		return 100;
	}
	if (bs->inventory[INVENTORY_HEALTH] < 40) {
		return 100;
	}
	if (bs->weaponnum == WP_MACHINEGUN) {
		return 90;
	}
	if (bs->inventory[INVENTORY_HEALTH] < 60) {
		return 80;
	}
	return 0;
}

The bot returns a "cowardice" value (the higher the value the more likely they will retreat) after certain checks. If they have only the Gauntlet or their health has lowered to 40HP they will be more prone to running away than if they have Gauntlet+Machinegun or their health is below 60HP. I was thinking that armor should be taken into consideration.

This "cowardice" value is then used in other two functions: BotGetLongTermGoal (for a check in Overload mode where the bot doesn't want to attack the enemy obelisk anymore) and BotWantsToRetreat (self-explanatory). In both cases, if the "cowardice" value is over 50, they will retreat.

[Neon_Knight]

So... I've separated each botfix into a separate branch. It's going to be better this way since we can focus on . Here are the fixes per branch:

- Bot chats: https://github.com/OpenArena/gamecode/pull/65
- Rocketjumping: https://github.com/OpenArena/gamecode/pull/66
- Grappling hook: https://github.com/OpenArena/gamecode/pull/67
- Domination: https://github.com/OpenArena/gamecode/pull/68
- Possession: https://github.com/OpenArena/gamecode/pull/69

[Neon_Knight]

One thing I've noticed is how the bots don't like to simply jump down to a lower level, they always walk to the edge, look down and slowly walk off the edge.  It is very unhuman like, and makes the vulnerable to attack while doing this.

Apparently this behavior is in the engine side of things. And I'm yet not confident enough to go to that area.

[andrewj]

One thing I've noticed is how the bots don't like to simply jump down to a lower level, they always walk to the edge, look down and slowly walk off the edge.  It is very unhuman like, and makes the vulnerable to attack while doing this.

Apparently this behavior is in the engine side of things. And I'm yet not confident enough to go to that area.

Perhaps the bspc tool could change TRAVEL_WALKOFFLEDGE into TRAVEL_JUMP? I will experiment with it today....

UPDATE: tried it just now, on a large drop they did jump but not on a small drop (128 units), however they still slowed down and looked down before jumping, so that behavior must be hard-coded in the engine.

[Gig]

About "taking some care before jumping down isn't human like"... well it depends from what you consider human! In real life, with no respawn, I would pay attention! ^_^
Sorry, I can't parkour!

  

I don't want to troll, just to have a smile!  

[Gig]

One thing I forgot to say yesterday: while spectating a bot which tried repeatedly to perform that hook use in the warehouse (it tried, bumped against the walls, falled down so got some damage and tried again, and again and again), I noticed that it continued trying up to dying by "cratering" at the end. It even had the portable medkit, but it didn't use it and just consumed all of his health a little at time.

I don't remember which bot it was, maybe sarge, grism or assassin...

[andrewj]

About "taking some care before jumping down isn't human like"... well it depends from what you consider human! In real life, with no respawn, I would pay attention! ^_^

Heh.

I just experimented with changing the engine code -- the function is BotTravel_WalkOffLedge() in the file ai_move.c -- and found I could improve behavior somewhat, though needed extra code to ensure the speed did not get too low, otherwise a bot could get to the edge and simply stop because speed became zero.

But I actually think this function should not be changed, not much anyway.  What probably needs to happen is make the bspc tool produce better end coordinates for TRAVEL_WALKOFFLEDGE, since at the moment they are typically very vertical (X and Y coordinates of start and end are only a few units apart) so the bot is doing a very vertical fall.  If the end coordinate was out from the wall by 32 or 64 units, then I think they could fall off without slowing down.

[Neon_Knight]

There's a fork created by Id's own Timothy Besset. And Sago also forked it.

[sago007]

i would prefer calling it "Improving the AI". "Fixing" suggest that the AI is broken and I don't see any major bugs. Most of the things are improvements and some seems to suffer from regressions.

Domination and Possession could really need some improvements though.

[andrewj]

There's a fork created by Id's own Timothy Besset. And Sago also forked it.

I am working on my own fork too, but it is not ready to go public yet.

[Neon_Knight]

Cleaned up ai_team.c

This should allows us to take a better look at how the bots are organized in team matches.

[Neon_Knight]

After so many attempts, I guess I have the AI for Double Domination and Domination tackled. Bots now respond to orders and have initial formations, though I had to include some files from the /botfiles/ folder, since they have the required code for some of the messages to be interpreted.

Here's a test pk3 as well.

What needs to be tested in this case?

• Botmatches in Double Domination and Domination.
• How bots react to orders.
• If the bots do indeed go to pick the DOM points.
• Anything else that I'm missing.

Additional notes

• Currently the bots have the following formations for the following cases in DD:
  • Both/No points taken: each bot will pick one (50% A/50% if only one teammate, it will pick one at random) and hold onto it. Odd numbers will have one bot roaming the map.
  • Point A taken: 60% of the teammates will take the B point while 40% will instead hold onto the A point.
  • Point B taken: 60% of the teammates will take the A point while 40% will instead hold onto the B point.
  • Domination: All bots are set to pick a random point and hold onto it.
• As for their behavior, for Domination, the bots will pick a random point (if it falls inside of both established map and gametype limits -WIP-) and act upon it. If the point is already taken, they can choose to hold onto that point, or go chase after another empty/enemy-controlled point. If all the points are taken, the bot will pick one at random and hold onto it.
• Double Domination has a similar logic to Domination, however if the bots already have a clear order about which particular point to hold on, they will stay on it.
• The generic "Defend/Attack the Base" orders have different behaviors in both gamemodes. In DD, "Defend the Base" will make the bots to go take and hold CP while "Attack the Base" will make the bots to go take and hold CP A. In DOM, both orders do the same thing: tell the bot to pick a random point and take/hold it. Each time any of those functions are called in DOM, however, the bot will pick a random point.
• In both gamemodes, absence of control points will make the bots to roam through the map. Also behavior in both modes now account for the presence of a team leader, last goal, aggression and preference.

To-Do

• Bots take a lot of time to reach the points and often get distracted in fights. They should go straight to the CPs, especially if they're empty.
  
• A feature I'd like to add, yet I haven't thought about how to put it in the code: if an info_camp entity is linked to a DOM point or a flag, the bot should use it as a defense point. Obviously the "angles" key should be pointing to the direction the bot should cover while defending.
  
• While checking ai_team.c, I figured out that bots have different team formations for aggressive and passive modes. I'd like to implement those as well in these modes.
  

[Neon_Knight]

Another separate AI improvement branch has started, this one focuses on the handling of both Personal Teleporter and Grappling Hook.

Here's a test pk3.

What needs to be tested in this case?

• AI behavior when using the Personal Teleporter
• AI behavior when using the Grappling Hook

Additional notes

The current implementation (of this PK3) has the following set of checks (some more may be needed):

• If a bot has both the Grapple and the PT, the GH checks run before the PT. The GH's first check checks if the bot has a PT. (Should this change in the future?)
  
• Bots will instead use the Grappling Hook if (in order): they don't have the Personal Teleporter, the general grappling setting for them is off, they have the GH in their inventory, they aren't in a lava or slime pit (they will ALWAYS use it in that case), and their personal bot settings allow for it.
  
• Bots will use the Personal Teleporter if (in order): they have the PT in their inventory, they aren't in a lava or slime pit (they will ALWAYS use it in that case), they're holding a key game objective (team/neutral flags, Harvester skulls...) and their personal bot settings tell if they're self-preserving.
  

To-Do

• Should the PT checks take priority over the GH checks?
  
• Account for death pits in both cases. The current checks for death pits check if the map triggers the sound "*falling1.wav", which excludes maps with death pits that don't trigger this sound. This isn't really useful and needs to be changed. They should ALWAYS use the teleporter when falling to the void/death pit.