Pages: [1] 2
  Print  
Author Topic: OpenArena Server Crash 0.8.5 on Debian  (Read 41779 times)
glabasnat
Nub


Cakes 0
Posts: 9


« on: September 02, 2010, 11:40:27 AM »

Hi,
My server OpenArena cresh randomly

My OS is Debian Lenny 5.0.5
Only OpenArena is from unstable version
openarena-data (0.8.5-3)
openarena-server (0.8.5-4 and others)

Older version 0.8.1 was okay

Log and config:
http://game1.xline.cz/oa/log/

Tomas
Logged
Gig
In the year 3000
***

Cakes 45
Posts: 4394


WWW
« Reply #1 on: September 02, 2010, 11:53:14 AM »

That crashlog.txt file reports 403 Forbidden, I can't read it.
Logged

I never want to be aggressive, offensive or ironic with my posts. If you find something offending in my posts, read them again searching for a different mood there. If you still see something bad with them, please ask me infos. I can be wrong at times, but I never want to upset anyone.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #2 on: September 02, 2010, 11:58:00 AM »

Sorry, is it log (crashlog.txt) OK
« Last Edit: September 02, 2010, 01:18:09 PM by glabasnat » Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #3 on: September 03, 2010, 04:28:42 AM »

Do these crash appear often ?

Seems unlikely, but do you add bots once the server is started ?
Logged

Todo: Walk the cat.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #4 on: September 03, 2010, 04:40:47 AM »

Yes,
If play boots or player, server crash in 30minutes - 2 hours
Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #5 on: September 03, 2010, 05:43:49 AM »

Maybe there is no bot support in one of the maps you play.
If the crash always happen on the same maps, that would explain.
Logged

Todo: Walk the cat.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #6 on: September 03, 2010, 08:32:58 AM »

Server crashes independently of the map Sad

Could be a old library of Debin Lenny? (That would be a problem)
Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #7 on: September 03, 2010, 03:41:41 PM »

If it happens often, you may try to install the openarena.ws release (not the debian one) and see if it solves anything.
I run Debian too on most of my servers, your problem is not likely due to a library, more likely due to some exotic setup (if I learned my lessons well, your crash comes from the virtual machine of the game, not from the engine/system itself)
Do you use some custom pk3s in your server ? (like mods, maps/ressources...)
Logged

Todo: Walk the cat.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #8 on: September 04, 2010, 06:57:58 AM »

This is default (clean) install from Debian.
Logged
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #9 on: September 04, 2010, 09:13:51 AM »

Test:
Debian Leny and OpenArena from unstable/testing repository - Server crash

Debian Testing and OpenArena from unstable/testing repository - Server OK crash

Where is error?
« Last Edit: September 04, 2010, 10:17:18 AM by glabasnat » Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #10 on: September 04, 2010, 06:12:08 PM »

And if you try with this+that ?

http://openarena.ws/download.php?view.2
http://openarena.ws/download.php?view.3
Logged

Todo: Walk the cat.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #11 on: September 24, 2010, 12:38:13 PM »

Server is OK on original file from http://openarena.ws/
Where is bug? (I prefer a package of Debian)
Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #12 on: September 25, 2010, 04:16:23 PM »

Hard to know the reason of the bug.
Check last error messages every time, try to reproduce the crash (try to know its conditions : specific maps, specific settings with bots, specific game modes...).
It's not even sure official OA release runs better than the Debian one, maybe you just got luck with the first.

Logged

Todo: Walk the cat.
sago007
Posts a lot
*

Cakes 62
Posts: 1664


Open Arena Developer


WWW
« Reply #13 on: September 25, 2010, 05:12:08 PM »

Debian has started to use compiled so files instead of qvms. The so-files was only meant for debugging and greatly increases risks of a crash.
Logged

There are nothing offending in my posts.
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #14 on: September 26, 2010, 03:15:24 AM »

Crash log often ends:
Kill: 2 2 26: Player_name killed Player_name by MOD_KAMIKAZE
Logged
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #15 on: October 11, 2010, 05:33:25 AM »

Thanks for reporting this. My guess would be that it's a NULL pointer dereference somewhere in qagame, possibly related to the kamikaze - NULL pointer dereferences will crash the native-code version, but would just quietly do the wrong thing in the bytecode interpreter, as far as I can see.

If you can rebuild the Debian version of openarena with debug symbols (DEB_BUILD_OPTIONS="noopt nostrip") and get a backtrace from a crash with gdb, that would be very useful. I realise those instructions are rather too short to be useful, I'll try to provide more detailed instructions later.

I assume you're using some sort of wrapper script to run the server repeatedly and capture crashlog.txt whenever it dies? If so, please send me a copy, and hopefully I can modify it to pick up backtraces from gdb automatically.

I had to remove the bytecode from the PK3 files in Debian for licensing reasons, but the engine still has the bytecode interpreter/JIT, so if you replace the PK3 files installed by openarena-data with the ones available from here, the Debian engine build is still able to run the bytecode versions of the game logic (if both bytecode and native-code are available, it'll normally prefer to run bytecode, unless configured specially). It'd be very useful to get a crash backtrace, though - any crash in the native-code build indicates that there's a bug somewhere.
Logged
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #16 on: October 11, 2010, 02:57:55 PM »

Here's a version built with debug symbols, with a script added to run it under gdb. Install the appropriate one of:

http://people.debian.org/~smcv/openarena-server_0.8.5-5~debug_i386.deb (32-bit)
http://people.debian.org/~smcv/openarena-server_0.8.5-5~debug_amd64.deb (64-bit)

From your crash log it looks as though you'll want the i386 version.

Instead of running openarena-server, if you run "/usr/share/games/openarena/openarena-server-gdb", it should add a backtrace to the crash log when it crashes. Please do that, try to crash it (adding a lot of players/bots might help to make it crash quicker), and send me the log.

To check that the crash logging is working, you can simulate a crash with:

pkill -ABRT openarena-server

Note that that command will "crash" every OpenArena server on your machine, if you're running more than one!

The log from simulating a crash like that won't be any help to me, but you can use it to check that the crash logging is actually working: if it is, you'll see a lot of gdb output at the end of the crash log, starting with something like this.

Program received signal SIGABRT, Aborted.
__strncpy_ssse3 () at ../sysdeps/x86_64/multiarch/strcpy.S:1107
1107   ../sysdeps/x86_64/multiarch/strcpy.S: No such file or directory.
   in ../sysdeps/x86_64/multiarch/strcpy.S
Logged
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #17 on: October 11, 2010, 05:07:42 PM »

While trying this out with many bots, I got one server crash caused by a missing NULL check: client was NULL in the backtrace I saw. I can't see how it would happen in practice (both players and bots have a client structure) and couldn't reproduce it, so this may or may not the same bug glabasnat saw, but the change would be good to have regardless.

I'm doing an updated debug build with this fixed. When it finishes compiling and uploading, it'll be at:

http://people.debian.org/~smcv/openarena-server_0.8.5-5~debug2_i386.deb

Code:
commit 80f2967f82314bf10e975d1b00678b9b4327eae9
Author: Simon McVittie <smcv@debian.org>
Date:   2010-10-11 23:28:51 +0100

    G_Damage: check before dereferencing targ->client, which may be NULL

diff --git a/game/code/game/g_combat.c b/game/code/game/g_combat.c
index 22ce65a..7d1fba7 100644
--- a/game/code/game/g_combat.c
+++ b/game/code/game/g_combat.c
@@ -1043,7 +1043,7 @@ void G_Damage( gentity_t *targ, gentity_t *inflictor, gentity_t *attacker,
         
         //Sago: See if the client was sent flying
         //Check if damage is by somebody who is not a player!
-        if( (!attacker || attacker->s.eType != ET_PLAYER) && client->lastSentFlying>-1 && ( mod==MOD_FALLING || mod==MOD_LAVA || mod==MOD_SLIME || mod==MOD_TRIGGER_HURT || mod==MOD_SUICIDE) )  {
+        if( (!attacker || attacker->s.eType != ET_PLAYER) && client && client->lastSentFlying>-1 && ( mod==MOD_FALLING || mod==MOD_LAVA || mod==MOD_SLIME || mod==MOD_TRIGGER_HURT || mod==MOD_SUICIDE) )  {
             if( client->lastSentFlyingTime+5000<level.time) {
                 client->lastSentFlying = -1; //More than 5 seconds, not a kill!
Logged
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #18 on: October 11, 2010, 05:18:20 PM »

Now uploaded. However, openarena-server-gdb turns out not to be quite right: you'll need to edit it and change where it says "/usr/games/openarena" to be "/usr/games/openarena-server". I'll fix that in the next build.
Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #19 on: October 12, 2010, 03:48:36 AM »

Heh, your efforts are quite appreciated Smiley
I had something in mind, which is: try to find configurations which make the server crash (simple). Right now you may be aware if a map doesn't have some bot script, and the server tries to add bot, then it crashes (or at least it complains a lot. There are also a few issues about "spawnpoints" which if they do not exist in certain gametypes (starting a deathmatch on a CTF map) will crash the server. They're not related to the engine but to the QVM, I didn't take that initiative from now because I wasn't sure what developers were able to do against it, to fix it, nor if it was worth, I'm also ready to start back using the debian package if I know there is care for it and if it's maintained. Goal would be to get better stability for OpenArena servers.
Logged

Todo: Walk the cat.
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #20 on: October 12, 2010, 05:25:43 AM »

They're not related to the engine but to the QVM, I didn't take that initiative from now because I wasn't sure what developers were able to do against it, to fix it

If you use native code for the game logic that'd normally be bytecode (the native-code version is qagamesomething.so or qagamesomething.dll, the bytecode version is qagame.qvm in the PK3 files), you're more likely to see hard crashes rather than things quietly going wrong... but if you can get a backtrace for those crashes, it's also easier to fix them :-)

We can't use the bytecode version in Debian/Ubuntu because there's no suitably licensed compiler for it, so we get more crashes, but more debuggability, as a side-effect. Hopefully that'll mean I send in a series of one-line patches and things get more reliable for everyone!

I've left an openarena-server running on a spare machine with 16 bots fragging each other at an accelerated g_speed, I'll see whether any of them manage to trigger another crash...
Logged
glabasnat
Nub


Cakes 0
Posts: 9


« Reply #21 on: October 13, 2010, 02:32:38 PM »

Good work,
I tested:
openarena-server_0.8.5-5~debug2_i386.deb
openarena-data (0.8.5-3)

Server is online 24 hours Smiley

Tomas
Logged
smcv
Nub


Cakes 5
Posts: 23


ioquake3/OA/Q3 Debian maintainer


« Reply #22 on: October 13, 2010, 02:50:18 PM »

I can't see how it would happen in practice

It turns out you can trigger this by setting off the Kamikaze close to a non-player object (e.g. anything you can pick up), so I'm fairly confident that this is what caused glabasnat's crashes.

sago007 applied it in svn as r239, and it'll be fixed in Debian as version 0.8.5-5. Hopefully I can get a freeze exception to get this into Debian 6.
Logged
Cacatoes
Banned for leasing own account
Posts a lot
*

Cakes 73
Posts: 1427


also banned for baiting another to violate rules


« Reply #23 on: October 25, 2010, 02:03:47 PM »

I gave this debug package a try but I didn't find as many problems as I was expecting.
I might install it on other played servers, but it happens my servers are quite stable these days.
Only crash I was able to reproduce was because of missing spawn points.

Take this map:
http://download.tuxfamily.org/openarena/autodownload/baseoa/schism-b2.pk3

Start this map with g_gametype 4 (CTF), and join either blue or red team (or ask a bot to do so). It will crash with:

Code:
ClientUserinfoChanged: 0 n\^7Caca^2t^7oes\t\1\model\arachna/red\hmodel\arachna/red\g_redteam\\g_blueteam\\c1\7\c2\5\hc\100\w\0\l\0\tt\0\tl\1\id\144F10ADA851FC79008571529BE91EE4
Playerstore: Nothing to restore. Guid: 144F10ADA851FC79008571529BE91EE4
********************
ERROR: Couldn't find a spawn point
********************
----- Server Shutdown (Server crashed: Couldn't find a spawn point) -----
Sending heartbeat to dpmaster.deathmask.net
Sending heartbeat to dpmaster.deathmask.net
==== ShutdownGame ====
ShutdownGame:
------------------------------------------------------------
recursive error after: Couldn't find a spawn point
openarena-server: code/sys/sys_main.c:149: Sys_Exit: Assertion `ex == 0' failed.
Received signal 6, exiting...
----- Server Shutdown (Signal caught) -----
Sending heartbeat to dpmaster.deathmask.net
Sending heartbeat to dpmaster.deathmask.net
==== ShutdownGame ====
ShutdownGame:
]DOUBLE SIGNAL FAULT: Received signal 11, exiting...

This is only an issue if maps are not properly made, which shouldn't happen with standard OA maps. When used on a server in a regular way (in deathmatch mode), a simple callvote to change gametype would then crash the server.

This bug has been mentionned several times on the forum but is not listed on the DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Bugs]Wiki as of now.

I also have a question about the OpenArena Debian package: would it be useful to have some default configuration for servers, and /etc/init.d scripts ? That would be another plus to encourage the use of Debian package over oa.ws ones when installing them on servers.
Logged

Todo: Walk the cat.
sago007
Posts a lot
*

Cakes 62
Posts: 1664


Open Arena Developer


WWW
« Reply #24 on: November 01, 2010, 12:03:42 PM »

This is only an issue if maps are not properly made, which shouldn't happen with standard OA maps. When used on a server in a regular way (in deathmatch mode), a simple callvote to change gametype would then crash the server.
The problem is that the game cannot do anything about the problem so it tries to shutdown the game in order to let the script that started the server do its magic. Or in case of a listening server it can drop back to the menu to let the player pick another map.

There are not many options: The map has no spawnpoints, so running the game logic is not an option. nextmap might not be set, so the game cannot go to the next map. Restarting the map has no effect as the problem will still be there. The default config cannot be loaded directly because it is only known to the startup script and the problem might have been caused by it anyway.
Logged

There are nothing offending in my posts.
Pages: [1] 2
  Print  
 
Jump to: