OpenArena Message Boards

pyqscore is a python script that reads OA logs and outputs an HTML file with numbers.


(http://img96.imageshack.us/img96/3628/screenffa.png)

(http://img585.imageshack.us/img585/6800/screenctf.png)


It has options to change how entries are sorted, eliminate players from output, set maximum number of entries and a couple of them more. More information in the README file. I expect it to contain some bugs that passed unnoticed given the limited number of log files I have. Download it here:

http://www.4shared.com/file/LvpOcmg4/pyqscore.html

I just linked it here:
(DO NOT LINK) h t t p s : / / openarena . wikia . com/wiki/Servers#Server_managers

If you will release a new version, please update the link there, too.  :)

PS: In your example screenshots, why does your name appear more times, even if written in the same way? Were there different colors, that are not shown in the table? But I see that some colors appear there...

Hi, thanks!

Good question about the names, I wondered that myself the first time I saw it. The example screenshot at the top was taken by processing my 'games.log', i.e. the log OpenArena writes when you play against bots. Unless I deleted something at some point, that log contains everything I did since I started playing, and so there are some variations of my nick there. Those two that look identical actually are not; one of them is "^4kernel panic", whereas the other is "^4 kernel panic" (notice the space). It doesn't bother me particularly, but it all has to do with how I parse the coloured codes, which could be improved a lot...

Maybe you could add a column (even at the end of the table) that shows the full "source code" of the names...

Maybe you could use the GUID to identify the player.
I suppose you're aware Sago made some stats tool, and aftershock did one too.

The thing is that, tecnically, pyqscore is doing what it should do. After all, both nicks are different. The problem, of course, is that they are different only because at some point many months ago I accidentally hit a space when entering my nick. I don't think it's a real issue, and probably it doesn't deserve a special case. A different situation would be if pyqscore were actually used by somebody and this turned out to be a problem.

What would need some attention is the case of repeated nicks. As all my logs are from 0.8.1 I wasn't aware of GUIDs--apparently this is logged since 0.8.5. This would be a partial solution, but not definite, because identical nicks would still show up in the final data as identical nicks. When writing this tool I decided that I wouldn't complicate matters trying to handle unsolvable problems (unsolvable this far down the ladder). Unfortunately, idiots impersonating other players are one of these, and server admins are the ones that should deal with them.

But thanks for the suggestion, I'd probably rely on GUIDs for something if I knew they existed. Then again, there are like 2 persons using pyqscore (me and my mate), and none of us are parsing logs of public servers filled with a lot of players, so...

As for sago's tool and AfterShock, yes, I was aware of them. I don't like the approach of oastat (I think that's the name), it reminds me of vsp, and I'm not even sure there is a frontend for it. AfterShock's solution--coding the whole thing in the mod--is the only right one. The rest, including pyqscore, are broken  : )

Using version 0.9 (19-04-2011) and Python 2.6.6 I get a couple of errors, depending on the log file I try to parse:



I actually believe that parsing the logs are the right way to do it. That way the games only responsibility it to write the all events to the log. The game does not do that at the moment but that is because the logging is flawed not the idea. OpenArena have added some extra logs (like CTF:) but even more is needed.

My needs as a user might be very different. For my own personal logs I would prefer a program like pyqscore to quickly parse my personal logs while on my server I would prefer to get the stats into a database, so I could make more advanced queries. 

Hi, sago

I was expecting some stuff to break when people tried some logs. Does this happen with all the logs you tried? Would you mind sending me these problematic logs somehow? I've introduced several changes in the script but I haven't uploaded anything yet. I would like to have a look.

Regarding the "right" way to deal with this, it's true that I'm biased against log parsing because of the flaws present in idtech3's implementation. Still, I don't see the advantage of doing it outside the game. Actually, my personal needs seem pretty similar to yours. I only wrote this to have some more or less nice looking stats for my games in some private server. A database solution is overkill for this problem. But even if I were running a public server with lots of players joining every month, I don't think I'd need more complex information. Sure, you can come up with pretty exotic queries, but those aren't really useful, are they? The most obvious thing pyqscore can't do at the moment that would be remotely useful is per map and item stats (the basis for the latter are actually in place). Other than that, I can't think of anything I would like to know out of my games...

Nice work KP.



Yes, very true.



But you posted the program for others to use so it would be nice to include GUID support.  Anyway, do continue to improve it, more options are always ideal for Server Admins.



Shameless plug:  Our solution in E+ which is compatible with a variety of games and mods - http://vsp.goquake.com/

Thanks!


The thing is that I don't see what fundamental problem GUIDs solve. I can only think of
two use cases (but I probably lack imagination and real world experience in dealing with
large server logs): a) identical nicks; b) gather statistics _per player_, as opposed to
_per nick_ (like your vspxp does I think). The first problem is not fully solved, and in
any case I don't think log massaging is the right approach to solve it, but I appreciate
that you may prefer trying than doing nothing. The second is not a problem per se, but a
question of how you want to deal with player data. Personally, I don't mind players changing
their nicks and counting their statistics separately (actually, I look at it as almost a
privacy issue).


Very nice, I din't know about this one. I tried vsp some time ago and found that it didn't
suit my needs. Something like pyqscore (or qscore) is much more manageable for private
servers (or for regular users wishing to parse their games.log files and check how bad they
fare against nightmare bots :-) But apart from its complexity, which renders it unusable for
all but the most determined players, what I didn't like of vsp was the closed license and
the ridiculous decision of its developer to obfuscate the code. I wonder whether this didn't
give you any headaches when you created vspxp.

hi kp,
nice job !

i'm using your script, i'll tweak-it soon to fit my site, ty for your effort.

there's a point i'll talk to you about, regarding the programming.
running your script on my server's log and having no player that used handicap, i've got an exception (line 297)

so i think it will be a good ideea to handle the exception with a try/except in a future release

thanks again

You are very kind, and you are very welcome!

Not that you need me to say it, but do introduce as many changes as you want, hopefully it will suit your needs. Your post forced me to do some quick stuff that was needed to release a new version containing changes I'd done weeks ago. So if you improve the script (and decide to release it), it would be best for both of us if you worked on the new version. It contains some bug fixes (I believe Sago's problems should be gone), some tiny improvements, and some minor changes not noticeable by the user. A list of changes is now included within the zip file.

You are absolutely right about the handicap issue. Since all my logs contain a handicap value, I did not realize this did not apply universally. I invite you to make the change yourself (I can't test this with my logs) and let me know if you obtain the expected behavior.

You can get the latest version here:

http://www.4shared.com/file/HKCDYEKJ/pyqscore_091.html

Cheers

Hi

First of all i wanted to thank Kernel Panic for making pyqscore. I'm using your script on a daily basis at http://linuxiuvat.de/stats/oa/daily.htm (http://linuxiuvat.de/stats/oa/daily.htm). It's a very useful and lightwight tool to analyse OpenArena logs and for creating a simple static statistics page. It definitely wins out over these PHP+MySQL solutions.

I've discovered some minor issues with your script which doesn't check very carefully about some special characters. I also use Tidy to clean up some minor html errors.

But one thing people should change in your script are these two lines at line 544 in your name_colour function.

limit = len(nick) -1
if nick.rfind('^') > -1 and nick.rfind('^') < limit:

Otherwise pyqscore would exit with an out of bound error if the ^ symbol is the last character in the player name. Besides from that it's perfect.  ;D

Hey, hi, Apo.

I'm very happy to see somebody using the script in a public server--and with some customisations, nice one!

Good catch about the nicks. I didn't think that through very well, did I?

By the way, do you think the server frag count is being reported correctly?

Cheers

Hello

sorry for the late reply. I didn't check the "notify me" box. Hopefully i will be notified from now on.

Well, there are some minor issues but in general pyqscore just works and it is very useful. 

I think the player names are a special case. It would be ideal if every player name could be stripped of special characters like <>. For example one player called himself <html5> thus one table cell was rendered incorrectly. At the moment i solve this issue by adding the name to the ignore list.

Yes, i think the server frag count is not being reported correctly because four bots play on the server. I didn't want pyqscore to count those frags so i put all bot names on the ignore list. I guess that's the reason why the frag count differs.

Cheers

Downloadlink renewal? It says the publisher can still access it.

If it's under a megabyte or 5, I can host it for lifetime if you want. Also if it's larger, but then the download takes forever (my speed goes up in a few months when we get fiber though).

You can download my slightly modified version of pyqscore here.

http://dl.linuxiuvat.de/oa/pyqscore.tar.gz

Feel free to mirror it or to use it under the GPL-2 license.

The result looks like this:

http://linuxiuvat.de/stats/oa/daily.htm

is some stats program for oa friendly for windows ?

Hi!
I've just updated the download link in the wiki (DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/Servers#Server_statistics]here (http://([b)).

Maybe the next time, update version infos, too.  :)

Thanks Gig for updating the links!

I think a version bump wasn't necessary and the changes were relatively small. Besides I'm not the original author. However I can recommend pyqscore to every OpenArena server admin who would like to keep it simple. Parsing the log files with pyqscore instead of installing a full blown php and mysql server is sufficient in my opinion.

@capodecima

You can install Python for Windows and run the script on your pc. Pyqscore is indeed very simple to use.

so bad there isnt any readme so i dont know how the program use :/

Gig,

I have to say the same :( I don't know how to use this program either. I have python fully installed, is there something else needed?

Sorry, I don't know what to tell you about usage of the program.

Maybe you may try to send PMs to Kernel Panic (http://openarena.ws/board/index.php?action=profile;u=999) and/or Apo (http://openarena.ws/board/index.php?action=profile;u=7673), asking them to reply here.

Thanks, I messaged Apo, I dont know how active he is with his inbox on here. I know Panic probably doesn't check about this most likely.

Well, also if Panic does not check the forum directly, and his email address is hidden, maybe he may get the "email notification" when he receives a PM...

Very nice!

Has anyone gotten in touch with Panic yet? I still haven't spoken to him in quite a long time.. I'm pretty sure I can ask him about this if I ever speak to him in IRC again. I hope it's soon.. I really want to test the program out on my machine.

Sometimes he connects to these forums, though it's been a long time since he posted AFAIK.

Thanks for telling. I hope we can get an update on this quick. I know he works alot though.. but maybe he will read the forum one day and see.

You may sending him a PM. This may trigger a notification email to him.

Sorry for my late reply. I changed my e-mail address but forgot to change it here too.

Just run


On Linux python is most likely already in your $PATH and you could install pyqscore.py to e.g. /usr/local/bin , so running


should be sufficient.

This command will parse your OpenArena log and write the output to daily.htm in the same directory which you can then view in your web browser of choice. If you don't like the colors you have to customize pyqscore_style.css.

I should receive e-mail notifications from now on. But if I'm too slow with replies you can also contact me directly. The information are available on this site.

http://linuxiuvat.de/contact.htm

Hi.

I don't really play or follow the forum anymore. I stumbled upon it the other day and saw this. I made some modifications to pyqscore to hopefully make it straigtforward to use. I also included the suggestions made in this thread by Akts^Itachi and Apo as well as some modification from Apo's file, and fixed some stuff.

Now one should be able to double-click on pyqscore.py and see a file dialog. Use it to open the games.log file which is found in a location similar to:

   C:\Documents and settings\username\Application data\OpenArena\baseoa\games.log

or:

   C:\Users\username\Appdata\Roaming\OpenArena\baseoa\games.log

   (~/.openarena/baseoa/games.log in Linux, of course)

That should be it. A browser window should display the generated HTML file...The script can still be run from the command line, and the Tkinter window deactivated in the options.

Download it from here:

http://sourceforge.net/projects/pyqscore/

If someone can update the wiki links it would be cool. Also, I noticed that the file provided by Apo does not include the icons, so it won't really work properly (at any rate, thanks for providing it).

@Apo: should you want to use this version (I recommend it, it fixes a few things), you'll notice you should do some edits again to accomodate your server's directory structure and probably CSS stuff, and take a look at the options again to get the same behavior : )

Cheers

kp

DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/index.php?title=Servers&diff=16545&oldid=16404]Done (http://([b). However, the wiki does not require registration (although registering is welcome!), so anyone can edit it.

Hi kp,

thanks for updating your script! Nevertheless I'm quite happy with my current version and it just works. So better don't touch. :)

Hi, Apo

That's wise. Although remember, if it ain't broken, tweak it!

Cheers

Hi ive downloaded the pyqscore code  but having issuses with an error in abit of the code & dont really understand whats its trying to do

def mainProcessing(log):
    '''Main processing function'''
    server = Server()
    cgames = []              # Cumulative list of games: instances of Game()
    N = 1                    # Game number
    lines = (line for line in log.values())
    for line in lines:
        if line.find(' InitGame: ') > 0 and lines.next().find(' Warmup:') == -1:
            # New game started (no warmup). Begin to parse stuff
            game = Game(N)
            N += 1
            game.pos = 1          # Player's score position
            game, server = lineProcInit(line, game, server)
            game, server, valid_game = oneGameProc(lines, game, server)
            if valid_game == True:
                server.time = server.time + game.time - min(game.ptime.values())
                cgames.append(game)               # Append game to list of games
    return server, cgames

The bit that giving me a headahce is " server.time = server.time + game.time - min(game.ptime.values()) " comes up with "ValueError: min() arg is an empty sequence"

& does not make the html file ? hope anyone can help


Chris

Hi, LBS,

Thank you for reporting this. Turns out I never considered the possibility of "empty" games where nothing happened. I uploaded a quick fix to sourgeforge, which is just these two extra lines:


Have fun!

KP