OpenArena Message Boards

OpenArena Contributions => Development => Topic started by: smcv on August 08, 2011, 12:30:29 PM



Title: Engine compatibility with ioquake3 1.37
Post by: smcv on August 08, 2011, 12:30:29 PM
I know ioquake3 1.37 hasn't been released yet, but it looks as though it's probably quite imminent, so now is the time to think about how to make/keep OA compatible with it... I've been uploading recent snapshots to Debian experimental, which might be useful as a reference.

master server (dpmaster) heartbeat

All released versions of OpenArena send the "QuakeArena-1" heartbeat string to the master server. This causes dpmaster to identify it as Quake III Arena (technically: game name "Quake3Arena").

Newer versions of ioquake3 use the "DarkPlaces" heartbeat string, which selects use of the DarkPlaces protocol. This is heavily based on the Q3A protocol, but instead of a hard-coded mapping from "QuakeArena-1" to "Quake3Arena", dpmaster looks for a "gamename" key in the server info, and uses that.

In a generic "any game" ioquake3 engine (like the one I upload to Debian), the gamename is set on the command line via "+set com_gamename Something". In a standalone build with things hard-coded (like the official OpenArena engine binaries), you'd redefine GAMENAME_FOR_MASTER, which just sets the default value of com_gamename (formerly cl_gamename).

When looking for servers, "getservers" guesses a gamename based on the protocol, and "getserversExt" explicitly sends one. This actually means that the engine supplied with OA 0.8.5 will never find any IPv6 servers - it queries for them with "getserversExt openarena ipv6" as a result of GAMENAME_FOR_MASTER having been defined to "openarena", but all current OA servers implicitly advertise themselves as "Quake3Arena", so they won't be found like that.

In Sago007's "updated engine" builds (which are compatible with, and intended to be used with, OpenArena 0.8.1 and 0.8.5), and in my Debian packages (likewise), I think we have no choice but to set com_gamename to "Quake3Arena" - this is the only way to have dpmaster.deathmask.net send us details of existing OpenArena 0.8.1 and 0.8.5 servers.

Next time OpenArena deliberately breaks network-compatibility with older versions (is this planned for 0.8.7, or 0.9.0, or what?), com_gamename should clearly change - it could become "OpenArena0.9" or something.

(I did propose some patches for dpmaster, to treat the combination of QuakeArena-1 + protocol 71 as meaning that gamename was "openarena", to work around the 0.8.1 and 0.8.5 binaries that most people will still be using, but the author of dpmaster rejected that idea.)

New network protocol

ioquake3 now supports up to two network protocols in parallel: the new protocol (currently 71, confusingly enough) and the legacy protocol (68, the same as Q3A 1.32). It always advertises the legacy protocol to the master server, unless configured not to support it at all. When a client actually joins, it will attempt to negotiate use of the new protocol instead.

In the new protocol, the client and server also compare their com_gamename (see above) in order to give a better error message if they're incompatible - in the distant future, joining a Quake III Arena server with OpenArena will give you a message like "that server is Quake3Arena, not OpenArena".

The conservative thing to do is to switch off support for the new protocol altogether, and keep using protocol 68 (re-labelled as 71 in OpenArena-land). You do that by setting the com_protocol cvar to the same value as com_legacyprotocol (in a standalone OA engine, you'd change some #defines to change the default values of those cvars, but the effect is the same). In my Debian packages I currently set both to 71, so they always use the legacy protocol, and are completely interoperable with 0.8.1 and 0.8.5.

Another possibility would be to say something like: "the OpenArena-0.8.5-compatible version of the "new protocol" will always be the engine's real protocol version + 85000". So with current svn snapshots of ioquake3 you'd have com_legacyprotocol = 71 and com_protocol = 85071, but if 1.37 gets released with com_protocol = 75, you'd use com_legacyprotocol = 71 and com_protocol = 85075 for that version of the engine. If you want to go that way, I have a patch to add a "com_protocoloffset" cvar, which would mean I'd set com_protocoloffset = 85000 in the Debian OpenArena package, and always get an appropriate version number for the engine (and I'll try to send that upstream to ioquake3, if it'll be useful).

When OpenArena next breaks compatibility (0.9.0 or whatever), I think the thing to do would be: keep increasing the legacy protocol number (perhaps to 90068), set the new-protocol number back to what the engine would "naturally" call it (so it stays 71 or 75 or whatever), and set com_gamename to "OpenArena0.9" or something as the way to discriminate between OpenArena and non-OpenArena clients.

ioquake3 upstream don't consider it to be useful to offer the legacy protocol in standalone games, but I'd appreciate it if you'd leave it in - that would mean that a client and a server with the "official" OpenArena 0.9.0 engine would use the "new protocol" that was current at the time that engine was released, but an updated client based on a newer version of ioquake3, with an even newer "new protocol", would still be able to join that server by falling back to the legacy protocol. The ioquake3 authors don't plan to support more than one "new protocol" at a time; if I need to, I could patch in support for that in Debian, but I'd really rather not!

(By the way, the legacy protocol can be disabled at runtime, by setting com_legacyprotocol to 0, if you're into that sort of thing.)


Title: Re: Engine compatibility with ioquake3 1.37
Post by: sago007 on August 09, 2011, 10:32:46 AM
I believe that there are quite a long time till network compatibility is broken again, so I have not thought that much about it.

In the beginning I was very interested in a new protocol that did not suffer from historical decisions. However I am not that keen to jump on a protocol that might loose support later.

I find the DarkPlaces heartbeat string very interesting, mainly because it solves the problem that was discussed back in 2007 that ended in the protocol number being boosted in lack of a better alternative.

Once that game is identified by the gamestring I see no reason to change the protocol anymore. Of course filtering of LAN games should be tested but so far I assume that is taken care of. I mean why couldn't it use the gamestring too?


Title: Re: Engine compatibility with ioquake3 1.37
Post by: smcv on August 09, 2011, 11:03:42 AM
In the beginning I was very interested in a new protocol that did not suffer from historical decisions. However I am not that keen to jump on a protocol that might loose support later.

Yeah, I was a bit worried about that too; that's why I suggested leaving in support for the legacy protocol, as a lowest-common-denominator way for any two compatible versions to interoperate.

Quote
I find the DarkPlaces heartbeat string very interesting, mainly because it solves the problem that was discussed back in 2007 that ended in the protocol number being boosted in lack of a better alternative.

Yes, but only after you break compatibility by changing your com_gamename...

(... which is why it might be a good idea to break compat somewhat sooner than you'd expected.)

Quote
Once that game is identified by the gamestring I see no reason to change the protocol anymore. Of course filtering of LAN games should be tested but so far I assume that is taken care of. I mean why couldn't it use the gamestring too?

Filtering of LAN games does indeed now use the com_gamename.

The problem with stopping re-numbering the "legacy protocol" is that older versions of Q3, OA, etc. don't discriminate by com_gamename (and I'm not sure whether newer clients using the legacy protocol *can* tell the server their own com_gamename - so the server wouldn't be able to tell the difference between a too-new client falling back to the legacy protocol because its dialect of the modern protocol differs from the server's, which should get allowed in, and a Q3 1.32 client, which should get disconnected).

Perhaps the solution is to try to talk the ioquake3 people into continuing to support the version of the "modern protocol" used by ioq3 1.37 (after it's released), in addition to the "legacy protocol" and any even newer protocols they add?


Title: Re: Engine compatibility with ioquake3 1.37
Post by: rm on August 11, 2011, 04:52:09 AM
Quote
When looking for servers, "getservers" guesses a gamename based on the protocol, and "getserversExt" explicitly sends one. This actually means that the engine supplied with OA 0.8.5 will never find any IPv6 servers - it queries for them with "getserversExt openarena ipv6" as a result of GAMENAME_FOR_MASTER having been defined to "openarena", but all current OA servers implicitly advertise themselves as "Quake3Arena", so they won't be found like that.
This is correct and I had to dirty-hack my IPv6 dpmaster (http://openarena.ws/board/index.php?topic=2260.msg39556#msg39556) a bit to make everything work.
Code:
diff -ru dpmaster-2.2/src//games.c dpmaster-2.2-rm2/src//games.c
--- dpmaster-2.2/src//games.c 2011-07-23 12:29:48.000000000 +0600
+++ dpmaster-2.2-rm2/src//games.c 2011-08-11 15:44:14.329620304 +0600
@@ -544,7 +544,7 @@
  GAMENAME_Q3A,
  2,
  {
- "protocols=66,67,68",
+ "protocols=66,67,68,71",
  "heartbeat=QuakeArena-1",
  },
  },
diff -ru dpmaster-2.2/src//messages.c dpmaster-2.2-rm2/src//messages.c
--- dpmaster-2.2/src//messages.c 2011-07-23 12:29:48.000000000 +0600
+++ dpmaster-2.2-rm2/src//messages.c 2011-08-11 15:42:20.811916626 +0600
@@ -41,6 +41,7 @@
 // Maximum size of a reponse packet
 #define MAX_PACKET_SIZE_OUT 1400
 
+#define GAMENAME_Q3A "Quake3Arena"
 
 // Types of messages (with samples):
 
@@ -377,7 +378,7 @@
  if (space)
  *space = '\0';
  msg_ptr = msg_ptr + strlen (gamename);
-
+    if (strcmp (gamename, "openarena") == 0) strncpy (gamename, GAMENAME_Q3A, sizeof (GAMENAME_Q3A));
  game_options = Game_GetOptions (gamename);
 
  // Read the protocol number
Good thing is that one can solve this at the dpmaster, no need to go around patching/updating every dedicated server in existence.


Title: Re: Engine compatibility with ioquake3 1.37
Post by: smcv on August 11, 2011, 09:01:47 AM
That's a pretty dirty hack, but probably the best that can be done until an official OA release breaks compatibility.

Would you like me to configure Debian's openarena package to have sv_master2 = dpmaster.version6.ru by default? If we ask him nicely, Hans would probably do the same in Fedora.


Title: Re: Engine compatibility with ioquake3 1.37
Post by: rm on August 12, 2011, 02:03:38 AM
Quote
Would you like me to configure Debian's openarena package to have sv_master2 = dpmaster.version6.ru by default? If we ask him nicely, Hans would probably do the same in Fedora.
I would certainly not mind, and I have no problem leaving this dpmaster server up for a long time to come.

However my primary hope is that dpmaster.deathmask.net will implement IPv6 in the same way, soon. And the main purpose of my own IPv4+IPv6 dpmaster set up was to prove this can be done without any service disruption to IPv4-only gamers.


Title: Re: Engine compatibility with ioquake3 1.37
Post by: Gig on November 12, 2011, 11:58:56 AM
This post from smcv sounds interesting:
http://openarena.ws/board/index.php?topic=1933.msg39469#msg39469

Uhm... now that I check, I see he wrote that before starting this thread...

Have there been some more infos?