Automatic download speed + fs_homepath error?

[Gig]

Hi guys!

I know that for fast downloads one should enable "www download" mode... but I want to ask a thing about the "standard" download. I'm doing some tests to act as server. With "\rate" (client-side) and "\sv_maxrate" (server-side) set to 25000, I supposed the download speed to be about 25k/sec... Instead, it is about 9-10 k/sec... Someone knows why?

I thought I read somewhere that, when playing over LAN (I did my test using two Openarena processes on the same machine, but the client one using a separate fs_homepath), the rate limit shouldn't apply (thus downloading at high speed)... but probably I remember wrong!


The other thing I noticed while doing this test: I'm using Windows XP, and my test to launch OpenArena.exe +set fs_homepath . (that DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/wiki/FAQ#Can_I_change_the_directory_to_where_configuration_and_autodownloaded_files_are_saved.3F]should place the config/autodownloaded files directly inside the installation folder, like the original Q3 did) was a failure: the program crashes. Specifying a different fs_homepath (example: c:\test) works, but with ".", it crashes immediately. Does this happen only to me, or is a general problem?

[Udi]

Specifying a different fs_homepath (example: c:\test) works, but with ".", it crashes immediately. Does this happen only to me, or is a general problem?

Use "./" instead of "."

[Gig]

With "./", it works.

So, is it "./" instead of "." for everyone (thus, the instructions in the FAQ would be wrong), or is it "./" for Windows users and "." for Linux (or Mac?) users (thus, the instructions in the FAQ would be incomplete)?

[Udi]

On Linux it works both with a "." and "./", probably that's the case on MacOSX too. So I would say always use "./", that's easier to remember and makes the scripts portable between platforms.

[Gig]

Ok, DO NOT LINK[/b]) h t t p s : / / openarena . wikia . com/index.php?title=FAQ&diff=6939&oldid=6933]I updated the FAQ.

Someone has got any idea for the reason why the "standard" download does not go up to 25 KB/s as expected?

[7]

Maybe you've got a weird network configuration?

The server has automatic LAN detection and sets the client rate to 99999 if the client connects from a private network address (IPs in ranges 10.0.0.0-10.255.255.255, 172.16.0.0-172.31.255.255 and 192.168.0.0-192.168.255.255) or the loopback address (127.0.0.1). If you connect from any other IP, client rate settings are in effect.

The Q protocol imposes an overhead (of 12 bytes if I remember correctly) on every packet also, so not reaching the maximal theoretical throughput is to be expected.

[Gig]

My IP class is 192.168.0.x, and I'm making the test with a single computer (I have only one).

[7]

Do you notice a speed increase when you set both rate and sv_maxrate to 99999?

If you do, either your localhost address isn't 127.0.0.1 or there is an error in the OA network code that wasn't there before. If you don't, either you've got a damn slow network stack or computer, something is really broken in your network or firewall configuration, or rate was recently hard-capped to 25000 somewhere in the OA network code.

(I can't eliminate the possibility of errors in the code because I haven't got the time to download and debug the OA 0.8.5 source at the moment.)

[Falkland]

Do you notice a speed increase when you set both rate and sv_maxrate to 99999?

But this is automatically done when dedicated is set to 1

[Gig]

I changed the server's sv_maxrate to 90000, and set rate 90000 in the test homepath in both baseoa and my test folders config...
The speed sometimes is about 11-12KB/s and sometimes about 19-20KB/s, I can't understand exactly why... (update: I'm not completely sure, but it seems that with the rate set to 90000, it goes to 19-20 KB/s, that, guessing some overhead, could be a reasonable download speed for a rate of 25000... but why this is not achieved with rate 25000, that goes at 11-12KBs instead?)

I've also noticed the existence of a variable called "sv_lanforcerate": it is set to 1... I tried to set it to 0, but the speed doesn't go up like it should over "local" connection.

Maybe the server and the client having the same IP address is a non-standard situation that the "lan speed" check does not expect?

UPDATE:
1) In the "multiplayer" menu, there is an option called "Optimize for LAN": I don't know what this exacly does, and to what Cvars it is related, thus I don't know how is it set in my "dedicated" server.

2) I just figured out that I forgot to ask an important question: the high download speed over LAN was already there in Quake 3 or has been added by ioquake3 or OpenArena? And in this case, is it expected to work also with old Q3A mods, or only with baseoa or OpenArena-specific mods? Because, to have something to download, I'm doing the test with a small Q3A mod (a single 3 MB additional pk3)...

[7]

But this is automatically done when dedicated is set to 1

If I understand Gig correctly, he's running two instances of OA on the same host, the server exe and the client exe.

The speed sometimes is about 11-12KB/s and sometimes about 19-20KB/s, I can't understand exactly why...

I suspect your network stack is slow, likely because you're totally maxing out your CPU. This would result in a slow connection with lots of packet loss on the loopback interface (127.0.0.1).

Maybe the server and the client having the same IP address is a non-standard situation that the "lan speed" check does not expect?

Nope, if server and client are on the same host the loopback interface is used, and there is a specific check for 127.0.0.1 in the OA network code.

Edit:

1) In the "multiplayer" menu, there is an option called "Optimize for LAN": I don't know what this exacly does, and to what Cvars it is related, thus I don't know how is it set in my "dedicated" server.

I think the most important thing the optimize for LAN option does is setting cl_packetdup to 0, but that's a client setting so I wouldn't suspect it to have any influence on download speeds.

2) I just figured out that I forgot to ask an important question: the high download speed over LAN was already there in Quake 3 or has been added by ioquake3 or OpenArena? And in this case, is it expected to work also with old Q3A mods, or only with baseoa or OpenArena-specific mods? Because, to have something to download, I'm doing the test with a small Q3A mod (a single 3 MB additional pk3)...

Don't worry, what you download is totally irrelevant, it's the client and server that make the speed and not the content that's being transferred.

[Gig]

But this is automatically done when dedicated is set to 1

If I understand Gig correctly, he's running two instances of OA on the same host, the server exe and the client exe.

Server:
Executable: C:\...\OA\oa_ded.exe
fs_homepath: standard (%appdata%\openarena)
Mod folder: C:\...\OA\bfg10k\

Client:
Executable c:\...\OACopy\openarena.exe
fs_homepath: modified (C:\test\openarena)
Mod folder, automatically created: C:\test\openarena\bfg10k\
I delete the client's automatically downloaded .tmp file between tests.

The speed sometimes is about 11-12KB/s and sometimes about 19-20KB/s, I can't understand exactly why...

I suspect your network stack is slow, likely because you're totally maxing out your CPU. This would result in a slow connection with lots of packet loss on the loopback interface (127.0.0.1).

This does not explain why, with rate set to 90000, it gains some speed (from 11-12 to 19-20 KBs). I hope to be able to do some tests on a better computer, anyway...

Maybe the server and the client having the same IP address is a non-standard situation that the "lan speed" check does not expect?

Nope, if server and client are on the same host the loopback interface is used, and there is a specific check for 127.0.0.1 in the OA network code.

The server "status" command does not identify the client as 127.0.0.1, but as 192.168.0.X (the real IP address).

[7]

This does not explain why, with rate set to 90000, it gains some speed (from 11-12 to 19-20 KBs). I hope to be able to do some tests on a better computer, anyway...

So there really is a speed increase, or is it just temporal fluctuations in speed? If there really is a speed increase this would imply a bug in the OA network code (because the server should have detected 192.168.0.X as a LAN address and should have set rate to 99999 automatically).

The server "status" command does not identify the client as 127.0.0.1, but as 192.168.0.X (the real IP address).

Ah ok, I guess you're connecting with \connect 192.168.0.X and not with \connect 127.0.0.1. That doesn't make much of a difference anyway, because the 192.168.X.X range is checked in the OA server network code just like 127.0.0.1 is. If there is any speed increase when you connect over 127.0.0.1 that would most likely have to do with internal routing in the XP network stack and not with OA.

[Gig]

I was using the "multiplayer" menu to find the server....

Connecting with \connect 127.0.0.1, the "status" command on the server says the client is 127.0.0.1, too... but, with rate 25000, the speed is stuck to 9-10 KBs anyway...

[7]

Yeah, looks like a bug in the OA code (assuming the speed increase is real) I have sv_maxrate and rate at 90000 in my server and client configs, so I didn't notice it before. I'll test it next weekend myself.

[Falkland]

But this is automatically done when dedicated is set to 1

If I understand Gig correctly, he's running two instances of OA on the same host, the server exe and the client exe.

When dedicated is set to 1 ( oa_ded.xxx +set dedicated 1 ... ) , the server rate is set to LAN rate ( 999999 ) ; sv_lanforcerate 1 is useful to force LAN default rate also on a public server ( when dedicated is 2 server rate is set to sv_maxrate )

[Gig]

I thought the only difference from dedicated 1 to dedicated 2 was the sending of the heartbeats o the master server... so I was wrong. My server was using dedicated 2 mode... I will do some tests with dedicated 1, too..

Anyway, since the "force" option was enabled, it should have been working at high speed anyway, right? But it didn't...

[Gig]

I think the most important thing the optimize for LAN option does is setting cl_packetdup to 0, but that's a client setting so I wouldn't suspect it to have any influence on download speeds.

Someone can confirm?
"Optimize for LAN" seems a server-side thing (its button is in Multiplayer-Create menu)... do you mean that it overrides the cl_packetdup value set by the clients?
Someone can confirm?

Anyway, I've just done some tests on a better machine.
OA 0.8.5, mod BFG10K Arena (3 MB download).
Windows XP, same machine, connecting to 127.0.0.1.
sv_forcelanrate is 1, the default.

Client running from a separate installation folder, with fs_homepath ./

Automatic download speed is about 19-20 KBs, with both rate and sv_maxrate set to 90000 or to 25000, no difference. Thus, it seems that the speed down to 11-12 KBs with rate 25000 and up to 19-20 with rate 90000 happens in my home computer, but not with this (better) machine. I don't know what to think.

Anyway, also here, the expected "lan boost 999999" does not occur at all, and I tried with dedicated 0 (openarena.exe), dedicated 1 and dedicated 2 (oa_ded) modes... Are you sure it should affect also the download speed and not only the effective transmission during the match?

[7]

"Optimize for LAN" seems a server-side thing (its button is in Multiplayer-Create menu)... do you mean that it overrides the cl_packetdup value set by the clients?

I glanced over the ui code and I retract my cl_packetdup statement It sets the dedicated cvar.

Automatic download speed is about 19-20 KBs, with both rate and sv_maxrate set to 90000 or to 25000, no difference. Thus, it seems that the speed down to 11-12 KBs with rate 25000 and up to 19-20 with rate 90000 happens in my home computer, but not with this (better) machine. I don't know what to think.

I think this is due to speed issues on your slower computer. I'll test things on a real LAN this weekend.

[Gig]

"Optimize for LAN" seems a server-side thing (its button is in Multiplayer-Create menu)... do you mean that it overrides the cl_packetdup value set by the clients?

I glanced over the ui code and I retract my cl_packetdup statement It sets the dedicated cvar.

Uh? The dedicated Cvar? I tried to change "Optimize for LAN", but "\dedicated" is still 0... (and I can still play locally).

[7]

Sorry, I was accidentally browsing the 0.7.7 sources again, ui_dedicated doesn' t even exist in 0.8.5.  I'll shut up until after the weekend, I make mistakes like these all the time when I'm in a hurry.

[Gig]

Q3A had the "dedicated" option in the GUI, and, from your words, i guess OA 0.7.7 still had it... I wonder why it has been removed in later versions...

Another question: since it seems to me that "download resuming" is not supported (if you begin to download a pk3, abort with ESC, then reconnect to the server, it seems that the download restarts from the beginning of the file), wouldn't it be better to automatically remove the .tmp files when you abort a download?

[sago007]

The build in autodownload is optimised for 28k modems.

I don't know exactly how things like transfer windows (important for speed) are handled but I believe not that well.

dedicated was removed from the menu because there was no guaranty for it returning to a usable console.

[Gig]

dedicated was removed from the menu because there was no guaranty for it returning to a usable console.

Uh? To return to a usable GUI, you mean? For that, wouldn't it be enough to tell the user "to stop the dedicated server, just type \dedicated 0 in console": this brings you back to the main menu...

Anyway, now I'm a bit more interested to understand what that "Optimize for LAN" option does...

.... and if the "network rate boost" that the engine should give when playing locally really works or is broken... and if it works, it affects only the actual game and not the automatic download (that would be an explaination...)

[sago007]

Uh? To return to a usable GUI, you mean? For that, wouldn't it be enough to tell the user "to stop the dedicated server, just type \dedicated 0 in console": this brings you back to the main menu...

When I activated the option the UI locked up and I could not shut it down or enter commands.

Anyway, now I'm a bit more interested to understand what that "Optimize for LAN" option does...

I believe it was supposed to ignore the value of rate on all players.

.... and if the "network rate boost" that the engine should give when playing locally really works or is broken... and if it works, it affects only the actual game and not the automatic download (that would be an explaination...)

Transferring files is much more than just having a high rate. Http-redirects use TCP and have all the benefits of that: Reliability, resizeable transfer windows, congestion control and the operating system can change the congestion protocol to a more optimal one if needed. idtech3 runs the reliable file transfers on its own network stack based on UDP. Apparently there is no congestion control and the game therefore uses "safe" values then pumping packets on the network. It is possible to implement the TCP-protocol in UDP but it is done in this engine. Even a simple TCP-Reno protocol should give much higher speeds.