Server failure

[saintk]

Hi,

Not sure where to post this exactly, so if it's in the wrong section please move it.

I run the specialattack.net hub on Debian Squeeze 64bit.

I noticed after running a few days the server errored out with the following error;

  Code:

2012-08-04 18-17-18 EXCEPTION: EAssertionFailed: Tried to access invalid client index (KM_NetServer.pas, line 152) 2012-08-04 18-17-18 Server restarting... 2012-08-04 18-17-18 Server destroyed 2012-08-04 18-17-18 Server: LNet: Error on bind [98]: Address already in use 2012-08-04 18-17-18 Server: Warning: Client 0 was already disconnected 2012-08-04 18-17-18 SERVER FAILED TO START! Exception: Server failed to start 2012-08-04 18-17-18 Stopped listening

Apparently it doesn't wait long enough between killing the server and restarting it and thus a manual restart was required.

Another thing I noticed is that I have to 'renice' the process to -20 to get somewhat performance out of it. The machine it's hosted on is a high performance machine located in a datacenter with a 100Mbit pipe straight to the AMS-IX.

The low performance I notice can't be in the machine or it's connection.

I am not sure where you stand on netcode development, but it looks like the game could do with a sort of buffer feature, or a mechanism if an update doesn't arrives on the server from a client, it won't pause the game, but try to receive the update on the next package. Currently there is quite some noticeable lag in the game seemingly being caused by slow connected clients.

Thumbs up for the remake, I'm really enjoying the multiplayer feature of this game!

Cheers,

Saint K.

[Lewin]

Hi Saint K,
Thanks for posting here about your problems.
I'm Lewin, one of the KaM Remake developers. I wrote most of the network/server code. Krom wrote most of the render code, and we write the game code about 50/50

I wasn't aware that the Linux version of the server was so unstable, we haven't received many other complaints about it. Unfortunately I'm not an expert on network code and writing servers. I'm not sure if I'm using the network library correctly, there are two separate libraries for Linux and Windows so the Windows one has been much more thoroughly tested because I only have Linux VMs. I spent the past few hours working on the network code to try and resolve your problems, I'll answer each one separately. Are you interested in helping sort this out? I've posted a "testing" version of the server at the end of this message which you can try it out and see if it solves your issues.

1. Exception: This is a bug. I don't know why it happened, I can't see any obvious reason by looking at the code, but I've made the server print out a full stack dump so I can see where it came from and hopefully figure out why it happened. If you can update to this testing version (which I will post below) and send me their log file after this exception occurs I should be able to figure out why and fix the bug.

2. Failing to restart after the exception: You are right, it should pause after closing the server before restarting. I've added a 5 second pause and fixed another bug so it now closes the server properly. I've also made it retry over and over if it fails to restart the server (with 5 second pauses between each attempt) Hopefully this is completely fixed.

3. Low performance: How are you measuring performance? By ping shown in the KaM Remake lobby? Ping measurements in the KaM Remake will be higher than in other games, because it's not just measuring the network delay, it's also measuring the time it takes for the client to respond. We don't use threads in networking so the game cycles between 3 states:
- Updating game state/menus
- Rendering
- Processing network messages
While it is in one of the first two states it won't be responding to pings, and because updating/rendering can take quite a few milliseconds you'll notice a higher ping than you would in another game. If we put networking in a separate thread it would show pings the same as other games, but it wouldn't actually improve the game quality because the network messages cannot be used until the game has finished updating/rendering (you can't update/render the game at the same time as changing it due to the network)

However, you said that renicing it improved the performance? In what way and by how much? (Lower pings? Or something else?) Renicing doesn't make any difference to me but it could be different on a server. I've tweaked the way the server idles (waiting between messages) so it doesn't actually sleep now, instead it just gives the OS a 1ms timeout when asking for updates about sockets, I didn't realise this was possible. Maybe because it was sleeping before the OS moved the process off the CPU (sleeping ends the allocated timeslice as far as I know) This might improve performance for you, please let me know.

Here is a testing version of the server: (Linux x86_64 version only, I can build the x86 one if you want it)
http://lewin.hodgman.id.au/temp/KaM_Ser ... x86_64.zip
It is for clients running r3392 (the current public release) so it should work just the same as the one on our download page, except hopefully it won't have some of the issues you posted about. Please let me know how it goes and send me your log file after a few days or if an exception/crash occurs.

If you're not interested in testing the new version (or don't have time) that's no problem, I can probably get someone else to test it during the beta phase of the next release.
Cheers,
Lewin.

[saintk]

Hi,

Thanks a lot for your reply! I'll go ahead and upload the test build and see how this runs!

The performance issues which are noticeable are not the ping times shown (I had read the reasons for that in the FAQ), it's the actual gaming experience. When there is one slower client, it seems to be slowing down all the other clients. If a client responds too late, it will pause the game for the other players and read "waiting for player X". Before renicing the process it also seemed a bit jerky in unit movements etc.

I'll start playing around with that build and post some feedback!

Cheers,

Saint K.

[Lewin]

The performance issues which are noticeable are not the ping times shown (I had read the reasons for that in the FAQ), it's the actual gaming experience. When there is one slower client, it seems to be slowing down all the other clients. If a client responds too late, it will pause the game for the other players and read "waiting for player X". Before renicing the process it also seemed a bit jerky in unit movements etc.

Unfortunately renicing won't effect that, the server doesn't know anything about the game, it just forwards packets around, all of the game simulation is done by the clients. After "waiting for players" has shown a few times it should increase the delay between sending commands and when they will be executed, so it goes away. Unfortunately if one player's computer is too slow (cannot run the game at normal speed) then it will effect everyone because you have to wait for that player to catch up when he gets behind. In the next version the game code has been optimised a lot so it will run much faster and hopefully lag like this will be lessened.

Thanks for trying out the test version, I hope it works

[saintk]

The performance issues which are noticeable are not the ping times shown (I had read the reasons for that in the FAQ), it's the actual gaming experience. When there is one slower client, it seems to be slowing down all the other clients. If a client responds too late, it will pause the game for the other players and read "waiting for player X". Before renicing the process it also seemed a bit jerky in unit movements etc.

Unfortunately renicing won't effect that, the server doesn't know anything about the game, it just forwards packets around, all of the game simulation is done by the clients. After "waiting for players" has shown a few times it should increase the delay between sending commands and when they will be executed, so it goes away. Unfortunately if one player's computer is too slow (cannot run the game at normal speed) then it will effect everyone because you have to wait for that player to catch up when he gets behind. In the next version the game code has been optimised a lot so it will run much faster and hopefully lag like this will be lessened.

Thanks for trying out the test version, I hope it works

Alright, clear story!

Thanks for the feedback. I'll post back here once I have some test results. If you need anything else tested server wise throw me a message.

Cheers,

Saint K.