Mirage Source

Free ORPG making software.
It is currently Tue Apr 23, 2024 12:38 pm

All times are UTC




Post new topic Reply to topic  [ 47 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Wed Dec 20, 2006 8:09 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
Originally posted by Verrigan

Difficulty: Medium 3/5

This tutorial is designed to show you how to put a buffer system in your game. Why would you want to do that? Think about it. When a player connects, there are a series of messages sent to that player, and in Mirage, they are sent one right after the other.. If you have large maps, or you have a bunch of items/npcs/spells/etc., the server will send all that data to a user who is connecting.. For small games with small sized maps, this isn't a problem.. Lag is minimal. However, for games with tons of items, spells, etc., you will experience lag when another player connects.

Okay, now introduce a buffering (queue) system.. The server form has a timer that calls a subroutine that is solely responsible for dropping the size of the queue. How does the queue get filled? Instead of immediately sending any data to a player, you make the server add it to the queue. The server then hits the timer's interval, calls the subroutine to drop the queue, which sends messages to each user that is waiting for those messages. The subroutine will go through all connections, check the queue, and deliver messages to each user up to a predefined size limit for each user.. (This tutorial limits that size to 32K per user, per loop, but you can easily customize that limit.)

All that said, I must say that I have only tested this buffer system with one connection, so it might need some work, which is why I set the difficulty at 3.. Some of you might think the difficulty should be 5, but that's why I'm going to tell you this: BACKUP YOUR SERVER BEFORE ADDING THIS TUTORIAL.

Now, on with the tutorial..

Files to Modify
  • frmServer.frm
  • modGeneral.bas
  • modServerTCP.bas
  • modTypes.bas
*** In addition to the above files, you will need to change any line from:
Code:
Call CloseSocket(<whatever variable is used... Index, i, etc.>)
to:
Code:
QueueDisconnect(<same variable name... Index, i, etc.>) = True


frmServer.frm
FrmServer is where timers and sockets are stored.. I don't like to do this, but my tutorials are based on the already existing code in the Mirage Source, not redoing the whole thing.. :P So...

Add a timer to the form named: tmrStartMsgQueue with an interval of 100. Then, in the code of frmServer, add the following code:
Code:
Private Sub tmrStartMsgQueue_Timer()
  Call SendQueuedData
End Sub


modGeneral.bas
ModGeneral is where general stuff happens.. The server is initialized here, the AI is in here, etc.

Add the following code somewhere:
Code:
Sub SendQueuedData()
  Dim i As Integer, n As Long
  Dim TmpStr As String
 
  For i = 1 To MAX_PLAYERS

    TmpStr = ""
    With ConQueues(i)
      If Not .Lock Then
        If frmServer.Socket(i).State <> 7 Then
          .Lines = ""
        End If
        If Len(.Lines) = 0 And QueueDisconnect(i) = True Then
          Call CloseSocket(i)
          QueueDisconnect(i) = False
        Else
          If Len(.Lines) > 0 Then
             If Len(.Lines) < MAX_PACKETLEN Then
               TmpStr = .Lines
             Else
               TmpStr = Left(.Lines, MAX_PACKETLEN)
             End If
             .Lines = Right(.Lines, Len(.Lines) - Len(TmpStr))
          End If
        End If
        If Len(TmpStr) > 0 Then
          Call frmServer.Socket(i).SendData(TmpStr)
        End If
      End If
    End With
  Next
  DoEvents
End Sub

In Sub InitServer
After this code:
Code:
    ' Init all the player sockets
    For i = 1 To MAX_PLAYERS
        Call SetStatus("Initializing player array...")
        Call ClearPlayer(i)
       
        Load frmServer.Socket(i)
add this code:
Code:
        ConQueues(i).Lines = "" 'Initialize the lines.


modServerTCP.bas
ModServerTCP handles the data being sent from/received to the server.

In Sub SendDataTo()
Change this code:
Code:
    If IsConnected(Index) Then
        frmServer.Socket(Index).SendData Data
        DoEvents
    End If
to this code:
Code:
    If IsConnected(Index) Then
        With ConQueues(Index)
          .Lock = True
          .Lines = .Lines & Data
          .Lock = False
        End With
        'DoEvents
    End If


modTypes.bas
ModTypes is where all the data types are stored, and other stuff...

Under this code:
Code:
Type GuildRec
    Name As String * NAME_LENGTH
    Founder As String * NAME_LENGTH
    Member(1 To MAX_GUILD_MEMBERS) As String * NAME_LENGTH
End Type
add this code:
Code:
Type ConDataQueue
  Lock As Boolean
  Lines As String
End Type

Public ConQueues(MAX_PLAYERS) As ConDataQueue
Public Const MAX_PACKETLEN As Long = 32768 'About 32K
Public QueueDisconnect(MAX_PLAYERS) As Boolean


This was tested in a vanilla Mirage Source 3.0.3. So, if you add this to your game, and I only tested this with one player connected.. (I don't have a bunch of users, cause I don't run a Mirage server.. :P) Anyways.. Just like any other tutorials on here... BACKUP YOUR SOURCE!!! I said it twice, now... Hopefully you will take my advice..

It is also possible that I failed to put something in here.. In the event that you find that I forgot something, or you tried this in a vanilla server and it didn't work.. Please post here, but I'm pretty sure I remembered everything.. :)

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 18, 2007 5:09 pm 
Offline
Pro
User avatar

Joined: Wed Sep 20, 2006 1:06 pm
Posts: 368
Location: UK
Google Talk: steve.bluez@googlemail.com
Has anybody tested this?


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 18, 2007 5:22 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
Yes.

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
 Post subject:
PostPosted: Thu Jan 18, 2007 11:24 pm 
Offline
Knowledgeable
User avatar

Joined: Mon Jul 24, 2006 2:04 pm
Posts: 339
I just want to throw in - this is a great idea, but not when using Winsock control. Reason for this is that the Winsock control has the Nagle Algorithm enabled by default, and I don't think you can turn it off.

The Nagle Algorithm is basically a packet buffer created for office programs where slight packet delays are hardly a problem. The Winsock control was designed for quickly made applications, not online games or anything, so I doubt turning it off was even a consideration in their design.

How it works is that it buffers your packets for about 300ms from the first call. So if you send a packet, it will take at most a bonus 300ms onto your network lag. Thats on top of everything else, not total. If you buffer yourself, you'll probably keep throwing your buffers into the same Nagle buffer, which means it'd have the exact same effect in the long run (almost) to just send a bunch of small packets.

You have to thank the Nagle algorithm, though, since if it wasn't there, any ORPG using the winsock control (Mirage, Elysium, etc) and without a buffer, would probably have a good 2-4kb/sec added on to the packet size just from headers. :wink:

By the way, TCP headers are 20 bytes, IPv4 headers are 20 bytes, so every packet you send is 40 bytes + the data.

_________________
NetGore Free Open Source MMORPG Maker


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 3:26 am 
Offline
Pro
User avatar

Joined: Wed Sep 20, 2006 1:06 pm
Posts: 368
Location: UK
Google Talk: steve.bluez@googlemail.com
Spodi, vbGORE uses the SOX method right? Which would you recommend added to MSE, the SOX method, or IOCP?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 5:04 am 
Offline
Knowledgeable
User avatar

Joined: Mon Jul 24, 2006 2:04 pm
Posts: 339
I really don't have a lot of IOCP experience, but I can't see many large benifits it'd make to a relatively small series of connections. SOX would be the easiest to use since it is designed to be enarly the exact same as Winsock, so the only change you have to make is using Shut instead of Close and not using an array on the control I believe. :wink:

_________________
NetGore Free Open Source MMORPG Maker


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 2:11 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
Spodi wrote:
I just want to throw in - this is a great idea, but not when using Winsock control. Reason for this is that the Winsock control has the Nagle Algorithm enabled by default, and I don't think you can turn it off.

The Nagle Algorithm is basically a packet buffer created for office programs where slight packet delays are hardly a problem. The Winsock control was designed for quickly made applications, not online games or anything, so I doubt turning it off was even a consideration in their design.

How it works is that it buffers your packets for about 300ms from the first call. So if you send a packet, it will take at most a bonus 300ms onto your network lag. Thats on top of everything else, not total. If you buffer yourself, you'll probably keep throwing your buffers into the same Nagle buffer, which means it'd have the exact same effect in the long run (almost) to just send a bunch of small packets.

You have to thank the Nagle algorithm, though, since if it wasn't there, any ORPG using the winsock control (Mirage, Elysium, etc) and without a buffer, would probably have a good 2-4kb/sec added on to the packet size just from headers. :wink:

By the way, TCP headers are 20 bytes, IPv4 headers are 20 bytes, so every packet you send is 40 bytes + the data.

So you think it's a bad idea to use this?

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 4:33 pm 
Offline
Regular

Joined: Fri Dec 01, 2006 12:33 am
Posts: 37
It shows people how packet buffers are created... I personally wouldnt use this tutorial but I got a good understanding of how I wanted mine to go by looking at this. This is good to learn from but I wouldnt go to any mesure on adding it into my engine or game.

A suitable packet buffer can be created for winsock users but I dont think this is it.

Still I belive/think it holds very good value for people to view and understand and create their own.


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 4:49 pm 
Offline
Submit-Happy
User avatar

Joined: Fri Jun 16, 2006 7:01 am
Posts: 2768
Location: Yorkshire, UK
TheRealDamien wrote:
It shows people how packet buffers are created... I personally wouldnt use this tutorial but I got a good understanding of how I wanted mine to go by looking at this. This is good to learn from but I wouldnt go to any mesure on adding it into my engine or game.

A suitable packet buffer can be created for winsock users but I dont think this is it.

Still I belive/think it holds very good value for people to view and understand and create their own.


What are you on about..?

Do you use winsock? Do you use the winsock control?


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 5:55 pm 
Offline
Regular
User avatar

Joined: Tue Aug 08, 2006 7:03 pm
Posts: 75
Kite wrote:
TheRealDamien wrote:
It shows people how packet buffers are created... I personally wouldnt use this tutorial but I got a good understanding of how I wanted mine to go by looking at this. This is good to learn from but I wouldnt go to any mesure on adding it into my engine or game.

A suitable packet buffer can be created for winsock users but I dont think this is it.

Still I belive/think it holds very good value for people to view and understand and create their own.


What are you on about..?

Do you use winsock? Do you use the winsock control?


Wooooow!!! Slow down there.

He needs to get Visual basic first. :roll:

_________________
Image


Top
 Profile  
 
 Post subject:
PostPosted: Fri Jan 19, 2007 11:51 pm 
Offline
Knowledgeable
User avatar

Joined: Mon Jul 24, 2006 2:04 pm
Posts: 339
No William, not useless. With the Winsock control, yes (unless you can turn off Nagling in it, often referenced as NoDelay or something), but with SOX and probably IOCP (depending on what the socket library offers), it is absolutely priceless. When you get your packets crunched down a lot on a simple 2d ORPG (since as we all know, not a whole lot of data ever has to be transferred), the big worry becomes sending data as little as possible, and not as much what you are sending. Custom packet buffers give you so much more flexibility on how often you send data. And its fun to play with! Weeee!!! :D

So I'd just add a quick short explanation at the top on why you wouldn't want to use it with WinSock, but that with anything else where you can turn off Nagling, is simply just awesome. :wink:

_________________
NetGore Free Open Source MMORPG Maker


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 12:28 am 
Offline
Submit-Happy
User avatar

Joined: Fri Jun 16, 2006 7:01 am
Posts: 2768
Location: Yorkshire, UK
Spodi wrote:
No William, not useless. With the Winsock control, yes (unless you can turn off Nagling in it, often referenced as NoDelay or something), but with SOX and probably IOCP (depending on what the socket library offers), it is absolutely priceless. When you get your packets crunched down a lot on a simple 2d ORPG (since as we all know, not a whole lot of data ever has to be transferred), the big worry becomes sending data as little as possible, and not as much what you are sending. Custom packet buffers give you so much more flexibility on how often you send data. And its fun to play with! Weeee!!! :D

So I'd just add a quick short explanation at the top on why you wouldn't want to use it with WinSock, but that with anything else where you can turn off Nagling, is simply just awesome. :wink:


That must give you such a massive headache.


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 12:54 am 
Offline
Knowledgeable
User avatar

Joined: Mon Jul 24, 2006 2:04 pm
Posts: 339
Actually it isn't too bad. Organization is the main key. You just have to look at the packets you have, and think which ones would suffice having a slight delay, and which ones would be fine not coming until any other packets come. For example, if you get a message on new mail, you can just attach it to the next packet instead of letting it send on its own, since a few second delay wont make any difference (unless they are waiting next to the mailbox doing nothing, no one / thing is in the screen, and no one is talking). Its really fun to see what you can come up with. Of course, you wont be able to find as much as you want just looking at the packets. :wink:

_________________
NetGore Free Open Source MMORPG Maker


Top
 Profile  
 
 Post subject:
PostPosted: Sat Jan 20, 2007 1:06 am 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
Well since Im using IOCP it's probably a good idea to use the packet buffer.

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
 Post subject:
PostPosted: Wed Mar 07, 2007 12:41 am 
Offline
Pro
User avatar

Joined: Wed Sep 20, 2006 1:06 pm
Posts: 368
Location: UK
Google Talk: steve.bluez@googlemail.com
I'll have to try and add IOCP again then.


Top
 Profile  
 
 Post subject:
PostPosted: Thu Mar 08, 2007 6:40 pm 
Offline
Knowledgeable
User avatar

Joined: Sun May 28, 2006 10:07 pm
Posts: 327
Location: Washington
I just wanted to say these things:

1) This is still a good idea even with Nagling turned on.. It was mainly created to keep the people who login from lagging everyone else in game while everyone waited for that person to receive all the item/npc/etc packets. It also (for an added benefit) helps to keep others from lagging when a player has to download a map on a game that has (excessively) large maps.
2) It is possible to turn off the Nagling using API calls on the Winsock control by using the SocketHandle of whatever winsock object you're using.. (I posted this somewhere... Can't remember if it was on this forum or not.. maybe I'll find it and post a link.)

And for an added bonus, I just want to say that Spodi is a kick-ass programmer, and I want to know if you (Spodi) do your own graphics! :) (No, I'm not trying to be funny.)

[Edit]
Here's the link I mentioned in item #2 above: http://key2heaven.net/ms/forums/viewtop ... =8235#8235


Top
 Profile  
 
PostPosted: Sun Sep 02, 2007 6:35 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
If adding this, disable the nagling algorithm and also change the timer interval from 100 to 25.

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
PostPosted: Sun Sep 02, 2007 7:29 pm 
Offline
Submit-Happy
User avatar

Joined: Fri Jun 16, 2006 7:01 am
Posts: 2768
Location: Yorkshire, UK
Ooh, gonna work on this once I finish converting to the API.

Lol @ Reece being naive.

"I'm sure Elysium fixed this"

lolagasm.

_________________
Quote:
Robin:
Why aren't maps and shit loaded up in a dynamic array?
Jacob:
the 4 people that know how are lazy
Robin:
Who are those 4 people?
Jacob:
um
you, me, and 2 others?


Image


Top
 Profile  
 
PostPosted: Tue Dec 09, 2008 2:33 am 
Offline
Persistant Poster
User avatar

Joined: Wed Nov 29, 2006 11:25 pm
Posts: 860
Location: Ayer
Apologies for necro..

But disabling the Nagle algorithm and implementing this Packet buffer

would reduce packet lag?

edit\
Why not use IOCP and packet buffering?

_________________
Image


Top
 Profile  
 
PostPosted: Tue Dec 09, 2008 7:20 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
Don't use IOCP and don't use a packet buffer.

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
PostPosted: Tue Dec 09, 2008 11:13 pm 
Offline
Pro
User avatar

Joined: Wed Jun 07, 2006 8:04 pm
Posts: 464
Location: MI
Google Talk: asrrin29@gmail.com
don't use IOCP? did something change since I've been gone?

_________________
Image
Image


Top
 Profile  
 
PostPosted: Tue Dec 09, 2008 11:21 pm 
Offline
Persistant Poster
User avatar

Joined: Wed Nov 29, 2006 11:25 pm
Posts: 860
Location: Ayer
William wrote:
Don't use IOCP and don't use a packet buffer.


Because?

_________________
Image


Top
 Profile  
 
PostPosted: Wed Dec 10, 2008 3:41 pm 
Offline
Community Leader
User avatar

Joined: Mon May 29, 2006 1:00 pm
Posts: 2538
Location: Sweden
Google Talk: johansson_tk@hotmail.com
IOCP is unstable, and the code verrigan provided isn't fully bug free. And IOCP is supposed to be used on a server with thousands of connections, a ms server is better without it. So just skip it.

_________________
I'm on Facebook!My Youtube Channel Send me an email
Image


Top
 Profile  
 
PostPosted: Wed Dec 10, 2008 10:23 pm 
Offline
Pro
User avatar

Joined: Wed Jun 07, 2006 8:04 pm
Posts: 464
Location: MI
Google Talk: asrrin29@gmail.com
I've had IOCP in my code for over a year and have never had any sort of hang ups with it. granted, I've not had more then 10 people online at a time, but unless I'm missing something there doesn't seem to be a problem.

_________________
Image
Image


Top
 Profile  
 
PostPosted: Wed Dec 10, 2008 10:53 pm 
Offline
Persistant Poster
User avatar

Joined: Thu Mar 29, 2007 10:30 pm
Posts: 1510
Location: Virginia, USA
Google Talk: hpmccloud@gmail.com
I took IOCP out of a game because it kept getting out of memory errors and players were always having account online problems and so on. The game has about 20 people on average.

_________________
Nean wrote:
Yes harold. Give it to me.

Image
Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 47 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group