How to handle netcode?

Steven Lu
  • How to handle netcode? Steven Lu

    I'm interested in evaluating the different ways that the netcode can "hook into" a game engine. I am designing a multiplayer game now, and so far I have determined that I need to (at the very least) have a separate thread to handle the network sockets, distinct from the rest of the engine which handles the graphics loop and scripting.

    I did have one potential way to make a networked game entirely single-threaded, which is to have it check the network after rendering each frame, using non-blocking sockets. However this is clearly not optimal because the time it takes to render a frame is added to the network lag: Messages arriving over the network must wait until the current frame render (and game logic) completes. But, at least this way the gameplay would still remain smooth, more or less.

    Having a separate thread for networking allows the game to be completely responsive to the network, for instance it can send back an ACK packet instantly upon receipt of a state update from the server. But I'm a little confused about the best way to communicate between the game code and the network code. The networking thread will push the received packet to a queue, and the game thread will read from the queue at the appropriate time during its loop, so we haven't gotten rid of this up-to-one-frame delay.

    Also, it seems like I would want the thread which handles the sending of packets to be separate from the one that is checking for packets coming down the pipe, because it wouldn't be able to send one off while it is in the middle of checking if there are incoming messages. I am thinking about the functionality of select or similar.

    I guess my question is, what's the best way to design the game for the best network responsiveness? Clearly the client should send user input as soon as possible to the server, so I could have net-send code come immediately after the event processing loop, both inside of the game loop. Does this make any sense?

  • I need to (at the very least) have a separate thread to handle the network sockets

    No you don't.

    I did have one potential way to make a networked game entirely single-threaded, which is to have it check the network after rendering each frame, using non-blocking sockets. However this is clearly not optimal because the time it takes to render a frame is added to the network lag:

    Doesn't necessarily matter. When does your logic update? There's little point getting data off the network if you can't do anything with it yet. Similarly there's little point responding if you have nothing to say yet.

    for instance it can send back an ACK packet instantly upon receipt of a state update from the server.

    If your game is so fast-paced that waiting for the next frame to be rendered is a significant delay, then it's going to be sending enough data that you don't need to send separate ACK packets - just include ACK values inside your normal data payloads, if you need them at all.

    For most networked games, it's perfectly possible to have a game loop like this:

    while 1:
        read_network_messages()
        read_local_input()
        update_world()
        send_network_updates()
        render_world()
    

    You can decouple updates from rendering, which is highly recommended, but everything else can pretty much stay that simple unless you have a specific need. Exactly what kind of game are you making?

  • However this is clearly not optimal because the time it takes to render a frame is added to the network lag: Messages arriving over the network must wait until the current frame render (and game logic) completes.

    That's not true at all. The message goes over the network while the receiver renders the current frame. The network lag is clamped to a whole number of frames on the client side; yes- but if the client has so few FPS that this is a big deal, then they have bigger problems.

  • Ignore responsiveness. On a LAN, ping is insignificant. On the internet, 60-100ms round-trip is a blessing. Pray to the lag gods that you don't get spikes of > 3K. Your software would have to run at very low number of updates/sec for this to be a problem. If you shoot for 25 updates/sec, then you've got 40ms max time between when you receive a packet and act on it. And that is the single-threaded case...

    Design your system for flexibility and correctness. Here is my idea on how to hook a network subsystem into game code: Messaging. The solution to a lot of problems can be "messaging". I think messaging cured cancer in lab rats once. Messaging saves me $200 or more on my car insurance. But seriously, messaging is probably the best way to attach any subsystem to game code while still maintaining two independent subsystems.

    Use messaging for any communication between the network subsystem and game engine, and for that matter between any two subsystems. Inter-subsystem messaging can be a simple as a blob of data passed by pointer using std::list.

    Simply have an outgoing messages queue and a reference to the game engine in the network subsystem. The game can dump messages it wants sent into the outgoing queue and have them automagically sent, or perhaps when some function like "flushMessages()" is called. If the game engine had one large, shared message queue, then all subsystems that needed to send messages (logic, AI, physics, network, etc.) can all dump messages into it where the main game loop can then read all of the messages and then act accordingly.

    I would say that running the sockets on another thread is fine, though not required. The only problem with this design is that it is generally asynchronous (you don't know precisely when the packets are being sent) and that can make it hard to debug and making timing related issues appear/disappear randomly. Still, if done properly, neither of these should be a problem.

    From a higher level, I'd say separate networking from the game engine itself. The game engine doesn't care about sockets or buffers, it cares about events. Events are things such as "Player X fired a shot" "An explosion at game T happened". These can be interpreted by the game engine directly. Where they are generated from (a script, a client's action, an AI player, etc) doesn't matter.

    If you treat your network subsystem as a means of sending/receiving events, then you get a number of advantages over just calling recv() on a socket.

    You could optimize bandwidth, for example, by taking 50 small messages (1-32 bytes in length) and have the network subsystem package them into one large packet and send it. Maybe it could compress them before sending if that was a big deal. On the other end, the code could uncompress/unpackage the large packet into 50 discrete events again for the game engine to read. This can all happen transparently.

    Other cool things include single player game mode that reuses your network code by having a pure client + a pure server running on the same machine communicating via messaging in shared memory space. Then, if your single player game operates properly, a remote client (i.e. true multiplayer) would operate as well. Additionally, it forces you to consider ahead of time what data is needed by the client since your single player game would either look right or be totally wrong. Mix and match, run a server AND be a client in a multiplayer game -- it all works just as easily.

  • Network communications should be batched. You should strive for a single packet sent each game tick (which is often when a frame is rendered but really should be independent).

    Your game entities talk with the network subsystem (NSS). The NSS batches up messages, ACKs, etc and sends a few (hopefully one) optimally sized UDP packets (~1500 bytes usually). The NSS emulates packets, channels, priorities, resend, etc while only sending single UDP packets.

    Read through the gaffer on games' tutorial or just use ENet which implements many of Glenn Fiedler's ideas.

    Or you can just use TCP if your game doesn't need twitch reactions. Then all the batching, resends, and ACK problems go away. You'd still want an NSS to manage bandwidth and channels, however.

  • Don't completely "ignore responsiveness". There is little to gain from adding another 40ms latency to already delayed packets. If you're adding a couple of frames (at 60fps) then you're delaying the processing of a position update another couple of frames. It is better to accept packets quickly and process them quickly so you're improving the accuracy of the simulation.

    I've had great success optimising bandwidth by thinking about the minimum state information needed to represent what is visible on the screen. Then looking at each bit of data and choosing a model for it. Position information can be expressed as delta values over time. You can either use your own statistical models for this and spend ages debugging them, or you can use a library to help you out. I prefer to use this library's floating point model DataBlock_Predict_Float It makes it really easy to optimise the bandwidth used for the game scene graph.

Tags
c++ architecture networking
Related questions and answers
  • to that Game. Since enet supports channels I was thinking of using 2 channels per table, one for the game messages and one for in game chat. Would something like this work? Does anyone have any advice / design ideas for a game with a lobby and many tables? Is there a usual way this is done that I'm overlooking? Any conceptual ideas or even c/c++ code examples would be very helpful. Thanks ... not need any information from the lobby, they see the card table and the data about the other players and so forth. I've programmed the server portion for the game itself. The clients connect to my

  • well... I'm building the animation system of my game engine (the skeletal and skinned animation stuff), and I came to a point where I added so much functionality in the frame and node structures... I was thinking about making smaller structures with the fTime variable each so I can have attribute keyframes, light keyframes and buffer keyframes, then make the node structure look like... know if my speculations are ok, as I don't have much experience with 3d animations yet. I want to make a well decision as any option I choose would require a lot of work to get it to render and I

  • networking testing phase. I'll probably end up using dead reckoning. However I have a couple questions: 1: I am using TCP for all my networking. However it jumps and skips if you drop any packets at all, so I'd like to switch to UDP and have a numbering mechanism to make sure all packets arrive. Is there a preferred Networking Schema for game communication? 2: For critical things like object hits...So I'm starting work on a multi-player space shooter. And I've run into some issues. As I type this, my system is currently: //Main While Loop Gather Input Update Own Sprites Based on Input Send

  • or easier way? If it can be multithreaded that would obviously be better, and the less main-thread calculation the better. (For anyone wondering, the project is in C++). ...? My idea is to have a separate thread that compares the space difference of all the players every few seconds or so and stores references to all nearby players in the player's object for easy access...I'm looking into designing and implementing an online RPG (or more accurately, redesigning an existing piece of server software). One of the problems is that of managing visibility. Update data

  • () { } Cube::~Cube() { pBuffer->Release(); // why does this only work when put here? because it's created here? I thnk so, why not iBuffer though? } void Cube::Draw() { render_frame(); } void Cube...Basically when placed in the same file this works fine, but if placed in separate files (and I have tested this) just after Init() is called, the pointer to ID3D10* device's value is suddenly...(); }; #endif If I call Init() from game.cpp, the code works fine, if I call cube->init(), just as this function is entered (not when the other one is left), the device pointer suddenly has an address

  • I am developing a multi-player on-line game. I just started coding the server but I have no idea how to do it. Do I have to use threads ? And if i do, do I need one thread for every client? Knowing that I am using UDP, can I use multiple sockets in different threads to send packets on the same port? edit: server does: -listen for client connections -when a client connects: -update files (send new files to the client) -retrieve data from database -loop to retrieve and update live data -when client disconnects: -update data in database

  • with the Physics correctly... So my main question is: What would be a good combination of libraries to make an online game with? Im sure that many people have good combinations of libraries for making a game... better. I would also like compatability, If Theres one that fits DirectX(at least 9) and OpenGL, then that would be good) 2D Graphics(I liked SFML, so it its possible to get something that works with SFML, that would be nice) Networking(a way to connect multiple clients to a server/database) and Physics(Ive worked with Bullet Physics, but I cant find a model setup that would fit with said physics

  • by the event to work out where everything was at that time, but it seem like a lot of work - probably because the way I have implemented everything on the update side - so is there a better way? I did... that I have generated. I handle the threads by using turn-based concurrency. My problem is I worry about lag of the input events when there are fast moving objects, and I am curious to see... created on the native side. So I feel that the timestamp is my best solution. If anyone has any better ideas, it would be great to hear them.

  • that. All this seems well and good, except I have one question, why not linked lists rather than arrays? Sure, if you had to make your own linked list class, I would be on board, but in c++ anyway, assuming.... And I could see a case there for using arrays rather than linked lists, because you would have to make your own (which while simple enough to do, could potentially lead to many more errors). I also... critical place in the code. He gave the example of loading data from a wad file in doom, which while fast, still took that naive approach (at least according to the talk, I haven't looked