Architecture

The Unity game server and uStream acts together as a single game server. uStream offloads the Game server and can utilize modern multicore processors to do packet routing to hundreds or thousands of players. The Unity game server is single-threaded, but uStream is highly optimized to run on multicore processors. This is why this setup is much more scalable than a setup without uStream. The following figure illustrates the network traffic between players and the game server is handled by uStream. The network traffic is uLink over UDP. uGameDB is not mandatory, it is included to give a complete picture of a “basic setup” with essential server side products.
The network communication between uStream and the Game server is uLink data packets over UDP. The network communication between the clients and uStream is also uLink data packets over UDP. uStream has one UDP port open for new connections from players, default is 4001. When a player connects, the server opens a dedicated UDP socket for the player and redirects the player to use that port. The redirect feature is part of uLink and it is triggered by the uStream server. The uLink client will reconnect to the dedicated port automatically. One port per player is one of the keys to get parallelism in uStream and thus effective usage of a multi-core processor. Another UDP port is open for the game server, 4002. The Game server is initialized as a uLink “cell server”. This makes the game server start a bit differently from a normal stand alone game server. The term “cell server” is borrowed from the Pikko Server architecture. When using uStream, a cell server is just a Unity game server. It is good to know this when using the uLink API since it includes the concept “cell server”.

Latency Addition

The latency addition for setting up uStream in front of a game server is a few milliseconds. The exact number depends on gameplay and server load, but 7-20 ms is a normal range during load tests when uStream and the game server is located on two separate hosts. We recommend running uStream and the game server on the same host to minimize the latency addition. With a setup on the same host, the latency addition will be even lower. Note: Better and more relevant latency numbers will be added here when tests have been done on the Windows release of uStream.

Game Type: FPS

uStream fits perfectly when you need high player density in a FPS game. The authoritative Unity server does physics and hit-box calculations and keeps track of scores and health. uStream is used to broadcast unreliable state synch, including position, health and more. Reliable RPCs are broadcast for events like hits, explosions and end of match. The area of interest feature can be used if the level is larger than the line of sight for players. And if the levels are small, turn off the area of interest calculations in uStream and use it only as a “mulit-core broadcast server” to offload the game server.

Game Type: Zoned MMO

In an MMO, you might want to add a zone with much higher density compared to other zones (sometimes called instances). This could be a city, a space station or a “battle area”. No matter why you need high player density for a zone, uStream will do the work. The redirect feature in uLink makes it very easy to transfer players between game servers. This makes it possible to build MMO games with “portals” between zones and if one of the zones is hosted with a uStream server in front of the game server, this transfer of players will work too. Read more about redirects, handovers, and sending RPCs between game servers in a peer-topeer network in the uLink manual. If a zone in the MMO game has a large map with lots of NPCs spread out over the map, uStream will help to reduce server load and reduce bandwidth. uStream will make sure a player only gets RPCs and state synch from the NPCs within the area of interest. The number of NPCs on a large map can be quite high with this setup, the bandwidth need per player will be OK anyway because of the area of interest calculations done by uStream. This setup doesn’t make the MMO world completely seamless, but the scalability of every game server is increased a lot (more NPCs, and more players). The next logical step for even more scalability is to use Pikko Server. Read more about Pikko Server at muchdifferent.com.

Server load

The exact number of players that can connect and get a “lag free” gaming session depends on several things, but the server load is one important thing to understand as a game developer. We always recommend doing load tests with a tool like uTsung to verify how many players and NPCs a setup with uStream can handle. The server load depends on the formula Load = (X_tot - X-invisible) * players * updates/s * objects size where X is the number of network instantiated objects with frequent updates. The same formula, a bit more in details: (total number of network instantiated objects - mean number of objects that are invisible for a player) * (number of players) * (number of updates per second) * (object’s mean serialized state size) The formula can be even more detailed, but the main purpose here is not go into all available parameters. Instead, this is a “high-level” reasonable formula for understanding the main factors that do generates server load. The server could be a Unity server with uStream or without uStream, the load formula is still the same. The big difference is that uStream can utilize all available cores on the host.

AOI and LOS

Two concepts are used extensively in the rest of this uStream documentation. Area of Interest (AOI) and Line of Sight (LOS). AOI is a circle around the player. LOS is also a circle around the player, but a bit smaller. All network instantiated objects will be considered by uStream and if they are withing the AOI the player will get state synch and RPC from these objects. The following figure illustrates Area of Interest and Line of Sight.
Network instantiated objects are usually player’s characters, NPCs and stateful scene objects like elevators and cannon towers. uStream does the hard CPU-consuming work to calculate what objects are within the AOI. The client does the LOS calculation. The LOS should always be a bit smaller than the AOI just like the picture. This is because moving objects (other players or NPCs) should enter the AOI first and then the Unity client will become aware of the objects’ fresh position and state. A few milliseconds later the moving objects can enter the LOS and then it will be rendered correctly in the client. The client code is responsible for doing do the LOS calculations. If objects are outside the LOS, they should not be rendered and they should not affect physics or gameplay. Be aware that if objects outside the LOS do get an RPC from the game server it will (and should) process the RPC and update it’s state. For example, if a door is opened, the server usually sends a reliable RPC to all clients and the state of the door game object will become “open”. This RPC should be executed in all clients no matter if the door is within the AOI or not. If the LOS is too big or game objects move too fast, the risk is that players will see game objects appear (pop into existence) INSIDE their Line of Sight. This is undesirable and it is the responsibility of the game designer to test the uStream server with the fastest moving objects in the game and verify that the AOI and LOS can handle the fastest significant objects. All state synchronizations are limited by AOI, but only some RPCs are limited by AOI. There is a new type of RPC available when using uLink and uStream. Put a plus sign (+) in the first character in the RPC name and the RPC will only be sent to players which have the object in question inside their AOI. We call them AOI-RPCs. Be careful when using state synch and AOI-RPCs with uStream. You can’t send all traffic from the server using these. Then the clients would miss some essential information. There are still many RPCs that need to be broadcast without AOI-calculations. For example RPCs like “WinningTeam” and “OpenDoor”.

What happens when all players gather in one place?

This is one of the first questions people ask when they hear about uStream. The answer always depends on gameplay. But two examples will shed some light on this challenging situation and how to handle it. First of all, use a modern multi-core server. The more cores, the better uStream will help you offload the game server.

Solution 1 - Set a player limit

This is the most common solution in MMO games. Set the maximum number of players per game server to a low limit. For example 20 players or 40 players. This way it is easy to test the load with the maximum number of players and verify that the server load and bandwidth need is OK for the planned hardware and bandwidth hosting cost. This also makes it easy to promise that players with a limited bandwidth will be able to play the game. NPCs are also evenly distributed across server to keep bandwidth needs for each player to a limited low amount no matter where the player goes in the virtual world. It is of course smart to only spawn and move NPC which are close to players, in order to avoid unnecessary load on the server. This is a safe “old-school” solution. With uStream the bar has been raised. It is now possible to set the player limit at hundreds of players or even thousands of players using this “old-school” solution.

Solution 2 - Gameplay limit for player density

Let gameplay reduce the chance of high player density or make a radical gameplay restriction that avoids too many players in one place. This can be done in a nice manner by giving extra bonus to players that move away from dense areas. For example, spawn new ammo packs only far away from player dense areas. Another somewhat nice trick is to remove all NPC from areas with many players. Otherwise there is always the option to to this in a not-so-nice manner. Teleport or kill players that try to enter a very player dense area. This is not very nice, but at least it keeps the server from being overloaded. Use this radical restriction only if players accept it and only if the nice manner restrictions didn’t help.

Solution 3 - Lower the sendrate

When player density is high, reduce the frequency for outgoing network traffic in the game server. This can be done in uLink by changing uLink.Network.sendrate. This solution might work very well in an MMO game with a crowded city where fights are not allowed at all. But it might not work at all in a fast paced FPS game that need fast updates.

Solution 4 - Lower the AOI

Make the area of interest for each player smaller when the player is in a high player density area. In the game, a visual effect like smoke or some kind of “fog of war” can be introduced to indicate that network instantiated objects in the distance are suddenly outside the line of sight. From the game server, it is possible to alter the radius for the area of interest for a player. This change is sent as a uLink internal RPC to uStream. This does reduce server load, but it is also good to use when a player is standing on top of a hill overlooking a large area. Then the players area of interest should be larger.