Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Gamasutra: The Art & Business of Making Gamesspacer
Scaling a Physics Engine to Millions of Players
View All     RSS
July 15, 2020
arrowPress Releases
July 15, 2020
Games Press
View All     RSS

If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Scaling a Physics Engine to Millions of Players

July 12, 2012 Article Start Previous Page 2 of 3 Next

From Centralized to Distributed Physics

Along with Assemblies, our shift from centralized to distributed physics has allowed more players and more physics simulations in a shared space. Roblox initially used centralized physics simulation, a model under which physics calculations are processed on the game server, and the results are constantly fed to players and reflected on screen.

While a centralized model works on a small scale -- meaning few players, few physics calculations or few simulated parts -- Roblox's implementation wasn't made to stack up against our growth and increasingly complex physics. The server must simulate too many parts and player interactions, causing motion slow down rather than run in real time.

We solved this problem by evolving our centralized engine in 2010 to a distributed physics engine.

This graphic is a simplified illustration of the distributed physics model. In practice, client-simulated regions are not necessarily uniform in shape.

Distributed physics distributes the physics simulation among the server and players. In the graphic above, the blue area represents the server (server-simulated region) and all its physical bodies and players. In the case that no player is within an area, the server handles the physics simulation and relays data to players, as in the centralized model.

The difference is that each player has a dynamic region of physical bodies -- those within close proximity -- that they simulate locally (client-simulated region), taking the strain off the server. Such distributed architecture allows us to scale to an almost infinite number of physically simulated bodies without degrading physics performance.

We implemented load-balancing algorithms to shrink or grow the region size of each player, based on their computation power and network bandwidth. Players with slow machines or internet connections will generally simulate fewer physical objects; the opposite is true for players with fast connections and hefty rigs. The key challenge behind distributed physics is simulating objects at the border of two players' simulation regions.

A strict division works in very simple circumstances; for example, an object is in one player's simulation region and not interacting with other objects. The player runs the rigid-body simulation and sends the results to the other players.

In a more challenging -- and very likely -- scenario, object A and object B are colliding with each other. Each object is in a separate player's region. Theoretically, we could let one player simulate object A and the other player simulate object B, then exchange simulation results. With an immediate network update, this works. But the network update interval is usually lower (e.g., 15Hz) than the rendering rate (e.g., 30Hz) to conserve bandwidth. Assuming our physics engine simulates at 30Hz, twice as fast as the network update frequency, the network update only occurs at every other step of physics simulation. This causes problems during high-speed collisions.

Here's an example:

A is a vehicle simulated by player A

B is a brick simulated by player B

The scenario: Vehicle A runs into brick B. Vehicle A bounces back as though it ran into a rubber wall. (We wrote an article about this phenomenon, for more background.) Vehicle A should slow down and brick B should at least get knocked over. The following steps illustrate why it breaks:

Simulation A (Vehicle A/Player A):

A touches B at high speed and generates contact force, separating A from B

A bounces back and moves away from B; sends update of A to B

A receives update from B; B is not moving

Simulation B (Object B/Player B):

B receives update from A; A is two steps away from B

A is one step away from B; sends update of B to A.

B receives update of A. A moves away from B. B is not moving.

Simulation B doesn't get a chance to witness the collision before Simulation A processes the collision and separates the vehicle from the brick. As a result, brick B behaves as though it was anchored and vehicle A bounces back.

For slow collisions or cross-boundary interactions, this situation does not occur because both sides have a chance to witness the interaction and generate a more believable response. Increasing the network-update frequency or reducing network-link latency will improve the results of high-speed collisions, but will not solve the problem: The two parties involved in a collision are not simulated on the same simulator.

Article Start Previous Page 2 of 3 Next

Related Jobs

Airship Syndicate
Airship Syndicate — Austin, Texas, United States

Junior to Mid Programmer
Disbelief — Cambridge, Massachusetts, United States

Mountaintop Studios
Mountaintop Studios — Los Angeles, California, United States

Engine/Systems Engineer (remote)
Mountaintop Studios
Mountaintop Studios — Los Angeles, California, United States

Graphics Engineer (remote)

Loading Comments

loader image