|
[Individual game developers take responsibilities for different parts of game development - sometimes leading to content mixups and bottlenecks where their work overlaps. In this in-depth article, originally published in Game Developer magazine, Mick West discusses how collaborative editing may be the future.]
Game development teams are getting quite large. The largest teams can have more than a hundred developers working on a game at any one point in the process. Yet many developers still use work practices that revolve around individuals taking the entire responsibility for large chunks of the game.
For many things, such as the creation of the 3D models and textures for individual game objects, this is still quite reasonable. However, for the creation of large game levels and continuous game worlds, this can create problems and it is worth investigating methods of content creation that are more collaborative in nature.
This article takes a look at some of the issues involved, and discusses a few technical problems from the tool programmer's perspective.
Code Conflicts and Bottlenecks
When multiple people are working on overlapping aspects of a game, there are two problems that arise: edit conflicts and edit bottlenecks. These problems are two sides of the same coin. Fixing edit conflicts can create edit bottlenecks, and vice versa.
An edit conflict happens when two people edit the same thing at the same time and their changes create two conflicting versions. The nature of this problem depends on how fine-grained the "things" are in the game being developed.
Programmers have always had this problem when two of them work on the same area of code at the same time. If both programmers make changes to the same file, an edit conflict occurs, and it needs to be resolved.
Generally, programmers will not be working on the exact same line of code, but rather will work on separate tasks, which use code that overlaps in various files. When a programmer wants to make a change to a file, he or she will check it out from the version control system (VCS), edit it until the changes are working, and then check it back into VCS.
The problem of code edit conflicts can be handled by allowing only one programmer to check out a particular file at one time (this is typically a setting in the VCS software). Since only one programmer can edit the code at a given time, there can never be an edit conflict. However, we still get bounced over to the other side of the problem -- the edit bottleneck.
An edit bottleneck occurs when two people want to edit the same thing at the same time, yet person B cannot edit it because person A is editing it. With code, this is particularly problematic when large sections of functionality are incorporated in key files (typically with rather overarching names, like: "player.cpp" or "globals.cpp"), which affect many areas of the game and need many changes.
The common solution is to allow for multiple checkouts. Thus, more than one person is allowed to check out the same file at the same time. They make their changes, and then check the file back in. If the changes conflict, then they have to be merged. This step can usually be done automatically, but occasionally it requires a little manual intervention. Sometimes programmers end up editing the exact same piece of code and have to actually talk to each other to figure out a solution.
Some programmers prefer to structure things in such a way that multiple checkouts are never required. Theoretically, we can achieve this solution by making individual code files as small as is practical, splitting up files that contain multiple functionality, and establishing programmer procedures for rapid iterations, minimizing the length of time files can be checked out.
In practice a combination of efficient code division and multiple checkouts is commonly used. While edit conflicts can occasionally create problems, these can generally be mitigated by keeping a reasonable amount of functional separation in your code organization, and ensuring programmers check in (or merge) their code reasonably often.
The problem of edit bottlenecks can stop programmers in their tracks, or at least seriously cramp their programming options, and the occasional problem caused by an edit conflict is well worth the extra flexibility that simultaneous editing can provide.
|
Building a real-time collaborative game asset editor will take time, especially if you consider the computing, storage, and bandwidth problems that would be encountered in the process. In the meantime, it would be more practical to reduce the amount of edit conflicts by reducing the dependencies and increasing the orthogonality of modules (as fine grained module division is usually not enough), and by improving the communication between team members.
When you think about it, VCSs shouldn't take the blame for most of the problems in version control (unless, of course, we're talking about VSS :P ).
Consider two *entwined* editing sessions, then ask yourself how you are going to "undo" some changes. Do you undo your own edits? The other guy's? The other guy's that happen to touch the objects you touched?
I'm betting these hard questions would result in the same solutions we have today. Keep out of my backyard.
I understand that in Hero, the recommendation is to put up yellow police-tape around the area you are working in so these collaborative-conflicts don't happen.
Thanks for the thoughtful and balanced analysis of this problem. I'm trying to keep an open mind about it, but there is a lot of disagreement about where it lands on the usefullness scale. And what size team it makes sense for, what genre of game...
@Eric Slick and others, The article was actually written for Game Developer a few years ago, this is a reprint, so HeroEngine was not so well known back then. But it does sound just like what I was suggesting.
Different granularity comes with tradeoffs, as the article describes; with low granularity, conflicts are hard(people problem), and with high granularity, scalability is hard(computer problem). Note that DVCS mostly arises of the "people problem," and horizontal-scaling, non-relational databases arise from the "computer problem." Most of the obvious ground seems to have been covered by a VCS or database system in some form, so unless there's a radical improvement in our concepts, probably the best working practice is to go for the maximum level of granularity and then work out ways to optimize it, given that computers are cheaper than people.
I politely disagree. On the last page, you stress that "We are in the latest version. We don't need to check things in or out, as we simply start editing them, and they get locked."
That is in my eyes a very simple source control paradigm. Subversion-like systems perform better than that, and the concepts behind distributed systems such as Git are even more flexible.
Granted, those systems lend themselves better to code control. Merging changes to binary data is all but impossible in most cases, so in game development, the criteria are mostly different from many other software projects.