Managing An International Remote Development TeamBy Max Meltzer
As the game industry continues to grow at a phenomenal rate, so too does the desire by people around the world to join the industry. As a result, talent is appearing around the world in the form of international remote contractors.
By definition, an international remote team works in a different country than the internal development team. A benefit for the contractor is that it works where it wants to, possibly benefiting from lower wages than that in the client's country (and passing that value along to the client). For the client, using a remote contractor can bring special technical know-how to the project, an international prospective on the game, and often lower costs than hiring new full-time internal employees.
This article focuses on effectively managing such a team. Managing a remote team in another country is a challenge, yet it has been repeatedly proven that when used correctly, a remote team can be instrumental in hitting project milestones on time and completing the game on budget. The article uses Ubi Soft's upcoming cel-shaded first-person shooter XIII, and a previously cancelled Xbox Live title called The Ire Sequence, as postmortem-style case studies to illustrate the ups and downs of managing remote teams. I also provide information and advice from other sources to support the issues I will relate.
Integral Studios began developing The Ire Sequence in February 2001, and had a 12-month internal project schedule. The limited internal core team was based in Indianapolis, Indiana, and worked in collaboration with multiple remote teams worldwide. Remote locations included the United Kingdom, South Africa, Ukraine, Russia and India. The publisher, due to financial difficulties, eventually cancelled the game, and the assets of the game now belong to a California-based game developer.
Ubi Soft France, the developer of XIII, contracted Southend Interactive in Sweden (the same team that developed the Xbox game Deathrow) to work on some aspects of the game. Working remotely, Southend developed the multiplayer functionality and special effects assets for the game. Southend Interactive's team consists of nine people.
What Went Right
1. Planning our communications. The pinnacle of any project manager's success comes from the three "P"s: planning, planning and planning. It's no different with remote management, though there was one thing that increased the long-term efficiency while in the development of The Ire Sequence: a communications protocol among the teams.
Every project manager is required to be an excellent communicator, but making your voice heard in multiple languages (and holding it at a steady octave) can be tricky when you're separated from a contractor by thousands of miles. That's why it's important to develop a detailed and lengthy communication plan.
During the development of the communications plan for The Ire Sequence, we explained the importance of having a communication hierarchy diagram. The benefit of this diagram let us clearly display who should contact whom in a variety of situations. It's important to be open with your external teams, and by letting them contact specific members of the internal in times of need, they can get the information they need to create the game assets on time and to specification. The benefit of preparing the diagram meant it eliminated the time required to relay technical problems to the right people. Likewise, the people with the answers were only contacted when their expertise was needed. This saved time, since technical questions weren't shuffled around in search of a person to answer them. From a project manager's point of view, enabling specific team members to display their expertise can help nurture the leadership and communication skills of talent. It also meant, in my case, that I didn't have to worry about the team wasting valuable time, nor spending my time sifting through feedback and passing it on to the right person.
On The Ire Sequence, one of the development teams we worked with was based in the Ukraine and only had a limited English vocabulary. So providing this team with a full communications plan in English was of limited value. As required though, we dealt with a project manger who spoke both fluent English and Russian, and we made it his role to translate the entire communications document and pass on to his team. (Note that a professional translator could have done this work, but we lacked the resources.)
We soon found out that this communications document became the team's bible. To a remote team, communication becomes almost the highest priority, just below doing the work itself. Each team member was provided a copy of the communications plan so that everyone knew exactly what to do when they encountered technical problems, design issues, problems with tasks or even basic questions related to the game. This made their ability to develop assets far more efficient, and brought my personal project-management style across the border. It also meant that both the core internal team and external groups ran under the same documentation and generally on same schedule that I set.
It wasn't until a rainy day in July (I was in England at the time) that we truly understood how important the plan had become to the team. The project manager of the Ukrainian team had fallen ill as we headed for a key milestone. As specified in the communication plan, my fallback point of contact was an artist on the team, who spoke good English. While the artist did his best, it was incredibly difficult for me to communicate ideas to the associate producer and rest of the team, and I feared that the team's progress would slow. Yet, when I returned the following morning and the project manager was still sick, I was pleased to discover all of the required assets from the team had been completed. Technical problems still came into our team, sometimes from developers who were handy with a Russian-to-English web translator, sometimes from Ukrainians who could speak some English, and sometimes from Ukrainians who could write English but not speak it. The team as a whole was not phased by the lack of communication while their project manager was sick. They simply followed the translated documents that displayed and described what they should do if the project manager wasn't available.
The communication guidelines continually played a large role in facilitating interactions between teams. It enabled the remote team to do their job under practically any conditions. One of the Ukrainian programmers even mentioned that his team followed the guidelines like a bible, making sure the correct people were contacted, helping everyone with tasks like uploading assets, and so on. He also told me that the communication plan made him feel as though he was part of the entire development team as a whole, since it made him aware of what to do and who to communicate with in any situation. There was a time when I thought that my distance from the Ukrainians would make it difficult to share my vision for the project. But I was wrong - documentation is the key.
Sketched Logo for cancelled Xbox Live title, The Ire Sequence.
2. Establishing a non-standard working schedule. One of the most common reasons game development companies reject the use of international remote teams is because of the impact it can have on working hours. When a contractor is many time zones away, the time difference can be extremely difficult to handle. On the other hand, some contractor teams don't work until late at night, so even if they are just an hour ahead, their working patterns may disrupt the way your internal team wants to work.
Steve Powers, Senior Game Designer at Ion Storm, has this to say about working with remote contractors and teams:
"When I have worked with virtual employees in the past, some of them have migrated to night-owl hours or worked at off-peak times. This complicates communication when schedules begin to diverge. For example, we would arrive at work in the morning, find that an off-site programmer had checked in broken code, and would not be available to repair it until late in the day. As a result, our local developers would be paralyzed during that workday".
This experience is not uncommon, and illustrates problems that can and do occur when using remote teams. So, how do you solve them? It's simple. As the Chartered Management Institute (http://www.managers.org.uk/institute) suggests, all work performed externally must suit the needs and wants of the internal team -- and that's exactly what you have to insist on in these situations.
As this relates to The Ire Sequence, we needed a team that would adapt to our schedule. This could have meant earlier or later starting times, or even in some cases, normal working hours in preference to off-peak times. A project manager must remember that the subcontracted team must suit the client's needs. You should only take on a remote team that is willing to work to your required specifications.
Fortunately, adaptable teams are much easier to find that you might expect. For example, the core of The Ire Sequence team was based on the East Coast of the United States, and we began working with a team from the United Kingdom. The UK-based team, we understood, enjoyed working at off-peak times and in relation to our internal team, we preferred them to do so as a group. So every day around noon, the British team awoke, had breakfast and got to work at around 3:00 pm -- about 10:00 am for us. Sometimes working odd hours like this can destabilize working habits and inhibit socialization, but the UK team was extremely efficient. Many of them were used to working these hours and were great at communicating with our team. The overlapping work hours meant that even over international distances we could efficiently communicate to solve problems, and neither team was left paralyzed at any point.
Sometimes the remote team doesn't need to adapt to extremely different working hours - small schedules shifts are sufficient. As Daniel Jeppsson, co-founder and lead programmer of Southend Interactive suggests, "Four shared hours [between teams] per day could be enough unless the work itself is closely connected…The remote team should also have their own tasks so communication could be cut down to a minimum." Asking the remote team to start work just an hour or two later is unlikely to be rejected, yet could give both teams more time to communicate on technical issues.
If you can contract a talented remote team that's willing to adapt to your project's needs, you should seriously consider using that team. Conversely, no matter how talented a remote team is, if it is unwilling or unable to accommodate your working hours, it will ultimately affect your schedule for the worse.
3. Assigning tasks that resulted in visible successes. A basic tool used by businesses is that of motivation diagrams, which indicate that self-appreciation and realization are the pinnacle of personal success. So you should try to achieve these feelings with external teams, too.
Daniel Jeppsson of Southend Interactive believes that the way Ubi Soft handled the scheduling of XIII significantly improved his team's drive to succeed. He noticed on his team, and within himself, that the way development tasks had been scheduled enabled Southend employees to feel appreciated, even though they weren't publicly recognized for their work on the game. Jeppsson believes early preparation of a schedule that factors in the remote team not only improves the remote team's efficiency; it benefits the employee morale and financial picture of the client.
"I think it is important to have clearly defined, separate goals so the remote team can feel proud of their part of the job and can easily verify that the job is actually being done," Jeppsson says. It sounds simple, but it's an important reason why Southend Interactive is successful as a remote team.
Many companies contract remote teams to work on a variety of assets that make up a small percentage of the overall collection of assets of a single objective. For instance, on XIII, Southend had specific objectives like creating the multiplayer aspects for the Xbox version. Those tasks made Southend employees feel involved and recognized for their work. Obviously this is good for the remote team, and it also played a role in reducing bugs in the game. For instance, if there were any bugs related to the multiplayer aspects, Ubi Soft would contact Southend Interactive directly. Bug reports were easily assigned to the right people on the right team. This is much better than a situation in which a remote team is working on a variety of assets for separate objectives, which can make finding the right person to fix a problem a lengthy process. I caution you against calling in a remote team to work on random "bits and bobs".
on, when Ubi Soft's XIII design team decided the game would include
multiplayer, the company quickly crafted a plan to use a remote team to
work on these aspects, and chose a team that had the expertise to do so.
With this schedule they clearly defined several objectives for the remote
team, which was successful. Once again this underscores the importance
of early and effective planning. Make yourself aware of the capabilities
of the remote teams in the early stages of development - it will help
you better prepare for the project and pave the way for a successful relationship
with the remote team.
Sketch of races available to player in The Ire Sequence.
4. Proper risk assessment. With every project comes risks. During the development of The Ire Sequence we encountered several unexpected problems while working with a remote team. When analyzing the project after it had finished, I believe we had to follow the same course that Ted Price of Insomniac Games stated in his Ratchet & Clank postmortem: "We had to fail to succeed".
I cannot stress enough the importance of analyzing the basic, critical issues related to using remote teams. As a project manager, you must identify potential problems that could arise with the external team. The difficulty is determining whether the remote team, no matter how talented, can succeed at your project. For a remote team to work well, you must assess their talents carefully so that your workflow and communications succeed from the beginning.
When working with a team in a foreign-language country, naturally it's critical to have a bilingual project manager on the remote team that you can work with. But that person also needs to have technical skills and be proficient at project management, too. Finding someone who meets all of these requirements is hard, but setting goals for what you want in each remote employee is an ideal way of culling prospects. Indeed, setting individual goals for each remote employee will give the remote team's management the opportunity to select the best team members for your project. You'll also be able to quickly establish exactly what you want from the remote team. Once these basic objectives are identified, you'll be able to narrow your field of candidates based on their budget, project schedule and team skills.
5. Effective communication. Communication is a recurring subject. It cannot be understated how important good communication processes are when managing a remote team. Additionally, initiating communication in the right way can be just as important as sustaining it, and you must invest in the right equipment and resources as needed.
I'm a strong believer in the potential of virtual development, in which everyone works remotely, usually from home. Virtual development is a development method, which has been used to make several retail games, including Tactical Ops. But even in an extreme like this, production will suffer if face-to-face communication doesn't occur at some stage. Face-to-face communication is a good way of creating and sustaining a friendly and a creative work atmosphere, even over large distances.
Serge Hascoet, Executive Director of Worldwide Content Strategy at Ubi Soft recently wrote an article describing how the company's creative studios benefited from face-to-face conventions. Similar communication techniques can be implemented with remote teams. Hascoet explained that Ubi Soft occasionally held an event for developers known as the "Academy of Experts". The events let teams share experiences and learn more about the production process. The days help create a sense of communal belonging and provide creative inspiration for future games. This is the sort of event that should be considered when working with remote teams. Even a meeting at E3 can be enough to boost communication and provide a common experience that's needed to support a good working relationship.
Initiating work in a friendly environment is a great way to build a relationship. With regards to XIII, Ubi Soft's regular face-to-face interaction with Southend Interactive lead to a successful relationship between the two companies. In fact, none of the Southend employees never felt as close to a publisher as they did with Ubi Soft throughout the entire development process, which inspired them. Daniel Jeppsson of Southend explained that his team and Ubi Soft communicated "…mostly by mail, but also by phone and sometimes people flying back and forth. We have patience and respect for the other parties' work, I guess. Like always in a creative business, people have egos and feathers can easily be ruffled unless people get to know each other [face-to-face] beforehand."
Jeppsson said the regular meetings with Ubi Soft quickly established a mutual respect of the teams' creative and technical abilities. It created an atmosphere that allowed both teams to communicate effectively, even during the early critique stages of the project. It broke the ice between the developer and remote team and created synergy between them.
Inter-team communications can also be handled via the Internet, as mentioned earlier. In particular, online forums are can facilitate a sense of community between the primary developers and remote team, especially if you're low on resources. (Although it will never compete with verbal communication for building relationships.)
Integral Studios used forums to communicate worldwide during development of The Ire Sequence and Serge Hascoet (Ubi Soft) recently said that the company's worldwide development studios use forums to communicate technical problems and share ideas. So whatever the size of your project or company, there are many tools that can improve the communication between far-flung teams.
Real-time image of The Ire Sequence terrain environment.
What Went Wrong
1. Travel expenses went over budget. While developing The Ire Sequence it was occasionally necessary to visit various remote teams. Often these were routine trips during pre-production and production, but there were also trips made during emergencies.
At any given time we were usually working with at least two remote teams, but the project was not budgeted for so many visits. As a small development team, the rising costs affected us. As a result, we had to forfeit specific assets to meet the schedule, potential hurting the game's quality.
To address the problem, we reduced our travel and began leveraging online methods for communicating, such forums. We also worked with the developer to create a schedule that included regular visits and took emergency travel into account. We also devised methods for solving some potential emergency situations. By doing so, we were able to cut down on costs by booking flights in advance.
Overlooking a full plan dedicated to scheduling and budgeting travel in relation to working with a remote team is a big mistake, as we found out on "The Ire Sequence".
2. Lack of passion. As I said earlier, passion was something we looked for in remote teams. A team that is passionate about playing and developing games will most likely produce the best assets. Those without passion are more than likely to lose interest in the project, lack determination, and fail to put in the hours required to reach its objectives.
While working on The Ire Sequence, we contracted a team that had a strong background in developing remote assets for several moderately successful titles, including Cossacks. When we visited their studio, we met some very intelligent and creative people, some of which had the passion and ability to work at some of the world's greatest games companies. But something we overlooked was the effect of that team's growth and employment searches. Only a month into development, the external project manager left for a larger company and was soon replaced. Soon afterwards their team brought in a new project manager, but we started to feel the effects of a project manager who had no passion for games. While it's true that a non-interested manager can succeed, problems began to arise that hadn't previously.
So we consulted the remote company and reiterated the goals we had set for them, and told them that the project manager had not met our requirements. We tried to keep in mind that while we had the right to demand whatever we wanted in relation to our project, we knew we couldn't tell them how to hire their employees.
It was when the company began to lay off talented workers that we had to draw the line. We told them that we wanted to be consulted about whom they were hiring and removing from their team, but language difficulties became problematic at this point. It was about then that we realized how instrumental a bilingual person on our own team would have been. But we knew it was our fault to have contracted a company that was becoming increasingly interested in web development rather than games (as they told us). As this was our first time working with a remote team, it stung us in ways we hadn't felt before. Yet it gave us the impetus to work with a more professional team, using knowledge and experience we gleaned from our prior problems, and understanding how to prepare for future incidents like this.
In-game screenshot of Ubi Soft's upcoming cel-shaded first person shooter, XIII.
3. Asset management problems. With every game development project, asset management is an important yet potentially difficult subject. During production of The Ire Sequence, we encountered problems while trying to share assets with the remote team. We began by using an old-fashioned method of sharing assets: via standard mail. The first problems that occurred were mailing the wrong assets, wrongly addressing packages and simply not sending the packages. But we also mishandled the assets once they arrived -- some packages were lost and/or went unnoticed for weeks.
We soon turned to developing procedures for handling mailed assets, and developed specific contingency plans for handling such incidents. But it didn't take long before we decided to migrate all of our asset management online. This meant reserving funds to purchase equipment for remote studios to upload assets to via FTP. This system gave us one place to store all our assets. The only problem was the speed at which it took to upload and download assets, some of which updates were gigabytes in size.
During the development of XIII, the remote team used FTP. The only major pain was uploading and downloading a 5-gigabyte file. Using online capabilities, it doesn't take long at all to prepare a steady asset system. I suggest using one at all times when working with a remote team.
4. Asset implementation setbacks. Whenever Southend Interactive uploaded their newly updated work, Ubi Soft had to download it and run regulatory checks. They would then compare it to the older versions and take the necessary steps to integrate, for example, new sections of code. There are two problems that can result from this process, which you should take into account if you're considering a similar method:
the amount of time that will be needed to integrate the new assets.
It is likely to take some time and may even become a full-time role
for one of your employees. For example, consider how much time is needed
before a new version can be delivered to the remote team when working
on a game for multiple platforms.
- Make sure to schedule periods when the remote team can work on assets that don't rely on an updated version of the game. For instance, Southend Interactive had to wait at least a few days before their uploaded assets were verified and Ubi Soft sent them a new package with the latest version. During this waiting period, Southend needed to work on other assets, but there were no difficulties because we had foreseen this problem.
5. Over management and under management. Who holds the ultimate control within the contractor-client relationship? The company that signs the paychecks, of course. It's important to initially have unwavering respect for each other, but there are times when it is necessary to step in -- and step out -- in relation to management issues.
For instance, we considered when to travel and when not to. Yes, face-to-face communication is important, yet when is it best to visit? Is it better to schedule travel meetings to assert authority or build relationships? From my experience working on XIII and The Ire Sequence, and talking with others in the industry, I've made a list to help schedule travel and meetings. (The following guidelines do not account for unforeseen emergencies within each phase, of course).
& Travel Priority: Highest.
Scheduling: Plan to travel and conduct meetings regularly.
Typical Reasons To Travel: To create a relationship with the remote team. You need to ensure that the contractor understands your project's scope and design. Meetings should be used to plan the best and most efficient methods to achieve your goals. Pre-production could be, and in some cases should be, the longest period of development. Consequently, people should be as ready for a significant number of meetings during this period.
Full Production Phase:
& Travel Priority: Average
Scheduling: Moderate amount of travel as circumstances warrant.
Typical Reasons To Travel: To make sure everything is running smoothly. At this stage you are working with the remote team's project manager to analyze the development schedule and any risks involved. You should also travel to help get the contractor (including the project manager, if necessary) back on track when development is lagging.
& Travel Priority: Low
Typical Reasons To Travel: To establish the QA process. Try not to schedule too many meetings during this phase; especially ones that involve programmers, since they're working hard to find and fix bugs and it's better not to have an employer looking over their shoulder.
The importance of meetings at each major phase.
The Bottom Line
Using international remote teams can potentially provide a fresh cultural prospective on your game's development, you can discover amazing talent in far-off countries, and it can be cheaper than hiring new internal staff. Both XIII and The Ire Sequence were able to draw from pool of talent at a lower cost, and saw no adverse effects in their quality. If you can source a team that's skilled and ready to work to your standards, you should seriously that option. It won't be long until more companies appreciate the potential of international outsourcing and leverage it.
Return to the full version of this article
Copyright © UBM Tech, All rights reserved