If you've booted up the PlayStation versions of Spelunky
or Jet Set Radio
recently, you probably know the burnt orange logo of the Blitworks
Co-founder and lead programmer Miguel Angel Horna actually gave a talk
at GDC Europe this week about the technical challenges BlitWorks faced in working with Polytron to rebuild Fez
to play nice with PSN. When I caught up with him last week to ask why, we wound up having a remarkably informative back-and-forth about how BlitWorks does business.
Developers, take note: Horna also shared some useful insight into how game creators can make their work easier to port to other platforms.
It's a notoriously troublesome process that companies like Unity and Epic are trying to capitalize on by expanding their engines to support cross-platform development. Horna and I also talked a bit about how that affects BlitWorks' plans for the future in the following email conversation, which has been edited for length and content.
Can you tell me about the history of BlitWorks, and why you decided to focus on port work?MGH:
What later would become BlitWorks started with a friend and me, both sharing long experience writing console and arcade emulators. This hobby work put us in contact with people at Sega and our relationship grew over time. At some point, due to our expertise with consoles, we were offered to do a port of the Sonic CD
remake that was already being done by Christian Whitehead. This was when we decided to join forces with other friends and start BlitWorks.
After Sonic CD
, we did a first build of Jet Set Radio
using our emulation technology and we demoed it to Sega. They were very interested in having Jet Set Radio
refreshed, but the emulated game required a lot of changes to reach current-gen standards (HD graphics, trophies, leaderboards, etc.) so we started thinking about another approach, based on the original source code.
With all the experience and knowledge of the Dreamcast we achieved doing its emulator, we built a new hybrid technology that allowed us to run the old code in current-gen consoles. So the final approach for Jet Set Radio
was halfway between using an emulator and a full port.
After Jet Set Radio
, we started getting more inquiries, so there was definitely a need that we were covering, and we decided it was worth focusing on game porting with the two values that made the initial ports a success: strong low-level programming skills and console knowledge, and a love for games that make us polish every port as if it was our own game. And this is how everything started.
What unique challenges did you face in porting a game like Fez to PSN?
Well, the most difficult challenge we had was that the original Fez
game was written in C#, and there was no C# support for PlayStation platforms when we started the port, so we faced the big decision of trying to port the whole Mono runtime or to convert the game to C++.
If we continued using C#, we would be hitting CPU performance problems, as even the original game on the Xbox 360 experienced some slowdowns. On the other hand, converting the game code (and Monogame itself) to C++ was a very long and tedious task, but it would offer us some unique optimization opportunities. As we wanted to achieve the best possible port, we finally opted for the C++ way.
Another big problem was graphics performance. Although it might look like a simple 2D game, Fez
worlds have a very complex 3D structure (pictured) -- there could even be pixel-sized polygons. We were using the PC version as a base for the port, and graphics performance was not a big problem on that platform as current video cards could already handle that workload quite easily, but the shaders and geometry caused us some performance trouble when running on previous-gen or portable consoles. So we had to rewrite some parts of the drawing code and optimize a few shaders to particularly fit each console.
The sum of both the CPU and the GPU optimizations has been the key for achieving the smoothness we reached.
What do you wish more developers knew about the process of porting games?
The porting process is usually approached in several steps, each one of them facing its own challenges:
- First: Making the game "run" (at least compile and somehow boot, usually with graphics call stubs) in the target platform. This step is sometimes complex and where we find the hardest issues, mainly due to dependencies on other libraries or middleware.
Sometimes games are built with closed-source tools, and if the tool doesn't support the target platform, we would have to start working on a remake instead of a straight port, having to rewrite the game in another language that supports the target platform.
Another usual problem is the use of middleware (especially for audio) or libraries. If they are closed-source, we have to make sure there is a version for the target platform. If they are open-source, we need to ensure there is a suitable license to use them, and that they compile and work on the target platform.
- The next step is adding proper graphics support, extracting the graphics calls to a common interface for all platforms. Games that are written with portability in mind will have this layer
"The number one showstopper in game porting is the usage of closed-source tools, engines or libraries."
already separated from the main code, but sometimes (usually games using OpenGL) direct graphics calls are interspersed in the code, and we have to extract them to a separate library, that we then implement for each target platform graphics API.
- When game and graphics are working, it won't usually be with desired performance. This is often true when porting from desktop to portable/mobile platforms. Games are built and tested for a specific platform, and when you move to a different platform, different challenges arise (CPU performance, shaders throughput, fillrate...) that can affect the game in ways that are hard to predict.
This is the most challenging part of the porting process, as you need a quite good understanding of the underlying hardware and how it's different than the one the game was originally built for, in order to find where the bottlenecks are and how to solve them. As we often say, there is no bad hardware -- just unoptimized software!
It's in this phase when using pre-built libraries can cause trouble, because there is no easy way to know what they are doing internally, and we can't optimize the way they work; that's why it's better to have all the dependency libraries in source code form.
- Finally, once it's all running and working at a good framerate, you have to deal with a host of small, and very time-consuming, problems. The most common are controls that need to be tweaked for the platform (touch, sticks, button remapping, vibration), UIs that have to be adapted to the screen size, resolution and input methods (for example changing a touch based UI to a controller based one is not an easy task.) Text and localizations have to be revised and updated to the platform naming conventions. Also with the resolution change, all texts have to be tested so they fit in their given space and they are readable in all resolutions.
Also, each platform has its own checklist of certification requirements that the game must comply with. Usually this requires changing the game menu flow or the saving system.
The first two phases can be simplified if the original game was written with portability in mind. This would greatly speed up the porting time spent in these two phases, but the last two ones are completely intrinsic to the game port. No matter how cross-platform or portable the game code is, they have to be done in a per-game, per-platform basis.
How can we, as an industry, improve the process of porting games between platforms?
The number one showstopper in game porting is the usage of closed-source tools, engines or libraries. Game developers should be aware of the technical decisions they are making, and how they will later affect portability of their game.
- Engine or library choice: In very small teams, it's usually necessary to use a pre-made engine or library in order to dedicate the programming resources to the actual game, but that could later impact portability of the game if they are closed-source. When developers choose a library they should bear in mind the availability of source code, or a pre-built version for the desired platforms (and support from the original developer).
- Language choice: Is it going to properly compile in all platforms? Will it perform well? Beware of "too new" languages with super features (for example C++11) that might not be completely supported in all platforms (as we have painfully realized).
Going through the subtleties of each language/framework/tool would need an entire article. Our point is that there is no silver bullet -- there is no perfect cross-platform language or tool. The
"As we often say, there is no bad hardware -- just unoptimized software."
most important thing is to know the weak sides of the system you're going to use before it's too late.
The second problem we usually find is resource usage. It's not a showstopper as you can always mitigate it, but it's good to have it in mind during all phases of the game development, for ease of porting:
- Use the memory resources sparingly. Just because PCs have multi-gigabyte memory and video cards, that doesn't mean you must use it all. Remember that some other platforms count their memory in megabytes and would later have great problems accommodating your game. And also, please, always remember to release your unused resources (textures, sounds) once they're not needed in-game.
- Keep the CPU usage under control. Each time you're using a library (physics, audio, xml parsing, etc), you're inheriting a lot of performance implications. Developing on a high-end platform can let you forget the limitations that you're carrying on. It's good to have low-profile devices around to be aware of potential performance problems as soon as possible.
- Try to make your code multithreaded from the beginning. On some platforms, a single CPU may not be able to perform all the processing, so it will have to be split in 2 or more threads, running on different CPUs. Having the code already multithreaded could save a lot of further work.
How do you think the rising popularity of engines that that can natively port across platforms (Unity, Unreal, etc.) is going to affect your business? Is it scary to specialize in porting games as more engine makers try to make it easier for devs to port games themselves?
Not at all! First of all, the sentence "This engine is cross-platform", does not imply that your game would automatically be cross-platform just by using that engine; there's a host of issues that can break the holy grail:
- Use of plugins: most of the time, games use specific plugins and some plugin vendors do not offer the same degree of compatibility as the tool itself.
- Use of platform-specific technical features: when developing a game there is so much pressure to complete that is very easy to forget about shader versions, texture formats or other features that may not be present in platforms that would have to be supported later.
If you're lucky enough and have not fallen into the previous pitfalls, then your game will run on the desired platform. You've gone through the first item in our previous list, but there are still a some more things to do:
- Adjust the performance or the visual quality.
- Tweak the input controls, sizing the UI, update localizations with platform naming conventions, etc.
- Platform specific certification requirements.
As we mentioned, these last steps is where we spend most of the time when doing a port, so these tools are not making us redundant at all, as there is still a lot of work to be done. And from the organizational point of view, some teams prefer to focus on creating the next game and leaving the burden of porting for a specialized team.
Anyway, the fact that those tools are not solving the full "cross-platform" problem (which we think is inherently unsolvable) does not mean they're bad. Actually, these tools are really good for indie teams, as you can start a project right away, without an upfront investment on the technology side.
The downside is that you're technologically in someone else's hands. Do you want to port your game to a platform not supported by your tool vendor? Is your game crashing on a specific platform? Do you need support for a specific feature no one else needs? You'll very probably be left out in the cold.
Luckily for the developers, inside this trend of cross-platform engines, there is an increasing portion of solutions offering full source code. Our advice is to take this into account when choosing an engine. This can be thought of as insurance for your future.