Our next question is from @DCharlieJP in Tokyo, Japan:
3. How aware are level designers of the limitation of the game engine? How is this factored in and/or communicated in the design process?
Steve Gaynor - Oh, very aware. The technical constraints of the engine define everything you can do as a level designer.
How it's factored in depends a lot on what state the tech is in -- if you're working with a stable, established engine, your constraints can be much more clear and top-down from the beginning; if the tech is still being assembled while the game is being designed, the dialogue between programming and design is more fluid, but can also be more uncertain and frustrating, if you don't know exactly what your constraints are.
But on some level, part of your job as an LD in this case is to help push the limits of the tech, and discover what it's capable of as well as what you would LIKE it to be capable of, in order to help figure out what the constraints will end up being when the engine does stabilize.
Jim Brown - If LDs aren't fully aware of their engine's capabilities, the project is at an extreme disadvantage. The last bit of polish at the end of any project is usually the most difficult -- and that's always expected -- but a lack of understanding that leads to building something entirely out of scope (or otherwise causes major redesigns) is unacceptable and wasteful; it can kill budgets, schedules, and careers.
You have to build within the framework of what your team and engine are capable of producing, and you have to keep those goals in mind even when prototyping. And of course, you have to ensure that project goals are aligned across the entire team.
With Gears, for example, we were just starting in on UE3, so we knew up front that we wanted advanced shaders and high-poly characters. As a group we agreed on a third person camera and close "intimate" combat distances to highlight those engine features. That obviously influenced design across the board, and had to be kept in mind at all times as it affected the number of enemies on screen, scale of architecture, and encounter scripting in big ways.
Neil Alphonso - Level designers need to be as familiar with the inner workings of their game engine as they can be, but the pace of technological change can make this very difficult! In the end, this is a responsibility that needs to be shared throughout the team; the tech leads need to provide guidelines for level and asset creation, the level designers need to provide a layout that can marry this with the environmental visual fidelity targets for the game, and the artists need to push as much quality as they can within that and still hit framerate goals.
Tools have made this somewhat easier in modern development, as automated processes can flag any problematic areas before it gets too painful to change them. Anything mechanically risky really needs to be addressed in a prototype well before production, because unless it is or becomes something that is used game-wide, the chances of development resources being dedicated to it for such isolated use are significantly lessened.
The next question comes from games industry veteran and Kabam General Manager, Mike Sellers:
4. Tools and Metrics: How do you know how players like the level, aren't getting lost, frustrated, etc? There are some good solutions for this but they're also unknown for a lot of people (even pros).
Joel Burgess - As early and as often as possible, get people in front of the level and watch them play it. Don't wait for the level to be polished or for your publisher/producer/whomever to arrange a playtest session. Grab somebody and sit them down with as little setup or guidance as possible. Encourage them to vocalize as they play. Then: Shut up. Don't interrupt, don't help, don't correct. Ignore direct questions unless absolutely necessary.
The unfiltered feedback you get from players will always be the best guiding light, and will often help you win internal arguments you had already been having.
Jim Brown - The simplest answer is to watch and pay attention. And while that sounds obvious, it's probably the most overlooked. It's not uncommon for designers to get too attached to their work. If you're too involved or too invested, you tend to lose sight of the bigger picture. A few years ago in my LDIAD talk I mentioned that the designer's job is to "be an advocate for the player" -- you can't just build things that you like, or lose sight of what the player's role is in experiencing your game. You're building for them, not yourself!
That said, usability testing, focus groups, heat maps, stat tracking, and any other number of analytical tools are incredibly useful and should be employed whenever possible. It's also well worth the time to read up on some basic psychology. The human brain is a crazy thing, and doesn't always work the way you would assume. Watch as other people play through your work, and keep an open mind. Getting into the head of the average gamer will make you a better designer.
Neil Alphonso - The lowest-cost method is simply watching somebody play, and diligently taking notes! Many studios even now use biometric data to help mine more useful information out of these sorts of tests. Tools for tracking metrics on a large scale have improved significantly over the years however, and can provide much more clinical information when a big enough audience sample size is provided. Valve's changes to Half-Life 2 and Team Fortress 2 that have been based on Steam metrics have shown that with enough actionable information, frustration points (or "shelf moments") can be lessened significantly.