So now that you know what a TCR is, how does Compliance test them? Each TCR has a corresponding Test Case which is currently maintained as a 1:1 relationship for simplicity. The aim of each Test Case is to break the requirement out into its key components and act as an independent script, or set of test steps for testing the TCR.

Some TCRs/Test Cases have a very broad scope, influencing large areas of the game and requiring a large amount of time and attention to insure full and accurate coverage. Others are more specific and less resource intensive, but no less important, and dictate how a game should react when certain events are triggered e.g. when a controller is disconnected or loses power during gameplay.

The submission is assigned to a team of Compliance testers that deal with executing the Test Cases. The team will use a tailored set of Test Cases that match the supported features of the game. For example, if testing a single player game that does not support Leaderboards, there is no need to check Leaderboard support or Game Invitations, so these will be omitted from the test cycle. With that said, there's still a lot to check, and a large proportion of the games maximize the use of platform features, so Compliance can run at a frenetic pace, all the while maintaining test accuracy and coverage.

When all of the Test Cases have been executed and the dust settles, it's time to go back and perform a regression check for all of the issues that were found during testing before finally publishing the Compliance report. In the case of a Pre-Cert, the publisher will receive a Compliance Test Complete report containing all issues that were found, and that will conclude Pre-Cert testing.

It's a different story for Final-Cert submissions, as the results of the Compliance testing will determine the submission progressing into Functional testing, which you'll read about further in this article. Regardless of the outcome, a Final Compliance report is issued upon completion of Final-Cert testing. In a nutshell, the primary responsibility of Compliance is to enforce the TCRs, which are a set of core ideals for the Xbox 360 platform. Before moving on, a final word on TCRs...

The TCRs and Test Cases represent an important tool for publishers and developers; Compliance uses the exact same TCRs and Test Cases internally as those that are published externally to the team's website, the platform's online resource for their publishing and development partners. The aim of this transparency is to promote predictable results.

Functional

So you're now a little more knowledgeable about what Mastering Lab do, and much more familiar with Compliance and the TCRs, but what about the other area within XNA Game Quality: Functional. What exactly do they do?

Well their primary role is to mimic the average gamer, and find those gameplay issues that the likes of you or I might encounter during a typical gaming session. They get to see the games for longer, and also get to play them a lot more than the other teams. Sounds like the perfect job I hear you say...

Functional's methodology for testing games is comprised of two distinctly different but combined approaches; the Functional Test Cases (FTCs), and the Game Specific Test Plan. So to begin with, let's take a closer look at the FTCs, which are a standard set of checks designed by the Functional area. Every game must adhere to them, and their aim is to ensure that a consistent technical quality bar is met by all games released on the Xbox 360 platform.

Similar in nature to Compliance's TCRs and their Test Cases, and in some instances based on particular TCRs and Test Cases that warrant the extra attention, FTCs focus on the end user experience, making sure the games interact in the ways they're intended to with the key platform differentiating features such Achievements, Rich Presence and the in-game Xbox Guide.

Whilst this duplication may be viewed by some as repetition, the principal reason for this doubling of effort is down to the simple fact that Functional have more time and resources to test these preset scenarios far deeper into a game than Compliance. The FTCs also take on a very similar look and format to the TCR Test Cases, including a title, description, remarks, preparation and test steps.

The reason for this is to maintain consistency and familiarity for the user. In addition to being used in-house in the same way as the TCRs and Test Cases, the FTCs are also published to the team's website. Their intention is to make their testing practices and procedures as transparent as possible, and to demonstrate they only have the same guidelines, requirements and tools at their disposal as their publishing and development partners have.

As well as being alike in many ways, the FTCs are a little different from the TCRs and Test Cases in other ways. The FTCs focus on testing known issues and specific user scenarios that resulted in previous games falling foul during the Functional test process. In addition, the FTCs are used by Functional to test both the processed game files, and the retail disc based media, which forms the bulk of their testing, and provides a far more "real-world" test bed.

The FTCs are sorted by category, of which there are currently several submission types and supported features. By categorising the FTCs in this way, it means they can be used in a modular fashion, allowing them to be mixed and matched to tailor the requirements of any game. It's very important to Functional that the FTCs are kept to a manageable number, not only for their own purposes, but for their partners' purposes, too; which is why an average FTC test suite takes relatively little time to complete. It's because of this that Functional conduct regular reviews of the FTCs to ensure their accuracy and efficiency at all times. Only those FTCs that test for serious issues make the final cut, as it's in no one's interests to test scenarios that don't matter.