Sound innovation is pretty tough because few dev teams can afford to devote much time to sound unless they're making a strongly sound-themed game. In a game jam I think you get points for recording your own effects and composing your own music, even if they aren't especially innovative. As for quality, well, if you do create your own effects and music they need to sound decent, and of course effects should play when they're supposed to and harmonize with the theme of the game. As with graphics, completeness again means that all the sounds that should be in are in.
Gameplay innovation is obviously a high priority, but it's hard to do in a game jam because time is so short, and judges are likely to disagree about whether a game is innovative or not. I've seen a lot of games in 42 years of playing them. Quality means "is it fun?" and is another debatable point, since different players like different kinds of things; but this also includes the ease of using the controls. Completeness refers to the number of play features included -- several of our games started with plans for more different kinds of challenges and actions than they could actually get into the game.
Completeness sort of depends on another variable, ambition. If you say you're going to make something complicated and you only get a quarter of it done, is that better than saying you're going to make something simple and getting it all done?
I discussed this with some colleagues on Google+, and several felt that, at a game jam, innovation is by far the most important factor and completeness shouldn't come into it much. Another thought that game jams are effectively rapid prototyping, and the result should be judged like a prototype, for its ability to give a feel of the real product.
It's possible that I allowed myself to be influenced by my other work. As a freelance professor I spend quite a lot of time looking at student projects, and in a student project, completeness is critical. A student needs to build a portfolio of finished work.
Publishers and developers hire new graduates for their ability to get things done, not for their imagination. It's painful, but it's true. But a game jam changes these rules: in 48 hours, we can't reasonably expect too much, and it's a good opportunity to go a little crazy.
This raises the question of how many different kinds of events there are. From shortest to longest development cycles, I've judged two-, four-, and eight-hour game design (no coding) workshops; a 48-hour game jam; three five-day student development festivals; numerous multi-week student class projects; and some professional, commercial games for a major award. The lighthearted awards are definitely the most fun and require the least work (from the judges).
In the end we were helped by the small number of teams; we didn't have many games to argue about. If there had been 20 games and several that were really close together, it might have been difficult to arrive at a conclusion that we could all feel confident about. It was also good that our judges had diverse backgrounds.
But if I'm invited to do this again, I'd like to sit down with the other judges in advance to discuss our expectations about the judging criteria. We'll probably get it done faster if we know in advance what we want to reward. And of course, if any of the developers are taking the awards seriously, they would like to know too.