Hits Elements
Does the game contain the ingredients the author chose? Are they nice and central? Is the use of them interesting?
Tightness
Is the game tight? Does it have no distracting bits that get cut off in actual play? Is the setting completely supported by the system? Is the system completely enclosed by the setting? Does the game do what it tries to do?
Style
Judging on polish of presentation is unfair for a short project like this. But still, I want to see a game with flair and interesting stuff and ideas.
Completeness
Can I pick this up and run with it? I don't mean "Did you remember the 'what is roleplaying' chapter" (unless you're making a very non-traditional game), but I do want it to have enough rules to be able to play, enough description to know how I am to play it, enough setting to know what I'm playing, and enough ideas that as a GM and a player I can think of at least a few different ways things could go.
Fun
The most important, but hardest to describe category. Is your game enjoyable? Is it pure awesome?