i judged the igf this year. it was a frustrating experience. i’m going to try and identify the biggest problems with the igf process and suggest some solutions. that’s if the igf is interesting in actually “rewarding innovation in indie games” (its claim) instead of simply being a press spectacle. the competition seems perfectly happy, at present, being a press spectacle.
when you, the judge, click on a game in the igf judge’s website, the first thing you see is a field in which to enter your scores for the game. below that is a screenshot of the game, the names of the people who worked on the game, the author’s personal description of the game, and a pile of serial numbers (in the case of commercial games) for judges to play the game. underneath all this, tucked at the very bottom of the web page, are the “judge notes”: the only avenue the igf provides for discussion of a game’s merits and value among judges. note, first of all, that this means a judge can score a game without even seeing what other judges thought of it.
there’s a link at the top of the page to the judge notes, yes. it says, “game doesn’t work? please leave a noe about it before scoring. we’ll check it out!” the judge notes are presented as, AND ARE USED BY MOST JUDGES AS, a way to report technical problems that keep the game from running correctly. this is a valuable function, and it’s important to have this function when fair judging is dependent on games actually working on a judge’s hardware, but the far more important function that these notes should be serving is to allow discussion to take place between judges so that a game’s values, merits and strengths can be assessed, argued, and some kind of consensus among judges can take place. the judge notes section of the site is MISCHARACTERIZED.
what i’d do: put the “judge notes” at the very top of the page, the first thing a judge sees upon clicking on a game. don’t call it “judge notes,” call it DISCUSSION. make it mandatory for a judge to post there at least once before voting. don’t open voting until a week before scores are due. force judges to actually participate in the process. don’t make it easy for them to shirk their duty.
i was assigned, at random, fourteen games entered into the competition to play and score. all the other games, though i was permitted to download them, play them, and leave judge notes (and i’ve explained how effective those are – often mine were the first, sometimes the only, notes on games i wasn’t assigned), i was not allowed to contribute to the score that decides their final standing in the igf. which means that whatever shmuck decided super meat boy is a more amazing and revelatory use of audio than AUDITORIUM i had no way of telling “no, you’re totally incorrect.”
there’s no reason to limit the number of eyes that look at a game. the games that are scoring the highest, in fact, are the ones that should be most scrutinized. and if i find value in a game that would otherwise be ignored by the judges, i should be able to express that value in the game’s overall score, rather than it being stoned to death for having drawn the wrong straw. if i think don’t shit your pants is a valuable piece of satire (certainly at least as much as that game about burning a rope that made finalist last year) or want to point out that, hey, edmund mcmillen (though he is a good friend bless his heart) has ALREADY WON the igf, my only avenue is to leave notes to the mystery judges who were randomly selected to judge those games (and which will probably be ignored).
i understand why judges have games assigned to them at random: so that each game is guaranteed to have at least a few judges look at it, instead of the most well-marketed games getting all the votes. but the ludum dare competition has a similiar system that works a lot better: games are randomly assigned to judges so that each game is guaranteed to have at least a few people play it, but judges are free BEYOND THAT to score any and every game other game in the competition that they play. then again, ludum dare has an advantage in that its judges actually care about the competition and its entries.
igf entries are rated in categories such as: excellence in audio. excellence in visual art. technical excellence. (remember when the categories were “innovation in” rather than “excellence in”? maybe they felt they were being dishonest.) why, as far outside of the big games industry and the enthusiast press as we supposedly are, are we still partitioning games like they do, as though a game’s graphics could be judged seperately from its worth as a whole? this is the independent games festival: are graphics and sound really the areas in which small creative authors and developers have the most to contribute?
true, there’s the “nuovo” award, the latest hasty attempt to patch the igf so that interesting games can actually have a place in a competition that claims to be about innovation (what was last year’s? the introduction of the “excellence in design” award?). why should a special case need to be created just so interesting games can compete on their own terms in a competition that ought to contain only interesting games? suggestion: expand the nuovo award to ten or so finalists. rename it “the igf.”
what if judges scored games in terms of actually being interesting instead of having the coolest graphics or gnarliest music? “how provocative is this game?” “how elegant is its design?” “how effective is it at telling its story?” “how well does it facilitate an experience between the players?” (yes, imagine that! an award exclusively for games for more than one player!) this is supposed to be the INDEPENDENT games festival. we can do this! why not? who are we beholden to?
the igf needs more perspectives. NOT more judges – more perspectives. it needs more people who do not share the same mindset. why even have more than one judge if every judge will value the same done-already physics game (joe danger) or bland, polished commercial title (cogs) the highest? why even have a competition?
to the igf’s credit, they asked me to recommend people for judges in the hopes of having a more diverse panel. against their credit, they failed to contact any of them. i recommended people like anne-marie schleiner, artist and activist, charles pratt, game design lecturer at nyu, librarian of their games library, and host of the another castle podcast, and scott nicholson, founder of syracuse university’s library game lab and published board game designer (and if you’re responding that board games have nothing to do with videogames then you’ve missed the point entirely). these are people with both academic backgrounds and design experience – imagine what they’d bring to the independent games festival.
independent games should have value not just to scene nerds, not just to gamers, but to people and players as a whole. why reward games based on how well they pander to their audience’s expectations when we should strive to reward them based on how much they have to contribute to the body and the study of games as a whole? why not? again, to whom are we beholden? what’s keeping us from making richer choices?
recognizing, of course, that many games live well after their first release and are continually added to, refined and readjusted based on the experience of their players. they’re living works. but if you’ve never released your game at all, i don’t think you have a place in the igf. the igf shouldn’t be a place for commercial titles to find publishers or to build their pre-release hype machines. as a judge, i refused to give any unreleased game i played over 75 points out of a hundred (knowing full well as i did that limbo would get in regardless).
the igf is a press event where we trot out our most mainstream creations in an attempt to get the mainstream press excited about them and, by extension, (in theory) about independent games as a whole. i would argue that that doesn’t happen, that the press only gets excited about super meat boy, and that the whole event exists only to serve those developers who are already entrenched in the business / marketing cycle. if the organizers of the igf are content with it being this way, then the only problem is that they’re dishonest about it, and that’s why i was so angry last year.
but my real anger, my frustration with the igf, is that it could be so much more than what it is. a celebration of the diversity of people who make games outside the big industry and outside of the mainstream. confront the people from the offices next door with genuinely interesting, provocative games and ask what’s taking them so long to catch up. we could prove, with the right selection of games, the value of our work for now and ever. but we don’t. if we’re truly independent, why do we settle for mediocrity?