The review philosophy around how we look at value and the evolution of game design.
Any time we publish a review that takes a strong stance – positive or negative – on an anticipated game, we get a lot of feedback from our community. Beyond the unavoidable differences of opinion on the quality of a given game (there’s never been a review in any publication that everyone agrees with), there are interesting philosophical questions that arise around how we make our evaluations.
In the case of last week’s Crackdown 3 review, two often-debated topics were raised: Why is a sequel that’s so similar to a well-regarded original now thought of as uninteresting, and what consideration should be given to a game that can be played for next to no cost? These are questions we actually spend quite a lot of time thinking about, and so now seems like a good time to shed light on how we approach those questions and the reasoning behind it so that you can have a better idea of how we review.
Value, Like an Opinion, Is Subjective
The concept of “value” in a game is a nebulous and divisive issue when it comes to reviewing because putting a price on art is always a sticky proposition. What it often boils down to is the number written on the price tag that’s attached to what amount of content, which is usually a decision made by business people rather than the actual game creators whose work we appreciate. That makes the idea of judging the creative work by that metric seem incongruous from an art criticism standpoint; few, for instance, would hold the rising price of a movie ticket against filmmakers, and (as a more absurd example) none would let the Mona Lisa’s $830 million valuation color their opinion of whether it’s a masterpiece. At the same time, the price of a game is a hugely important factor to gamers who, more often than not, are not made of money and must decide between dozens of different games all vying for their attention.
Judging a creative work by its price seems incongruous from an art criticism standpoint.
That’s why it’s easy to understand where people are coming from in a common piece of feedback on our Crackdown 3 review, in which they say our score of 5.0 is too harsh for a game that would normally go for $60 but is alternatively available as part of Microsoft’s $10-per-month GamePass service, and thus very inexpensive for subscribers. It’s not an unreasonable argument: after all, while there’s a wide range of valid opinions on whether Crackdown 3 is good or not, one thing its fans and most of its detractors can agree on is that it’s better than playing nothing at all. For frugal gamers who aren’t into the free-to-play competitive scene of Fortnite, Apex Legends, and their ilk, that’s something to be valued and appreciated.
For our purposes at IGN, though, when we’re reviewing and assigning scores we strongly prefer to decouple price from our criticism of the games themselves (except in extreme cases) in order to emphasize quality over quantity. Our philosophy on this topic has to do with keeping our reviews relevant to a wide variety of perspectives. For example, someone who has a tight gaming budget – including most students out there – will be more inclined to dive into a game that’s not particularly groundbreaking or exciting, but is still reasonably entertaining and doesn’t ask for a lot of cash in exchange for many hours of entertainment. Meanwhile, older gamers with full-time jobs and families making demands on their time might have enough disposable income that the difference between a $60 game and a $15 game is negligible but their gaming hours are more precious. Anecdotally, they will often want the highest-quality experience they can find to fill those hours, even if it’s relatively expensive. Those two perspectives can lead to very different takes on how much a game should cost and how much that should matter in an evaluation.
Does a game’s score dramatically rise when it goes on sale for 75% off on Steam?
A close relationship between price and score also causes major problems because, now more than ever, games’ prices are in flux. Retailers like Amazon and GameStop (until recently) offered games for 20% off – should that result in a 20% increase in score? Does a game’s score dramatically rise when it goes on sale for 75% off on Steam? Are there different scores for a Microsoft game if you buy a retail copy versus getting access through GamePass? Should a free-to-play game never score lower than a 7.0 because you don’t have to pay for it at all? We think not, because getting a good deal on a bad or mediocre game does not make it any more fun or interesting to actually play.
It’s certainly true that lowered expectations can make a run-of-the-mill game more enjoyable by removing the element of disappointment. But we see it as our job as critics to help you to lower those expectations for games we find underwhelming, and the more we award high scores to games that simply provide more value instead of memorable experiences, the more we inflate expectations and create disappointment.
We hope that this philosophy keeps our scores useful to you even when a game goes on deep discount because, regardless of your financial situation, you’ll be able to use them as a guide for how we recommend you spend your time. Naturally, everyone has their own threshold for what they’re willing to spend and for what caliber of gaming experience. So when you see a game that we gave a 9.0 or higher drop to a low price, you’ll know you’re getting something we thought was an outstanding experience for a song; when you see a game we thought was just okay (that’s 6.0-6.9 on our scale) but you think might be up your alley for a pittance, trying it for yourself to see if you agree with us or not becomes much safer without the fear of a $60 disappointment.
What’s Old Isn’t Always New Again
As for the question of why Crackdown 3 is treated more harshly by modern critics than Crackdown was in 2007 despite closely resembling a (moderately) visually updated version of that game, it’s simply a victim of progress. Countless lessons were learned in the following 12 years, and it’s no secret that some game designs hold up better than others. While it’s easy to pick up a classic like 2007’s Portal or even 1991’s The Legend of Zelda: A Link to the Past and enjoy them as though they’d come out yesterday, those are the exceptions to the rule. Most games show their age in the absence of features, subtle touches, and quality-of-life improvements we’ve all become accustomed to that were introduced by either their own sequels or similar games they inspired.
The bar for design in open-world action games has been raised by leaps and bounds since Crackdown came out. Think about it: during that time we’ve seen a massive influx of games that includes not only Crackdown 2 but Grand Theft Auto IV and V; Red Dead Redemption 1 and 2; the entire Assassin’s Creed series (including 11 mainline games); all four Batman Arkham games; Watch Dogs 1 and 2, Just Cause 2, 3, and 4; Saints Row 2, 3, and 4; Infamous 1 and 2; Spider-Man; Horizon: Zero Dawn; The Legend of Zelda: Breath of the Wild; Far Cry 3, 4, 5, Primal, and New Dawn; Prototype 1 and 2; Fallout 3, New Vegas, and 4; Skyrim; Middle-earth: Shadow of Mordor and Shadow of War; and literally dozens of others. Some – specifically Spider-Man, Batman, Infamous, Prototype, and Saints Row 4 – are even directly comparable to Crackdown in that they’re focused on superheroes who run, swing, leap, fly, and zip around cities.
The bar for design in open-world action games has been raised by leaps and bounds since Crackdown came out.
Regardless of what you think of the quality of any of those individual games or series, it’s fair to say that each of them has had at least a few good ideas about how to create a large map and fill it with interesting, varied activities, and that the expectations for how densely populated and diverse a typical world should be have shot up. Game development doesn’t happen in a vacuum, and most game designers working on a new open-world project would play at least a fair number of their competition and be influenced by them, integrating the best innovations, concepts, and conveniences into their own designs. And having played a wide variety of those games that have all stood on their predecessors’ shoulders to some extent, it can be difficult to go back to a game that tries to pretend all of that never happened.
Crackdown 3, by many people’s judgment, is that type of game. In our review and others, its open-world map has been criticized as lacking in detail and liveliness, its mission variety as monotonous, its story as throwaway, its combat as simplistic, its weapon balance as tilted too heavily toward a few favorites, its visual quality as flat, its driving as weak and useless, its reliance on collectibles as too heavy, and its Terry Crews as criminally underused. All of those (with the exception of Terry Crews, I suppose) are relative complaints – not relative to the original Crackdown, but to its modern competition in its crowded genre.
To say that Crackdown 3 is “a good Crackdown game” and should be scored as such is to pretend every series exists in a universe by itself, wherein only games with the same name may set the standards by which a new one can be judged. The reality is that we all live and play games in a universe where that’s not true, and countless other games are out there to compare it against. So yes, Crackdown 3 is in some ways an improvement over the original, but if that same original game were to have come out today, it would likely fare poorly by modern standards.
Naturally, there’s no one “right” way to review and score video games, but to us, this approach makes the most sense at this point in time. As new and aggressive forms of monetization (like loot boxes and other microtransactions) are experimented with and generous services like GamePass catch on, the question of how price relates to the quality of games – which we consider to be works of art – remains one that will certainly be debated both internally at IGN and out there. As always, we enthusiastically welcome all respectful discussion on these and any other topics and encourage you to leave your thoughts in the comments.
Dan Stapleton is IGN’s Reviews Editor. You can follow him on Twitter to hear gaming rants and lots of random Simpsons references.