One of the most important jobs of a games journalist has always been to honestly convey information to their audience, in as accurate as a way possible. With that the Daily Reaction crew of Seb and Dan go over the issues that surround having differing opinions when it comes to what is important for a game’s score, and why the power of public opinion can both hurt and help the process.
Dan: One of the things that has always been interesting has been the way we as an audience perceive the meaning of the number behind a review. It’s as if all games are only supposed to fall between some form of magical scale that must hold true for all titles, past and present. This number or letter becomes the steadfast to the grade, while the text behind it becomes a justification for the score itself. It’s something that is barely read, as the general audience looks to justify their predisposition on the quality of a title that most have yet to play. This is all the more apparent in big budget AAA titles like GTA V, where the internet would explode if the game was given a 5/10 by a reviewer, and the game isn’t even out. Public opinion as a whole drives a game’s score well before it is even released, as a games heritage and budget become factors that should not determine its true quality. Whether this is due to gamers wanting to justify their purchase, or because they need to feel like they are correct, is difficult to determine, but most do not understand the infection this causes to the industry as a whole.
Games journalists who set out to review a game are always going to fight an uphill battle to determine the validity of a title’s ‘score’, as they must contest with opinions of hundreds of other writers who are also hoping for some consensus. But this hope to fall along the mean of grades is a mentality that not only hurts gamers, as they are getting a tainted point of view, but also tears apart the fabric this industry relies upon. Journalists act as a median between the advertisers pushing out ‘the best thing ever’, and the audience who is simply looking to invest their hard earned cash wisely. So, if we start off with a false premise regarding what a game should score, we not only are lying to you, but we are also destroying the fate of something people have dedicated their lives too. Does this mean we should lie to save careers? Of course not, because criticism is the driving force to improve this industry we care about so much about, but we need to be critical in an ethical manner.
Once reviews start hitting the web, there is a public opinion that seems to gain a life of its own, as people seem to know where a title ‘should’ fall on the magical scale. Yet, as reviews are still being released long after public opinion has already been swayed, where are these reviews supposed to fall if they differ in opinion? Are journalists supposed to say ‘no, I liked the title?’ Even though the masses have determined the score, long before they were able to pass judgment? The answer falls in the integrity of the journalist and the publication they work for, as an opinion needs to be voiced, even if it goes against the grain. The only qualification is that, that voice, needs to be able to justify their opinion in a reasonable manner.
Seb: That’s the problem – there are different kinds of critics. There are those that don’t want to stray too far from the herd, there are those that have no idea what they are doing, there are those that try to be honest and, sadly, there are those that purposely court controversy for traffic. Unfortunately, it can at times be hard to tell the last two apart – sometimes a reviewer does actually think a game everyone else is giving a 10/10 is worthy of a 5/10. That’s opinion after all. But, giving a game like, say, Uncharted or Journey 5/10 will mean a ton of traffic. And that’s mostly why you see it happen. Occasionally someone will give an unusual review simply because they believe in it, but the majority of the time it’s simply hit-bait.
As for the reviewers that don’t know what they’re doing, there is a worrying trend of rushing to meet the embargo deadline – the date and time set by a publisher for when reviews are allowed to go live. I’ve been working at PSLS for over 3 years now, and it’s abundantly clear that if a review is published in time for an embargo it does exponentially better than if it comes out later. The thing is, however, publishers don’t always give you enough time to fully play a game before the embargo is up. Sometimes it is physically impossible to complete the game in time, even if you play it non-stop (and that’s not counting any time taken for eating/sleeping/personal life/actually writing it). So we miss the embargo, we lose out on a bunch of traffic and think “that’s annoying”.
Unfortunately, not every site is willing to miss out on that traffic, and neither are journalists that are paid per hit. Corners are cut, parts are missed, people are cheated.
EGM recently reviewed Aliens Colonial Marines, and were one of the first to go live. They gave it a 9/10, compared to an average review score of around 4/10. The problem is, with all these different kinds of critics, it’s hard to outright say that this guy is an utter hack who doesn’t know what he’s doing. While the review reads like an Amazon Product Description and suggests that he never actually played it, we couldn’t say that. It could be his opinion.
Usually, readers can decide whether a review is fishy (in this case they believe it was a paid review, although Sega should really have paid a few more people then…). Unfortunately, it’s not just the reader that suffers from an unjust score. Publishers want to incentivize developers to make a good game (obviously because it means both more immediate sales, and more long-term sales for the franchise) and so they pay bonuses to developers who make quality products, but they can’t always internally judge a game because the developer wouldn’t trust the publisher when money is on the line. The public also can’t fully be trusted because they’ll likely only buy a game that they already think they’ll like. So publishers turn to reviewers. We talked about Bungie’s Destiny yesterday, and the same court documents revealed that if Destiny scored 90 or higher on GameRankings (a website that aggregates and averages review scores), Activision would pay them a bonus of $2.5 million. That’s no small figure, and it’s a practice that tons of publishers adopt.
In theory, paying bonuses based on reviewed quality seems reasonable. But if you then take a look at some of the sites that GameRankings uses, the fact that it’s used to help determine the future of a developer is bone-chilling. Complete idiots that don’t actually play the game, or are simply looking for a quick buck by being controversial, are instrumental deciding factors for publishers when it comes to ensuring, or condemning, the livelihood of developers.
It’s not all bad, thankfully. I like to repeatedly point out the many games journalists and outlets that are downright terrible, but the fact that tons of publications that reviewed ACM came out with low scores for the embargo (which means they couldn’t have seen each other’s reviews), it’s clear that most games journalists do actually play the game and then review it.
But as long as there are those that abuse the system, and are still accepted by GameRankings/Metacritic and publishers, games critics will never be able to be taken seriously.
Dan: Ditto… Now I have to go and review ACM….hopefully I make the embargo date 😉
Seb: And we didn’t even get around to mentioning the whole Kane & Lynch/Tomb Raider scandal at GameSpot which highlighted the power of advertisers over some publications, as well as the possibility of bribes. Why don’t we ever get bribe offers? I feel insulted.
What is more important to you for a review, the score? Or the text? Is it ok for different sites to have majorly different scores? Let us know in the comments below what you think, and feel free to review us on Twitter at Seb and Dan, just know we don’t pay for reviews.
Be sure to email DR ideas, podcast comments and checks for “reviews” to [email protected].
