Mistakes Critics Make (and Solutions)

It’s foolish as a writer to criticize the critics. I know this. But regardless of what happens to me after this essay, it’s hard to argue with the observation most reviews of most things are not very good.I suspect even most critics will agree, since, after all, they’re critics and one assumes they’re as critical of each other as of the rest of us.

Critics and reviews on the web seems worse than print, at least in part because of volume. Now everyone is a critic on amazon.com or yelp.com or on their own blog. But even in old-school print media, in the past and the present,  there are a small number of frequent mistakes many critics, amateur or professional, tend to make.

These include:

  • Confusing proving how smart you are, with helping the reader
  • Assuming the reader has, or exclusively wants to hear about, your tastes
  • Forgetting the reader may not have read, heard or seen what you have
  • Slamming things because you don’t like them, rather than because they’re poorly made

I’ve written about how to give good criticism before, and care about this topic because I care about the future. A good review, in the sense it’s constructed well, is a bet that the reader will be smarter or wiser after reading it, and making more informed choices in the future. Criticism and reviews of any kind are noble, as the motivation is to help people find the “good”, point them away from the “bad”, and teach them to think for themselves about the difference. Good criticism is a way to teach people aesthetics. Yay.

But good criticism is rare because it hinges on the critic having a strong sense of themselves without forgetting to put the reader first. If a critic loves chocolate, but the reader loves vanilla, a good critic can covey the vanilla-ness of a cone of ice-cream sufficiently well for vanilla loving readers to realize they want one. Since a critic doesn’t know his reader, s/he should be assuming many different kinds of readers and writing with sufficient care so they all think he’s writing for them. For example, I was ready to leave the film 300 after 20 minutes (it was amazing for that long, and then suddenly quite boring for me) – but my experience wouldn’t prevent me from recommending it to young men aged 13-21 of any sexual preference who don’t mind fantastically produced cartoon violence, as it’s the best film ever made for them (as perhaps Conan the Barbarian was when I fit that profile decades ago).

Why review averages are often useless

This begs the question of the 4 star or 5 star review – as what does that mean exactly? Most people give these scores based on how much they liked it, rather than any sense of how much they expect anyone else to like it, meaning the raw average of 100 reviews doesn’t tell you very much. It tells you an average of how people who may or may not share your preferences liked it for themselves, and if your tastes are not like the average of those people, that score is of limited use. For vacuum cleaners or toaster ovens it’s safer to assume most people have the same tastes, but the closer you get to the arts, the less that’s true. In some cases a movie or book that gets all 1 star and 5 star reviews, indicating something bold and strong in one direction, is a better bet for my enjoyment, than one that gets all 4 star reviews (a distinction hard to make on some websites).

Some highbrow reviewers, like those for The New Yorker or The New York Times, use the guise of reviewing a film as a way to explore aspects of culture and history. This is fine, provided the reader gets what they want: some sense of whether they want to see the film or not. If they don’t get that, don’t call it a review. Call it an essay, or an article, or something.

Can a review elevate the reader?

The ideal review should compel me to see, or buy, something, even if the critic hated it, if it in fact would satisfy my different tastes, without disparaging me as the reader for that divergence in taste.  I might just happen to like cheezy pop dance music. It’s possible I have learned how to enjoy Michael Bay films, in full awareness of their flaws. A great critic, living up to the term critique (the art of discerning),  will identify the core characteristics with sufficient clarity than they are almost reporting on the film, or book, or product, rather than on it’s ability to match their personal preferences. An exceptional critic helps me widen what I might enjoy, expanding my range of what I’m capable of enjoying. Or at minimum, articulating what it is I don’t enjoy so I understand it and myself better.

If a critic is truly a master of their trade, whether it’s art, or film, or food, they should be fluent in a wide range of genres and styles, and recognize what is good or bad for that style, independent of whether they like, or do not like, that kind of work. If they can’t do that either they don’t know their stuff, they don’t respect other people’s tastes, or they’re writing for themselves and not their readers.

Why friends make for bad reviewers

For this reason it’s often useless to ask a friend who hates horror films what they thought of the new horror film that came out. If they saw it, they’re likely to tell you not to see it because they hated it (and aren’t you just exactly like them?) It’s a common social response to assume friends share tastes in everything, but it’s not very useful here. The equation of a (what was the goal of the film) + b ( how do my film preferences match yours relative to a) is never considered, and that’s a core equation you need to get data on the thing, which is what criticism is supposed to do.

It’s also often pointless to ask a friend who is a connoisseur of mindless action movies what he thought of the latest Jason Stratham film when you’ve never seen his work before, as he’ll tell you how annoyingly derivative it is of his other work, work you’ve never seen, and therefore wouldn’t even know what you’re supposed to be annoyed by. Alternatively, watching Quentin Tarantino post Pulp Fiction has been largely an exercise in seeing films made by a brilliant filmmaker who happens to love obscure and mostly bad films, trying to make a good film that will work or not mostly based on how many bad films he’s referencing that you have seen. I found Kill Bill hard to watch, but with a different film viewing history, I suspect it would have been different.

The secret thing critics never say

The taboo subject among critics hidden in my dislike of Kill Bill is this: I saw it alone, in a bad mood, at a empty theater, in a matinee showing, and that may have changed my opinion ( compared to having seen it with merry friends after a few drinks). It’s obvious a person’s mood and mindset changes their experience of everything, yet critics and reviewers never, almost as a rule, include their personal context. If you’re in the right mood some art can work for you, if you’re in the wrong mood it will seem like trash no matter what it is. But this important variable is never stated, leaving you to assume the critic is immune to the opinion shifting forces of mood. This includes reviewers explaining their expectations for the thing before they experienced it. If you expected a McDonald’s happy meal and got a five star meal, you experienced it differently, than if you expected a five star meal and got McDonald’s. But this is information, the reviewers expectations which the movie/book/thing is being judged against, is rarely shared, allowing everyone to pretend the reviewer is somehow more objective than the rest of us.

Reviews as revenge!

On the web, sites like amazon.com, yelp, and tripadvisor, have made everyone a critic, for better and for worse. It’s a boon to small businesses to democratize this sort of thing, as anyone can vote (and I’ve been one to ask people to vote via reviews myself). But pure democracy can lead to mob rule, at least in some corners. Clearly some use reviewing as venting, writing as if they are the only people in the world. Their reviews are their way to seek revenge for a bad meal, a ruined evening, or a tough night sleep. And I always wonder: why do they never comment on how everyone else at these places seemed be doing? Was everyone else as miserable and mistreated and furious as you were? If not, perhaps their experience was uncommon, or they’re pickier about some things than others, both important variables to note in a good review. Reading the reviews on these sites is an exercise in detective work, trying to extrapolate from the review to what actually happened, and how trustworthy the reviewer is compared to the other reviews. Some are obviously written by insane, angry people who don’t write well, while others seem only to know how to give 5 stars reviews to things.

Over on amazon.com, reviewers often forget to take into account potential readers may or may not have read as many books on the subject as they have. And if a slam is in order, which does have its place, why is it so rare to see a sentence or two to recommend a better book or movie of the appropriate kind given their wide ranging knowledge? (It’d seem amazon would want this for 2 or 1 star reviews – at least they might buy something else) The reader doesn’t care which project management book they read, as long as it’s a good one for them, so why not help them on their way? I haven’t studied it too closely, but for all the systems I’ve seen, it’s very hard to structure a system that rewards the right kinds of contributions in reviews in the right way. It’s too subjective, for the recursive problems of reviewing reviews.

Tricks with suspension of disbelief

As I’ve grown as a writer I’ve increasingly tried to put my imagination cap on when watching movies, plays or TV shows. Whenever my suspension of disbelief wanes, or my active brain cells feel offended for being active, I put my writer cap on. I invent some reason, or explanation, or workaround to explain the offending gap flagged by my critical brain, and pretend the workaround had been used. I patch the script or story in real time. Or in other words, I try to watch, and read, generously. It’s in my benefit to do so, as I get to scratch my creative itches, while simultaneously extracting maximum pleasure from whatever it is I’m looking at (I can’t seem to do this with food however).

How to become your own best critic

In this line of thought, I recently found this great comment on mefi about how to be critic independent, which inspired this post. Idiopath wrote:

Here are some criteria I personally think are really helpful when looking at art:

  • Which of my criteria did I inherit from someone else (for example my parents or a critic) without any choice of my own?
  • Which of my criteria did I borrow from a friend or someone I admire?
  • Which of my criteria are ones I chose for myself because I liked their consequences (ie. to better observe another more important criterion).
  • Which of my criteria are subverting my other criteria?

And spurred on by his line of thinking, I can think of a few bigger ones:

  • What was the creator trying to do? Did they achieve that whether I liked it or not?
  • Who do I know that might like this? And why? Will they be reading what I write?
  • How can I look at this, from what angle, perspective, or assumption, so I enjoy it more?

Any questions like these lead any reviewer into more of a critique kind of mind, which I’m convinced leads to a better sense of what the review should contain to be of use to its readers. And selfishly, can helps maximize the enjoyment you get from anything put in front of you.

7 Responses to “Mistakes Critics Make (and Solutions)”

  1. Smaranda

    Some good points – it partly explains why I, the movie lover that I am, never read any reviews before seeing a movie.

    One additional point: in some industries, such as film, the ones who produce a piece and the ones who criticize it are part of the same ecosystem. They go hand in hand and you can’t have one without the other – for reasons that have to do with press, publicity, PR, money etc. I won’t go into any details here, but one can watch “How to Lose Friends and Alienate People” to get a vague idea of how it works. I agree to the fact that reviews (like everything else) must be put into context. And this is an important part of the context: seeing critics as actors in an industry. Consider this: what if Ms Lindsay Lohan cleaned up her act for exactly 3 months and played the most amazing role of her career and then went back to drinking/partying/poll dancing? Would the role itself be enough for her to get a good review? I wouldn’t bet on it.

    I don’t know how/if this works with books. But I’m sure there’s an angle there…

    That being said, this was a good article – but a bit of a harder read than usual. Given the topic though… hey, it’s fine.. we all have to vent! :))

    Reply
  2. Chris

    Hmmm. Not so sure about this. There is more than a little value in expertise; some people’s opinions of a piece of art, or a film, are better informed than the opinions of others. This is because the experienced ‘opinion’ of a 20-year film critic, who has seen films from 100 countries in dozens of genres, is better informed than the ‘opinion’ of the Internet blogger who thinks Michael Bay invented movies.
    Criticism is wasted where it is not needed, including those Baysian noisemakers. I believe a key critic mistake is bothering to even review a movie like Transformers. It is a movie only in the sense it plays in theatres and is shot in 35 mm. But in this day and age, when everyone has not only an opinion but an outlet for that opinion, even a H. L. Mencken would likely have blogged about Megan Fox.

    Reply
  3. Joe McCarthy

    It does seem that we would be able to make better informed choices if more reviews were more informational, and I think one way to encourage better reviews is to enable readers to review – or rate – reviews.

    I like Amazon’s mechanism for rating the helpfulness of reviews, and its leaderboard for “Top N Reviewer”. I find Yelp’s restriction of review ratings to exclusively positive labels – “useful”, “funny” and “cool” – to be somewhat hypocritical given the negativity and even vehemence often articulated in the reviews themselves; there are many rants and raves I would like to rate as “unhelpful” or “not useful”.

    My favorite reviewing site is Netflix. I have not liked any movie that has a personalized rating – for me – of less than 4 stars, and their similarity scores on individual reviews – where they say how similar a reviewer’s overall ratings are to mine – are extremely helpful in providing context for the reviews (I pay more attention to the reviews by people who are 80% like me than those by people who are 20% like me).

    Finally, I wanted to share one of my favorite papers from the recent CSCW 2010 conference: Understanding Deja Reviewers, by Eric Gilbert and Karrie Karahalios. The authors distinguish between “pro” and “amateur” reviewers, and note the game mechanics that motivate the former, and the more authentic “almost visceral reactions” that characterize most of the latter. Although it’s an academic paper, it’s exceedingly well written, and only 4 pages long … and as you can probably surmise, I highly recommend it :). I posted a review of this, and two other papers (and associated presentations), in some Notes from CSCW 2010.

    Reply
  4. idiopath

    I am glad you liked my answer, but it looks like my point got scrambled in your editing process. I listed two different concerns: the selection of criteria (which are the basis for forming judgments) which you quote as a list, and then some specific things to apply those criteria to – you use the heading from that, but don’t list any of my suggestions.

    Reply
  5. Mura

    Scott, This post prompts some questions
    a) Can you name some critics who you think adhere to your standard.
    b) In my mind, there will hardly be any who adhere to these ideals. I will leave to the reader to read reviews of the kinds he is agreeable with i.e., if you go to movies to think, read the reviewers who align with those tastes
    c) On the averaging of reviews, i believe your interpretation is incorrect. Review average more works like electoral democracy. We all know that the person who is president of US is NOT the best president of US. He is the person who was able to garner most votes at that point of time. It helps to know that say 1000 other liked a book or a movie. In fact, i look at these rating for a sanity check. If 1000 others believe that a book is poor i would hestitate to buy it.
    Only a person in strong denial can say that these 1000 people have different tastes from mine and take the plunge

    Reply

Leave a Reply

* Required