When Data Goes Wrong

By Rupert Burnham | September 4, 2017

In the age of the data science universe and with the level of insight that it can provide, many believe that data is everything and that it can do no wrong.  
On the surface the amount of data and insights available are being used to produce a better quality and standard of content.  This stands to reason, a clearer picture of what is and isn’t liked and is and isn’t successful is available.  The audience is better known, more quantifiable and feedback is more readily accessible.  In short, better data and better analytics equals better shows and content.
But if you thought that better data and analytics automatically results in an ever increasing amount of better content being produced of an increasingly higher quality and rating, then you’d be wrong.  This is evidenced by the recent approach of Amazon Studios in the creation and production of their not so hit show ‘Alpha House’. 
All production companies want the show they choose to produce to be a ratings hit, think ‘Breaking Bad’ or ‘Game of Thrones’.  Amazon, in a ground breaking approach towards commissioning (and potentially distributing) a new hit show held a competition.  The eight best contenders were chosen, and pilots of each were made and these were then made available to watch on Amazon Prime for free by all.  Millions of people watched them and Amazon captured as many data points as they could to glean as much insight into audience preferences as was possible.  They captured and crunched and concluded that to make a ratings hit they needed a sitcom about four Republican US Senators.  This they duly produced, and it was called ‘Alpha House’.  The fact that it’s not well known is testament to the fact that it has an average rating – certainly not the hoped for smash hit.
A different approach to using the data and analytics available to produce a show was taken by rivals Netflix that resulted in a multi award winning series, they looked at the massive data set that they already had.  This includes all sorts of information on ratings viewers have given, viewing histories, preferences and so forth.  Within this they were able to delve deeper to see what would play well with their audience, what were the favourite actors, affiliated shows, the preferred producers and more.  What they did next with this insight required a little leap of faith.  They gave a project based on their extrapolations a green light, and a hugely successful drama series about one senator was commissioned, namely ‘House of Cards’ starring Kevin Spacey.
So why did one approach only partly succeed whilst the other was a runaway success?  The answer lies in how data analysis is used and applied.  It seems that whilst data analysis is extremely good at deconstructing data to reveal hidden and useful insights, its not nearly as good at putting the pieces back together again.  Successful decision making using data, the taking it apart, the analysing it and putting it back together again needs something else other than data and computers.

It just so happens that we have something else that’s much better than a computer at putting it all back together again, the human brain – in fact it excels at it (especially if it’s the brain of an expert).  In terms of problem solving, it is currently the human mind that is the required essential ingredient.

This is why Netflix succeeded where Amazon failed.  Netflix used the data to understand more about their audience, but then used their critical thinking, their brains, risk, gut instinct or whatever you want to call it to make the decision to produce ‘House of Cards’.  It was a risky and expensive decision, but it was a team of individuals who used the data and took the decision to commission the show.  Amazon by comparison did the reverse, they used data all the way through to drive the decision making.  This was safe approach but it didn’t lead to an exceptional success, and in a very real sense, it stifled the potential and creativity of the idea.
Insight can be gleaned from data to enable us to take risks, and these are needed for success.  However, despite the availability of insight, intuition is required and this in itself is inherently risky, and in terms of producing a show, very expensive.  If it works?  Great, plaudits all round.  If it doesn’t?  Expensive ratings flops can abound.  It seems that certainly in the context of creating a producing hit TV shows, decisions driven by individuals rather than the data are essential for originality and a successful TV series.
Data analysis works best when used as a framework within which to construct and allow for creativity.  As in architecture, a space is provided within which to build and create.  This space of course has constraints, however you know that if you want to build a two bedroom house, then you’ll need a door, windows, bedrooms etc.  There are certain prerequisites and requirements in order for it to be a successful building, but it is the architect who orchestrates, interprets and creates the vision and building to build.  In the case of TV shows, it is the director and/or producer who are wrestling with an idea to create something special and unique.  This then is what data can provide, a brief in which to design.  It takes human intuition and interpretation to turn it into something special.  If this special something really breaks free of constraints or completely reinterprets something (if it’s not a complete disaster) it is usually defined as being radical or revolutionary.
Competitive, data-savvy companies like Amazon, Google and Netflix have learned that data analysis and the collection of data alone doesn't always produce optimum results, a little something else is required to successfully use the data.

Of note within this, is the fact that studios are littered with projects that were never the successes that they should have been.  One can argue that the proliferation of blockbusters and sequels that exist is because these represent the safe money. The numbers add up and there's already an existing audience keen to see it, so it’s viable enough for studios to recoup any investment made.  Oh, and the marketing has already been done.  The risk has already been negated and mitigated.  A word of caution though, too much data and too little risk makes Jack a very dull boy.

Imagine, just imagine the calibre of film that might be produced were studios able to really analyse their audience in the same way as Netflix did, imagine the films that we could be seeing were that the case! However with current models of distribution and the way in which viewing habits are changing, it might be that this will never come to pass and this particular set of data points will never be fully analysed.  It is interesting to see that two of the recent submissions to this year’s Cannes Film Festival were Netflix productions namely ‘Okja’ and ‘The Meyerowitz Stories’.  This in itself caused some furore due to neither having a traditional theatrical distribution, though judging by the critical acclaim that they received, this is a great shame for audiences worldwide.

In conclusion, we are provided with more data/insight than ever before and it appears that this data can reduce the risk for the creative, though it still requires human intervention to deliver the rating successes that is desired.
To quote Sir Francis Bacon “knowledge is power” however it’s what you do with it that counts.

Tags: amazon data film netflix tv