Latest News About England: Netflix gathers detailed viewer data to guide its search for the next hit

Streaming giant, which makes House of Cards, feeds response from 44 million subscribers into programme commissioning

When Netflix decides to sink $100m into House of Cards without even seeing a pilot, or plans a move into Games of Thrones territory with the upcoming Marco Polo, or pulls off its first Oscar nod after gambling on a documentary charting political unrest in Egypt, the company isn't making blind bets.

The US streaming giant spends $3bn a year on TV and film rights – with about $300m of that ploughed into making its own shows – using decisions based on a meticulous analysis of the viewing habits of its 44 million subscribers worldwide. Todd Yellin built the engine that cleverly recommends TV shows and films that users might like – it can deliver more than 76,000 different genre types such as "alien films from the 1970s" – and feeds their viewing habits back into the data-driven Netflix programme commissioning machine.

"We own the Netflix customer experience from the moment they sign up, for the whole time they are with us, across TV, phone and laptop," says Yellin, vice president of product innovation. "We climb under the hood and get all greasy with alogrithms, numbers and vast amounts of data. Getting to know a user, millions of them, and what they play. If they play one title, what did they play after, before, what did they abandon after five minutes?"

Netflix users watch 2bn hours of programming each month and it is Yellin's ability to micro-analyse that data that allows the company to turn much of the traditional TV series commissioning model on its head. Instead of making a show and then hoping it catches on with a big audience, Netflix crunches its subscriber base viewing data to identify fans of specific genres and then looks at TV formulas that it already knows are likely to appeal to them.

"We have an immense amount of data, we see everything our subscribers are watching," says Cindy Holland, head of original content at Netflix. "We can identify subscriber populations that gravitate around genre areas, such as horror, thriller and supernatural. That allows us to project a threshold audience size to see if it makes for a viable project for us."

One formula that has proved to be a reliable winner is banking on serialised scripted TV content, shows that Holland says "suck you in from the beginning" and "tick a lot of boxes", often with a charismatic lead and a famous director or showrunner, and using pre-existing material. Netflix's runaway success House of Cards is the embodiment of the model – loosely based on the 1990s BBC series, starring Kevin Spacey, directed by The Social Network's David Fincher – and there are a slew of new Netflix TV series on the way that bear similar hallmarks.

Friday Night Lights star Kyle Chandler is to appear in a psychological thriller from the creators of Damages, the legal thriller starring Glenn Close; a number of the directors involved in Game of Thrones are making Netflix's first foray into big budget historical productions with Marco Polo; and bizarre animated, Family Guy-type comedy BoJack Horseman will see Breaking Bad's Aaron Paul team up with Arrested Development's Will Arnett.

So has Netflix found the data-driven holy grail of how to make a guaranteed hit TV show? "I don't believe so, [all the] information doesn't give us a crystal ball," says Holland. "Look at Orange Is The New Black. There was no [specific] data to say that a comedic drama set in a women's prison, with mostly unknown actors, would definitely work."

The project that has produced arguably the biggest vindication of Netflix's ability to mine user data is The Square, a documentary shot over three years about the political unrest in Egypt, which has picked up the company's first Oscar nomination.

The US streaming giant has confounded the established norms of TV broadcasting, arguing that measuring success by audience size is meaningless, and eschewing the episode-a-week law by allowing viewers to watch entire series in one shot on the day of launch.

"It is a beautiful thing being a subscription service," says Yellin. "We have nothing to do with advertising, it becomes less about ratings. The days of pure popularity [as a yardstick of success] are over. It leaves the individual quirks and quirks of people's taste in the dust. We share all the data with the [Los Angeles programming] team, to see how it compares to the shows they are thinking about. User data helps us decide to initially buy the show and to renew it for another season. Traditional networks and cable networks don't know that stuff."

Netflix may not fear rivals such as HBO or the BBC , but the might and ambition of digital-savvy Amazon perhaps poses the biggest threat to the company. This week Amazon will hit back at its international expansion – Netflix recently raised $400m to fund its European ambitions – by rebranding its LoveFilm streaming service and launching a greatly enhanced "one stop" subscription offering.

"I feel like sometimes I'm on the bridge of the starship Enterprise," says Yellin, describing the experience of charting Netflix's course in a fast-changing digital universe. "Is Netflix future-proofed? Google isn't future proofed. No one is. Companies need to keep innovating to secure the future."

Mark Sweney