2015 Schedule
Interactive: March 13–17  •  Film: March 13–21  •  Music: March 17–22

Netflix Shares a Decade of A/B Test Learning

Saturday, March 14
9:30AM - 10:30AM

JW Marriott
Salon D
110 E 2nd St

In order to continually improve the customer experience the Netflix product, design and algorithm teams have run over a thousand A/B tests with tens of millions of users. We have had many successes and, even more importantly, many more failures. If too many of our tests are wins then we’re not pushing hard enough and not learning enough.

For years we made steady progress but 2010 was a turning point. As we started doing large scale UI testing on the “ten foot experience” we learned a smarter way to do A/B testing, rapidly accelerated the pace of product innovation.

Todd Yellin will share an assortment of design and algorithm wisdom gleaned from getting at the truths that can only be gained from focusing on actual user behavior. He’ll share some less obvious lessons about interaction design gained from some supposedly sure bets that flopped. And other more successful tests run on the interfaces and algorithms from sign up to content discovery to actual viewing experience.


#sxsw #abtesting


Todd Yellin

VP Prod Innovation


Todd S. Yellin has enjoyed a career dedicated to film and TV entertainment. He started working at Netflix nine years ago. His current role as VP of Product Innovation revolves around changing the...

Show the rest


Interactive Badge, Gold Badge, Platinum Badge
Solo / Dual / Panel
Design and Development

Thanks to our sponsors

Miller Lite Esurance AT&T IFC Mazda USA Monster Energy McDonalds logo CapitalOne logo PepsiCo logo Austin Chronicle

Music sponsor

Sonic Bids

Interactive sponsor

Phillips logo