How testing in unnatural environments backfires

If you’ve ever wondered why every poster and every trailer and every TV spot looks exactly the same, it’s because of testing. It’s because anything interesting scores poorly and gets kicked out Now I’ve tried to argue that the methodology of this testing doesn’t work. If you take a poster or a trailer and you show it to somebody in isolation, that’s not really an accurate reflection of whether it’s working because we don’t see them in isolation, we see them in groups. We see a trailer in the middle of five other trailers, we see a poster in the middle of eight other posters, and I’ve tried to argue that maybe the thing that’s making it distinctive and score poorly actually would stick out if you presented it to these people the way the real world presents it. And I’ve never won that argument.