How to measure creative risk: A story of Casual Fridays

Grace
Share on Facebook0Tweet about this on Twitter

Flashback to June 2016: minutes before our big premiere. For the past few months, our office had been turned upside down as conference rooms became video sets and work spaces became dressing rooms. Sure, we were pretty proud of the videos we created, but what if we were the only ones who thought we were funny? What if nobody even kind of liked them?

It’s no big deal, right? We’re only in a screening room filled with over 120 of our closest friends, family, colleagues and clients. You know, the people who matter the most to us.

See, our agency had gone through quite the transition the past few years as we went from a traditional advertising agency no one had really heard about to a digital shop that prides itself on its unique culture. As part of this transformation, we created a new video series called Casual Fridays. And we invited everyone we knew to a season two premiere to show off our work.

“We created Casual Fridays to humanize our brand,” Frank Conjerti, our creative director, says. “We wanted to reintroduce ourselves to clients and change Quinlan’s perception.”

Each Friday, we’d launch a new episode highlighting advertising topics ranging from legendary ads we didn’t know were ads, to data vs. creativity, getting the most out of the writer in your life and beyond. And we were really successful – more than half of season one’s videos had an average of 75 percent viewer engagement.

But what did that mean? Was it our creative risk that was bringing people back? Some studies have found that creative messages lead to positive attitudes about the products being marketed, but no one is really looking to see if that risk pays off (possibly because they’re afraid there may be no correlation).

So with that nagging question in mind, we put on our analytical hats, devised a plan to measure our creative risk and buckled up for a productive season two.

Casual Fridays Season Two: Better quality, bigger budget and more weird jokes.

 

We learned a lot about branding from season one of Casual Fridays, but we knew we had to turn it up a notch for season two. We upped our production quality, increased media spend (boosting our Facebook promotion by over 1,200 percent) and made the decision to move to a fully scripted sitcom.

“The first time around, we were really focused on injecting humor into thought leadership segments,” Frank says. “For season two, the humor took over. We reviewed the detailed video analytics from season one and that influenced our creative decisions for season two. When we took more creative risks, our viewer engagement numbers skyrocketed – that, and making a sitcom sounded a lot more fun.”

Those few months may’ve been a little hectic, but there was no doubt in our minds we were having fun. But then came the hard part: looking at the creative and performance analytics to evaluate if our creative risk actually paid off.

How the hell do you quantify creativity?

Yeah, we know. Creativity is subjective. But in order to effectively measure something, you have to put a number on it. After some thoughtful deliberation, we came to the conclusion that the best way to rank each episode was by creating an episode scale – a “scale-o’-weirdness,” if you shall.

How we did it: We came up with seven categories. These included how safe or unique our plotlines were, the structure of the scripts themselves and a number of smaller variables (think curse words, weird jokes, farts, obscure Simpsons references, etc.) and calculated how many times each variable occurred in each episode. By adding up all of the tallies, we could calculate a score. We then used that score to rank each episode in order from the least to the most creative.

Breaking down episode analytics.

Our initial plan was robust. We were testing a number of different things, including what hosting platforms worked best for the show’s overall performance. The episodes lived on our site through Wistia, but also lived independently on Facebook, YouTube and Twitter.

How we did it: Instead of trying to find some logical way to make sense of every bit of data we had, we thought it best to stick to the one we thought gave us the best picture overall: Facebook. Not only was the sample size much larger, but Facebook’s promotion and timing were much more consistent. Similarly to our creative scale, we were able to look at video views, post interactions, post reach and other factors to rank each episode in terms of performance.

Here’s what we came up with.

measuring creative riskThe scales may not correlate perfectly, but hey – they’re pretty damn close. As you can see, Deus Ex Frankina, Generation Gaps and Buzzwords line up perfectly. Workaholic, Dress Code and Jillian Wins an Award are only off by one and Chili Cook-Off is off by two.

While Chili Cook-Off is a bit of an anomaly (we’d line up almost exact without it), we think it’s important to note that this video went live the week after July Fourth – the chunk of time otherwise known as the most popular week for summer travel. Similarly, Dress Code, having been released the week of July 4, was another slight deviation that was affected by the travel week.

“Ideally, we try to avoid weeks with lower-than-average media performance, but we didn’t want to defer our schedule for this campaign,” Malorie Benjamin, our media director, says. “We also planned an equal spend budget for each episode, which means we didn’t spend more money on this episode to offset the holiday.”

At the end of the day, we’d say it’s fair to assume this correlation means something. The numbers are there: people watched, shared and loved the episodes with the weirdest storylines, craziest interactions and most obscure references.

What we learned from it all.

Casual Fridays was a huge step forward for Quinlan. It taught us how to push the bar and rethink creative limits for our agency, our clients and ourselves personally.

“Traditionally, there’s been this archaic assumption that self promotion isn’t important,” Franks says. “But experience really is the best teacher there is. By using ourselves as the guinea pig, we were able to experiment, learn from our mistakes and have a solid strategy in place for clients.”

It also broke the mold on how we approached branded content. Sure – a few people may have rolled their eyes at our corporate cursing and office nudity, but the overwhelming majority of our new business contacts are familiar with the series before we even meet them. And while that may not mean they’re quite ready to strip down to their tightie whities or ask their employees to be subjected to fart jokes, it has served as a way to vet which clients are a good fit for our culture.

And we’re not done just yet. We’re continuously testing as we create new things and adapting our methods to reflect the new information we find. We’ve come to value the balance between data and creativity, and our work is getting stronger with each project.

about quinlan

Want to learn more about Casual Fridays? Wondering how to measure creative risk for your brand? We could seriously talk about this all day. Tweet us @QuinlanCompany.

Leave a Reply

Your email address will not be published. Required fields are marked *