Good Survey Questions: Master the Art of the Ask

“Surveys are the most dangerous research tool,” says Erika Hall, author of Just Enough Research. “If you write bad survey questions, you get bad data at scale with no chance of recovery.”

With people’s attention spans getting shorter, there is growing pressure to make surveys lean. More and more people are taking surveys on their phones and if the survey is too long, they drop out.

How can we make the survey short and still get the information we need?

By writing better questions.

Why Run Surveys?

Surveys help you find answers to business questions. It’s that simple.

Of course, the higher quality questions you ask, the better answers you tend to get. And in the context of digital marketing, and in particular conversion optimization, asking the wrong questions can lead you down the wrong path.

Therefore, for your conversion research efforts, it helps to spend a bit more time on question development.

As David F. Harris said, “If we make questions clear, answerable, easy and unbiased, we can reduce the survey length, improve the respondent experience, completion rates and the quality of data.”

Better questions aren’t just about getting you better data for better decisions, either. Surveys affect the company’s reputation.

How would people describe your surveys?

  • “This is the kind of survey I really enjoy… the best survey I have completed in the last year.”
  • “I took this survey from company X, and it was horrible! I got so annoyed I just quit.”

All the more reason to take writing customer surveys more seriously.

Image Source

For each type of survey we run as part of conversion research, some questions are more effective than others.

Good Survey Questions by Survey Type

There are no universal guidelines for good survey questions. Of course, it first depends on your research goals – what you actually hope to learn with this information. Your survey questions should also be designed in context of both your company and your customers. Then, the type of survey matters, too.

We’ll organize broadly good survey questions by type of survey here.

Survey of Current Customers

Surveying your customer base can help clarify personas, pinpoint new personas, identify stumbling blocks in the user experience, inform product and content decisions, inspire marketing messaging, and the list goes on.

You can get valuable insights by asking these questions:

  • What can you tell us about yourself? can also be framed as In 1 sentence, describe yourself.
  • What are you using [the product] for? What problem does it solve for you?
  • How is your life better thanks to [the product]?
  • What made you buy [the product]? What convinced you that it was a good decision?
  • What doubts or hesitations did you have before buying?
  • What questions did you have that you couldn’t find answers to?
  • Did you consider alternatives? How many websites did you visit before buying from us? which ones?
  • What was your biggest challenge, frustration or problem during your visit to our website?
  • Anything else you would like to tell us?

David Darmanin from HotJar has analyzed hundreds of user surveys and zeroed in on some of the most insightful questions you can ask current customers:

  • Where exactly did you hear about us?
  • How would you describe us to a friend?
  • What would you miss the most if you could not use us anymore?
  • What’s the one big thing we’re missing?
  • What are your biggest everyday challenges?

All of these questions are open-ended, which is tricky. Most people don’t like to write paragraph after paragraph. If you want to ask more than 5 questions, add an incentive like an Amazon gift card or a chance to win a valuable prize (HotJar was giving out an iPad mini).

Mix Closed-Ended and Open-Ended Questions

Tamara Mendelsohn, VP and General Manager, Consumer at Eventbrite, has successfully run a surveys containing a mix of open-ended and closed ended questions. At True University 2015, she presented her strategy, consisting of the following questions:

  • How do you feel about Eventbrite?
    • I love it and would recommend it to a friend
    • The app suits my needs, but I’m open to other solutions
    • I’m really dissatisfied with the product.
  • What originally prompted you to use the app? (include a list of value proposition items and have them choose all that apply)
  • After using Eventbrite, what is the greatest value you are getting from it?
  • What words do you think represent Eventbrite?

The responses Tamara and her team received helped crystallize the brand identity and value proposition.

How To Get Better Answers

If you are feeling innovative, try the Love/Breakup letter technique.

You ask the respondents to imagine as if your product were a person, and ask them to write either a Love letter or a Breakup letter addressed to that person. This technique strips away all the armor and makes for genuine and emotionally colorful responses.

To get more in-depth answers, Talia Wolf from GetUplift suggests adding personal questions to your surveys that offer a window into the customer’s emotional triggers and intent. For instance:

  • What is the biggest challenge you’re facing right now? (best for B2B)
  • Where do you see yourself in 3 years from now?
  • If [company name] were a person, what kind of person would it be? (or how would you
    describe them?)
  • Who is your role model?

According to Talia, the last question in particular helped her client achieve a 62% boost in sales.

Here’s the story:

taliaTalia Wolf:

“While working with an Ecommerce site, we sent out a survey including the question, “Who is your role model?” and received extremely interesting responses.

Over 90% of respondents claimed their role model was a close family member while the vast majority said it was a parent.

Using this insight we crafted our message, chose an image strategy, planed our social proof strategy and chose our color scheme to highlight family values above all and to create a sense of a close community. This change in strategy increased purchases by 62%.”

The first step Liston Witherill from GoodFunnel takes when coming up with effective survey questions is contemplating the specific question he’s attempting to answer, and he holds that as his North Star. Next, he found that focusing on the drivers of people’s behavior yields rich material.

listonListon Witherill:

“The questions that we find the most helpful usually have to do with motivation. So a question like “What are you hoping the [product/service] will do for you?” or “What happened that made you look for [product/service]?”

As much as you may think you know your customers, you’ll be quite surprised by the variety of answers you get.”

Sometimes asking the right follow-up questions can make huge difference in the outcome of the survey.

Sean Campbell – CEO of Cascade Insights – has a story to prove it:

seanSean Campbell:

“During a recent study, we surveyed our client’s current customer base and uncovered that while 92% of the market segment we were targeting were satisfied with their current solution, 40% expressed the likelihood they would switch providers in the upcoming year.

If we had merely stopped at the first question, the client would have been left with the belief that keeping current clients wasn’t going to be a challenge. But that obviously wasn’t the case.”

How to ask the customer to participate in a survey

For the best response rates, frame the survey request as a personal favor. Alex Turnbull, CEO of Groove, has nailed the art of The Ask. The template below is based on Alex’s outreach messages:

At [company], there is nothing that keeps us up at night more than thinking about how we can make a better product for you.

But one of the most important lessons we’ve learned over the years is that what WE think is best for the product doesn’t really matter. What matters most is the challenges our customers – that’s you – are facing, and how we can better solve them.

We want [product] to be as useful and easy to use as it can possibly be.

We want it to be the best [category] product on the planet. Will you help us do that?

Just click here right now, and answer a few quick questions.

It won’t take you more than … minutes.

I—and the entire [company] team—would really, really appreciate it.

This request is so personal – it is hard to ignore.

Getting Ideal (but Prospective) Customers to Take Your Survey

With the right approach, it is even possible to get your prospective customers to participate in a survey, even if they’ve never engaged with you before.

Alyssa Ackerman from EverThere has mastered the art of sending “cold” survey invites. The response rate she gets is surprisingly high. How does she manage to get complete strangers to take her survey? By being respectful and genuine. Here is her magic “cold” outreach template:

Hi [Name], I found your profile on LinkedIn and wondered if you’d be interested in participating with a research study I’m running geared towards [their field of expertise].

It will take less than 10 minutes of your time and comes in the form of a survey. Let me know!

Thanks,
[your name]

If they agree to participate, you follow up with:

Hey [Name], thanks for connecting.

I wondered if as a [their field of expertise] you have you ever [done something your product is helping with].

If so I’d love your input on a research study I’m running. It will take less than 10 minutes and you can find the survey here: … The goal of the research is to improve [their experience] and push innovation in the space.

Please let me know if you have any questions. I really appreciate any time you can offer this effort.

This kind of outreach has had a fortunate secondary effect – after participating in the survey, prospective customers actually become interested in the service Alyssa’s company is offering. A win all across the board!

Survey of Engaged Prospects

Rand Fishkin from Moz has an interesting approach to conversion. Here’s what he said in a podcast with Alex Harris: “We don’t want to convert too fast! I want them to become part of the community, I want them to get educated, I want them to have a great experience with Moz.”

Rand has found that this strategy maximizes customer LTV (lifetime value) in the long run. After the potential customer has been engaged for a while, has been reading the blog and participating in free webinars, Rand sends them a probing survey asking:

Have you considered becoming a customer? What is stopping you?

The answers help make the pre-conversion on-boarding experience better and ultimately create loyal customers.

Post-Conversion Survey

The customer has just bought from you and there is positive momentum. You can use this time to refine your value proposition and discover new areas you can fix.

For B2B, post-conversion survey could be part of an on-boarding email. I like how Alex Turnbull, CEO of Groove, does this:

Subject: You’re in 🙂 PLUS, a quick question…

Hey [name],

I really appreciate you joining us at [company], and I know you’ll love it when you see how easy it is to [do the job your product was hired to do].

We built [product] to help businesses [reach a goal], and I hope that we can achieve that for you.

If you wouldn’t mind, I’d love it if you answered one quick question: why did you sign up for [product]?

I’m asking because knowing what made you sign up is really helpful for us in making sure that we’re delivering on what our users want. Just hit “reply” and let me know.

Thanks,
[Your name],
[Your company]

Put a Survey on Your Thank You Page

Image Source

Brian Massey from Conversion Sciences says the Thank You page survey is his favorite type, for two reasons:

  1. Completion rates are high thanks to the “Liking” effect. When someone chooses you, you instantly become better in their minds, so they are more willing to help.
  2. The question is posed to actual buyers/leads. These are people who have made it through the process successfully.

Brian found that the information he gets is rich and full of hypotheses for optimization. Here is a story Brian shared of a Thank You page survey bringing valuable insights:

brianBrian Massey:

“One of our clients sells two versions of a product for automobiles, a Pro and Lite version. We used a thank-you page survey to ask visitors why they chose the version they bought. The survey was slightly different for the Lite product thank-you page and the Pro version thank-you page.

We asked “If you don’t mind us asking, what made you choose Lite over Pro?”

Response rates were good, and the results were very helpful for refining our message and adding content. The key differentiator for this product was a 3G connection that didn’t require communicating through a phone. It became clear that we weren’t communicating the benefits of this, and addressing issues such as security.

This also helped us identify key segments of buyers to address in our copy.”

Other questions you could ask in the Thank You page survey:

  • Why did you decide to sign up/purchase from us today?
  • What almost stopped you from signing up/buying from us today?
  • What are the top 3 things that persuaded you to use us?
  • What are the top 3 things that nearly stopped you from using us?
  • What could we have done to make your decision easier?

Post-User Testing Survey

You’re running user tests, right?

Much of the insight comes from analyzing the user behavior, of course, but you can also learn a lot from doing a post-test survey.

Once the participant has completed the actions on your website you have outlined in your script, follow up with a few questions while their impressions are still fresh:

  • What was the worst thing about your experience?
  • What aspects of the experience could be improved?
  • What did you like about the website?
  • What other comments do you have?

Survey of Customers Who Churned

Clearly, a good point to gather insights would be when you’ve failed: when a user churns. The more insight here, the better, as churn is a huge problem with growth. 

For this, there’s another email template from Alex Turnbull, CEO of Groove. This email gets a nearly 14% response rate and Alex has gotten some incredible insights from it.

Hey …,

I noticed that you didn’t upgrade your account. I completely understand that [the product] isn’t the best fit for everyone, and there are no hard feelings on my end 🙂

But if you’re willing to answer, I have a quick question: what could we have done better to keep you as a customer?

Just hit “reply” and let me know.

Thanks! [Your name]

On-Site Surveys

On-site surveys can provide valuable context that is often missing from click-stream data.

Image Source

Avinash Kaushik is convinced that nothing beats the Three Greatest Questions Ever:

  • What is the purpose of your visit to our website today? It can also be framed as:
    • What is the reason for your visit today?
    • What task are you looking to accomplish on our website today?
    • Why are you here today?
  • Were you able to complete your task today?
  • If you were not able to complete your task, why not? It can also be framed as:
    • If you were able to not complete your task please explain
    • Why were you not able to complete your task on our website today?
    • How can we improve our website to ensure you are able to complete your task?

Peep Laja, founder of CXL, recommends asking these 2 questions on your site:

  • What is stopping you from completing your purchase today?
  • Are there any questions you are not finding the answers to? (if they say Yes, ask which ones)

Experiment to see which one gets a better response rate for you.

NPS (Net Promoter Score) Survey

Well, first ask the classic question:

On a scale from 1 to 10, with 10 being “Extremely likely,” how likely are you to recommend our product to a friend or a colleague?

But don’t stop there, also follow up with:

Please explain why you have chosen [number].

If you don’t ask why, the numbers are not as useful – there is no clear way to act on them.

Survey Questions to Inform Content Strategy

On their blog page, HotJar asks just 1 question:

What else would you like to learn about?

Great way to focus your content efforts.

Kissmetrics asks their customers:

  • What are your most burning questions about_____?

  • What’s your preferred method of learning (reading, watching or listening)?

As a result, they can optimize both content and format.

Alyssa Ackerman from EverThere discovered what they should be focusing on in their content when running a completely different survey.

The survey was intended to test EverThere’s value proposition. Alyssa’s goal was to find out if the target customers understood the value of the service EverThere was offering – “Qualified leads at a fraction of the price you pay for leads coming from other acquisition channels.”

It turned out that the majority of marketers did not know what their cost per lead was, so the value proposition didn’t resonate with them.

Alyssa realized that top priority for EverThere when it comes to content was educating marketers about the average lead cost different acquisition channels offer, and how it compares to EverThere.

Conclusion

The results of surveys can be eye-opening and explain a lot – if you ask the right questions.

If you ask the wrong questions, the wrong people, or at the wrong time, you’re likely wasting your time. Your surveys are only as good as the questions you ask, and those should purely reflect your business goals and challenges. Take a step back to think about what problems you want to solve, and ask the right questions to tease out some answers. Hopefully this post gives you some inspiration for your next survey.

The post Good Survey Questions: Master the Art of the Ask appeared first on CXL.

Source: Conversion XL

Be the first to comment

Leave a Reply

Your email address will not be published.


*