How to Structure a Survey Using Block Randomization

If you’re using surveys to get data, it’s almost always a good idea to use randomization. But why?

Well, we’ve talked before about how the order of questions in your survey can bias your results. Did you know that the order of your survey’s pages can affect your data too?

A common method used to counteract this type of bias is to randomize the page order of your survey. But hold up. What if certain pages need to always be together?

This is where Block Randomization comes in. Block Randomization allows you to tie together survey pages so they stay together and never get broken up.

Why use Block Randomization?

Block Randomization is useful in market research and in particular—ad testing. For example, say you’re an ad agency and you have three different ad concepts. You want to gather feedback about which ad people like the most.

You carefully choose the right target demographic and send out your survey using SurveyMonkey Audience. The survey shows respondents each of your ad concepts on separate pages.

For each page that shows an ad concept, there’s a follow-up page that asks the respondents questions about it. But the order the ads appear in could affect the way respondents answer the questions. Do they like the first ad the most just because it’s the first one they saw?

On the other hand, simply randomizing the page order of your survey isn’t an option. You’d run into strange situations like asking respondents to answer questions about ads they haven’t seen yet.

Block Randomization, which is available to Platinum customers, is designed to solve exactly this problem.

Using Block Randomization, you can tie the first ad page to the page containing follow-up questions about it. That way, the two pages will always be stuck together so that follow-up questions will then immediately follow the ad concept’s accompanying image.

A group of pages that are tied together in this manner is called a block.

When you repeat the process for the second and third sets of ad and follow-up question pages, you have 3 blocks which you can then randomize. This way, the order in which respondents see the ads is random, but you ensure that respondents are answering questions about the correct ads.

Here’s an example of an image-testing survey that uses block randomization. If you take the survey multiple times, you’ll notice that the order in which you see the images changes each time.

How Block Randomization works

If you’re familiar with using Skip Logic on SurveyMonkey, it will take no time at all to learn how to use Block Randomization. And if you’re not, this section is for you!

The first step is to fully design each of your survey pages and have a clear strategy for the structure of your survey (it helps to draw it out). Now you can build the blocks and apply the randomization.

You can find Block Randomization under the Logic Tab.

1. Click Block Randomization
2. Click + New Block
3. Select the consecutive pages you want to include in each block, then click Add Block
4. Once you’re done creating all your blocks, click Done.

Pro tip: You can add multiple pages to your block by selecting the first page you want in the block, holding down the shift key, and selecting the last page you want in your block.

Even though the feature is called Block Randomization, there’s a lot more functionality available than merely randomization. You can select any of the following:

• Randomize blocks: Blocks will appear in a random order for each new respondent.
• Flip blocks: On average, half of respondents will see the block order flipped. For example: 3, 2, 1 instead of 1, 2, 3.
• Rotate blocks: Each successive respondent will see a different version than the last, so the block that appeared first in the previous survey appears at the end. For example: The first respondent will see 1, 2, 3, 4, 5; the second respondent will see 2, 3, 4, 5, 1; and the third respondent will see 3, 4, 5, 1, 2.

Pro tip: When you export your data, you can see which respondent saw which block first. This helps out your analysis process a ton.

Whichever way you do it, changing up the order of your blocks can help you reduce bias and give you better data.

Questions about anything? Let us know in the Comments below! 

The post How to Structure a Survey Using Block Randomization appeared first on SurveyMonkey.