A lack of understanding of the benefits of block randomization and how it can be implemented in the context of online survey experiments have limited its use in survey research. The goal of this memo is to address this gap. I describe how to implement discrete and continuous blocking in the context of online survey experiments. All the solutions described in the memo are easily implementable using common tools such as Qualtrics. They require only minimal coding skills thanks to R packages developed by other scholars (Moore and Schnakenberg 2012; Leeper 2015). The memo is written with the most commonly used respondents recruitment platforms (RRP) in mind (MTurk, SSI, Prolific, Lucid, YouGov, GfK or NORC). It is also designed to address the most common practical considerations rarely discussed in methodological papers regarding when and how to use block randomization.
If you end up using the advice in this memo, I would be very grateful if you could mention it somewhere as follow:
Cavaille Charlotte, “Implementing Blocked Randomization in Online Survey Experiments”, Methodological Note available at https://charlottecavaille.com.
If you end up using the API, please acknowledge Diag Davenport’s contribution as follow:
Source code by Diag Davenport available at https://github.com/diagdavenport/rpy2-heroku
Charlotte Cavaille 