Experiment smartphone-first questionnaire layout

This study investigates the effects of six experimental conditions:

1. A smartphone-first developed login screen versus the regular login screen.
2. A smartphone-first developed questionnaire layout versus the regular questionnaire layout.
3. Four designs for grids: three smartphone-first developed options for grid design (stem fix, carrousel and accordion) versus the regular design (classic table for large screen and stem fix for small screen).
4. The use of smileys and icons in answer options: text only, text and smileys/icons and smileys only.
5. Showing an instruction on smartphones to encourage the use of speech to text for open questions versus not showing such an instruction.
6. A shorter (17 minutes) versus a longer version (25 minutes) of the questionnaire.

Across a broad set of response behavior indicators (e.g., logging in, break-off, choosing ‘no answer’, straightlining, mean scores on various scales, and consistency with register data for several items), as well as indicators of respondent satisfaction, the experiment revealed remarkably few significant effects. This is reassuring, as it suggests that respondents provide answers largely independent of the tested design alternatives. The smartphone-first design will be gradually implemented in CBS questionnaires, as it aligns with the corporate style guide, improves accessibility, and ensures a consistent experience across devices.

Giesen, D., Kompier, M. & van den Brakel, J. (2025). Experiment Smartphone-First Questionnaire Layout. Discussion paper, Statistics Netherlands, The Hague/Heerlen.