Different survey questions on the same topic

Abstract

Survey data are often used for comparison purposes, such as comparisons across nations or comparisons over time. Ideally, this would require equivalent questions and equivalent responses options to these questions. Yet there is a lot of variation in the response scales used, which, for example, differ in the number of response options used and the labelling of these options. This difference in items is no problem when surveys are analysed separately, but it limits the comparability of findings gathered in different surveys that used different items for the same topic. This reduces our accumulation of knowledge and calls for methods to transform ratings on different scales to attain comparable results and to correct for effects of changes in measurements and other influencing factors. Conventional methods to transform ratings on different response scales to a common one, such as the commonly used Linear Stretch Method, fall short to overcome the comparability problem caused by the non-uniformity of survey items. The weaknesses of these early transformation methods also appear when the transformed scores are compared to average ratings on 0-10 numerical scales in the same country in the same year. The shortcomings of conventional methods instigated the development of new techniques, which will be discussed in this thesis. This thesis is divided into four parts. In the first part we give a comprehensive description of the comparability problem and why conventional methods fall short to solve this problem. Each of the three parts that follow focuses on a successive innovation to improve the comparability of survey findings with different survey items.