RUSSET Survey Design

16 september 2005

SURVEY DESIGN

The survey was designed according to the basic idea behind the study (see study design), keeping in mind that the interviews were done at the respondents home in a face-to-face situation, and that the study was a panel study – re-interviewing the same respondents for several times.

Development of the questionnaires

There are 3 broad topics - economics, politics, and satisfaction - covered in the panel for which the experts in the Research Team were responsible. They were asked to develop and supply questions on their field of specialization, keeping in mind the basic idea behind the questionnaire wherever possible. The lists of questions were then put together to form the initial questionnaire. It was then discussed which questions only need to be asked once at the beginning of the panel and which question needed to be asked every time. That proposal was discussed with the Russian researchers from CESSI, because cultural differences could make some questions or combinations of questions inappropriate. This process took some time and resulted in a English worded questionnaire. That questionnaire contained two types of questions. First questions that belong to the so-called panel core, these questions were asked in an unchanged format for the whole duration of the panel study. Second there were questions that are not intended to be asked multiple times, these are the so-called specific questions. The latter type of questions allow the researchers to monitor specific events in society such as elections or the introduction of the new rubble. It is also possible to extend the core questions for a ‘short’ time, so that theories can be enriched with new explanations. The other side of the coin is specific questions can easily blow up the questionnaire which will increase panel drop out or in the better scenario item non-response. So the specific questions should be excluded as soon as possible, but that is not an easy task, because who ever introduced the question will argue in favor of leaving that question in the questionnaire. 

Core questions
The strong feature of a panel study is that the same questions can be asked to the same persons across time. Most of the core questions followed the basic idea behind the study, that is (1) we ask for the present situation, then (2) the satisfaction with the present situation, (3) and finally we ask whether a actions will be taken to change the situation. The core of  questions cover the following topics:

1.          Social contacts

2.          Marriage and family life

3.          Housing

4.          Work/Job

5.          Income

6.          General satisfaction

For the variables measured within these topics we have trend tables available online.
 

Specific questions
The specific questions that have been asked during the panel study cover the following topics:

7.        Recall of satisfaction on different domains of life for 1988 and 1993

8.        Personality

9.        Health

10.      Politics

11.      Ethnocentrism

12.      Background

For these questions no trend tables are available online, however, one could download the RUSSET trend report that contains all distributions of all questions asked in the RUSSET study. The report is however quite lengthy, 200 pages, so it is advisable to just look at the report on screen.

Experimental questions: Methodological Research
The questionnaires also contain sections for methodological research. There were two types of experiments conducted: multitrait multimethod (MTMM) and split ballot experiments. In most cases they do not interfere with the other questions in the questionnaires, although they sometimes coincide with the other questions. MTMM experiments require a repeated measure in the same questionnaire. These repeated measures were generally put at the end of the questionnaire. An overview of these experiments can also be found on this website.
 

Due to these experiments, there are sometimes multiple questionnaires for one wave. The questionnaires only differ with respect to the experimental questions, that are generally found at the end of the questionnaire.  There were only multiple questionnaires, when an experiment required (random) subgroups. This was the case in wave 3 and 4 where the sample was randomly divided into three subgroups, named A, B, and C. Each subgroup was given a different subset of questions at the end of the questionnaire, that were also asked at the beginning of the questionnaire (to all groups). The subgroups are the same across wave 3 and 4. Thus group A in wave 3 contain the same people as group A in wave 4. In wave 5, 6, and 7 multitrait multimethod and split ballot experiments were also executed. In these waves two random samples, named A and B, were used as subgroups for the experiments. These random samples coincided with the original sample (1993) and the refreshment sample (1997).

Translation Procedure

The questionnaires were developed in close collaboration with CESSI (Fieldwork organization in Russia, Moscow). Each questionnaire was first written in English, and later translated into Russian, and finally this Russian version was again translated into English to check whether the translation worked.

For the finalization of the panel study we spend a month on checking the wording of the questions together with native speaking Russians from CESSI. This did not reveal serious problems, but we found some translations errors which were immediately corrected.

Nevertheless, it may still be possible that wording of the questions may still differ across the questionnaires. But that will in most cases be a real difference in the wording of the questions, i.e. not caused by translation errors. If one wants to be certain about what the questions measure, i.e. the meaning of the content, then one should read the Russian questionnaires which are also available online.