Measurement and Evaluation

Framing Effectiveness and individual willingness-to-pay in the case of nuclear energy policy The project examines the relationship between different kinds of framing (multiple, competitive framing) as applied to nuclear energy policy and the differential propensity of individual attitude change. Our main focus lies on a special form of attitude inconsistency: the existence of voters with policy preferences, which are at least objectively in contrast to the declared policy positions of their most preferred party (e.g. voters of the CDU who disapprove nuclear energy). This discernible group of voters is compared to so-called attitude consistent voters, who share the policy preference with their most preferred party, and to so-called attitude indifferent voters, who have no distinct preference in a contested issue. Based on first secondary analyses and experiments, we will investigate the impact of competitive framing, i.e. the simultaneous confrontation with conflicting partial aspects of the policy. Additionally, we will vary the type of the frame sender (political parties, scientists) etc.. In a second module we link our insights to the question how individuals willingness-to-pay for a nuclear phase-out and the use of renewable energy sources, vary with different framing strategies. A proposal for a first series of exploratory experiments has been submitted to MELESSA (Munich Experimental Laboratory for Economic and Social Sciences). Further laboratory analyses as well as embedded experiments in access panels as well as in representative opinion polls are currently prepared.

Thurner, Dr. Martin Binder

Zuverlässige Inferenz aus Umfragedaten – Eine Synthese aus statistischer Theorie und sozialwissenschaftlicher Forschungspraxis

Augustin, Braun

Response bias in surveys Background, objectives and applied methodology The statistical and econometric analysis of households’ decision, such as consumption and saving, requires reliable measures of variables that respondents cannot easily recall during a survey interview (a leading example of such a variable is “total expenditure on nondurable consumption items in the previous month”). Research on survey response behaviour has shown that survey respondents use heuristics to construct their answers during the interview process using information they can recall easily from memory. These heuristics can result in systematic response bias that does not conform to the traditional statistical model of classical measurement error. The aim of this project is to integrate research by social scientists, economists, and statisticians to (i) develop statistical models of survey response behaviour that take the survey response process into account, and (ii) develop survey designs that minimize response bias and produce variables that are required to estimate structural models of survey response behaviour. Methods that exploit experiments embedded into surveys and the unique possibilities of interactive internet interviews are of particular interest. Integration of the project / project network Joachim Winter is a collaborator in two international projects that are funded by the U.S. National Institute on Aging (NIA): “Internet Interviewing and the HRS” (P.I. Arie Kapteyn, RAND, Santa Monica) and “Measuring the effect of aging on perceptions and behaviour” (P.I. Daniel McFadden, University of California, Berkeley). Other co-operation partners are the economist Thomas Crossley (University of Cambridge), the econometrician Stefan Hoderlein (Brown University), the social psychologist Norbert Schwarz (University of Michigan). As a research professor at the ifo Institute, Munich, Joachim Winter is involved in the methodological advancement of ifo Institute’s business surveys. He is also a member of the advisory board of the Economic & Business Data Center (EBDC).

Winter, Tannhof, Hoffmann, Siflinger

Veränderungsmessung in der Psychologie

Küchenhoff, Bühner

New Methods for Item Response Theory Item Response Theory (IRT) is one of last century's most important achievements of psychological diagnosis and empirical educational research: It allows for an objective measurement of latent person characteristics by means of separating item and person parameters. Moreover, as opposed to the classical psychological measurement theory, IRT provides the opportunity to statistically test the model assumptions. The Rasch model, that has been used in the PISA-study, is the most well-known IRT model. In an intervention study on the effectiveness of competence-supporting learning environments the Rasch model was emplyed to measure mathematical comptetency for using diagrams and models in statistical contexts. In this study, both the objective measurement of the students' competencies and the modelling of the treatment effects is of major interest. Methodological challenges arise from the heterogeneity of the sample: The data have a hierarchical structure, because students are grouped in classes, classes in schools and so forth. Moreover it is necessary to check whether the measures are comparable for students from different groups, or if differential item functioning occurs. The aim of the project is to develop and apply new IRT Methods to account for the sample heterogeneity: For the diagnosis of differential item functioning methods from machine learning will be used in combination with latent-class-approaches. The hierarchical data structure will be accounted for by means of mixed models.

Strobl, Lindmeier, Reiss, Tutz

The influence of response sets on the results and quality of multivariant analyses This project demonstrates the vulnerability of several statistical tests and procedures towards response sets. Using data sets from three standardised surveys, secondary data analyses were conducted. Given the importance of the mode of data collection, as it can alleviate or exacerbate response sets, we used data gathered in face-to-face interviews as well as via mail and online surveys. As expected, we found that the overall number of response sets was low, with the highest number found in the online survey. Further testing showed, however, that despite their low number response sets are far from negligible: whilst simple bivariate tests such as cross tabulations and means response sets are hardly affected due to the low share of invalid answers results of multivariate procedures are considerably altered. Our results show that a share of 0.1 percent of respondents using response sets change the results of cluster analyses significantly. Based on these findings, we finally discuss how to identify and deal with respondents who used response sets.

Brosius, Jandura