Nonresponse In Sample Surveys

EUROSTAT L-2920 Luxembourg -Tel. (352) 43 01-1 -www.europa.eu.int/comm/eurostat/ Eurostat is the Statistical Office of the European Communities. Its task is to gather and analyse figures from the different European statistical offices in order to provide comparable and harmonised data for the European Union to use in the definition, implementation and analysis of Community policies. Its statistical products and services are also of great value to Europe's business community, professional organisations, academics, librarians, NGOs, the media and citizens.

This FORS guide discusses the effect of incentives on survey completion/response rate, sample composition, and response quality. The guide particularly emphasizes the effects of different types of incentives – conditional vs. unconditional, monetary vs. non-monetary – for encouraging survey participation and reducing panel survey attrition in the Swiss context.

Time orientation is an unconscious yet fundamental cognitive process that provides a framework for organizing personal experiences in temporal categories of past, present, and future, reflecting the relative emphasis given to these categories. Culture lies central to individuals' time orientation, leading to cultural variations in time orientation. For example, people from future-oriented cultures tend to emphasize the future and store information relevant for the future more than those from present-or past-oriented cultures. For survey questions that ask respondents to report expected probabilities of future events, this may translate into culture-specific question difficulties, manifested through systematically varying " I don't know " item nonresponse rates. This study drew on the time orientation theory and examined culture-specific nonresponse patterns on subjective probability questions using methodologically comparable population-based surveys from multiple countries. The results supported our hypothesis. Item nonresponse rates on these questions varied significantly in the way that future orientation at the group as well as individual level was associated with lower nonresponse rates. This pattern did not apply to nonprobability questions. Our study also suggested potential nonresponse bias. Examining culture-specific constructs, such as time orientation, as a framework for measurement mechanisms may contribute to improving cross-cultural research.

For longitudinal surveys, there is little discussion on how call record data are able to account for household nonresponse. This paper uses call records as well as observed data from Understanding Society’s Wave 1 to model Wave 2, Wave 3 and Wave 4 household contact and cooperation propensities. Multi level logistic models are used to account for the nested structure of the data (households within interviewers). Results indicate that households which repeated unproductive contacts, broke appointments, registered above median proportion of "no replies", or began the call sequence with an unproductive contact in Wave 1 are at risk of future nonresponse.

We examine the effects of interviewer-respondent familiarity on both response patterns and rates of item nonresponse when self-administered questions (SAQs) are used. We use SAQ data from a survey in which the researchers experimentally ensured that there would be varying degrees of familiarity between interviewers and respondents. Our results reveal only minimal differences in response patterns by degree of prior acquaintance between interviewer and respondent, indicating that SAQs are effective at eliminating potential bias stemming from such relationships. Our results for item nonresponse depend on how we measure the relationship between interviewer and respondent; but in all cases, it is clear that prior knowledge of one another, far from harmful, leads to low nonresponse rates to SAQs. Thus, researchers using SAQs may not need to adhere to the norm that interviewers and respondents must be strangers, with practical and cost-effective consequences for data collection.

classical intra-interviewer correlation (ρ int ) provides survey researchers with an estimate of the effect of interviewers on variation in measurements of a survey variable of interest. This correlation is an undesirable product of the data collection process that can arise when answers from respondents interviewed by the same interviewer are more similar to each other than answers from other respondents, decreasing the precision of survey estimates. Estimation of this parameter, however, uses only respondent data. The potential contribution of variance in nonresponse errors between interviewers to the estimation of ρ int has been largely ignored. Responses within interviewers may appear correlated because the interviewers successfully obtain cooperation from different pools of respondents, not because of systematic response deviations. This study takes a first step in filling this gap in the literature on interviewer effects by analyzing a unique survey data set, collected using computerassisted telephone interviewing (CATI) from a sample of divorce records. This data set, which includes both true values and reported values for respondents and a CATI sample assignment that approximates interpen-Brady T. h i n t e r v i e w e r v a r i a n c e i s n o n r e s p o n s e e r r o r v a r i a n c e ? 1005 etrated assignment of subsamples to interviewers, enables the decomposition of interviewer variance in means of respondent reports into nonresponse error variance and measurement error variance across interviewers. We show that in cases where there is substantial interviewer variance in reported values, the interviewer variance may arise from nonresponse error variance across interviewers. 1. Additional interviewer-level information, such as interviewing experience, is not available.

Survey researchers often assume that respondent burden is an important determinant of survey participation propensity and that interview length is a good indicator of burden. However, there is surprisingly little evidence of the effect of length of a completed interview on subsequent participation propensity, particularly in the case of face-to-face surveys. This article presents results from a large-scale randomised experiment in which respondents experienced interviews of different lengths at wave 1 of a panel survey. Subsequently, respondents were asked to complete a self-completion questionnaire and then to take part in further waves of the survey. The article compares completion rates for each of these subsequent tasks between those administered the shorter and longer versions of the wave 1 interview. No evidence is found that wave 1 interview length affects subsequent participation propensity.

A common hypothesis about practices to reduce survey nonresponse is that those persons brought into the respondent pool through persuasive efforts may provide data filled with measurement error. Two questions flow from this hypothesis. First, does the mean square error of a statistic increase when sample persons who are less likely to be contacted or cooperate are incorporated into the respondent pool? Second, do nonresponse bias estimates made on the respondents, using survey reports instead of records, provide accurate information about nonresponse bias? Using a unique data set, the Wisconsin Divorce Study, with divorce records as the frame and questions about the frame information included in the questionnaire, this article takes a first look into these two issues. We find that the relationship between nonresponse bias, measurement error bias, and response propensity is statisticspecific and specific to the type of nonresponse. Total bias tends to be lower on estimates calculated using all respondents, compared with those with only the highest contact and cooperation propensities, and nonresponse bias analyses based on respondents yield conclusions similar to those based on records. Finally, we find that error properties of statistics may differ from error properties of the individual variables used to calculate the statistics.

A common hypothesis about practices to reduce survey nonresponse is that those persons brought into the respondent pool through persuasive efforts may provide data filled with measurement error. Two questions flow from this hypothesis. First, does the mean square error of a statistic increase when sample persons who are less likely to be contacted or cooperate are incorporated into the respondent pool? Second, do nonresponse bias estimates made on the respondents, using survey reports instead of records, provide accurate information about nonresponse bias? Using a unique data set, the Wisconsin Divorce Study, with divorce records as the frame and questions about the frame information included in the questionnaire, this article takes a first look into these two issues. We find that the relationship between nonresponse bias, measurement error bias, and response propensity is statisticspecific and specific to the type of nonresponse. Total bias tends to be lower on estimates calculated using all respondents, compared with those with only the highest contact and cooperation propensities, and nonresponse bias analyses based on respondents yield conclusions similar to those based on records. Finally, we find that error properties of statistics may differ from error properties of the individual variables used to calculate the statistics.

The paper systematically reviews existing literature on the relationship between the level of effort to recruit a sampled person and the measurement quality of survey data. Hypotheses proposed for this relationship are reviewed. Empirical findings for the relationship between level of effort as measured by paradata (the number of follow-up attempts, refusal conversion and time in the field) and question-specific item non-response rates, aggregate measures of item non-response rates, response accuracy and various measurement errors on attitudinal questions are examined through a qualitative review.

When potential respondents consider whether or not to participate in a telephone interview, they have very little information about the interviewer, aside from how he/she sounds, speaks and interacts. Yet interviewers vary widely in how often their invitations lead to participation, suggesting that potential respondents may give considerable weight to interviewers’ verbal attributes. We examine the impact of three interactional behaviors: disfluencies (um, uh), backchannels (mm-hmm, I see), and simultaneous speech, on the outcome of specific telephone survey invitations, agree-to-participate, scheduledcallback, and refusal, in a corpus of 1215 audio-recorded survey introductions (contacts). Agreement was highest when interviewers were moderately disfluent. Further, in contacts where “answerers” ultimately agreed to participate, they displayed more backchannels than when they refused. Finally, there was more simultaneous speech in contacts where answerers ultimately refused to partic...

In this article a two-stage method for the on site collection of data on a theatre audience is introduced and a number of techniques, acknowledged to reduce unit nonresponse in household surveys, are applied. More particularly, three experimental designs are used to test the impact of a number of survey design features on response rates and sample composition. The various survey design features included are the way in which the sample units are contacted on site, the use of incentives and the specific sequence of the questions in the questionnaire. Only type of contact proved to have a significant impact on the response rate. A selection bias with regard to age and topic salience was found. Suggestions for further research are presented.

Data collected by interview may be influenced by the interviewers who collect the data. Such effects are proposed to arise from characteristics of interviewers (e.g., dress, sex, race) as well as interviewer behaviors (e.g., reading questions as worded, establishing rapport with the respondent). The latter category of causes highlights the dyadic nature of interviewer effects specifically that they arise from the interaction of two independent-yet-related actors engaged in a structured conversation. This conceptual presentation will focus on theory related to interviewer-respondent interaction as a special type of dyadic data, and will motivate the statistical analysis of micro-level behaviors (e.g., conversational turn and utterance-level level behaviors) that can shed light on how interviewer effects arise. This approach will be compared to traditional measures of interviewer effects, such as rho-int, which focus on interviewers as a source of additional variance.

En este artículo consideramos un estimador de Bouza-Herrera (2013) en el caso de no respuesta. Justificamos que este estimador es condicionalmente sesgado para estimar la media poblacional. Obtenemos su sesgo y su varianza condicionales. Proponemos otro estimador insesgado en las mismas condiciones, calculamos su varianza y damos un estimador insesgado de la varianza del estimador propuesto. Comentamos el interés científico de los resultados en las encuestas.

There are a few sampling methods available to survey households in situations where sample frames are either unavailable or are unreliable. The most popular of these methods is the expanded programme of immunization (EPI) sampling method, which has been used in low-income countries. The purpose of this paper is to explain how mathematical programming can be used to optimize EPI and other household survey sampling methods in these situations.

The UK Office for National Statistics provides a thin -client remote laboratory service for secure research on confidential microdata. This technological solution is allied to tight procedural environment and compulsory training of researchers to achieve an effective mix of practicality and security. The result has been a ten-fold increase in the use of the ONS business data by external researchers, and a significant increase in ONS' in-house capabilities and projects. This solution raises several potential problems: the irreducible person risk from providing access to identifiable data; a need for "intelligent" disclosure control policies; the management of off-site access to data; and the possibility of "unwanted" recreation of official data. This paper discusses how these have been addressed.

We examine the effects of interviewer-respondent familiarity on both response patterns and rates of item nonresponse when self-administered questions (SAQs) are used. We use SAQ data from a survey in which the researchers experimentally ensured that there would be varying degrees of familiarity between interviewers and respondents. Our results reveal only minimal differences in response patterns by degree of prior acquaintance between interviewer and respondent, indicating that SAQs are effective at eliminating potential bias stemming from such relationships. Our results for item nonresponse depend on how we measure the relationship between interviewer and respondent; but in all cases, it is clear that prior knowledge of one another, far from harmful, leads to low nonresponse rates to SAQs. Thus, researchers using SAQs may not need to adhere to the norm that interviewers and respondents must be strangers, with practical and cost-effective consequences for data collection.

Here, small area estimation is applied in the sense that we are "borrowing strength" from data outside of given subpopulations for which we are to publish estimated totals, or means, or ratios of totals. We will consider estimated totals for establishment surveys. A subpopulation for which we wish to estimate a total will be called a "publication group" (PG), and data that may be modeled together, using one regression, will be called an "estimation group" (EG). See Knaub(1999, 2001, 2003) regarding this for a more complex application. When a PG consists of a set of EGs, that is stratification. When an EG contains PGs, this is a simple form of small area estimation because we are using data outside of a given publication group to help estimate statistics/parameters for that model, used to estimate for each impacted PG. (In Knaub(1999, 2001), there are overlapping 'areas' as well.) Here we consider very small areas (PGs), which may fall within a &...

Trust plays a crucial role in survey participation, among other factors. Acting upon Eleanor Singer's calling for more theory, our proposal for workshop discussion is the applicability of recent advances in trust theory to survey incentive usage. Current approaches are mainly based on Social Exchange Theory, and its obscure trust notion emerging out of reciprocal relationships. Trust is assumed to be learnt in the course of recurring cost-benefit calculations in social exchange situations. However, the cognitive approach of trust conceptualisation to which this concept belongs to has been since proven to describe a calculative, strategic, interest-driven, shrewd, and/or manipulative frame of mind-the direct opposite of trust. Designing incentive strategies on such bases may well result in (unintended) manipulation. Any perception of manipulative intent can lead to reactance, alienation, and the feeling of exploitation, inducing subjective isolation from the perceived manipulator. Even the subliminal 'nudges' of behavioural economic choice architectures are manipulative, for they are to reduce subjective relevance of objective alternatives, thus also harming the autonomy of prospective respondents. Instead of counting on ill-assumed learnt trust mechanisms, incentive design could be based on the concept of 'built-in' basic trust described by the non-cognitive approaches. Just like doubt, distrust is learnt rather than trust. It may keep innate human trusting in latency, therefore incentive usage should be shifted from 'trust-triggering/provoking' design to engender the irrelevance of distrust regarding the response situation. The situation remains non-intrusive, even trust-assuming if the institute demonstrates trust, thus empowers the respondent and facilitates her relaxing into trust. In contrast with attempting to (gently) force out reciprocity, respondent autonomy is left unharmed. A properly tailored leverage-saliency design may be adequate for this purpose, but the appropriate form of incentives is yet to be found. Our early-stage research focuses on the development and testing of this approach.

From InterStat, http://interstat.statjournals.net/
Weighted least squares regression through the origin has many uses in statistical science. An important use is for the estimation of attribute totals from establishment survey samples, where we might use quasi-cutoff sampling. Two questions in particular will be explored here, with respect to survey statistics: (1) How do we know this is performing well? and (2) What if the smallest members of the population appear to behave differently? This review article contains a summary of conclusions from experimental findings, and explanations with numerous references.
Key Words: Establishment Surveys, Heteroscedasticity, Model-Based Classical Ratio Estimator, Multiple Attributes, Nonsampling Error, Prediction, Regression Through the Origin, Total Survey Error, Weighted Least Squares Regression

Statistical examinations of deterministic and stochastic response propensity assert that a sample case's propensity is determined by fixed respondent characteristics. The perspective of this article, that of dynamic response propensities, differs, viewing sample cases' propensities as evolving over the course of the data collection. Each sample case begins the data collection period in a "base" response propensity. Each change in the data collection protocol which the survey organization subsequently makes might change that base propensity. This article examines four questions: (1) Is there any evidence that the average response propensities of sampled individuals vary over the data collection? (2) Is there any evidence that propensities are influenced in accordance with specific actions taken by the survey recruitment protocol?

This FORS guide discusses the effect of incentives on survey completion/response rate, sample composition, and response quality. The guide particularly emphasizes the effects of different types of incentives – conditional vs. unconditional, monetary vs. non-monetary – for encouraging survey participation and reducing panel survey attrition in the Swiss context.

When potential respondents consider whether or not to participate in a telephone interview, they have very little information about the interviewer, aside from how he/she sounds, speaks and interacts. Yet interviewers vary widely in how often their invitations lead to participation, suggesting that potential respondents may give considerable weight to interviewers' verbal attributes. We examine the impact of three interactional behaviors: disfluencies (um, uh), backchannels (mm-hmm, I see), and simultaneous speech, on the outcome of specific telephone survey invitations, agree-to-participate, scheduledcallback, and refusal, in a corpus of 1215 audio-recorded survey introductions (contacts). Agreement was highest when interviewers were moderately disfluent. Further, in contacts where "answerers" ultimately agreed to participate, they displayed more backchannels than when they refused. Finally, there was more simultaneous speech in contacts where answerers ultimately refused to participate, but interviewers interrupted answerers more when they scheduled a callback, perhaps reflecting their attempt to salvage the contact. We conclude by discussing next analytic steps, as well as practical implications for interviewer hiring and training.

As survey response rates decline, correlates of survey participation may also be changing. Panel studies provide an opportunity to study a rich set of correlates of panel attrition over time. We look at changes in attrition rates in the American National Election Studies from 1964 to 2004, a repeated panel survey with a two-wave pre-post election design implemented over multiple decades. We examine changes in attrition rates by three groups of variables: sociodemographic and ecological characteristics of the respondent and household, party affiliation and political and social attitudes recorded at the first interview, and paradata about the first wave interview. We find relatively little overall change in the pre-post election panel attrition rates, but important changes in demographic correlates of panel attrition over time. We also examine contact and cooperation rates from 1988 to 2004.

When potential respondents consider whether or not to participate in a telephone interview, they have very little information about the interviewer, aside from how he/she sounds, speaks and interacts. Yet interviewers vary widely in how often their invitations lead to participation, suggesting that potential respondents may give considerable weight to interviewers' verbal attributes. We examine the impact of three interactional behaviors: disfluencies (um, uh), backchannels (mm-hmm, I see), and simultaneous speech, on the outcome of specific telephone survey invitations, agree-to-participate, scheduledcallback, and refusal, in a corpus of 1215 audio-recorded survey introductions (contacts). Agreement was highest when interviewers were moderately disfluent. Further, in contacts where "answerers" ultimately agreed to participate, they displayed more backchannels than when they refused. Finally, there was more simultaneous speech in contacts where answerers ultimately refused to participate, but interviewers interrupted answerers more when they scheduled a callback, perhaps reflecting their attempt to salvage the contact. We conclude by discussing next analytic steps, as well as practical implications for interviewer hiring and training.

There are a few sampling methods available to survey households in situations where sample frames are either unavailable or are unreliable. The most popular of these methods is the expanded programme of immunization (EPI) sampling method, which has been used in low-income countries. The purpose of this paper is to explain how mathematical programming can be used to optimize EPI and other household survey sampling methods in these situations.

Data editing, a crucial task in the data production process, has received little scientific attention. Consequently, there is no consensus among social scientists about how data should be edited or by whom. While some argue that it should be left to data managers and data users, others claim that it is primarily a task for fieldworkers. The authors review these divergent approaches to editing and evaluate the underlying theoretical arguments. Results are reported from a methodological experiment in which different types of actors who are party to the data production and research process were asked to solve artificially generated inconsistencies in real survey data. Results are informative on two counts. First, the least accurate editors were the researchers with no field experience in the survey sites. Second, when provided with only partial information on which to make editing decisions, fieldworkers edited more accurately than both data managers and data users.

This paper discusses recent developments in the UK Office for National Statistics in linking business data. The regulatory background, the history of the Business Data Linking Project, and a number of important results arising are described. Creating good microdata from survey data collected for aggregate analysis causes some problems, and other difficulties with generating usable data from micro studies are considered. The paper notes recent developments which improve the analytical process and offers suggestions to improve the effectiveness of microdata linking. Zusammenfassung: In dieser Studie werden neuere Entwicklungen der nationalen Statistikbehörde (UK Office for National Statistics) zum Zusammenführen von Wirtschaftsdaten diskutiert. Die Studie beschreibt den rechtlichen Hintergrund bezüglich Regulierungsgesichtspunkten, die geschichtliche Entwicklung des Projekts "Business Data Linking Project" sowie einige wichtige bisher erzielte Ergebnisse. Thema sind etwa das...

When potential respondents consider whether or not to participate in a telephone interview, they have very little information about the interviewer, aside from how he/she sounds, speaks and interacts. Yet interviewers vary widely in how often their invitations lead to participation, suggesting that potential respondents may give considerable weight to interviewers' verbal attributes. We examine the impact of three interactional behaviors: disfluencies (um, uh), backchannels (mm-hmm, I see), and simultaneous speech, on the outcome of specific telephone survey invitations, agree-to-participate, scheduledcallback, and refusal, in a corpus of 1215 audio-recorded survey introductions (contacts). Agreement was highest when interviewers were moderately disfluent. Further, in contacts where "answerers" ultimately agreed to participate, they displayed more backchannels than when they refused. Finally, there was more simultaneous speech in contacts where answerers ultimately refused to participate, but interviewers interrupted answerers more when they scheduled a callback, perhaps reflecting their attempt to salvage the contact. We conclude by discussing next analytic steps, as well as practical implications for interviewer hiring and training.

From InterStat, 2011,  http://interstat.statjournals.net/  -   
Abstract: Establishment surveys collected weekly are obvious candidates for cutoff sampling with prediction for estimation of the universe and estimation of relative standard errors for estimated totals, if there are good regressor data available. In the case investigated here, there are monthly census data that may be used for regressor data. The same data element/attribute collected in a previous census is often the best single regressor, but there may be others.  For example, crude oil imports by country of origin sampled weekly may use the same data element from a monthly sample as one regressor, but may also use imports from all other countries as a second regressor, as suppliers may change. This is analogous to fuel switching for electric power plants, Knaub (2003), where multiple regression estimation has been used. Small area estimation is used there, and may also be used here. A number of optional estimators for totals and their variances are suggested, and some of them are examined closely below for one regressor. This is preliminary work in an attempt to improve estimation, and to provide estimates of relative standard errors (RSEs) for better decision making by US Energy Information Administration (EIA) staff, regarding what data to collect, and for customers of EIA publications to make informed use of these results. Currently, forecasting is used for imputation for the weekly petroleum survey considered here, but prediction has been proven, see Knaub (1999, 2009), to perform well for this also, and includes the impact on the RSE estimates, and will cause results to better reflect the current status of the petroleum market. Methods developed for these surveys, could easily apply to other establishment surveys. The simplest option here is also the newest method being used for a survey of natural gas. This is part of ongoing work at the US EIA. 

Key Words: Quasi-cutoff sampling, small area estimation, borrowing strength, prediction, weighted least squares
regression, combined estimator

Coca is a native bush from the Amazon rainforest from which cocaine, an illegal alkaloid, is extracted. Asking farmers
about the extent of their coca cultivation areas is considered a sensitive question in remote coca growing regions in Peru. As
a consequence, farmers tend not to participate in surveys, do not respond to the sensitive question(s), or underreport their
individual coca cultivation areas. There is a political and policy concern in accurately and reliably measuring coca growing
areas, therefore survey methodologists need to determine how to encourage response and truthful reporting of sensitive
questions related to coca growing. Specific survey strategies applied in our case study included establishment of trust with
farmers, confidentiality assurance, matching interviewer-respondent characteristics, changing the format of the sensitive
question(s), and non enforcement of absolute isolation of respondents during the survey. The survey results were validated
using satellite data. They suggest that farmers tend to underreport their coca areas to 35 to 40% of their true extent.