Paradata and Survey Process Data

Сбор данных, несомненно - ключевой элемент во всем социальном обследовании, поскольку именно он, в конечном
счете, влияет на качество данных. В последнее время все больше и больше опросов проводится с использованием компьютерной техники, причем не только для обработки и анализа
данных, но и для их сбора и фиксации. Телефонные опросы не
являются исключением. В данной статье мы постараемся обозначить необходимость анализа дополнительной информации, собираемой в процессе телефонного интервью, получившей в зарубежном научном мире название "параданных". До сих пор в зарубежной, не говоря уже об отечественной методологии, не создана система комплексного анализа данных телефонных интервью, которая включала бы в себя сведения о самой процедуре опроса. Мы постараемся продемонстрировать
некоторые возможности, которые открывают параданные, с
точки зрения оценки работы звонкового центра и интервьюеров.

Modern technology allows for surveying through different media (e.g. Internet, mobile phones, tablets), which may influence the quality of collected data with additional effects due to the survey mode and should be pretested with the purpose of avoiding effects that would deteriorate the quality of collected data. In the present study, we analysed the technological development of surveying tools by testing the applicability of cognitive interviews on several different survey modes. We focused on the importance of the cognitive interviewer and the effect of reducing their degree of involvement on the quality of the interview results. We carried out personal interviews, interviews using voice-over-Internet protocol, interviews using programs for instant messaging, and web-based interviews; these enabled us to analyse the quality of each survey mode and recognize their advantages and deficiencies. Through the comparison of these modes and their assigned technique, we showed that the role of a cognitive interviewer is important for the quality of interviews regardless of the degree of their involvement in the survey process. However, the requirement for the pretesting situation to resemble the actual final process leads to a necessity to develop new, enhanced approaches to cognitive interviewing on trending survey modes.

Income nonresponse is a significant problem in survey data, with rates as high as 50%, yet we know little about why it occurs. It is plausible that the way respondents answer survey questions (e.g., their voice speech, and question-answering behavior) can predict
whether they will provide income data, and reflect the psychological states that produce this decision. Five questions each from 185 recorded interviews conducted by the Surveys of Consumers were selected. One was the annual household income question. Exchanges between interviewers and respondents were transcribed and coded for respondent speech and question-answering behavior. Voice pitch was extracted
mechanically using the Praat software. Speech, voice, and question-answering behaviors are used as verbal paradata; characteristics of the survey process that are not captured by
default. Verbal paradata are hypothesized to reflect respondents' affective and cognitive states, which then predict income nonresponse. It was hypothesized that indicators of respondent affect (e.g., pitch) and cognitive difficulty (e.g., disfluency) would be affected by sensitive and complex questions differently, and would predict whether respondents provide income in a dollar
amount, a bracketed range of values, or not at all. Results show that verbal paradata can distinguish between income nonrespondents and respondents, even when only using verbal paradata that occur before the income question. Income nonrespondents have lower affective involvement and express more negativity before the income question. Bracketed respondents express more signs of cognitive difficulty. Income nonresponse is predicted by behavior before the income question, while bracketed response is predicted by indicators on the income question itself. Further, question characteristics affect respondent paradata, but largely in unpredicted ways. There is evidence for psychological resource and conversationality mechanisms through which respondents reduce verbal paradata when questions are demanding, rather than increasing it as signs of trouble. The results have implications for theory of income nonresponse, specifically the role of question characteristics and respondent paradata in understanding what subjective psychological states respondents are experiencing when they answer survey questions, and how those states predict whether income is reported. There are also potential extensions to interviewer training and design of interventions that could produce more complete income data.