Christian University of Indonesia

Parlindungan Pardede

Definition
Survey research designs are procedures in quantitative research in which investigators administer a survey to a sample or to the entire population of people to describe the attitudes, opinions, behaviors, or characteristics of the population. It is a useful design to use when researchers seek to collect data quickly and economically, study attitudes and opinions, and survey geographically dispersed individuals.

Goals of Surveys
To describe trends, such as high school students preferences in having native speaker teachers or nonnative ones. To determine individual opinions about policy issues, such as whether English should be taught in primary schools. To identify important beliefs and attitudes of individuals, such as college students perceptions on the use of blogs to develop writing skills. To get information necessary to evaluate programs in schools, such as the success of using multimedia laboratory in English teaching.

Key Characteristics
1. Sampling from a population 2. Collecting data through questionnaires or interviews 3. Designing instruments for data collection 4. Obtaining a high response rate

Population, Target Population, and Sample

Senior High School Students in Jakarta Students of SMAN 16 Jakarta 11th Graders of SMAN 16 Jakarta

Types of Sampling Methods

Sampling Techniques Random Sampling Techniques Non-Random Sampling Techniques

Convenience Sampling

Judgmental Sampling

Quota Sampling

Snowball Sampling

Simple Random Sampling

Systematic Sampling

Stratified Sampling

Cluster Sampling

Other Sampling Techniques

Convenience Sampling
Convenience sampling attempts to obtain a sample of convenient elements. Often, respondents are selected because they happen to be in the right place at the right time. use of students, and members of social organizations mall intercept interviews without qualifying the respondents department stores using charge account lists people on the street interviews

Judgmental Sampling
Judgmental sampling is a form of convenience sampling in which the population elements are selected based on the judgment of the researcher. test markets purchase engineers selected in industrial marketing research bellwether precincts selected in voting behavior research expert witnesses used in court

Quota Sampling
Quota sampling may be viewed as two-stage restricted judgmental sampling. The first stage consists of developing control categories, or quotas, of population elements. In the second stage, sample elements are selected based on convenience or judgment. Population composition Percentage 48 52 ____ 100 Sample composition Percentage 48 52 ____ 100 Number 480 520 ____ 1000

Control Characteristic Sex Male Female

Snowball Sampling
In snowball sampling, an initial group of respondents is selected, usually at random. After being interviewed, these respondents are asked to identify others who belong to the target population of interest. Subsequent respondents are selected based on the referrals.

Simple Random Sampling

Each element in the population has a known and equal probability of selection. Each possible sample of a given size (n) has a known and equal probability of being the sample actually selected. This implies that every element is selected independently of every other element.

Systematic Sampling
The sample is chosen by selecting a random starting point and then picking every ith element in succession from the sampling frame. The sampling interval, i, is determined by dividing the population size N by the sample size n and rounding to the nearest integer. When the ordering of the elements is related to the characteristic of interest, systematic sampling increases the representativeness of the sample. If the ordering of the elements produces a cyclical pattern, systematic sampling may decrease the representativeness of the sample. For example, there are 100,000 elements in the population and a sample of 1,000 is desired. In this case the sampling interval, i, is 100. A random number between 1 and 100 is selected. If, for example, this number is 23, the sample consists of elements 23, 123, 223, 323, 423, 523, and so on.

Stratified Sampling
A two-step process in which the population is partitioned into subpopulations, or strata. The strata should be mutually exclusive and collectively exhaustive in that every population element should be assigned to one and only one stratum and no population elements should be omitted. Next, elements are selected from each stratum by a random procedure, usually SRS. A major objective of stratified sampling is to increase precision without increasing cost.

Stratified Sampling
The elements within a stratum should be as homogeneous as possible, but the elements in different strata should be as heterogeneous as possible. The stratification variables should also be closely related to the characteristic of interest. Finally, the variables should decrease the cost of the stratification process by being easy to measure and apply. In proportionate stratified sampling, the size of the sample drawn from each stratum is proportionate to the relative size of that stratum in the total population. In disproportionate stratified sampling, the size of the sample from each stratum is proportionate to the relative size of that stratum and to the standard deviation of the distribution of the characteristic of interest among all the elements in that stratum.

Cluster Sampling
The target population is first divided into mutually exclusive and collectively exhaustive subpopulations, or clusters. Then a random sample of clusters is selected, based on a probability sampling technique such as SRS. For each selected cluster, either all the elements are included in the sample (one-stage) or a sample of elements is drawn probabilistically (two-stage). Elements within a cluster should be as heterogeneous as possible, but clusters themselves should be as homogeneous as possible. Ideally, each cluster should be a small-scale representation of the population. In probability proportionate to size sampling, the clusters are sampled with probability proportional to size. In the second stage, the probability of selecting a sampling unit in a selected cluster varies inversely with the size of the cluster.

Types of Cluster Sampling

Cluster Sampling

One-Stage Sampling

Two-Stage Sampling

Multistage Sampling

Simple Cluster Sampling

Probability Proportionate to Size Sampling

Three Types of Information Surveys Can Provide

Factual information, like the characteristics of individual teachers and learners (e.g., students age, gender, ethnicity, language background, proficiency level, etc.). Behavioral information, i.e. the one that describe what students or teachers have done or regularly do in terms of their language teaching and learning, such how often students look up unfamiliar words in a dictionary or make an outline before they write an essay. Attitudinal information, which depict the opinions, beliefs, or interests of teachers or learners. These questions are often used in needs analysis when researchers want to gather information on such topics as what learning goals students have or what skill areas they are most interested in.

Types of Surveys

CrossSectional
Survey Longitudinal

data are collected at one point in time

data are collected over time

CROSS-SECTIONAL vs. LONGITUDINAL SURVEY

Types of Survey: CROSS-SECTIONAL

A cross-sectional study is one that produces a snapshot of a population at a particular point in time. The researcher collects data at one point in time in order to: (1) to examine current attitudes, beliefs, opinions, or practices; (2) to compare two or more educational groups (students with students, students with teachers, students with parents) in terms of attitudes, beliefs, opinions, or practices; (3) to measure community needs of educational services as they relate to programs, courses, or school facilities projects; (4) to evaluate a program, such as a survey that provides useful information to decision makers.

Types of Survey: LONGITUDINAL

The researcher collects data to study individuals over time. This design is differentiated into: (1) Trend study, involves identifying a population and examining changes within that population over time, e.g., Gallup Poll, which is used during elections to monitor trends in the population of voters from the primary to the final election; (2) cohort study, in which a researcher identifies a subpopulation based on some specific characteristic and then studies that subpopulation over time. For instance, a group of 18-year-old students is studied in the year 2001. Five years later (in 2006), a group of 23-year-olds is studied. (They may or may not be the same individuals studied in 2001.) Five years after that (in 2011), a group of 28-year-olds is studied.

Questionnaire Telephone
Surveys

Mail Web/Email
Face-to-Face Interviews

Survey Procedure

Questionnaire Design
Tips for designing questionnaire: 1) Consider whether a survey instrument is available to measure your variables 2) Consider modifying an existing instrument 3) design your own instrument by following 3 steps: Write different types of questions, including personal, attitudinal, and behavioral questions; sensitive questions; and closed-and open-ended questions. Use strategies for good question construction, i.e. using clear language, making sure the answer options do not overlap, and posing questions that are applicable to all participants. Perform a pilot test of the questions, and make revision based on obtained feedback.

Guidelines for Designing Questionnaire

Keep the questionnaire sufficiently short (30 minutes maximum). Avoid jargon. Seek simplicity but avoid being condescending. Keep questions short, as long and complex questions are difficult to understand. Split double-barrelled questions, e.g. How long have you studied English and been in receipt of Government support? into two questions. 5. Avoid leading questions which encourage a particular answer, e.g. Do you agree that your supervisor is supportive? 6. Avoid negatively framed questions or statements which are difficult to understand, particularly when asked to agree or disagree. 7. Make sure respondents have the knowledge, otherwise you may get false answers. 8. Check terms are suitable for the context in which they are used, as meanings may vary for different age groups, religions, cultures etc. 9. Ensure the frame of reference for each question is clear e.g. when asking for the frequency of an event, specify the time period. 10. Avoid creating opinions. Respondents do not necessarily hold opinions on topics. Allow a no opinion alternative. 11. Use personal wording if you want the respondents to express their feelings. 1. 2. 3. 4.

A good questionnaire
is complete, i.e. gets all the data you need; is short, i.e. doesn't abuse the respondents time or concentration; asks only relevant questions; gives clear instructions; has precise, unambiguous and understandable questions; has objective questions, i.e. doesn't suggest answers; starts with general questions; has appropriate questions; puts sensitive questions at the end; is complete, i.e. gets all the data you need; is short, i.e. doesn't abuse the respondents7 time or concentration; asks only relevant questions; 0 gives clear instructions; has precise, unambiguous and understandable questions; has objective questions, i.e. doesn't suggest answers; starts with general questions; has appropriate questions; puts sensitive questions at the end;

Data Analysis Techniques

DESCRIPTIVE. Deals with the question of what things are like, not why they are that way, and includes means, standard deviations, frequency counts, graphs, and charts. ANALYTICAL. Seeks to explain relationships, causes or consequences, and include bivariate and multivariate analyses such as correlations, cross-tabulations and regressions CONTEXTUAL. Narrows down the context by reinterpreting the data for subgroups. E.g. EFL students vs. ESL students, academic vs nonacademic employees.

Instrument Validity
Construct validity: Does the questionnaire really measure the construct being examined? Criterion-related validity. Does the instrument accurately predicts (predictive validity) or diagnoses (concurrent validity) some particular variable (criterion). Content validity. Does the contents of the questionnaire really measure the variable being measured ? To achieve this, compare your questionnaire to existing related instruments. If none exists, gather expert opinion on each question on the instrument to determine whether or not it actually tests what it is supposed to.

Measures for Assuring a Surveys Reliability

Giving the same survey on two occasions to the same individuals and checking the consistency of the same response to the same item. Having the same individuals taking two forms of a survey. check the internal consistency of responses in a survey, i.e. seeing how consistently the same respondents answer similar questions formulated in different forms

References
McKay, S. L. (2006). Researching second language classrooms. New Jersey: Lawrence Erlbaum Associates, Inc., Publishers Burns. A. (2010). Doing action research in english language teaching: A guide for practitioners. New York: Routledge: Creswell, J. W. 2008. Educational research: Planning, conducting, and evaluating quantitative and qualitative research. New Jersey: Pearson Denscombe, M. (2010). The good research guide for small-scale social research projects. New York: McGraw-Hill Goddard, W & Melville, S. (2006). Research methodology: An introduction. Lansdowne: Juta & Co, Ltd. Ross, Kenneth N. (ed.). (2005). Educational research: Some basic concepts and terminology. Paris: International Institute for Educational Planning/ UNESCO.

Examples
A Survey of English Language Teaching in Lithuania: 2003-2004 by Ministry of Education and Science of the Republic of Lithuania. (Available online at http://old.smm.lt/en/stofedu/docs/edu_reform/english%20languag e%20teaching.pdf) A Survey on the Iranian ELT Community's Attitudes to Critical Pedagogy by Hossein Davari, Abutaleb Iranmehr, Seyyed Mahdi Erfani. (Available online at http://www.ccsenet.org/journal/index.php/elt/article/view/14567) Survey of The English Language Needs of the Senior Students of Physical and Biomedical Sciences by: Inga Rozgien and Albina Tre iokait . (Available online at http://www.uki.vu.lt/file/Verbum/2011/2/rozgiene_treciokaite.pdf)