NAVR 611 Theory: Additional notes from

Practical Research: Planning and Design, Ninth Edition Paul D. Leedy and Jeanne Ellis Ormrod Chapter One: What is research? What research is not: Mere information gathering Mere transportation of facts from one location to another Merely rummaging for information A catchword used to get attention Research is: A systematic process of collecting, analyzing, and interpreting information (data) to increase understanding of a phenomenon about which we are interested. Characteristics of research Originates with a question or problem Requires clear articulation of a goal Requires a specific plan for proceeding Usually divides the principal problem into more manageable sub-problems Is guided by the specific research problem, question, or hypothesis Accepts certain critical assumptions Requires the collection and interpretation of data Is, by its nature, cyclical or helical Hypothesis: A logical supposition, a reasonable guess, an educated conjecture Provides a tentative explanation for a phenomenon under investigation May direct thinking to possible sources of information necessary to resolve the research problem and its sub-problems. Theory: An organized body of concepts and principles intended to explain a particular phenomenon Tentative explanations that new data either support or do not support Apt to drive further research Assumptions: Self-evident truths The bedrock upon which the study must rest Vitally important to the quality of the study

Methodology: Dictates how data are acquired Arranges data in logical relationships Sets up the approach for refining and synthesizing data Suggests how data will be interpreted Yields one or more conclusions that lead to expansion of knowledge Research: Developmental steps An initial question is asked. The initial question is formally stated as a problem. The problem is divided into sub-problems. Preliminary data are gathered. A tentative hypothesis is formed. Data are systematically collected. Data is processed and interpreted. A discovery is madea conclusion is reached. The tentative hypothesis is supported or not supported. The cycle is complete.

Chapter three: The research problem- The heart of the research process Finding research projects Research projects can be one of two types: Basic research intended to enhance basic knowledge about the physical, biological, psychological, or social world or to shed light on historical, cultural, or aesthetic phenomena. Applied research intended to address issues that have immediate relevance to current practices, procedures, and policies; intended to human decision making about practical problems; occasionally address questions in ones immediate work environment (action research). Criteria for identification of a suitable research problem The research problem should address an important question so that the answer will make a difference The research problem should advance the frontiers of knowledge by leading to
3

new ways of thinking, suggesting possible applications, or paving the way for further research in the field. Situations to avoid when considering a research problem Research projects should not be a ruse for achieving self-enlightenment. A problem whose sole purpose is to compare two sets of data is not a suitable research problem. Calculating a correlation coefficient between two sets of data to show a relationship between them is not acceptable as a problem for research. Problems that result in a yes or no answer are not suitable problems for research. Finding a Legitimate Research Problem Look around you. Read the literature. Attend professional conferences. Seek the advice of experts. Choose a topic that intrigues and motivates you. Choose a topic that others will find interesting and worthy of attention. Guidelines: Stating the Research Problem State the problem clearly and completely. Think through the feasibility of the project that the problem implies. Say precisely what you mean. -Absolute honesty and integrity are the rule! State the problem in a way that reflects an open mind about its solution. Edit your work. Dividing the Research Problem Into Sub-problems Sub-problem: the subparts of the main problem that are an integral part of the main problem. vs. Pseudo-sub-problems: procedural issues that involve decisions that must be made before resolving the research problem and its sub-problems. Characteristics of sub-problems Each sub-problem should be a completely researchable unit. Each sub-problem must be clearly tied to the interpretation of the data. The sub-problems must add up to the totality of the problem. Sub-problems should be small in number. State the hypothesis and/or research questions Hypotheses are essential to experimental research; research questions are more common in qualitative research.
4

Both hypotheses and research questions provide guidance for the kind of data that should be collected. Both hypotheses and research questions suggest how data should be analyzed and interpreted. Hypotheses and research questions may originate in the sub-problems. Hypotheses and research questions provide a position from which the researcher may initiate an exploration of the problem. Hypotheses and research questions act as checkpoints against which to test the findings that the data reveal.

Delimit the research Delimitations of the research are statements about what the researcher is not going to do. What the researcher will not do is to become involved in data extraneous to the research problem. The researcher must distinguish between what is and is not relevant to the problem. Define the terms The researcher must precisely define the terms in the problem and the subproblems. Each term should be defined as it will be used in the researchers project. In defining a term, the researcher makes the term mean whatever he or she wishes it to mean within the context of the problem and its sub-problems. operational definition = the definition of a characteristic or variable in terms of how it will be measured in the research study. State the assumptions Assumptions are basic to the research problem. All assumptions that have a material bearing on the problem should be openly and unreservedly set forth. A statement of the assumptions is necessary for others to evaluate the conclusions of the study. A statement of the assumptions reveals what the researcher may be taking for granted with respect to the problem.

Chapter four: Review of the related literature Benefits of conducting a literature review It can offer new ideas, perspectives, and approaches that may not have occurred to you. It can inform you about other researchers who conduct work in the same area. It can show you how others have handled methodological and design issues in studies similar to your own. It can reveal sources of data that you may not have known existed. It can introduce you to measurement tools that other researchers have developed and used effectively. It can reveal methods of dealing with problem situations that may be similar to difficulties you are facing. It can help you interpret and make sense of your findings and, ultimately, help you tie your results to the work of those who have preceded you. It will bolster your confidence that your topic is one worth studying, because you will find that others have invested considerable time, effort, and resources in studying it. In short: the more you know about investigations and perspectives related to your topic, the more effectively you can tackle your own research problem. Evaluating, organizing, and synthesizing the literature Never take other peoples conclusions at face value; determine for yourself whether their conclusions are justified based on the data presented. Organize the ideas you encounter during your review. Synthesize what youve learned from your review: o Compare and contrast varying theoretical positions on the topic. o Show how approaches to the topic have changed over time. o Describe general trends in the research findings. o Identify and explain discrepant or contradictory findings. o Identify general themes that run throughout the literature. Guidelines: Writing a clear and cohesive literature review Get the proper psychological orientation. Have a plan. Emphasize relatedness (i.e., how the literature is related to the problem). Give credit where credit is due; use appropriate citations. Review the literature, dont reproduce it. Summarize what youve said. Remember that your first draft will not be your last draft. Ask others for advice and feedback.

Chapter five: Planning your research project Identifying appropriate measurement instruments We pin down data by measuring it in some way. Measurement instruments provide a basis on which the entire research effort rests. A research effort employing faulty measurement tools is of little value in solving the problem under investigation. In planning the research project, the nature of the measurement instruments should be clearly identified. Instrumentation should be described in explicit, concrete terms. Instruments should have a reasonable degree of validity and reliability. The validity of measurement instruments Face Validity: the extent to which an instrument looks like its measuring a particular characteristic; relies on subjective judgment. Content Validity: the extent to which a measurement instrument is a representative sample of the content area being measured. Criterion Validity: the extent to which the results of an assessment correlate with another, related measure. Construct Validity: the extent to which an instrument measures a characteristic that cannot be directly observed but is assumed to exist (a construct, such as intelligence). Determining the reliability of a measurement instrument Inter-rater reliability: the extent to which two or more individuals evaluating the same product or performance give identical judgments. Internal consistency reliability: the extent to which all of the items within a single instrument yield similar results. Equivalent forms reliability: the extent to which two different versions of the same instrument yield similar results. Test-retest reliability: the extent to which a single instrument yields the same results for the same people on two different occasions. Quantitative vs Qualitative approaches Quantitative Research: involves looking at amounts, or quantities, of one or more variables of interest. Researchers attempt to measure variables in some way. Qualitative Research: involves looking at characteristics, or qualities, that cannot easily be reduced to numerical values. Researchers attempt to examine nuances and complexities of a particular phenomenon. Quantitative and Qualitative Processes: formation of one or more hypotheses
7

review of the related literature collection and analysis of data Quantitative vs Qualitative research: A Comparison Purpose: quantitative -- seeks explanations and predictions that are generalizable qualitative -- seeks better understanding of complex situations Process: quantitative -- methods defined before study begins; allows objective measurement; researchers remain detached qualitative -- holistic/emergent; researchers remain open and immerse themselves in the complexity of the situation and interact with participants; variables emerge from data. Data Collection: quantitative -- data collected in form that is easily converted to numbers qualitative -- researcher is the research instrument; verbal and nonverbal data collected. Reporting Findings: quantitative -- data reduced to averages; scientific styles of reporting qualitative -- construct interpretive narratives from data Internal vs External validity Internal Validity: the extent to which the design and data of a research study allow the researcher to draw accurate conclusions about cause-and-effect and other relationships within the data. External Validity: the extent to which the results of a research study apply to situations beyond the study itself; the extent to which conclusions can be generalized. Strategies to increase internal validity Conduct a controlled laboratory study Conduct a double-blind experiment Use unobtrusive measures Use triangulation Strategies to increase external validity Conduct the study in a real-life setting. Use a representative sample. Replicate the study in a different context Validity in qualitative research Use triangulation. Spend extensive time in the field.
8

Conduct a negative case analysis. Use thick description. Seek feedback from others.

Ethical Issues in Research Assurances Honesty with professional colleagues Protection from harm Informed consent Right to privacy Professional code of ethics Chapter six: Writing the research proposal Practical application: Writing your proposal Use a computer or other word processor. Adhere to guidelines required by the institution, organization, or funding agency to which you are submitting the proposal. When writing the first draft, focus more on organization and logical thought sequences than on exact wording, grammatical correctness, and spelling. Present the research problem at the beginning of the first chapter. Provide a context for your research problem. Assume that your readers know nothing about your proposal. Communicate that you have an open mind about what you will find. Describe your proposed methodology with as much detail and precision as possible. If you intend to use data that already exist, describe where the data are located and how you plan to obtain them. Describe how you will use the data to answer your research problem. Use appendices to present informed consent letters, specific measurement instruments, and other detailed materials. Guidelines: Revising your proposal Set the proposal aside for a few days. Read a paper (rather than an electronic) copy of your first draft. Carefully scrutinize what you have written, looking for disorganized thoughts, illogical thinking, and inconsistencies in terminology. Look for places where you are not as clear as you might be. Keep your sentences simple and straightforward. Choose your words carefully. Check carefully for errors in grammar, punctuation, and spelling. Make sure that items in bulleted lists are parallel in structure. Make sure there is a one-to-one correspondence between the in the text and the references in the reference list.
9

Consider the feasibility of your project once again. Print out your second draft and read your proposal carefully once again. See the feedback of others.

Common Weaknesses in Research Proposals Weaknesses related to the research problem: unclear purpose of the research unimportant problem poorly supported hypothesis problem is too complex problem has limited relevance to the field Weaknesses related to the research design and methodology: vague and unfocused description of design and method data inappropriate for the research problem methodology/procedures inappropriate for the research problem appropriate controls lacking or inadequate outdated or inappropriate equipment poorly considered statistical analysis Weaknesses related to the investigator: insufficient training or experience for the proposed research unfamiliar with the literature relevant for the proposed research insufficient time to devote to the project Weaknesses related to resources: institutional setting is unfavorable for the proposed research proposed use of equipment, support staff, or other resources is unrealistic.

Detractions from proposal effectiveness: Any research proposal The statement of the problem is vague. The methodology is incompletely described. The proposed treatment of each sub-problem is general and cursory. Criteria for the admissibility of the data are weak or nonexistent. The proposal lacks sharpness. The proposal is phrased in terms that are not useful for evaluation. The format of the proposal deviates from institutional guidelines.

10

Chapter seven: Qualitative research Approaches to Qualitative Research Focus is on phenomena that occur in natural settings Involves studying those phenomena in all of their complexity Researchers recognize the multifaceted form of the issue they are studying The researcher is the instrument The nature of the research problem and planning in qualitative research Qualitative researchers formulate only general research problems and ask only general questions about the phenomena they study. Qualitative researchers may not identify ahead of time the exact methods they will use. The methodology may continue to evolve over the course of the study. Qualitative research requires considerable preparation and planning. Qualitative studies do not allow the researcher to identify cause-and-effect relationships. Qualitative Research Designs Case Study Ethnography Phenomenological Study Grounded Theory Study Content Analysis Case Study Definition: In a case study, a particular individual, program or event is studied in depth; findings may not be generalizable. Method: The researcher collects extensive data on the individual(s), program(s), or event(s) on which the investigation is focused. Data Analysis: involves (1) organization of details about the case, (2) categorization of data, (3) interpretation of single instances, (4) identification of patterns, and (5) synthesis and generalizations. Research Report: includes (1) a rationale for studying the case, (2) adetailed description of facts related to the case, (3) a description of data that was collected, (4) a discussion of patterns found, and (5) a connection to the larger scheme of things. Ethnography Definition: In ethnography, the researcher looks in depth at an entire group that shares a common culture; it is especially useful for understanding complexities of a particular, intact socio-cultural group. Method: Site-based fieldwork is the essence of any ethnography. The researcher depends on a gatekeeper and key informants and is a careful observer.
11

Data Analysis: Data collection and data analysis occur somewhat simultaneously. Steps in data collection include (1) description, (2) analysis, and (3) interpretation. The research strives for rigorous subjectivity. Research Report: Often written in a personal, literary style and includes (1) an introduction providing a rationale for the study, (2) a description of the setting and methods, (3) an analysis of the group studies, and (4) a conclusion.

Phenomenological Study Definition: A phenomenological study attempts to understand peoples perceptions, perspectives, and understandings of a particular situation.The researcher tries to answer the question, What is it like to experience such-andsuch? Method: Phenomenological researchers depend almost exclusively on lengthy interviews with a carefully selected sample of participants. The phenomenological interview is often unstructured in which the researcher and participants to arrive at the heart of the matter. Data Analysis: The central task in data analysis is to identify common themes in peoples descriptions of their experiences. Steps include (1) identifying relevant statements, (2) grouping statements into meaning units, (3) seeking divergent perspectives, and (4) constructing a composite. Research Report: There is no specific structure but the report should produce understanding of the experience. Grounded Theory Study Definition: The major purpose of a grounded theory study is to begin with the data and use them to develop a theory. The study uses a prescribed set of procedures for analyzing data and constructing a theoretical model from them the theory is grounded in the data. Method: Data collection is field-based, flexible, and likely to change over the course of the study. Interviews typically play a major role. A constant comparative method is used in that data analysis drives later data collection. Data Analysis: Data analysis includes (1) open coding, (2) axial coding, (3) selective coding, and (4) development of a theory. No matter what form the theory takes, it is based entirely on the data collected. The Research Report: Writing is objective and impersonal and includes(1) a description of the research question, (2) a review of the related literature, (3) a description of methodology and data analysis, (4) a presentation of the theory, and (5) a discussion of implications. Content Analysis Definition: A content analysis is a detailed and systematic examination of the contents of a particular body of material for the purpose of identifying patterns, themes, or biases. A content analysis is typically performed on forms of human communication and involves the greatest amount of planning at the front end of the project.
12

Method: A content analysis is systematic and includes (1) identification of the material to be studied, (2) definition of the characteristics to be studied, (3) a breakdown of complex items into smaller segments, and (4) scrutiny of material for identified characteristics under study. Data Analysis: Data analysis involves tabulation of the frequency of each characteristic found in the material studied. Tabulations and statistical analyses are used to interpret the data. The Research Report: The research report includes (1) a description of the material studied, (2) precise definitions of material characteristics, (3) the coding or rating procedures, and (4) tabulations for each characteristic. Data Collection in Qualitative Research Multiple forms of data are used. Data collected early in the investigation may influence subsequent data. Potential sources of data are unlimited. Data collection takes a great deal of time. Data collection methods should be consistent with ethical principles of research studies. Data Collection Procedures Sampling: choice of sample depends on the research question tends to be nonrandom in selection of data sources is purposeful Observations: researcher may be an outsider or a participant observer intentionally unstructured and free-flowing can be problematic and lack objectivity Interviews: can yield a great deal of information tend to be unstructured and open-ended occasional use of focus groups Guidelines: Conducting interviews Identify some questions in advance. Consider participants cultural backgrounds on their responses. Make sure interviewees are representative of the group. Find a suitable location. Get written permission. Establish and maintain rapport. Focus on the actual rather than on the abstract or hypothetical. Dont put words in peoples mouths. Record responses verbatim. Keep your reactions to yourself.
13

Remember that you may not be getting the facts. When conducting a focus group, take group dynamics into account

Organizing and Analyzing Data Employ a data analysis spiral: Organize the data; Peruse the data set to get a sense of what it contains as a whole; Identify general categories or themes (and possible subthemes), then classify data accordingly; Integrate and summarize the data. Chapter ten: Experimental and Ex Post Facto Designs Independent and dependent variables Variable: any quality or characteristic in a research investigation that has two or more values. Cause-and-effect relationship: the extent to which one variable (the cause) influences another variable (the effect). Independent variable: a variable that the researcher studies as a possible cause of something else; the variable that the researcher directly manipulates. Dependent variable: a variable that is potentially influenced by the independent variable; a variable that is influenced by and to some extent depends on the independent variable. The Importance of Control Internal validity: the extent to which the design of a research study and the data it yields allows the researcher to draw accurate conclusions about cause-andeffect and other relationships. Without internal validity in experimental designs, the results are not interpretable. Confounding variables: account for differences in two or more groups that are not attributable to the particular treatment or intervention being studied. Strategies for Controlling Confounding Variables Keep some things constant. Include a control group. Randomly assign people to groups. Assess equivalence before the treatment with one or more pretests. Expose participants to all experimental conditions. Statistically control for confounding variables.

14

Categories of Experimental Designs Pre-Experimental Designs Not possible to show cause-and-effect relationships because (a) the independent variable doesnt vary or (b) experimental and control groups are not comprised of equivalent or randomly selected individuals. True Experimental Designs Compared to pre-experimental designs, experimental designs offer a great degree of control and greater internal validity. Quasi-Experimental Designs Randomness is not possible or practical; cant control for all confounding variables. Ex Post Facto Designs The researcher identifies events that have already occurred or conditions that are already present and then collects data to investigate a possible relationship between these factors and subsequent characteristics or behaviors. - like correlational research, ex post facto research involves looking at existing circumstances; - like experimental research, ex post facto research has clearly identifiable independent and dependent variables; - unlike experimental research, ex post facto research involves no direct manipulation of the independent variable the presumed cause has already occurred. Factorial Designs Examination of the effects of two or more independent variables in a single study. Chapter Eleven: Strategies for Analyzing Quantitative Data Exploring and Organizing a Data Set Statistics: a group of computational procedures that enable us to find patterns and meaning in numerical data. Principles Exploration of Data: 1. Where two variables are concerned, one of the variables becomes dominant and governs meaning that emerges from the other. 2. Whatever the researcher does with the data to prepare it for inspection or interpretation will affect the meaning that the data reveal. Therefore, every researcher should be able to provide a clear, logical rationale for the procedure used to arrange and organize the data.

15

Using the Computer to Organize and Analyze Data Electronic Spreadsheet: a software program that allows a researcher to manipulate data displayed in a table. Uses of electronic spreadsheets: - sorting data - searching for desired information - recoding data - graphing from the data - calculating formulas - employing trial and error explorations Functions of Statistics Two Major Functions of Statistics: 1. They describe what the data looks like; this is the function of descriptive statistics. 2. They allow us to make inferences about large populations by collecting data on relatively small samples; this is the function inferential statistics. Characteristics of Statistics Estimates of population parameters Different statistics are appropriate for different kinds of data Single-group versus multi-group data Continuous versus discrete variables Scales of measurement (nominal, ordinal, interval, ratio) Normal and non-normal distributions Parametric versus nonparametric statistics Measures of Central Tendency Mode: the single number or score that occurs most frequently. Median: the numerical center of a set of data. Mean: the arithmetic average of the scores within the data set. Geometric Mean: a measure of central tendency based on a geometric progression, such as growth. Note: The configuration of the data dictates the measure of central tendency most appropriate for that particular situation. Measures of Variability: Dispersion and Deviation Range: indicates the spread of data from lowest to highest value (highest score lowest score) Average Deviation: the average of differences of each score in a set of scores and the mean score. Standard Deviation: the measure of variability most commonly used in statistical procedures; the square of the score-mean differences. Norm-Referenced Scores: scores that reflect where each person in the group is relative to other members of the group.
16

Standard Score: tells us how far an individuals performance is from the mean with respect to standard deviation units.

Choosing Appropriate Statistics Statistics related to central tendency and variability provide a beginning point from which to view data. Statistical manipulation of the data is not research. Research demands interpretation of the data. Measures of Association: Correlation Correlation: a measure of the relationship between two variables; correlation indicates the strength of the relationship. Correlation Coefficient: a number between -1 and +1; most correlation coefficients are decimals (positive or negative) somewhere between these two extremes. Positive Correlation: as one variable increases, the other variable also increases. Negative Correlation: as one variable increases, the other variable decreases. Pearson r: the most widely used statistic for measuring correlation. Note: Correlation does not necessarily indicate causation. Point Versus Interval Estimates Point Estimate: a single statistic that is taken as a reasonable estimate of the corresponding population parameter; will typically not correspond exactly with its true equivalent in the population. Interval Estimate: more accurate estimate of the population parameter. Confidence Interval: a range within whose limits a population parameter probably lies; provides a certain level of confidence that the estimated range is correct. Inferential Statistics Two main functions of inferential statistics: 1. To estimate a population parameter from a random sample. 2. To test statistically based hypotheses. Estimating Population Parameters In conducting research, we use a sample to learn about the larger population from which the sample was drawn. Inferential statistics inform how closely sample statistics approximate parameters of the overall population. Statistical estimates of population parameters are based on the assumption that the sample is randomly chosen and representative of the total population. Error: the difference between the population mean and the sample mean.
17

Standard Error of the Mean: indicates how much a particular mean is likely to vary from one sample to another when all samples are the same size and are drawn randomly from the same population.

Testing Hypotheses Null Hypothesis: a statistical hypothesis which postulates that any result observed is the result of chance alone. Testing the Null Hypothesis: the process of comparing observed data in a research study with what we would expect from chance alone. Significance Level: the probability that researchers use as a cutoff point to decide that a result has not occurred by chance. Making Errors in Hypothesis Testing Type I Error: the erroneous conclusion that a result was not due to chance when in fact it was due to chance; incorrectly rejecting the null hypothesis. Type II Error: the erroneous conclusion that a result was due to chance when in fact it was not; incorrectly failing to reject a null hypothesis that is actually false; also known as a beta error. Suggestions for Increasing the Power of a Statistical Text Use as large a sample as is reasonably possible. Maximize the validity and reliability of your measures. Use parametric rather than nonparametric statistics whenever possible. Note: Whenever we test more than one statistical hypothesis, we increase the probability of making at least one Type I error. Interpretation of the Data Interpreting the data means several things: Relating the findings to the original research problem and to the specific research questions and hypotheses. Relating the findings to preexisting literature, concepts, theories, and research studies. Determining whether the findings have practical significance as well as statistical significance. Identifying limitations of the study.

18

Chapter Twelve: Technical details: style, format, and organization of the research report Planning a Research Report Four Objectives of a Research Report: It should give readers a clear understanding of the research problem and why it merited an in-depth investigation. It should describe exactly how data were collected in an attempt to resolve the problem. It should present the data precisely and completely. The data presented in the report should substantiate all the interpretations and conclusions that the report contains. It should interpret the data for the reader and demonstrate exactly how the data resolve the research problem. Description of the Research Problem Statement of the problem and other information needed to understand it should comprise the first section of the final report. The main purpose of the first section of the research report is to create a meeting of the minds between the writer and the readers of the report. The research report provides no opportunity for imprecise thought or inexact expression. Description of the Method The method used to collect data, including the sample, assessment instruments, and procedures should be described with utmost precision. The design of the study should be clear. In particular, the researcher should state whether quantitative or qualitative methods (or both) were used and what particular research traditions were followed. Presentation of the Data The data are presented in terms of the problem. There is a one-to-one correspondence between the data and the problem and each of the sub-problems. A separate section should be devoted to each sub-problem and its pertinent quantitative and/or qualitative data. Aside from the data, present all other items of interest. Use figures and tables to display certain types of data. Present a rationale for the statistical approaches that were used. Present data as evidence for the conclusions that you draw. Indicate whether or not data supported the hypotheses. Interpretation of the Data Interpretation of the data is the essence of research.
19

Without inquiring into the intrinsic meaning of the data, no resolution of the research problem or its sub-problems is possible. Exploit the data fully. Dont go too far beyond the data and lose sight of actual findings. Dont draw conclusions about causation or influence when the design of the study doesnt warrant such conclusions. Avoid the word prove. Report honestly what the data have revealed.

Concluding the Report Conclusion should bring closure to the interpretation of the data. Concluding discussion is often in a separate section or entitled Summary, Conclusions and Recommendations. All loose threads should be gathered together. Conclusions should be entirely supported by the data presented. Final steps: state whether or not the hypotheses have been supported, identify weaknesses and limitations in the study, identify possible practical implications of the results, make recommendations for further research.

20