Creating Surveys: Evaluating Programs and Reading Research

This book is copyrighted and available as an eBook or Paperback at AMAZON English Language Book AMAZON U.S. AMAZON UK AMAZON GERMANY AMAZON FRANCE AMAZON India Kindle Version AMAZON Canada Kindle Version AMAZON Australia Kindle Version And more.. Sample read follows “This resource provides practitioners and students a systematic, easy-to-read overview of what surveys are and how to use them. Even seasoned researchers could benefit from reviewing this book and keeping it handy for reference, but undergraduate and master’s students should find it particularly useful for grasping basic research constructs and designing simple survey projects. Not only does the book explain important principles, but it also provides many clear, concrete examples and links to additional resources that the reader will find helpful.” —Joe D. Wilmoth, Ph.D., Associate Professor of Human Development and Family Science, Mississippi State University “Many researchers find themselves in contexts where they have access to groups of people whose values, opinions, behaviors, emotions, and beliefs are worth studying, but too many then use surveys that are poorly constructed or inappropriately worded. Sutton’s Creating Surveys is a treasure trove of valuable resources and easy to follow instructions that has been created to increase the validity and reliability of survey research. It is a long overdue addition to the researcher’s toolkit” —Johan Mostert, Ph.D., Director of Research, CompACT "Creating Surveys is a practical and readable handbook for researchers providing a broad look at the many facets of survey research. I highly recommend this text for doctoral students and researchers utilizing and creating survey research. Dr. Sutton provides objectives to establish a clear purpose for each chapter. He brings to life quality survey components with clear examples from a variety of topics and fields of study." —Shonna Crawford, Ph.D., Associate Professor of Education in Literacy, Education Department Chair, Evangel University “Having guided dozens of Doctor of Ministry students through research projects for over ten years, I can attest to the fact that achieving quality design of assessment tools—and accomplishing proper interpretation of survey results—can present considerable challenges to students. Geoff Sutton’s Creating Surveys, with each chapter’s clear objectives, well-organized content, and chapter summaries, will serve research students, professionals, and community leaders well in their efforts to better understand the necessary components and methodologies for gaining desired information and presenting it in a way that reflects critical thinking about survey results.” —Dr. Lois E. Olena, Associate Professor and Doctor of Ministry Project Coordinator, Assemblies of God Theological Seminary "In this day and age, being on the cutting edge is more important than ever. Science allows us to better understand the world around us and effectively impact change, innovation, and growth. Yet, many are intimidated by the confusing process of gathering and interpreting empirical data and give up before they even begin. Creating Surveys simplifies this process and can help researchers, professionals, students, and lay people alike easily access data that can be used to affect change in the future. I only wish that this resource was around when I was a graduate student!" —Luke J. Davidiuk, PsyD, Capt., USAF, BSC ADAPT Element Chief, Vandenberg AFB, CA CREATING SURVEYS Evaluating Programs and Reading Research Geoffrey W. Sutton CREATING SURVEYS Evaluating Programs and Reading Research Geoffrey W. Sutton Sunflower Press Copyright © Geoffrey W. Sutton, Ph.D. 2018. All rights reserved. No portion of this book may be reproduced or transmitted in any form or by any means without the prior written permission of the author. ISBN: 9781522012726 To those who have encouraged me in conducting research— Thurman Vanzant, Evangel University James R. Koller, University of Missouri-Columbia Everett L. Worthington, Jr., Virginia Commonwealth University E. Grant Jones, Evangel University Table of Contents Introduction 8 Part I: Planning a Survey 1. Focusing on a Purpose 19 2. Learning from Previous Research 3. Writing Items and Questions 25 40 4. Organizing and Formatting Surveys 55 5. Getting Approval: Ethical Thinking 64 6. Selecting Respondents 69 Part II: Using Surveys to Understand People 7. Considering Multiple Dimensions 8. Assessing Social Context 9. Assessing Cognition 80 93 102 10. Assessing Behavior Patterns and Personality 11. Assessing Physical Health 111 12. Assessing Emotions and Attitudes 13. Assessing Spirituality 106 117 121 Part III: Understanding Basic Designs and Statistics 14. Using Surveys to Evaluate Workshops 126 15. Understanding Survey Results: Part 1 134 16. Understanding Survey Results: Part 2 143 17. Assessing Survey Reliability 151 18. Assessing Survey Validity 156 19. Presenting Survey Results 163 References 171 Research Vocabulary 180 Appendix A: Survey Approval Checklist Appendix B: Online Ethics Resources 200 202 Appendix C: Survey Consent Example 204 Appendix D: Survey Debriefing Example 207 Appendix E: Organizing a Spreadsheet 209 Appendix F: Examples of Survey Items 210 Appendix G: Sample Workshop Evaluation Table of Contents: Expanded Acknowledgements About the Author 228 229 219 216 Introduction People have been conducting surveys since the first person asked a question. We want to know what others think and feel. We want to know the best places to visit and where to get a good deal. Whether your school, organization, or place of worship has ten or ten thousand people, their ideas, beliefs, attitudes, practices, opinions, and commitments can help leaders make informed decisions. Surveys also help us understand what people have learned following a workshop, seminar, or other educational program. Of course, surveys are also part of formal research projects. Surveys are ways to collect information. The information obtained from surveys can be used to describe, compare, contrast, predict, or explain something about how people think, feel, and act. This book is about creating surveys and reading survey research. It is aimed at leaders and students in many fields who have not had extensive coursework and experience in research methods and statistics. What is the purpose of this book? You have probably seen a lot of surveys and found some are written or organized better than others. My primary purpose in this book is to help people create better surveys. I also hope to help readers evaluate news stories and reports of surveys. Even if you never create a survey, I hope you learn more about the usefulness and limitations of surveys. The readers most likely to find this book helpful are leaders in schools, universities, or organizations that serve people in such places as small businesses, community centers, and churches. Another category of readers who can benefit from this book is students—in a variety of fields—who will conduct a survey as a part of a research project in undergraduate or graduate programs. Readers need not have prior knowledge of statistics or research methods to benefit from this book. I hope this book will be of value to the following people. Administrators Leaders in any organization often read and participate in surveys. This book can help you better understand survey research whether you are reading the work of others or creating your own projects. In addition, many school administrators, agency CEOs, and clergy receive requests to conduct research on their campuses. This book will help you evaluate such requests. Health Care Professionals Counselors, social workers, psychologists, chaplains, and other licensed professionals are often conducting survey research or serving as team members on research projects. This book will help you review basic concepts typically taught in research methods textbooks. If you are consulting on a project, you may find this book useful to recommend to team members who are new to survey methods. Educational Professionals If you have not had coursework on survey research, this book can help you create projects to engage students, supplement your own research, or contribute to other projects in your school or professional organization. Students Undergraduate and graduate students can benefit from a review of the basic concepts covered in this book. This book does not cover the details you would find in a graduate research methods textbook, but it does review many of the key concepts needed to read and understand research and create survey projects. I hope this book is especially useful to students in disciplines that do not typically include an intensive sequence of research and statistics courses. Community Leaders Chances are you have seen a survey report in a news source or heard someone quote the results of a survey. This book can help you learn more about the survey process, participate as a team member on a research project, and think more critically about published research. How can I use this book? This book is designed to help people create surveys and think critically about survey results. Surveys can provide useful information, but the results can also be misinterpreted. Following are some ways to use this book. Create surveys to evaluate a workshop or other program Many organizations offer beneficial programs and services to the public, but many lack data demonstrating the effectiveness of their services. Leaders need to decide whether to renew or change existing programs or create new ones. People vote with their feet. Programs and services that are not helpful die for lack of support. Surveys can provide vital information about the needs of a student body, a community, or the membership in a church before creating a program that ignores critical needs. Surveys can help assess the importance of a program from the perspective of those it is designed to serve. This book offers guidance on different ways to evaluate a workshop or other educational presentations and community services. Create surveys to assess changes in attitudes, beliefs, or knowledge I have attended many educational programs. Some are better than others. Following a program, I have often been asked to give my opinion about the speaker, objectives, and setting. Sometimes I have been asked what I have learned. Educational programs help people learn more about their profession as well as life-changing events. People may learn to cope with loss, forgive others, prepare for marriage, improve their marriage, or become better parents. Programs can help lay people become better teachers, leaders, and crisis workers. Programs may also address other important issues, like sexual harassment, money management, and church security. This book includes examples of survey items that address opinions, attitudes, beliefs, and knowledge. Create surveys to help with decisions about a building project Organizations often need to make decisions about new buildings, remodeling, and relocating. Well-designed surveys can be quick and effective methods to obtain useful information. In addition to gauging attitudinal and financial support, survey items can help leaders decide on important features to include in a new project. Often, people who are not directly involved in building plans have specific knowledge that can be of great benefit before planning has gone too far. At later stages, people may be asked to vote on color schemes or other details involving options. This book can help you select ways to word your survey questions. Create surveys to understand people As organizations grow it can be useful to better understand members by collecting some basic information. Collecting information need not be intrusive. Just knowing some facts like age, sex, education, ethnicity, and occupation can provide some basic ways to think about “who we are as an organization.” In a short survey, it is easy to add a few questions to better understand the members’ opinions regarding important aspects of the organization. By the end of this book, you will find many examples of questions you may want to ask. Learn ways to evaluate survey research Survey reports are everywhere. The findings from surveys are commonly reported in news media, magazines, and books. This book can help you think more critically about research reports. You will learn how surveys may be helpful. And you will learn how survey results can be misinterpreted. You will gain a basic conceptual understanding of the statistics found in journal articles. What are some reasons not to buy this book? It may sound strange to talk about reasons not to buy this book, but the fact is, this book is not for everyone. I refer to various statistical procedures, but I do not explain how to do the calculations. Instead, I provide a reader’s conceptual approach to statistics. Similarly, if you need a detailed text covering a variety of research designs, this book will not meet that need. I refer to basic experimental studies with two or three groups and only briefly comment on advanced research methods like Structural Equation Modeling. This book can be used in undergraduate and graduate courses where students either do not need to know how to design and analyze complex research studies or get those advanced skills in other courses. How are surveys administered? Surveys can be administered in several ways. After an online survey has been created, links can be posted on online pages or in online groups. Survey links can be sent in emails or messages. Some companies offer easy-to-create surveys within their online pages. Alternatively, people can complete surveys on any available screen in a classroom, hallway, or public place. Surveys may be presented in a more personal way. For example, an interviewer can meet with people online, by phone, or in person. The interview can be conducted with one person or a group. Older methods of conducting surveys include sending paper questionnaires through the mail or asking people to complete paper forms in a public place. These older methods continue to exist because they reach people who may be missed by other methods. What is a survey? In this book, I primarily use the word survey to mean an organized set of items designed to collect information from respondents—people who respond to the items. A survey item is usually either a statement or a question. People use many different terms for groups of items that can be called a survey. Here are a few examples: polls, questionnaires, quizzes, tests, inventories, scales, and measures. A poll is often associated with elections but can simply mean a set of questions used to gather an opinion. Questionnaire and inventory are other words for sets of items used to collect responses. Measure is a generic term used by researchers who report the way they assessed characteristics of people in a research study. The measures may include collections of items called surveys, inventories, scales, and tests. The term quiz is usually reserved for a brief, informal, assessment of knowledge. A test is like a quiz but is usually longer. Tests cover a wide range of human characteristics, including knowledge, intelligence, memory, personality, and athletic skills. Many tests are published after research methods have established that their scores are reliable and valid for the purposes claimed by the test publisher. How is this book organized? I organized the chapters in this book into three sections. In Part I: Planning a Survey, I present information to guide you through the process of creating a survey and getting it ready to collect information. You will learn to create a purpose statement, strategies for reviewing previous research, how to write items, tips on organization and formatting, and elements of common approval processes, which include considering survey ethics. The last chapter in part one suggests ways to get respondents. In Part II: Using Surveys to Understand People, I present a six-dimensional model that addresses most aspects of human nature. I refer to the six dimensions using the acronym SCOPES: Social Context (e.g., age, sex, gender, ethnicity, other cultural factors), Cognition (thinking, beliefs, opinions), Observable behavior patterns or traits, Physical health, Emotions and feelings, and Spirituality or religiosity. This multidimensional model is sensitive to people from diverse cultural backgrounds. Each chapter includes examples of survey items that measure different aspects of the six dimensions. The final section, Part III: Understanding Basic Designs and Statistics, begins with an example of how to evaluate a workshop or similar educational program. Then I present a summary of basic statistics with an emphasis on reading research results rather than calculating them. Two chapters deal with topics of reliability and validity. And the final chapter offers some tips on presenting your survey results. Key Features of Creating Surveys • • • • • • • • • • • • • • Learning objectives for each chapter Concise chapter summaries Short chapters covering essential survey topics Many items to use in building your own projects Examples of text to introduce and close surveys How-to guide for evaluating workshops and seminars New concept list following chapter summaries Numerous references to online resources Reader’s guide to statistics and data analyses Information on ethics and project approval Examples of tables and charts for reports Extensive research glossary Extended Table of Contents Link to book website resources Style Note This book is written in a modified APA style designed to be reader-friendly. For the benefit of those familiar with APA style, I will note some differences. I use italics rather than quotation marks for emphasis as well as for terms and statistics. I limit the use of quotation marks to only indicate direct quotes. Rather than interrupt the text with long reference citations, I have mostly limited the in-text citations to one or two authors with dates, except when more authors are needed to identify the full reference, which you will find in the reference section at the back of this book. Similarly, I have placed most lengthy internet links in notes at the end of a chapter on in an appendix rather than interrupt the text. The tables and figures are designed for this publication format and do not follow the APA manuscript guidelines. Summary The primary purpose of this book is to help readers create better surveys. A secondary purpose is to help readers learn how to evaluate survey reports and contribute as knowledgeable team members on survey projects. Creating Surveys offers a readers’ guide to basic research methods and statistics. Survey is a broad term that includes polls, questionnaires, quizzes, tests, inventories, scales, and measures.