COH 609 Dec 2011

National University School of Health and Human Services Department of Community Health Masters of Public Health COURSE OUTLINE
COH 609 - Public Health Program Evaluation DECEMBER, 2011 Class Meetings: November 21, 23, 28, 30, December 5, 7, 12, 14, 17, +online exam Professor: Office: Suzanne Evans, MS, EdD, NBCT [email protected] National University Technology and Health Sciences Center, Room 114 3678 Aero Court, San Diego, CA 92123-1788 858-309-3484; cell: 847-431-0343 M & W 3:00-5:00 p.m. or by appointment
Office Hours:
TEXTBOOKS: Required Textbook: y Harris, M. (2010). Evaluating public and community health programs. San Francisco, CA: Jossey Bass. ISBN: 978-0-470-40087-6 y Internet Access and e-mail Access y E-College Ecompanion Required Evaluation Guides - available on ecollege y MMR. (1999). Framework for Program Evaluation in Public Health http://www.cdc.gov/mmwr/preview/mmwrhtml/rr4811a1.htm y CDC. (2005). Introduction to Program Evaluation for Public Health Programs http://www.cdc.gov/eval/evalguide.pdf y Kellogg Foundation. (2004). Evaluation Handbook http://www.ojp.usdoj.gov/BJA/evaluation/links/WK-Kellogg-Foundation.pdf y Kellogg Foundation. (2004). Logic Model Development Guide http://www.uwsa.edu/edi/grants/Kellogg_Logic_Model.pdf y OTHERS AS ASSIGNED Recommended Textbooks American Psychological Association. (2010). Publication manual of the American Psychological Association. Washington, DC: APA.
COH 609 Outline December 2011- page 1
COURSE DESCRIPTION: Fundamentals of evaluation methods applied to public health interventions. Effective use of measurement tools to evaluate achievement of program goals and objectives. Includes analysis of validity and reliability of measurement instruments. Emphasis on reach, effectiveness, acceptance, implementation, and maintenance of community programs. Includes fundamentals of proposal development. COURSE LEARNING OUTCOMES: Upon successful completion of this course, students will be able to: y Define program evaluation components including pretests, posttests, comparative designs, control groups, reliability, and validity. y Evaluate the internal and external validity of a measurement instrument. y Calculate the reliability of a measurement instrument. y Describe the components of the RE-AIM framework for public health program evaluation. y Design evaluation measurement instruments for the components of RE-AIM. y Develop a proposal for a public health program in response to a request for proposals or applications. PROGRAM MISSION: MASTER OF PUBLIC HEALTH The National University MPH program prepares educated, ethical and high-functioning public health professionals that serve the global community by advancing health and social justice. The NU MPH program employs collaborative administration where faculty, students, and public health professionals collaborate to disseminate public health scholarship through teaching, research, and community service. PROGRAM LEARNING OUTCOMES:
PLO 1. 2. 3. 4. 5. 6. 7. 8. Assess individual and community needs for health education. Plan health education strategies, interventions, and programs. Implement health education strategies, interventions, and programs. Conduct evaluation related to health education. Administer health education strategies and interventions. Serve as a health education resource person. Communicate and advocate for health and health education. Apply appropriate research principles and techniques in health education. COH 609 Mastery Mastery Developed Mastery Developed Developed Developed
PROGRAM LEAD FACULTY & DEPARTMENT CHAIR: GinaMarie Piane, DrPH [email protected] (858) 309-3474
TENTATIVE SCHEDULE:
Instructor reserves the right to change the timetable in order to better address the learning needs of the students in the class.
DATE
TOPIC
READING ASSIGNMENT
Due next class session
WRITTEN ASSIGNMENT
ACTIVITIES/ ASSIGNMENTS
Week 1
Monday Nov 21 y y y y y y y y Wednesday Nov 23 INTRODUCTIONS CHAPTERS 1, 2 Overview Framework For Program Evaluation STEP 1STAKEHOLDERS CASE STUDY ACTIVITIES SYLLABUS AND EXPECTATIONS PROCESS Review STEP 1 Stakeholders STEP 2 -DESCRIBE PLANNING OBJECTIVES SMART GOALS CASE STUDY ACTIVITIES READING: Harris - Chapters 1,2,3,5 MMR Framework pp 1-34 CDC Intro.to Program Eval Intro & Steps 1 CDC Intro.to Program Eval Step 2 CDC Evaluation Guide Smart Goals pp. 1-7 WRITING ASSIGNMENT: READING Response/Activity Step 1- Activity 1A & B Draft Section VStakeholders
y y y
y y
READING: Harris - Chapter 5 CDC Intro.to Program Eval Step 2 Kellogg Logic Model Development GuideInnonet Logic Guide CDC Logic Model Development
WRITING ASSIGNMENT: READING Response/Activity Step 2- Activity 2A & B Draft Section VIDescription LOGIC MODEL
Week 2
Monday November 28 y y PROCESS Review STEP 2 OBJECTIVES SMART GOALS CASE STUDY ACTIVITIES LOGIC MODELS Logic CHOOSING DESIGNS STEP 3 - FOCUS THE DESIGN The Question CASE STUDY ACTIVITIES READING: Harris - Chapter 6 CDC Intro.to Program Eval Step 3 WRITING ASSIGNMENT: READING Response/Activity Logic Models Draft Section VIDescription LOGIC MODEL WRITING ASSIGNMENT: READING Responses Step 3- Activity 3A & B Draft Section VIDesign
y y y Wednesday Nov 30
y y
READING: Harris - Chapters 7 & 9 Harris Chapter 8 & 10 CDC Intro.to Program Eval - Step 4 & 5
y y
Week 3
Monday Dec 5 y y y y y y Wednesday Dec 7 QUANTITATIVE DATA QUALITATIVE DATA STEP 4- GATHER EVIDENCE STEP 5- JUSTIFY CONCLUSIONS CASE STUDY ACTIVITIES Working Session Complete Steps 1-5 final Complete survey Prepare presentations PRESENTATIONS 1-3 Share responses STEP 6 SHARE & USE CASE STUDY ACTIVITIES STANDARDS OF GOOD EVALUATIONS READING: Harris - Chapters 7 & 9 Harris Chapter 8 & 10 CDC Intro.to Program Eval - Step 4 & 5 WRITING ASSIGNMENT: READING Responses Step 4 - Activity 4 A& B Step 5 - Activity 5 Draft Section VIIGather evidence Draft Section VIIIJustify
READING: Harris - Chapters 11 CDC Intro.to Program Eval Step 6
Step 4 - Activity 4 A& B Step 5 - Activity 5 Evaluation Plan Section I-VIII Complete Survey
Presentations
Week 4
Monday Dec 12 y y y y y Presentations READING: Harris - Chapters 4 Kellogg Evaluation HandbookREAIM WRITING ASSIGNMENT: READING Responses Step 6 - Activity 6A & B Draft Section IX Ensure Use
Wednesday Dec 14
Saturday December 17
PRESENTATIONS 4-6 Share responses PEER REVIEWS y REAIM y MANAGING EVALUATIONS Financial Ethical considerations y CASE STUDY y ACTIVITIES y PRESENTATIONS 7-8 Share responses PEER REVIEWS y CULTURAL COMPETENCY y PARTICIPATORY EVALUATIONS y Review y Final Examination Online by 12/19 y
READING: Cultural Articles as assigned
WRITING ASSIGNMENT: EVAL PLAN
Presentations Preparation for final
READING: Review for final
WRITING ASSIGNMENT: Submit Evaluation Plan online in dropbox
Preparation for final Final Exam
COURSE REQUIREMENTS: The readings, discussions, and assignments will focus on the exciting field of public health evaluation. Students are expected to complete all reading assignments prior to attending class. Participation in discussions and class activities is essential. Written assignments must be submitted on the due date unless special circumstances are discussed with the professor in advance. COH 609 Specific Course Requirements: Professionalism: Students are expected to: y demonstrate dispositions expected of students pursuing professional degrees y demonstrate professionalism by participating in all class sessions/discussions and submitting work on time y be honest in all coursework. Attendance: It is expected that students will: y attend all class sessions -an absence is assessed each time a student is not in attendance during a regularly schedule class period, whether or not it is an excused absence. y acknowledge that in accordance with National University policy, more than three absences from class or discussions, excused or unexcused, will result in a less than satisfactory grade. Written Assignment Expectations: All assignments completed out of class are to be: y typed/word-processed, font size 12, double spaced y written in standard English, error free in sentence construction, grammar, punctuation, and spelling. y APA format Late Work: Guidelines for late work are as follows: y all assignments are due at date indicated. y unless prior permission has been given to the candidate, late work will not be accepted and no credit will be given for that assignment. y with prior permission late work is accepted and will receive a deduction of 20% in the grade Assignments:
This course enables the students to participate in the following required learning experiences and assignments:
1. Classroom Attendance, Dialogue, Discussion and Discourse: You will complete assigned readings in the text and evaluation guides, view lectures and participate in discussions. Credit is given for attendance and it is expected that everyone attend class on time and actively participate in class activities and discussions. It is expected that the assigned readings will be completed prior to the scheduled class so you can engage in class discussions. Class will begin promptly at the designated start time. Habitually arriving late or leaving early will result in an absence. Students who miss class are responsible for all material presented in class and are advised to consult with their fellow classmates to find out what was covered during their absence. Up to 5 points, each class session may be earned for attendance and participation (40 points).
2. Planning Exercises: You will be completing written exercises to apply understanding of evaluation concepts. The assigned readings are outlined in the schedule and the activity sheets provided. Exercises will be submitted each class session. (30 points). 3. Logic Model: You will design a logic model and incorporate the model into the Evaluation Plan. Development of the Logic Model (30 points).Signature Assignment 4. Survey Questionnaire Project: You will plan the steps in the creation of a questionnaire. The project will include a short paper describing the process and the creation of a questionnaire and cover letter to be used in your evaluation plan (20) points. 5. Evaluation Plan: You will design a program evaluation on a public health topic or on the grant proposal developed in COH 607. Complete a detailed paper in APA format. Process Outlines will be due during the development process. (60 points). Signature Assignment 6. Evaluation Plan Presentation: PowerPoint Presentation for the class on the Evaluation Proposal will be shared. (30 points). Signature Assignment Exams: Comprehensive Final Examination: Using knowledge and application complete a multiple choice / essay exam.(50 points). Specific questions used as direct measures. GRADING FACTORS
Grades are based on points accumulated from all assignments, collaborative group activities, discussions and written assignments. 1. Classroom Attendance, Dialogue, Discussion and Discourse 40 2. Planning Exercises 30 3. Logic Model 30 4. Survey 20 5. Evaluation Plan 60 6. Evaluation Plan Presentation 30 7. Comprehensive Final Examination 50 TOTAL 260
Course Grading (expressed as percentages): A 96-100% B+ 87-89% C+ A90-95% B 84-86% C B80-83% C-
77-79% 74-76% 70-73%
D+ D D-
67-69% 64-66% 60-63%
Definition of Grades: Graduate Courses:

A Outstanding Achievement B Commendable Achievement C Marginal Achievement D Unsatisfactory * F Failing * y *Student receiving this grade in a course that is required for his/her degree program must repeat the course.
Definition of Grades:
y Incompletes (I) A grade of I may only be issued when a student has attended at least two-thirds of the course sessions and is unable to complete the requirements due to uncontrollable and unforeseen circumstances. Students must communicate these circumstances (preferably in writing) to the instructor prior to the final day of the course. If an instructor decides that an Incomplete is warranted, he/she issues a grade of I and notates the conditions for removal of the Incomplete in the students record. Students receive notification via e-mail about the assignment of an Incomplete, including the conditions for its removal. The information remains in place until the Incomplete is removed or the time limit for removal has passed. An Incomplete is not issued when the only way the student could make up the work is to attend a major portion of the class the next time it is offered. Students must resolve Incomplete grades no later than six months after the official course end date. Students can be required to remove an Incomplete in a shorter period of time at the discretion of the instructor. The Office of the Registrar sends a courtesy e-mail reminder regarding the Incomplete on record when four months has elapsed since the course end date, provided the instructor has not specified a shorter end date. An I that is not removed within the stipulated time becomes an F or a U based on the grading criteria of the course. No grade points are assigned. The F is calculated in the grade point average. U Unsatisfactory A permanent grade indicating that a credit attempt was not acceptable. An Unsatisfactory grade merits no grade points and is not computed in the grade point average. W Withdrawal Signifies that a student has withdrawn from a course after midnight of the ninth day of the class session. A Withdrawal is not allowed after the twenty-first (21st) day of the session. This is a permanent mark with no grade points assigned. S Satisfactory: Credit is granted but no grade points are assigned. IP In Progress: A designation representing a project course that allows up to six months or more for completion. No grade points are assigned for the IP grade.
y y
BIBLIOGRAPHY: Websites & Toolkits

American Evaluation Associationwww.eval.org Outcome Mapping, 2001 International Development Research Centre www.idrc.ca The Handbook for Evaluating HIV Education. Centers for Disease Control and Prevention www.cdc.gov/nccdphp/dahs/publications/hiv_handbook/index.htm What, Me Evaluate?, 1986 National Crime Prevention Council www.ncpc.org Association for the Study and Development of Community (June 2001). Principles for evaluating comprehensive community initiatives. Washington, D.C.: National Funding Collaborative on Violence Prevention.www.peacebeyondviolence.org Centers for Disease Control and Prevention (1999). Framework for program evaluation in public health. www.cdc.gov/eval/framework.htm#cbph. COH 609 Outline December 2011- page 7
Community Toolbox (online). Evaluating community programs and initiatives. http://ctb.lsi.ukans.edu/tools/EN/part_1010.htm Innovative Network (online). Helping agencies succeed. www.innonet.org. National Science Foundation (2002). The 2002 User-Friendly Handbook for Project Evaluation www.nsf.gov/pubs/2002/nsf02057/start.htm. Office for Substance Abuse Prevention (1991). Prevention Plus III: Assessing alcohol and other drug prevention programs at the school and community level.www.ncadi.samhsa.gov. Mob erg, DUPL. (1984). Evaluation of prevention programs: A basic guide for practitioners.www.uhs.wisc.edu/who. Sierra Health Foundation (2000). We did it ourselves: An evaluation guide book.www.sierrahealth.org. United Way of America (1996). Measuring program outcomes. Arlington, Va. University of Wisconsin-Cooperative Extension (1998). Evaluating collaborative. O.K. Kellogg Foundation (1998). Evaluation handbook. Battle Creek, MI.www.wkkf.org. O.K. Kellogg Foundation Evaluation Toolkit: http://www.wkkf.org/default.aspx?tabid=75&CID=281&NID=61&LanguageID=0 University of Kansas Community Tool Box: Evaluation Model: http://ctb.ku.edu/tools/en/sub_section_main_1007.htm Re-AIM http://www.re-aim.org/
Books & Journals

Butterfish, F. D. (2006). Process evaluation for community participation. Annual Review of Public Health, 27(1), 323-340. Cameron, R., Joplin, M. A., Walker, R., McDermott, N., & Gough, M. (2001). Linking science and practice: Toward a system for enabling communities to adopt best practices for chronic disease prevention. Health Promotion Practice, 2(1), 35-42. Cargo, M., & Mercar, S. L. (2008). The value and challenges of participatory research: strengthening its practice. Annual Review of Public Health, 29(1), 325-350. Cargo, M., Deformer, T., Levesque, L., Horn-Miller, K., Comber, A., & Macaulay, A. C. (2008). Can the democratic ideal of participatory research be achieved? An inside look at an academic-indigenous community partnership. Health Educ. Res., 23(5), 904-914. Clinton-Sherrod, A.M., Morgan-Lopez, I.E., Gibbs., Hawkins, S.R., Hart, L., Ball, B., Irvin, N., & Littler, N. (2009). Factors contributing to the effectiveness of four school-based sexual violence interventions. Health Promotion Practice. 10(1), 19S-28S. Coffield, A., Maiese, D. R., & Fox, E. (1996). Objectives-Setting for Improved Health The Public Health Service Healthy People Program. [Washington, D.C.]: Dept. of Health and Human Services, Office of Public Health and Science, Office of Disease Prevention and Health Promotion. http://purl.access.gpo.gov/GPO/LPS30295. Cronbach, L.J. and associates (1980). Toward reform of program evaluation. San Francisco: Jossey-Bass. Dunton, G.F., Lagloire, R., & Robertson. T. (2009). Using the RE-AIM framework to evaluate the statewide dissemination of a school-based physical activity and nutrition curriculum: "Exercise Your Options&quot. American Journal of Health Promotion: AJHP. 23(4), 229-32. evaluation. San Francisco: Jossey-Bass Publishers, 1994. Flicker, S. (2008). Who benefits from community-based participatory research? A case study of the positive youth project. Health Educ Behav, 35(1), 70-86. Flicker, S., Maley, O., Ridgley, A., Biscope, S., Lombardo, C., & Skinner, H. A. (2008). e-PAR: Using technology and participatory action research to engage youth in health promotion. Action Research, 6(3), 285-303. Flicker, S., Savan, B., Kolenda, B., & Mildenberger, M. (2008). A snapshot of community-based research in Canada: Who? What? Why? How? Health Educ. Res., 23(1), 106-114. COH 609 Outline December 2011- page 8
Glasgow, R. E., Klesges, L. M., Dzewaltowski, D. A., Bull, S. S., & Estabrooks, P. (2004). The future of health behavior change research: What is needed to improve translation of research into health promotion practice? Annals of Behavioral Medicine, 27(1), 3-12. Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM Framework. American Journal of Public Health, 89(9), 1322-1327. Goodstadt, M., Hyndman, B., McQueen, D. V., Potvin, L., Rootman, I., & Springett, J. (2001). Evaluation in health promotion: Synthesis and recommendations. In I. Rootman, M. Goodstadt, B. Hyndman, D. McQueen, L. Potvin, J. Springett & E. Ziglio (Eds.), Evaluation in health promotion: Principles and perspectives. Copenhagen: World Health Organization. Hallfors, D., & Godette, D. (2002). Will the `Principles of Effectiveness' improve prevention practice? Early findings from a diffusion study. Health Educ. Res., 17(4), 461-470. Hawe, P. P., & Potvin, L. P. (2009). What is population health intervention research? Canadian Journal of Public Health, 100(1), I8. Ioannidis, J. P. A. (2005). Contradicted and initially stronger effects in highly cited clinical research. JAMA, 294(2), 218-228. J.S. Wholey, H.P. Hatry, and K.E. Newcomer, (Eds.), Handbook of practical program Kelly JT. (2009). Evaluating employee health risks due to hypertension and obesity: self-testing workplace health stations. Postgraduate Medicine. 121(1), 152-8. Koelen, M. A., Vaandrager, L., & Colomer, C. (2001). Health promotion research: dilemmas and challenges. J Epidemiol Community Health, 55(4), 257-262. Kuczmarksi MF, & Cotugna N. (2009). Outcome evaluation of a 3-year senior health and wellness initiative. Journal of Community Health. 34(1), 33-9. Lavis, J. N., Posada, F. B., Haines, A., & Osei, E. (2004). Use of research to inform public policymaking. The Lancet, 364(9445), 1615. Lipsey, M. W., & Cordray, D. S. (2000). Evaluation methods for social intervention. Annual Review of Psychology, 51, 345-375. Lovisi Neto BE, Jennings F, Barros Ohashi C, Silva PG, & Natour J. (2009). Evaluation of the efficacy of an educational program for rheumatoid arthritis patients. Clinical and Experimental Rheumatology. 27(1), 28-34. Macaskill, L., Dwyer, J. J. M., Uetrecht, C., Dombrow, C., Crompton, R., Wilck, B., & Stone, J. (2000). An evaluability assessment to develop a restaurant health promotion program in Canada. Health Promotion International, 15(1), 57-69. McDonald, D. (1999). Developing guidelines to enhance the evaluation of overseas development projects. Evaluation and Program Planning, 22(2), 163-174. McKenzie, J. F., Neiger, B. L., & Smeltzer, J. L. (2005). Planning, implementing, and evaluating health promotion programs: A primer. San Francisco: Pearson/Benjamin Cummings. McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your program's performance story. Evaluation and Program Planning, 22(1), 65-72. McQueen, D. V., & Jones, C. M. (2007). Global perspectives on health promotion effectiveness. New York, NY: Springer. McVey G, Gusella J, Tweed S, & Ferrari M. (2009). A controlled evaluation of web-based training for teachers and public health practitioners on the prevention of eating disorders. Eating Disorders. 17(1), 1-26. Meltzer Graffunder C, & Charles D. (2009). Empowerment evaluation: building prevention science and organizational capacity to prevent sexual violence. Health Promotion Practice. 10(1), 71S-73S. Nastasi, B., & Hitchcock, J. (2009). Challenges of evaluating multilevel interventions. American Journal of Community Psychology, 43(3-4), 360-376. Nutbeam, D. (1996). Achieving "best practice" in health promotion: Improving the fit between research and practice. Health Education Research, 11(3), 317-326. Nutbeam, D. (1998). Evaluating health promotion--progress, problems and solutions. Health Promotion International, 13(1), 27-44. COH 609 Outline December 2011- page 9
Nutbeam, D. (1999). The challenge to provide "evidence" in health promotion. Health Promotion International, 14(2), 99-101. Opatz, J. P. (1987). Health promotion evaluation: Measuring the organizational impact. Stevens Point, Wis: National Wellness Institute, University of Wisconsin-Stevens Point Foundation. Parkinson, S. (2009). Power and perceptions in participatory monitoring and evaluation. Evaluation and program planning, 32(3), 229-237. Patton, M.Q. (1997). Utilization-focused evaluation (3rd edition). Newbury Park, CA: Sage Publications. Pinto, R. M. (2009). Community perspectives on factors that influence collaboration in public health research. Health Educ Behav, 36(5), 930-947. Rada, J., Ratima, M., & Howden-Chapman, P. (1999). Evidence-based purchasing of health promotion: Methodology for reviewing evidence. Health Promotion International, 14(2), 177187. Raphael, D. (2000). The question of evidence in health promotion. Health Promotion International, 15(4), 355-367. Rossi, P.H., Freeman, H. L., & Lipsey, M.W. (1999). Evaluation: A systematic approach (6th Edition). Thousand Oaks, CA: Sage. Salihu HM, Mbah AK, Jeffers D, Alio AP, & Berry L. (2009). Healthy start program and fetoinfant morbidity outcomes: evaluation of program effectiveness. Maternal and Child Health Journal. 13(1), 56-65. Saunders, R. P., Evans, M. H., & Joshi, P. (2005). developing a process-evaluation plan for assessing health promotion program implementation: a how-to guide. Health Promot Pract, 6(2), 134-147. Scarinci, I. C., Johnson, R. E., Hardy, C., Marron, J., & Partridge, E. E. (2009). Planning and implementation of a participatory evaluation strategy: A viable approach in the evaluation of community-based participatory programs addressing cancer disparities. Evaluation and program planning, 32(3), 221-228. Schensul, J. (2009). Community, culture and sustainability in multilevel dynamic systems intervention science. American Journal of Community Psychology, 43(3-4), 241-256. Scott SA, & Proescholdbell S. (2009). Informing best practice with community practice: the community change chronicle method for program documentation and evaluation. Health Promotion Practice. 10(1), 102-10. Scriven, M. (1991) Evaluation thesaurus (4th Edition). Thousand Oaks, CA: Sage. Shadish, W. R., & Cook, T. D. (2009). The renaissance of field experimentation in evaluating interventions. Annual Review of Psychology, 60(1), 607-629 Shadish, W.R., Cook, T.D., and Leviton, L.C. Foundations of program evaluation:Theorists and their theories. Newbury Park, CA: Sage, 1991. Smith, M. K. (1998). Empowerment evaluation: Theoretical and methodological considerations. Evaluation and Program Planning, 21(3), 255-261. Speller, V., Learmouth, A., & Harrison, D. (1997). The search for evidence of effective health promotion. British Medical Journal, 315, 361-363. Steckler, A. B., & Linnan, L. (2002). Process evaluation for public health interventions and research. San Francisco, Calif: Jossey-Bass. Thayer, C. E., & Fine, A. H. (2001). Evaluation and outcome measurement in the non-profit sector: stakeholder participation. Evaluation and Program Planning, 24(1), 103-108. Valente, T. W. (2002). Evaluating health promotion programs. New York: Oxford University Press. van Osch L, Lechner L, Reubsaet A, Steenstra M, Wigger S, & de Vries H. (2009). Optimizing the efficacy of smoking cessation contests: an exploration of determinants of successful quitting. Health Education Research. 24(1), 54-63. Vijgen, S. M. C., van Baal, P. H. M., Hoogenveen, R. T., de Wit, G. A., & Feenstra, T. L. (2008). Cost-effectiveness analyses of health promotion programs: a case study of smoking prevention and cessation among Dutch students. Health Educ. Res., 23(2), 310-318. COH 609 Outline December 2011- page 10
Wallerstein, N. B., & Duran, B. (2006). Using community-based participatory research to address health disparities. Health Promotion Practice, 7(3), 312-323. Wallerstein, N., Polascek, M., & Maltrud, K. (2002). Participatory evaluation model for coalitions: The development of systems indicators. Health Promotion Practice, 3(3), 361-373. Wandersman, A. (2009). Four keys to success (theory, implementation, evaluation, and resource/system support): High hopes and challenges in participation. American Journal of Community Psychology, 43(1-2), 3-21. Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A resultsbased approach to accountability. Evaluation and Program Planning, 23(3), 389-395. Webb, D., & Wright, D. (2000). Postmodernism and health promotion: Implications for the debate on effectiveness. In J. Watson & S. Platt (Eds.), Researching health promotion . London: Routledge. Weiss, C.H. (1997). Evaluation (2nd edition). New York: Prentice-Hall. World Health Organization European Working Group on Health Promotion Evaluation. (1998). Health promotion evaluation: Recommendations to policymakers. Copenhagen: World Health Organization Regional Office for Europe. Worthen, B.R. & Sanders, J.R., & Fitzpatrick, J. L. (1996) Program evaluation:Alternative approaches and practical guidelines (2nd Edition). New York:Addison-Wesley. Wurzbach, M. E. (2004). Community health education and promotion: A guide to program design and evaluation. Sudbury, Mass: Jones and Bartlett. Journals: American Journal of Health Behavior American Journal of Health Education (formerly Journal of Health Education) American Journal of Health Promotion American Journal of Health Studies American Journal of Public Health American Journal of Public Health California Journal of Health Promotion Canadian Journal of Public Health European Journal of Public Health Family and Community Health Health Education and Behavior(formerly Health Education Quarterly); Health Education Research (online version available) Health Promotion Practice Journal of Epidemiology and Community Journal of Health and Social Behavior Journal of School Health Journal of Public Health and Policy Journal of Public Health Management and Practice Leadership in Public Health Morbidity, Mortality Weekly Report Public Health Reports Organizational Resources: American Association for Health Education AAHE. http://www.aahperd.org American School Health Association ASHA. http://www.ashaweb.org American College Health Association ACHA- Health Education section: http://www.acha.org Agency for Healthcare Research and Quality. http:www.ahrq.gov American Public Health Association. http://www.apha.org American Cancer Society (http://www.cancer.org/docroot/home/index.asp) American Heart Association (http://www.americanheart.org/) Association of Schools of Public Health. http://www.asph.org COH 609 Outline December 2011- page 11
Association of Community Health Improvement (http://www.communityhlth.org/) Association of State and Territorial Health Officials. http://www.astho.org CA Department of Health (http://www. dph.ca.gov) CA. Department of Health Care Services (http://www.dhcs.ca.gov) Centers for Disease Control and Prevention. http://ww.cdc.gov Center for Law and the Publics Health. http://www.publichealthlaw.net Community Tool Box (http://ctb.ku.edu/en/) Council on Education for Public Health. http://www.ceph.org Center for Minority Health (http://www.cmh.pitt.edu/) Environmental Protection Agency. http://www.epa.gov Eta Sigma Gamma http://www.cast.ilstu.edu/temple/esg.htm Health Resources and Services Administration. http://www.hrsa.gov Healthy People 2010 (http://www.healthypeople.gov/) Healthy People 2010 Tool Kit: A Field Guide to Health Planning (http://www.healthypeople.gov/state/toolkit/default.htm) International Union for Health Promotion and Education. www.iuhpe.org International Association of National Public Health Institutes (http://www.ianphi.org/) Kaiser Family Foundation State Health Facts (http://www.statehealthfacts.org/) Making Health Communication Programs Work (http://www.cancer.gov/pinkbook) National Association of County and City Health Officials. http://www.naccho.org National Association of Public Health Policy ( http://www.naphp.org/) National Center for Health Statistics (www.cdc.gov/nchs) National Coalition of Health Education Organizations, NHEO. http://www.hsc.usf.edu/CFH/cnheo/index.html National Environmental Health Association. http://www.neha.org National Heart, Lung, and Blood Institute. http://www.nhlbi.gov National Institute of Occupational Safety and Health. http://www.niosh.gov National Institutes of Health. http://www.nih.gov National Library of Medicine. http://www.nlm.gov National Public Health Performance Standards. http://www.cdc.gov/od/ocphp/nphpsp Occupational Safety and Health Administration. http://www.osha.gov Pan American Health Organization. http://ww.paho.org Robert Wood Johnson Foundation. http://www.rwjf.org Rockefeller Foundation. http://www.rockfound.org Society for Public Health Education. http://www.sophe.org Substance Abuse and Mental Health Services Administration. http://www.samhsa.gov W.K. Kellogg Foundation. http://www.wkk.org World Health Organization. http://www.who.org U.S. Census Bureau (http://www.census.gov/) U.S. Department of Health & Human Services (www.cdc.gov/nchs/hus.htm)
SCHOOL OF HEALTH AND HUMAN SERVICES REQUIREMENTS

GENERAL POLICIES: Academic Integrity: Students are required to submit their own, original work that is written or developed for this particular class. Attendance: Students are expected to attend all class sessions. An absence is assessed each time a student is not in attendance during a regularly scheduled class period, whether or not it is an excused absence. An instructor may withdraw a student from class prior to the sixth session in graduate courses if there are more than two unexcused absences. Students who have more than three absences, excused or unexcused, cannot be given a satisfactory grade. Cell Phones: Cell phones may not be used during class. Students should shut off or switch phones to silent mode. Laptop Computers: Laptop computers may be used in class for taking notes and accessing materials posted on eCompanion. Instructors may ask students to close computers if use becomes disruptive to learning. Civility: As a diverse community of learners, students must strive to work together in a setting of civility, tolerance, and respect for each other and for the instructor. Rules of classroom behavior (which apply to online as well as onsite courses) include but are not limited to the following: y Conflicting opinions among members of a class are to be respected and responded to in a professional manner. y Side conversations or other distracting behaviors are not to be engaged in during lectures, class discussions or presentations y There are to be no offensive comments, language, or gestures Diversity: Learning to work with and value diversity is essential in every degree program. Students are required to act respectfully toward other students and instructors throughout the course. Students are also expected to exhibit an appreciation for multinational and gender diversity in the classroom and develop leadership skills and judgment appropriate to such diversity in the workplace.
Disability: National University complies with the Americans with Disabilities Act of 1990 and Section 504 of the Federal Rehabilitation Act of 1973. If you need accommodations due to a documented disability, please contact the Office of Scholarships and Special Services at (858) 642-8185 or via e-mail at [email protected]. Information received by this office is confidential and is only released on a 'need-to-know' basis or with your prior written consent. Accommodations can only be granted upon approval by the Committee for Students with Disabilities (CSD). Ethics: Ethical behavior in the classroom is required of every student. Students are also expected to identify ethical policies and practices relevant to course topics. Plagiarism: Plagiarism is the presentation of someone else's ideas or work as one's own. Students must give credit for any information that is not either the result of original research or common knowledge. If a student borrows ideas or information from another author, he/she must acknowledge the author in the body of the text and on the reference page. Students found plagiarizing are subject to the penalties outlined in the Policies and Procedures section of the University Catalog, which may include a failing grade for the work in question or for the entire course. The following is one of many websites that provide helpful information concerning plagiarism for both students and faculty: http://www.indiana.edu/~wts/pamphlets/plagiarism.shtml Quoting, Paraphrasing, and Summarizing (http://owl.english.purdue.edu/handouts/research/r_quotprsum.html ) Technology: Students are expected to be competent in using current technology appropriate for this discipline. Such technology may include word processing, spreadsheet, and presentation software. Use of the internet and e-mail may also be required. The following website provides information on APA writing and citation style required for papers: http://www.nu.edu/LIBRARY/ReferenceTools/citations.html Writing Standards of SHHS: Students are expected to demonstrate writing skills in describing, analyzing and evaluating ideas and experiences. Written material must follow specific standards regarding citations of authors' work within the text and references at the end of the paper. Students are encouraged to use the services of the University's Writing Center when preparing materials (see National University Services below). Grades will be assigned for written material in accordance with the University catalog and the following general criteria adopted by the School of Health and Human Services.
NOTE: Letter grades may be assigned for any or all of the following reasons:
"A" Range: Outstanding achievement, significantly exceeds standards Unique topic or unique treatment of topic, takes risks with content; fresh approach. Sophisticated/exceptional use of examples. Original and "fluid" organization; all sentences and paragraphs contribute; sophisticated transitions between paragraphs. Integration of quotations and citations is sophisticated and highlights the author's argument. Confidence in use of Standard English; language reflects a practiced and/or refined understanding of syntax and usage. Sentences vary in structure, very few, if any mechanical errors (no serious mechanical errors). "B" Range: Commendable achievement, meets or exceeds standards for course. Specific, original focus, content well handled. Significance of content is clearly conveyed; good use of examples; sufficient support exists in key areas. Has effective shape (organization), effective pacing between sentences or paragraphs. Quotations and citations are integrated into argument to enhance the flow of ideas. Has competent transitions between all sentences and paragraphs. Conveys a strong understanding of standard English; the writer is clear in his/her attempt to articulate main points, but may demonstrate moments of "flat" or unrefined language. May have a few minor mechanical errors (misplaced commas, pronoun disagreement, etc.), but no serious mechanical errors (fragments, run-ons, comma splices, etc.) "C" Range: Acceptable achievement, meets standards for course Retains overall focus, generally solid command of subject matter. Subject matter well explored but may show signs of underachievement. Significance is understood, competent use of examples. Structure is solid, but an occasional sentence or paragraph may lack focus. Quotations and citations are integrated into argument. Transitions between paragraphs occur but may lack originality. Competent use of language; sentences are solid but may lack development, refinement, style. Occasional minor mechanical errors may occur, but do not impede clear understanding of material. No serious mechanical errors (fragments, run-ons, comma-splices, etc.). "D" Range: Unsatisfactory achievement; does not meet acceptable standards. Note: The "D" grade is a passing grade for undergraduate work; work that is not of "passing quality" should receive grade "F". Significance of content is unclear. Ideas lack support, elaboration. Lacks sufficient examples or relevance of examples may be unclear. Support material is not clearly incorporated into argument. Expression is frequently awkward (problematic sentence structure). Mechanical errors may often impede clear understanding of material. May have recurring serious mechanical errors (fragments, run-ons, comma splices, etc.). "F" Range: Fails to meet minimal standards Ignores assignment. Lacks significance. Lacks coherence. Includes plagiarized material (intentional or unintentional). Lacks focus Difficult to follow due to awkward sentence or paragraph development. Mechanical errors impede understanding. Problems with writing COH 609 Outline December 2011- page 15
National University Library: National University Library supports academic rigor and student academic success by providing access to scholarly books and journals both electronically and in hard copy. Print materials may be accessed at the Library in San Diego or through document delivery for online and regional students. Librarians are available to provide training, reference assistance, and mentoring at the San Diego Library and virtually for online or regional students. Please take advantage of Library resources: y URL: http://www.nu.edu/library. y Contact the Library: o [email protected] o (858) 541-7900 (direct line) o 1-866-NU ACCESS x7900 (toll free) y Use the Library Training Tools (on the Library Homepage) for additional help o Recorded class presentations o Tutorials & Guides (APA/MLA, Peer-Review, and more)
ASSIGNMENT: READING RESPONSES & EXERCISES

You will be responding to various texts and completing written exercises to initiate the process of an evaluation. The assigned readings are outlined in the schedule and the activity sheets provided. Responses will be submitted each class session and do not need to be typed. EXERCISE 1- 5 POINTS CDC Eval Guide- Step 1- Activity 1A & B EXERCISE 2- 5 POINTS CDC Eval Guide- Step 2- Activity 2A & B EXERCISE 3- 5 POINTS CDC Eval Guide Step 3 - Activity 3 A & B EXERCISE 4- 5 POINTS CDC Eval Guide Step 4 - Activity 4 A & B QUANTITATIVE CDC Eval Guide Step 5 - Activity 5 EXERCISE 5- 5 POINTS CDC Eval Guide Step 4 - Activity 4 A & B QUALITATIVE CDC Eval Guide Step 5 - Activity 5 EXERCISE 6- 5 POINTS CDC Eval Guide Step 6 - Activity 6A & B
ASSIGNMENT LOGIC MODEL

You will design a logic model as a part of the public health program evaluation document. A logic model presents a picture of how your effort or initiative is supposed to work. It explains why your strategy is a good solution to the problem at hand. In its simplest form, a logic model is a graphic representation that shows the logical relationships between: inputs: resources that go into the program; outputs: activities the program undertakes; outcomes: changes or benefits that result. Effective logic models make an explicit, often visual, statement about what activities will bring about change and what results you expect to see for the community and its people. Having a logic model keeps everyone moving in the same direction, providing a common language and a common point of reference. The Logic Model should answer these questions: Does your organization have adequate resources to implement the activities and achieve the desired outcomes? If you need further resources, is that reflected in your activities? Have you included all the major activities needed to implement your program and achieve expected outcomes? Would the activities list enable someone who is unfamiliar with your program to understand its scope? Have you expressed your outcomes in terms of change? Have you identified who/what will experience that change, and over what time period? Do activities, outputs, and short- and long-term outcomes relate to each other logically (the if-then relationship)? Does your logic model clearly identify the scope of your programs influence? Have you considered a variety of perspectives? It is a good idea to get feedback from colleagues and stakeholders. (remember, the online logic model builder makes collaboration easy, and gives you a Is it visually engaging- simple, yet contains the appropriate degree of detail for the purpose. Does it provokes thought, triggers questions. Is it meaningful, does it make sense, is it doable, and can it be verified? Does it provide the program description that guides the evaluation process; help match evaluation to the program, help know what and when to measure, help focus on key, important information Remember, the logic model is a depiction of the program evaluation needs to match the program. Tailor questions and indicators to particular program and purpose of the evaluation. Match evaluation questions to the program stage of program development and purpose: improvement, assess worth/merit; new knowledge; compliance Are there indicators- determine how we will measure it. What would be the indicators for the outcomes and process variables you have selected? How would you know it? The Logic Model will be a flow chart that includes:
1. Problem statement or situation: The originating problem, or issue, set within a complex of sociopolitical, environmental and economic circumstances. The situation is the beginning point of Logic Model development. Clearly articulates the problem your work is trying to solve. It frames a particular challenge for the population you serve and that your work will try to solve. 2. Goal: Represents a general, big picture statement of desired results. It is the overall purpose of what you are trying to measure (your program, intervention, etc). Answers COH 609 Outline December 2011- page 18
what you are trying to accomplish. The answer to this question is the solution to your problem statement, and will serve as your goal. Goals serve as a frame for all elements of the logic model that follow. They reflect organizational priorities and help you steer a clear direction for future action. Goals should include the intended resultsin general termsof the program or initiative and specify the target population you intend to serve. 3. Inputs or Resources: Identify the available resources for your program. This helps you determine the extent to which you will be able to implement the program and achieve your intended goals and outcomes. List the resources that you currently have to support your program. Include such resources as staff, time, money, staff, space, technology, research base, partnership, equipment, and materials. Identify the major resource categories for your program and be specific about these resources. 4. Outputs: What we do and whom we reach: activities, services, events, products and the people reached. Outputs include such elements as workshops, conferences, counseling, products produced and the individuals, clients, groups, families, and organizations targeted to be reached by the activities. Activities are the actions that are needed to implement your programwhat you will do with program resources in order to achieve program outcomes and, ultimately, your goal(s). Common activities include: developing products (e.g., promotional materials and educational curricula); providing services (e.g., education and training, counseling or health screening); engaging in policy advocacy (e.g., issuing policy statements, conducting public testimony); or building infrastructure (e.g., strengthening governance and management structures, relationships, and capacity). It is often helpful to group related activities together. The number of activity groups depends on your programs size and how you administer it. They lead to desired outcomesbenefits for participants, families, communities, or organizationsbut are not themselves the changes you expect the program will produce. They do help you assess how well you are implementing the program. Whenever possible, express outputs in terms of the size and/or scope of services and products delivered or produced by the program. 5. Outcomes /Impacts: What results: the value or changes for individuals, families, groups, agencies, businesses, communities, and/or systems. Outcomes express the results that your program intends to achieve if implemented as planned. Outcomes are the changes that occur or the difference that is made for individuals, groups, families, organizations, systems, or communities during or after the program. Outcomes answer the questions: what difference does the program make? What does success look like? They reflect the core achievements you hope for your program. Outcomes should: represent the results or impacts that occur because of program activities and services ; be within the scope of the programs control or sphere of reasonable influence, as well as the timeframe you have chosen for your logic model ; be generally accepted as valid by various stakeholders of the program ; be phrased in terms of change ; be measurable. 6. Chain of Outcomes: Outcomes include short-term, interndiate and long-term benefits. Some outcomes must occur before others become possible. This is referred to as the chain of outcomes. Short-term outcomes: what change do you expect to occur either immediately or in the near future? Short-term outcomes are those that are the most direct result of a programs activities and outputs. They typically focus on changes in awareness, knowledge, skills, attitudes, opinions and intent. Intermediate outcomes: what change do you want to occur after that? Intermediate outcomes are those outcomes that link a programs short-term outcomes to long-term outcomes. They typically include changes in behaviors, decision-making or policy change, and actions. COH 609 Outline December 2011- page 19
Long-term outcome: what change do you hope will occur over time? Long-term outcomes are those that result from the achievement of your short- and intermediateterm outcomes. Often called impact, they include such as changes in social, economic, civic, and environmental conditions. Often long-term outcomes will occur beyond the timeframe you identified for your logic model. 7. Assumptions: The assumptions are the beliefs, principles, ideas we have about the program, the people involved and the way we think the program will operate. Assumptions underlie all that we do. Assumptions that underlie a programs theory are conditions that are necessary for success, and you believe are true. Assumptions include our ideas about the problem or situation; the way the program will operate; what the program expects to achieve; how the participants learn and behave, their motivations, etc.; the resources and staff; the external environment; the knowledge base; and the internal environment. Your program needs these conditions in order to succeed, but you believe these conditions already exist they are not something you need to bring about with your program activities. In fact, they are not within your control. These assumptions can refer to facts or special circumstances in your community, region, and/or field. Faulty assumptions are often the reason for poor results. 8. External Factors: Aspects external to the program that influence the way the program operates, and are influenced by the program. Dynamic systems interactions include the cultural milieu, Biophysical environment, economic structure, housing patterns, demographic makeup, family circumstances, values, political environment, background and experiences of participants, media, policies and priorities, etc. These are elements that affect the program over which there is little control.
See Template in e-College
RUBRIC LOGIC MODEL

Marginal 0 Acceptable 1-2
Although problem articulated, minimally addressed, generic Not well detailed, demonstrates thought.
Commendable 3-4
The originating problem articulated. Frames a particular challenge for the population that work will try to solve. Detailed, demonstrates thought.
Outstanding 5
The originating problem set within sociopolitical, environmental and economic circumstances. Clearly articulates the problem trying to solve. Frames a particular challenge for the population that work will try to solve. Complete, detailed, -demonstrates thought. Specific and connected, clear and logical. Clearly and precisely stated, thorough, & logical. Demonstrates clear understanding. Well articulated. Identifies the available resources for the program. Includes multiple resources such as staff, time, money, staff, space, technology, research base, partnership, equipment, and materials. Specifically identifies the major resource categories. Explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding. Purpose and description is evident and well articulated. Includes variety of outputs including activities and people. Address activities, products services, policy advocacy, & infrastructures. Outputs lead to desired outcomes. Specific, addressed with clarity and appropriateness. Explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding. Purpose and description is evident and well articulated. Clearly expresses the results the program intends to achieve if implemented as planned. Outcomes reflect the core achievements; represent the results or impacts that occur; are within the scope of the programs control and timeframe; phrased in terms of change; and are measurable. Includes clear shortterm, intermediate and long-term benefits. There is a clear chain of outcomes-discussed in detail, and fully represented in flowchart, map, or table. Demonstrates clear understanding. Well articulated.
1. Problem statement or situation X .5 2. Goal X1 3. Input or Resources X1
Limited or not complete
Limited; unclear . Stated yet is imprecise.
Generic description - all areas not addressed.
Imprecise.-some information expressed. Somewhat connected and logical. Demonstrates understanding. Identifies some of the available resources for the program. Includes some resources . Identifies some major resource categories..Explained & represented in flowchart, map, or table. Some understanding is evident. Needs some clarification.
Mostly specific and connected, clear and logical. Demonstrates understanding. Articulated Identifies most of the available resources for the program. Includes most resources. Identifies most major resource categories.. Explained & represented in flowchart, map, or table. Demonstrates understanding. Purpose and description is evident. Includes outputs including activities and people. Outputs address most desired outcomes. Addressed with clarity and appropriateness. Represented in flowchart, map, or table. Demonstrates understanding. Purpose and description is evident.
4. Outputs X1
Unclear and/or have some ambiguity. Detail is needed to provide exactness in meaning.
Includes some outputsdoes not address activities and people. Mostly represented in flowchart, map, or table. Demonstrates minimal understanding. Does not address all areas. All information not directly related to the topic.
5. Outcomes /Impacts 6. Chain of Outcomes X 1.5
Unclear or not stated. Detail is needed to provide exactness in meaning. Does not portray the sequence of outcomes.
Addresses some of the results the program intends to achieve. Outcomes somewhat reflect the core achievements; the impacts that occur- & are sometimes measurable. Includes short-term, intermediate and longterm benefits. Minimal chain of outcomes. Mostly represented in flowchart, map, or table.
Addresses most of the results the program intends to achieve . Outcomes mostly reflect the core achievements; represent the results or impacts that occur; phrased in terms of change; and are mostly measurable. Includes short-term, intermediate and long-term benefits. There is a chain of outcomes represented in flowchart, map, or table.
7.Assumptions X.5
Unclear or not stated. Detail is needed to provide exactness in meaning. All areas not addressed
Provides generic assumptions that do not fully reflect the beliefs, principles, ideas about the program, the people involved and the way the program will operate. analysis and synthesis, and interpretations. Does not clearly link to problem or goal. Minimal detail .
8.External Factors X.5
Unclear or not stated. Detail is needed to provide exactness in meaning. All areas not addressed.
Minimally identifies aspects external to the program . Few elements that affect the program over which there is little control shared. Generic ideas.
Addresses assumptions that reflect the beliefs, principles, ideas about the program, the people involved and the way the program will operate. Addresses some ideas about the problem ; the way the program operates& expects to achieve; how the participants learn and behave; the staff & resources; the external and the internal environment. Lacks consistent detail . Identifies aspects external to the program . Some elements that affect the program over which there is little control shared. Addresses limitations, scope.
Clear assumptions that reflect the beliefs, principles, ideas about the program, the people involved and the way the program will operate. Addresses multiple ideas about the problem or situation; the way the program will operate; what the program expects to achieve; how the participants learn and behave, their motivations, etc.; the resources and staff; the external environment; the knowledge base; and the internal environment. Consistently relevant detail makes the meaning exact
Clearly identifies aspects external to the program . Dynamic elements that affect the program over which there is little control shared. Addresses limitations, scope, in detail.
Total 30 POSSIBLE
Comments
Survey Questionnaire Project

1. Plan these Steps in the Survey Questionnaire Project a. Establish the goals of the questionnaire - What you want to learn b. Determine your audience/ sample - Whom you will interview c. Choose interviewing methodology - How you will interview d. Create a letter to accompany the questionnaire e. Create your questionnaire - What you will ask f. Determine how you will collect and analyze the data- What does it mean g. Determine how you will use the data- How will you use the data 2. In a short paper, utilizing the following headings, explain your process and plan Goals Audience & Sample Methodology Collection and analysis of Data Use of data 3. Appendix attached final copies of the following Letter Questionnaire
RUBRIC SURVEY QUESTIONNAIRE PROJECT

Criteria & scoring Audience X .5 Purpose X .5 Marginal 1
Limited understanding of the audience Limited; lack of understanding of use and dissemination not clear. Imprecise.
Acceptable 2
Some understanding of the audience Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used. Generic letter or plan for administration Topics and ideas discussed somewhat randomly. Missing clear introduction and/or conclusion. Survey questions are limited and offer some information required to assess. Survey questions are not listed or asked in a logical format. Format has some inconsistencies and missing areas. Writing lacks clearly defined organization, ineffective Some careless spelling and punctuation errors- 4 or more misspellings and/or grammatical errors Generic plan for collection, analysis or use. Topics and ideas discussed somewhat randomly.
Commendable 3
Clear understanding of the audience Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence. Clear letter and plan for administration Logical paragraph order, good development of ideas in clear structure. Introduction and/or conclusion. Survey items sufficiently cover important items for assessment Survey questions are asked in logical sequence which respondents can follow. The format is correct, title is appropriate. Logical writing, clear and functional, clear structure. Few spelling and punctuation errors -no more than two misspellings and/or grammatical errors Clear plan for collection, analysis and use. Logical order and clear structure.
Outstanding 4
Thorough understanding of the audience Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence. Clear letter and detailed plan for administration Very well organized, good development of ideas, strong sentences, effective. Clear introduction and/or conclusion. Survey items are well thought out and include all important items to adequately assess. Survey questions are asked in a logical and well thought out sequence which respondents can follow. The format is correct, title appropriate, and laid out wisely. Very well organized, strong sentences. Flawless spelling, punctuation, and capitalization.
No clear letter or plan
Methodology & for administration. Missing clear letter x.5 Survey Development Survey Format
introduction and/or conclusion
Survey Mechanics
Collection, analysis & use x.5 Total Possible 20
Survey questions do not present enough information to assess. Survey questions are not appropriate for requested information. Format is sloppy, margins, layout is inconsistent. Writing is unstructured, inconsistent. Many spelling and punctuation errors, many sentence fragments, run-ons. No clear or limited plan for collection, analysis or use
Clear l and detailed plan for collection, analysis and use . Very well organized, effective.
Comments
ASSIGNMENT PROGRAM EVALUATION PLAN You will design a Public Health Program Evaluation Document. The document will be a continuation of the grant proposal students developed in COH 607 or an alternative topic. Be sure to develop an evaluation plan to ensure your program evaluations are carried out efficiently in the future and to ensure your evaluation plan is documented so you can regularly and efficiently carry out your evaluation activities. Plans must include the following sections: I. Title Page (name of the organization that is being, or has a product/service/program that is being, evaluated; date) II. III. Table of Contents Executive Summary (one-page, concise program background , type of evaluation conducted, what decisions are being aided by the findings of the evaluation, who is making the decision Engagement of Stakeholders
y y y All groups identified - those involved in program operations; those served or affected by the program; and primary intended users of the evaluation. Address rights of human subjects, human interactions, conflict of interest Cultural competency addressed
IV.
V.
Description of the Program

y Statement of need -describes the problem, goal, or opportunity that the program addresses; the nature of the problem or goal, who is affected, how big it is, and whether (and how) it is changing; problem/opportunity to which program is responding to, the programs specific objectives Expectations - program's intended results-what the program has to accomplish to be considered successful; background about organization and program that is being evaluated; organization description/history. Activities - everything the program does to bring about changes- describe program components, elements, strategies, and actions; principal content of the program, delivery model Resources- include the time, talent, equipment, information, money, and other assets available to conduct program activities include program costs and the cost-benefit ratio as part of the evaluation; staffing (description of the number of personnel and roles in the organization that are relevant to developing and delivering the program Program's stage of development - reflects program maturity; address three phases of development: planning, implementation, and effects or outcomes; program documentation. Program's context - the environment in which the program operates; the area's history, geography, politics, and social and economic conditions, and what other organizations have done. Logic model - sequence of events input, output, and long-term goals, short-term; flow-chart, map, or table to portray the sequence of steps leading to program results; COH 609 Outline December 2011- page 25
clear description of program inputs, activities/processes; clear description of outcomes and impact.
VI. Evaluation Design

y y y y y y y Purpose - general intent of the evaluation. (gain insight, improve how things get done, determine what the effects of the program are, affect those who participate) Users - specific individuals who will receive evaluation findings. Uses - what will be done with what is learned from the evaluation. Answer specific questions- clarity and appropriateness of research questions, including which stakeholders will utilize the answers Methods-(experimental, quasi-experimental, and observational or case study designs) Agreements- summarize the evaluation procedures and clarify everyone's roles and responsibilities; describe how the evaluation activities will be implemented. Addresses evaluation impact ,practical procedures, political viability, cost effectiveness, service orientation, complete and fair assessment, fiscal responsibility .
VII. Gathering Evidence

y Indicators - general concepts about the program and its expected effects into specific, measurable parts; description of independent and dependent variables and how they will be measured and analyzed. Sources of evidence -people, documents, or observations, criteria used to select sources clearly stated; who participated. Quality -the appropriateness and integrity of information gathered Quantity-the amount of evidence gathered Logistics - the methods, timing, and physical infrastructure for gathering and handling evidence time periods sampled, data collection method and tools, any limitations caused by this method and tool Address Information scope and selection; information sources, valid and reliable information
y y y y
VIII. Justification of Conclusions

y y y y y y Standards -the values held by stakeholders about the program. Analysis and synthesis -methods to discover and summarize an evaluation's findings; detect patterns in evidence, by analysis, synthesis, mixed method Interpretation - figure out what the findings mean; interpretations and conclusions Judgments - statements about the merit, worth, or significance of the program, compare against one or more selected standards. Recommendations-actions to consider as a result of the evaluation; recommendations regarding the decisions that must be made about the service/program. Addresses identification of values, analysis of quantitative and qualitative information
IX. Use and Dissemination of Lessons Learned

y y y y y Design -how the evaluation's questions, methods, and overall processes are constructed. Preparation - steps taken to get ready for the future uses of the evaluation findings. Feedback - communication that occurs among everyone involved in the evaluation. Follow-up - the support that users need during the evaluation and after they receive evaluation findings. Dissemination - the process of communicating the procedures or the lessons learned from an evaluation to relevant audiences in a timely, unbiased, and consistent fashion. COH 609 Outline December 2011- page 26
X. Reflection on Standards for "good" evaluations Utility Standards

1. Stakeholder Identification: People who are involved in (or will be affected by) the evaluation should be identified, so that their needs can be addressed. 2. Evaluator Credibility: The people conducting the evaluation should be both trustworthy and competent, so that the evaluation will be generally accepted as credible or believable. 3. Information Scope and Selection: Information collected should address pertinent questions about the program, and it should be responsive to the needs and interests of clients and other specified stakeholders. 4. Values Identification: The perspectives, procedures, and rationale used to interpret the findings should be carefully described, so that the bases for judgments about merit and value are clear. 5. Report Clarity: Evaluation reports should clearly describe the program being evaluated, including its context, and the purposes, procedures, and findings of the evaluation. This will help ensure that essential information is provided and easily understood. 6. Report Timeliness and Dissemination: Significant midcourse findings and evaluation reports should be shared with intended users so that they can be used in a timely fashion. 7. Evaluation Impact: Evaluations should be planned, conducted, and reported in ways that encourage follow-through by stakeholders, so that the evaluation will be used.
Feasibility Standards
1. Practical Procedures: The evaluation procedures should be practical; to keep disruption of everyday activities to a minimum while needed information is obtained. 2. Political Viability: The evaluation should be planned and conducted with anticipation of the different positions or interests of various groups. This should help in obtaining their cooperation so that possible attempts by these groups to curtail evaluation operations or to misuse the results can be avoided or counteracted. 3. Cost Effectiveness: The evaluation should be efficient and produce enough valuable information that the resources used can be justified.
Propriety Standards
1. Service Orientation: Evaluations should be designed to help organizations effectively serve the needs of all of the targeted participants. 2. Formal Agreements: The responsibilities in an evaluation (what is to be done, how, by whom, when) should be agreed to in writing, so that those involved are obligated to follow all conditions of the agreement, or to formally renegotiate it. 3. Rights of Human Subjects: Evaluation should be designed and conducted to respect and protect the rights and welfare of human subjects, that is, all participants in the study. 4. Human Interactions: Evaluators should respect basic human dignity and worth when working with other people in an evaluation, so that participants do not feel threatened or harmed. 5. Complete and Fair Assessment: The evaluation should be complete and fair in its examination, recording both strengths and weaknesses of the program being evaluated. This allows strengths to be built upon and problem areas addressed. 6. Disclosure of Findings: The people working on the evaluation should ensure that all of the evaluation findings, along with the limitations of the evaluation, are accessible to everyone affected by the evaluation, and any others with expressed legal rights to receive the results. 7. Conflict of Interest: Conflict of interest should be dealt with openly and honestly, so that it does not compromise the evaluation processes and results.
8. Fiscal Responsibility: The evaluator's use of resources should reflect sound accountability procedures and otherwise be prudent and ethically responsible, so that expenditures are accounted for and appropriate.
Accuracy Standards
1. Program Documentation: The program should be described and documented clearly and accurately, so that what is being evaluated is clearly identified. 2. Context Analysis: The context in which the program exists should be thoroughly examined so that likely influences on the program can be identified. 3. Described Purposes and Procedures: The purposes and procedures of the evaluation should be monitored and described in enough detail that they can be identified and assessed. 4. Defensible Information Sources: The sources of information used in a program evaluation should be described in enough detail that the adequacy of the information could be assessed. 5. Valid Information: The information gathering procedures should be chosen or developed and then implemented in such a way that they will assure that the interpretation arrived at is valid. 6. Reliable Information: The information gathering procedures should be chosen or developed and then implemented so that they will assure that the information obtained is sufficiently reliable. 7. Systematic Information: The information from an evaluation should be systematically reviewed and any errors found should be corrected. 8. Analysis of Quantitative Information: Quantitative information - data from observations or surveys - in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered. 9. Analysis of Qualitative Information: Qualitative information - descriptive information from interviews and other sources - in an evaluation should be appropriately and systematically analyzed so that evaluation questions are effectively answered. 10. Justified Conclusions: The conclusions reached in an evaluation should be explicitly justified, so that stakeholders can understand their worth. 11. Impartial Reporting: Reporting procedures should guard against the distortion caused by personal feelings and biases of people involved in the evaluation, so that evaluation reports fairly reflect the evaluation findings. 12. Metaevaluation: The evaluation itself should be evaluated against these and other pertinent standards, so that it is appropriately guided and, on completion, stakeholders can closely examine its strengths and weaknesses.
RUBRIC - EVALUATION PLAN

Marginal 1 Title page, Contents & Summary X1.5 Stakeholders X2
Title Page, Table Of Contents And Executive Summary not complete
Acceptable 2
Title Page, Table Of Contents And Executive Summary minimally addressed, generic
Commendable 3
Title Page, Table Of Contents And Executive Summary completely addressed - all items covered in detail Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Most have supporting evidence. Stakeholders appear to be understood but some links are not often evident.
Outstanding 4
Title Page, Table Of Contents And Executive Summary complete, detailed, summary demonstrates thought. Specific and connected, clear and logical. Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information on stakeholders are expressed clearly and critiqued with supporting evidence. Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained in detail and fully represented in flowchart, map, or table. Demonstrates clear understanding. Purpose and description is evident and well articulated. Purpose, users, uses described. Specific questions addresses with clarity and appropriateness. Completely explains methods and agreements, describes how the evaluation activities will be implemented. Addresses evaluation, impact procedures, effectiveness, and viability in detail. Clearly addresses . Indicators & sources of evidence clearly addressed. Quality, quantity, and logistics discussed in detail. Addresses limitations, scope, and selection in detail. Supporting evidence is present. Consistently relevant detail makes the meaning exact
Limited; stakeholders appear to be understood but some links are not often evident. Stated yet is imprecise.
Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used.
Description X2
Generic description - all areas not addressed.
Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained and represented in flowchart, map, or table. Some understanding is evident. Needs some clarification.
Statement of need, expectations, activities, resources, stage of development and program context addressed. Logic model explained and fully represented in flowchart, map, or table. Demonstrates understanding. Purpose and description is evident and articulated.
Evaluation Design X2
Unclear and/or have some ambiguity. Accuracy of statements is not clear. Detail is needed to provide exactness in meaning.
Purpose, users, uses described. Specific questions addressed. Explains some methods and agreements, mentions evaluation activities .Does not addresses all areas: evaluation, impact procedures, effectiveness, and viability All information not directly related to the topic
Purpose, users, uses described. Specific questions addressed with clarity and appropriateness. Mostly explains methods and agreements, describes how the evaluation activities will be implemented. Addresses evaluation, impact procedures, effectiveness, and viability .
Gathering Evidence X2
Unclear or not stated. Supporting evidence is often missing. Detail is needed to provide exactness in meaning.
Indicators & sources of evidence mostly addressed. Quality, quantity, and logistics vaguely discussed in detail. Limitations, scope, and selection mentioned. Supporting evidence is usually present
Indicators & sources of evidence addressed. Quality, quantity, and logistics discussed . Addresses limitations, scope, and selection . Relevant examples explain most ideas Supporting evidence is present
Justifications & Conclusions X1
Unclear or not stated. Supporting evidence is often missing. Detail is needed to provide exactness in meaning. All areas not addressed Recommendation weak.
Addresses standards. Provides generic analysis and synthesis, and interpretations. Does not clearly substantiate judgments. Provides basic recommendations not fully linked to evaluation Minimally identifies uses in design and preparation. Provides some feedback. Lists basic follow-up ideas. Generic ideas for dissemination.
Addresses standards. Provides analysis and synthesis, some interpretations and does substantiate judgments. Provides recommendations based on evaluation
Addresses standards. Provides clear analysis and synthesis, solid interpretations and substantiate judgments. Provides detailed recommendations based on evaluation Clearly identifies uses in design and preparation. Delineates detailed feedback among everyone involved in the evaluation. Lists strong follow-up ideas. Creative and relevant ideas for dissemination. Fully addressees all standards; Utility, Feasibility, Propriety, and Accuracy. Reflects on each component addressing strengths and weaknesses, Demonstrates clear understanding and application All sources are accurately cited in the body and reference section using current APA style. Headings follow APA format Very well organized, good development of ideas, strong sentences, effective and varied transitions. Clear introduction and conclusion. Flawless spelling, punctuation, and capitalization, varied and interesting sentence structure.
Use & Dissemination x1
Unclear or not stated. Supporting evidence is missing. Detail is needed to provide exactness in meaning. All areas not addressed.
Completely identifies uses in design and preparation. Delineates feedback among everyone involved in the evaluation. Lists substantiated follow-up ideas. Relevant ideas for dissemination. Addresses all standards; Utility, Feasibility, Propriety, and Accuracy. Reflects on all components addressing strengths and weaknesses, Demonstrates an understanding and application Most sources are accurately cited, in the body and reference section using current APA style. Most Headings follow APA format Logical paragraph order, clears and functional transitions, good development of ideas in clears structure . Introduction and conclusion. Few spelling and punctuation errors, sentences structure mostly correct
Reflection on Standards
X2
Mentions without reflection. No demonstration of understanding and application
Addresses some standards; Utility, Feasibility, Propriety, and Accuracy. Reflects generically with little understanding and application
Organization
Mechanics and Grammar X .75 APA Format X .75
Some sources are accurately cited in body and reference section using current APA style. Headings inconsistent and do not follow APA format Writing is unstructured, inconstant paragraph order, transitions missing Missing clear introduction and/or conclusion. Many spelling and punctuation errors, many sentence fragments, and run-ons.
Some sources are accurately cited in body and reference section using current APA style. Some Headings follow APA format Topics and ideas discussed somewhat randomly, writing lacks clearly defined organization, ineffective paragraph orders, and few transitions Missing clear introduction and/or conclusion. Some careless spelling and punctuation errors, some errors in sentence structure
Total 60 possible
Comments
ASSIGNMENT EVALUATION PLAN PRESENTATION

Class teams will present their Program Evaluation Plan in presentations to the entire class. A PowerPoint Presentation will be used to share information about the Evaluation Plan. Create a visual PowerPoint presentation that identifies and explains your Evaluation Plan.
The presentation may be only 15-20 minutes in length.
Presentation must include y y y y y y

y
Engagement of Stakeholders Description of the Program Evaluation Design Gathering Evidence Justification of Conclusions Use and Dissemination of Lessons Learned Summary
Peer evaluations will be conducted on the presentations.
RUBRIC - EVALUATION PLAN PRESENTATION

Criteria & scoring Stakeholders X1 Description X1 Evaluation Design X1 Gathering Evidence Justifications & Conclusions X1 Use & Dissemination X1 Presentation X2.5 Marginal 1
Limited; stakeholders appear to be understood but some links are not often evident. Imprecise. Limited; description limited & imprecise.
Acceptable 2
Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used. Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used. Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used. Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used.
Commendable 3
Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence. Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence. Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence. Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence.
Outstanding 4
Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence. Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence. Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence. Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence.
Limited; design lacking all components Imprecise.
Limited; gathering evidence and justifications limited . Imprecise.
Limited; lack of understanding of use and dissemination - not clear. Imprecise. Knowledge and comprehension are evident. Organization is lacking. Details should be added. Greater clarity or accuracy is needed in the explanation within some slides. Background and graphics distract from the presentation Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) detract from the understandability of the presentation, and speaker appears uncomfortable.
Imprecise.-some information expressed clearly and critiqued. Some supporting evidence is used. Knowledge and comprehension are evident. Organization is evident. Most ideas are clearly identified, interpreted, applied, explained and/or critiqued Background and graphics do not support the presentation Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation understandable, and speaker appears tentative.
Clearly stated. Analysis is significant. Information expressed clearly and analyzed. Supporting evidence. Some higher order thinking is evident Organization is evident. Most ideas are clearly identified, interpreted, applied, explained and/or critiqued Accurate explanation issues and interventions shared. Background and graphics support the presentation Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation interesting, and speaker appears comfortable.
Information is clearly and precisely stated. Analysis is thorough, logical, and significant. Information expressed clearly and critiqued with supporting evidence. Higher order thinking is evident and presented in a clear logical flow. Insightful and thorough interpretation, application, explanation, and/or critique of ideas. Clear, precise, accurate, and detailed explanation of issues and interventions Background and graphics enhance the presentation Delivery techniques (posture, gesture, eye contact, and vocal expressiveness) make the presentation compelling, and speaker appears polished and confident.
Total Possible 30
Comments

COH 609 Dec 2011

Uploaded by

Copyright:

Available Formats

COH 609 Dec 2011

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

COH 609 Dec 2011

Uploaded by

Copyright:

Available Formats

National University School of Health and Human Services Department of Community Health Masters of Public Health COURSE OUTLINE

COH 609 Outline December 2011- page 2

COH 609 Outline December 2011- page 3

READING: Harris - Chapters 11 CDC Intro.to Program Eval Step 6

READING: Cultural Articles as assigned

WRITING ASSIGNMENT: EVAL PLAN

Presentations Preparation for final

READING: Review for final

WRITING ASSIGNMENT: Submit Evaluation Plan online in dropbox

Preparation for final Final Exam

COH 609 Outline December 2011- page 4

Course Grading (expressed as percentages): A 96-100% B+ 87-89% C+ A90-95% B 84-86% C B80-83% C-

77-79% 74-76% 70-73%

67-69% 64-66% 60-63%

COH 609 Outline December 2011- page 6

Definition of Grades: Graduate Courses:

BIBLIOGRAPHY: Websites & Toolkits

Books & Journals

COH 609 Outline December 2011- page 12

SCHOOL OF HEALTH AND HUMAN SERVICES REQUIREMENTS

COH 609 Outline December 2011- page 13

COH 609 Outline December 2011- page 16

ASSIGNMENT: READING RESPONSES & EXERCISES

COH 609 Outline December 2011- page 17

ASSIGNMENT LOGIC MODEL

See Template in e-College

COH 609 Outline December 2011- page 20

RUBRIC LOGIC MODEL

1. Problem statement or situation X .5 2. Goal X1 3. Input or Resources X1

Limited or not complete

Limited; unclear . Stated yet is imprecise.

Generic description - all areas not addressed.

5. Outcomes /Impacts 6. Chain of Outcomes X 1.5

COH 609 Outline December 2011- page 21

8.External Factors X.5

COH 609 Outline December 2011- page 22

Survey Questionnaire Project

COH 609 Outline December 2011- page 23

RUBRIC SURVEY QUESTIONNAIRE PROJECT

No clear letter or plan

introduction and/or conclusion

Collection, analysis & use x.5 Total Possible 20

COH 609 Outline December 2011- page 24

Description of the Program

VI. Evaluation Design

VII. Gathering Evidence

VIII. Justification of Conclusions

IX. Use and Dissemination of Lessons Learned

X. Reflection on Standards for "good" evaluations Utility Standards

COH 609 Outline December 2011- page 27

COH 609 Outline December 2011- page 28

RUBRIC - EVALUATION PLAN

Generic description - all areas not addressed.

COH 609 Outline December 2011- page 29

Justifications & Conclusions X1

Use & Dissemination x1

Mentions without reflection. No demonstration of understanding and application

COH 609 Outline December 2011- page 30

ASSIGNMENT EVALUATION PLAN PRESENTATION