Module 8.evaluation
Module 8.evaluation
Module 8.evaluation
LEARNING OUTCOMES
When you complete this module will be able to:
8.0 Introduction
1
In Module 7, we discussed the implementation
of the curriculum plan. We looked at why people resist
change, the role of teachers, students, administrator and
parents in ensuring the successful implementation of
change. In this chapter, we will focus on determining
whether the curriculum plan implemented has achieved
its goals and objectives as planned. In other words, the
curriculum has to be evaluated to determine whether all
the effort in terms of finance and human resources has
been worthwhile. Various stakeholders want to know
the extent to which the curriculum has been successfully
implemented. The information collected from evaluating a curriculum forms the basis
for making judgements about how successfully has the programme achieved its intended
outcomes and the worth or value of the programme.
ACTIVITY 8.1
Read the newspaper report at the beginning of the chapter and answer the
following questions
Do you think physical education be made an ‘examination subject’?
Do you agree with the writer’s opinions on the state of sports in schools?
McNeil (1977) states that “curriculum evaluation is an attempt to throw light on two
questions: Do planned learning opportunities, programmes, courses and activities as
developed and organised actually produce desired results? How can the curriculum
offerings best be improved?” (p.134).
2
Gay (1985) argues that the aim of curriculum evaluation is to identify its weaknesses
and strengths as well as problems encountered in implementation; to improve the
curriculum development process; to determine the effectiveness of the curriculum
and the returns on finance allocated.
3
example, so they have the relevant prerequisites and are they motivated to learn. From
these formative reviews, problems may be discovered. For example, in curriculum
document may contain spelling errors, confusing sequence of content, inappropriate
examples or illustrations. The feedback obtained could be used to revise and improve
instruction or whether or not to adopt the programme before full implementation.
SELF-TEST
SELF-TEST 8.1 8.1
Identify
Identify the
the key
key words
words in
in the
the five
five definitions
definitions of
of curriculum
curriculum evaluation.
evaluation.
Why do you need to evaluate curriculum?
Why do you need to evaluate curriculum?
What’s
What’s the
the difference
difference between
between formative
formative and
and summative
summative evaluation?
evaluation?
How should you go about evaluating curriculum? Several experts have proposed
different models describing how and what should be involved in evaluating a
8.3 Curriculum Evaluation Models
curriculum. Models are useful because they help you define the parameters of an
evaluation, what concepts to study and the procedures to be used to extract important
data. Numerous evaluation models have been proposed but three models are discussed
here.
4
about the programme introduced. For evaluations to be done correctly and aid in the
decision making process, curriculum evaluators have to:
first delineate what is to be evaluated and determine what information that has
to be collected (eg. how effective has the new science programme has been in
enhancing the scientific thinking skills of children in the primary grades)
second is to obtain or collect the information using selected techniques and
methods (eg. interview teachers, collect test scores of students);
third is to provide or make available the information (in the form of tables,
graphs) to interested parties. To decide whether to maintain, modify or eliminate
the new curriculum or programme, information is obtained by conducting the
following 4 types of evaluation: context, input, process and product.
Context
Input FORMATIVE
Process
+
Product SUMMATIVE
5
The techniques of data collection would include observation of conditions in the
school, background statistics of teachers and interviews with players involve in
implementation of the curriculum.
c) Process Evaluation (Is it being done?) is the provision of periodic feedback while
the curriculum is being implemented.
Context
Input
GOALS PLANS
CORE
VALUES
OUTCOMES ACTIONS
Product
Process
SELF-TEST
SELF-TEST 8.2 8.2
What
What is the difference between
is the difference between context
context evaluation
evaluation and
and input
input evaluation
evaluation
according
according to to the
the CIPP
CIPP model?
model? Give
Give specific
specific examples
examples
What
What isis the
the difference
difference between
between process
process evaluation
evaluation and
and product
product
evaluation
evaluation according
according to
to the
the CIPP
CIPP mode?
mode? Give
Give specific
specific examples.
examples.
6
8.4.2 Case Study:
Evaluation of a Programme on Technology Integration in Teaching and
Learning in Secondary Schools
The integration of information and communication technology (ICT) in
teaching and learning is growing rapidly in many countries. The use of the internet
and other computer software in teaching science, mathematics and social sciences is
more widespread today. To evaluate the effectiveness of such a programme using the
CIPP model would involve examining the following:
Input: Examine what resources are put into technology integration (Identify the
educational strategies most likely to achieve the desired result)
Is the content selected for using technology right?
Have we used the right combination of media? (internet, video-clips, etc)
ACTIVITY 8.2
With reference to Case Study 8.4.2
Suggest other questions you would ask regarding process
evaluation (i.e. implementation issues).
What data collection techniques would you recommend for
carrying out product evaluation to determine the teaching and
learning outcomes of technology integration?
7
8.4.3 Stake’s Countenance Model
The model proposed by Robert Stake (1967) suggests three phases of curriculum
evaluation: the antecedent phase, the transaction phase and the outcome phase. The
antecedent phase includes conditions existing prior to instruction that may relate to
outcomes. The transaction phase constitutes the process of instruction while the
outcome phase relates to the effects of the programme. Stake emphasises two
operations; descriptions and judgements. Descriptions are divided according to whether
they refer to what was intended or what actually was observed. Judgements are
separated according to whether they refer to standards used in arriving at the judgements
or to the actual judgements.
INSTRUCTION
8
He proposed the Connoisseurship Model in which he claimed that a
knowledgeable evaluator can determine whether a curriculum programme has been
successful, using a combination of skills and experience. The word ‘connoisseurship’
comes from the Latin word cognoscere, meaning to know. For example, to be a
connoisseur of food, paintings or films, you must have knowledge about and experience
with different types of food, paintings or films before you are able to criticise. To be a
food critic, you must be a connoisseur of different kinds of foods. To be a critic, you
must be aware and appreciate the subtle differences in the phenomenon you are
examining. In other words, the curriculum evaluator must seek to be an educational
critic. When employing the procedure of educational criticism the following questions
may be asked:
What has happened in the classrooms as a result of implementation of the new
curriculum?
What are some of the events that took place? (eg. more students are participating
in field work, more students are asking questions in class, even academically
weak students are talking in group activities)
How did students and teachers organise themselves in these events?
What were the reactions of participants in these events? (eg. students enjoyed
working collaboratively in projects)
How can the experiences of learners be made more effective as suggested by
students, teachers and administrators? (eg. more resources are needed for
fieldwork, more computers are needed to integrate the internet in teaching and
learning).
You will notice that these questions places more emphasis on the process of learning
and the quality of experiences by those involved in the implementation of the
curriculum; namely, students, teachers and administrators. According to the
Connoisseurship Model, evaluators provide a description and interpretation of the
curriculum plan implemented:
1) Description: The evaluator records the actions, the features of the environment
and experiences of students, teachers and administrators. People who read the
evaluation report will be able to visualise what the place looks like and the
processes taking place. The aim here is to help the reader “see” the school or
classroom and get a feel of what the curriculum evaluator or critic is attempting
to understand and help others understand.
9
o The evaluator keeps notes of what is done, what is said and more
importantly what is not said. The evaluator should strive to describe the
tone of the curriculum in action (Ornstein and Hunkins, 1998).
o The evaluator interview students, teachers and administrators about the
quality of the curriculum
o The evaluator would analysis student’s work
.
One of the great benefits of Elliot W. Eisner's activities has been the way in
which he has both made the case for a concern with connoisseurship and criticism, and
mediated these concerns for educators and researchers. The importance of his advocacy
of these ideas cannot be underestimated - especially at a time when rather narrow
concerns with instrumental outcomes and an orientation to the technical dominate.
Together they offer educators a more helpful and appropriate means to approach
evaluation, for example.
Over the time that Eisner has been writing there have been significant shifts in
the context in which schools have to operate. While there have been other voices
calling for changes in the culture of schooling (notably Howard Gardner in this
arena), the impact of globalization, growing centralization in many schooling systems,
reaction against more process-oriented forms of pedagogy, and a growing
instrumentalism education have served to make Eisner's message both more pertinent
to schools, and more difficult to respond to.
10
8.5 Phases of Curriculum Evaluation
The evaluator determines what is to be evaluated which may
be the total school system, a particular district, a
particular grade level or a particular subject. The objectives
1. Aspects of the of the evaluation activity are clearly stated.
curriculum to be
evaluated
11
the information obtained may not be accurate as it relies how truthfully subjects respond
to the questions. There is also the fear that the wordings used can bias client's responses.
Questionnaires are impersonal and since only a sample of subjects are given the
instrument, we not get the full story.
8.6.2 Interviews
Interviews are usually one-on-one situations in which an individual asks
questions to which a second individual (which may be a teacher, principal, student,
parent) responds. The person asking the questions is called the interviewer while the
person giving answers to the questions is called the interviewee. Interviews are used
when you want to fully understand someone's impressions or experiences, or learn more
about their answers to questionnaires. There are two general types of interviews
depending on the extent to which the responses required are unstructured or structured.
In an unstructured interview, the interviewer does not follow a rigid script and
there is a great deal of flexibility in the responses. For example; “Why do you think the
recommended textbook for the course is difficult for low ability learners? The teacher
responding to such a question will give a variety of reasons. Some of the reasons given
may be of a general nature while others may be specific to certain sections of the
textbook. This makes the task of keeping track of responses more difficult. The open-
endedness of the question will require that the interviewer record all responses and
make sense of it later. The advantage of the unstructured interview is that it allows the
evaluator to gather a variety of information, especially in relation to the interviewee’s
knowledge, beliefs or feelings toward a particular situation.
In a structured interview, the questions asked usually require very specific
responses. For example, “Is the recommended textbook difficult for low ability learners
because: a) there is too much content; b) the language used is beyond the
comprehension of low ability learners, c) or there are too few examples and illustrations.
Regardless of which type of interview is used, evaluators should ensure that each
question is relevant for its intended purpose. In the end, the data must be translated into
a form that can be analysed and this has to be dome carefully to preserve accuracy and
to maintain the sense of the data. The advantage of interviews is that it can get a full
range and depth of information and it develops a relationship with teachers and students
and it is more flexible. However, interview can take much time, can be hard to analyze
and compare, can be costly and interviewer can bias client's responses.
8.6.3 Observations
To gather accurate information about how a program actually operates,
particularly about processes. In other words to view operations of a program as they
are actually occurring. For example, can the people involved adapt to events as they
occur.
8.6.4 Documents
When we want impressions of how a programme operates without interrupting
the programme; we can review the memos, minutes, etc to get a comprehensive and
historical information about the implementation of the programme. However, we
should be quite clear about what looking for as there may be a load of documents.
12
Method Overall Purpose Advantages Challenges
-might not get careful
-can complete anonymously
feedback
-inexpensive to administer
when need to quickly -wording can bias client's
questionnaires, -easy to compare and analyze
and/or easily get lots of responses
surveys, -administer to many people
information from people in -are impersonal
checklists -can get lots of data
a non threatening way -in surveys, may need
-many sample questionnaires
sampling expert
already exist
- doesn't get full story
13
8.3 Case Study: Evaluation of a Mathematics Curriculum in South Africa
Method: Based on project workers’ observation and written field notes made during
the implementation of the MALATI curriculum the following findings were obtained:
Findings:
a. a number of teachers had not yet received the most basic communications
issued to schools regarding Curriculum 2005
b. teachers had difficulty interpreting certain aspect of the official curriculum
document. Lack of clarity led to confusion
c. the curriculum document had content errors
d. content knowledge of teachers was not adequate to handle some of the topic in
the curriculum such as statistics.
e. learners did not have the prior experience assumed in the curriculum eg. in
grade 9, the teaching of probability assumes that learner had done some
statistics in the earlier grades
f. teachers are continuing to teach the topics they are used to and are reluctant to
use the MALATI materials
g. the curriculum suggested that group work be used in teaching probability and
data handling. Learners were not accustomed to group discussion and listening
to one another.
h. the teaching of the topic took a longer time as teachers struggled to deal with
learners’ everyday experiences in the teaching of probability
Recommendations:
1) Teachers need workshops on selected aspects of the content
2) Selected parts of the curriculum documents need to be rewritten to reduce
confusion
3) To convince teachers not to treat the teaching of probability and statistics as
“new content” but teach it for its mathematical value
[Source: Karin Brodie and Craig Pournara, 2003. Towards a framework for developing and
researching groupwork in mathematics classrooms .http//www.hsrcpress.ac.za.
ACTIVITY 8.2
What are some of the problems identified with the implementation of the MALATI
programme?
Based on the findings list the recommendations made.
14
DISCUSSION
DISCUSSION QUESTIONS:
QUESTIONS:
Identify
Identify some
some problems
problems in
in the
the implementation
implementation of
of the
the Primary
Primary School
School Integrated
Integrated
Curriculum
Curriculum (KBSR) and the Secondary School Integrated Curriculum (KBSM)?
(KBSR) and the Secondary School Integrated Curriculum (KBSM)?
Describe
Describe how
how the
the teaching
teaching of
of science
science and
and mathematics
mathematics in
in English
English was
was
implemented
implemented in
in your
your school?
school?
“New
“New curriculum
curriculum often
often fail
fail to
to become
become established
established in
in schools
schools because
because the
the
importance and complexity of the implementation phase is not understood”
importance and complexity of the implementation phase is not understood”
Discuss.
Discuss.
READINGS
15