12-6&9-21 (Handout) Final-MUBalagtas-DepEd Bulacan-SOLO

Use of the SOLO Framework
in Test Development in the

New Normal
Presented this 6th day of December 2021
at DepEd Bulacan//Via Zoom
MARILYN UBIÑA-BALAGTAS, Ph.D.

University Professor, College of Graduate Studies and Teacher Education Research, Philippine Normal University
President, Philippine Educational Measurement and Evaluation Association. Inc. (PEMEA)
[email protected]
Mood Setting
Use of the SOLO Framework in Test Development in the New Normal

Dr. Marilyn Ubiña-Balagtas
2
December 2021
SESSION TARGET
The participants are expected to use the SOLO Framework in test
development in the new normal.
Topics to Discuss
1. The SOLO Framework and Use in Item Writing, Scoring and
Communication of Test Results
2. The SOLO Framework Combined with the Revised Bloom’s
Taxonomy for Test Development
3. Preparation of the Table of Specifications for Test
Development

3
December 2021
Pre-Test
https://forms.office.com/r/5yPqL5GdtM
4
December 2021
What is SOLO?
vSOLO to mean Structure of Observed Learning
Outcomes was popularized by Kevin F. Collis and
John B. Biggs in 1982 in their book “Evaluating the
Quality of Learning: The SOLO Taxonomy.
v It is a framework for understanding with five levels,

where each level embraces the previous level but adds
something more.

5
December 2021
What is SOLO?
Understanding/Competence/Proficiency
Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631

6
December 2021
What is SOLO?
Understanding/Competence/Proficiency
Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631

7
December 2021
What is SOLO?

8
December 2021
How is SOLO used in test construction?
v Use SOLO in structuring responses to questions in a

multiple-choice test format.
v Use SOLO in structuring rubrics in describing
responses to questions in an open-ended or
constructed-response type of test.
v Use SOLO to describe the degree of correctness of
responses given to questions in a test.
v Use SOLO in scoring, interpreting and
communicating test results.

9
December 2021
Level of SOLO Levels of
Code Meaning
Understanding Thinking
Pre-structural (P) 0 I am not sure about this topic
Uni-structural (U) 1 I have one idea about this topic Surface
Multi-Structural (M) 2 I have several ideas about this topic
Relational (R) 3 I can link the ideas together to see the big
picture
Deep
Extended Abstract 4 I can look at this idea in a new and different
(EA) way.

10
December 2021
Levels of Level of
Code Meaning
Proficiency Understanding
One does not know any idea about the lesson
Not Proficient Pre-structural 0
assessed.
Somewhat Uni-structural 1 One knows an idea about the lesson
assessed.
Proficient
Nearly Proficient Multi-Structural 2 One knows several ideas about the lesson
assessed but could not link them together.
Proficient Relational 3 One knows several ideas about the lesson
assessed and can link them together.
Highly Proficient Extended Abstract 4 One can link several ideas about the lesson
assessed and can represent it in a new and
different way/situation.

11
December 2021
Levels of Level of Level of
Code Meaning Understanding Test Format
Not 0 One does not know any idea Multiple Constructed-
Proficient Pre-structural about the lesson assessed.
Choice Response
Somewhat 1 One knows an idea about the
Proficient Uni-structural lesson assessed. Surface
2 One knows several ideas Understanding
Nearly Multi- about the lesson assessed
Proficient Structural but could not link them
together.
3 One knows several ideas
about the lesson assessed
Proficient Relational and could link them
together.
Deep
4 One could link several ideas Understanding
Extended about the lesson assessed
Highly
and could represent them in a
Proficient Abstract new or a different
way/situation.
12
December 2021
Levels of Level of Level of
Code Meaning Understanding Scoring Options
Not 0 One does not know any idea
Proficient Pre-structural about the lesson assessed. 0 0
Somewhat 1 One knows an idea about the
Proficient Uni-structural lesson assessed. Surface 1 1
2 One knows several ideas Understanding
Nearly Multi- about the lesson assessed
Proficient but could not link them 2 3
Structural
together.
3 One knows several ideas
about the lesson assessed
Proficient Relational 3 5
and could link them
together.
Deep
4 One could link several ideas Understanding
Extended about the lesson assessed
Highly
and could represent them in a 4 6
Proficient Abstract new or a different
way/situation.
13
December 2021
Let’s have an example
of questioning
in a test that is in
constructed-response
format.

December 2021 14
Study the table below. What percent of the population of the USA got infected of COVID-19 compared
with that of the Philippines?
As of June 30, 2021

15
December 2021
Who among the four students gave the correct answer?
Student 1 Student 3
Philippines USA Philippines USA
1, 408,058 = 0.01268% 34, 527, 493= 0.1037% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
Student 2 USA Student 4

Philippines 34, 527, 493= 10.37% Philippines USA
1, 408,058 = 1.27% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
332,929,050
111,025,935 332,929,050
111,025,935
10.37% - 1.27% = 9.1%
Answer: 10.37% of the population of USA got infected of
COVID-19 compared to the Philippines with only 1.27%.

December 2021 16
How do the students differ in their understanding of the problem based on their answers?
Student 1 Student 3
1, 408,058 = 0.01268% 34, 527, 493= 0.1037% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
This student knows how to compute the rate and how
This student knows how to compute the rate but to express it in percent but did not provide clearly the
does not know how to express this in percent. correct answer.
Student 2 USA Student 4
Philippines Philippines USA
34, 527, 493= 10.37%
1, 408,058 = 1.27% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
332,929,050
111,025,935 111,025,935 332,929,050
10.37% - 1.27% = 9.1% Answer: 10.37% of the population of USA got infected of
COVID-19 compared to the Philippines with only 1.27% .
This student knows how to compute the rate and how to express it in This student knows how to compute the rate and how to
percent but misunderstood the problem as it does not require subtraction. express it in percent to answer the problem.

December 2021 17
Which of the SOLO levels does each response represent?
Student 3
Student 1 Philippines USA
Philippines USA 1, 408,058 = 1.27% 34, 527, 493= 10.37%
1, 408,058 = 0.01268% 34, 527, 493= 0.1037% 111,025,935 332,929,050
111,025,935 332,929,050
This student knows how to compute the rate and how to express it in
This student knows how to compute the rate but does not percent but did not provide clearly the correct answer.
(MULTISTRUCTURAL)
know how to express this in percent. (UNISTRUCTURAL)
Student 4
Student 2 USA Philippines USA
Philippines 34, 527, 493= 10.37% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
1, 408,058 = 1.27% 332,929,050 332,929,050
111,025,935
111,025,935
10.37% - 1.27% = 9.1% Answer: 10.37% of the population of USA got infected of
percent but misunderstood the problem as it does not require This student knows how to compute the rate and how to
subtraction. (MULTISTRUCTURAL) express it in percent to answer the problem. (RELATIONAL)

December 2021 18
How could Extended Abstract in the SOLO framework look like?
Student 1 Student 3
1, 408,058 = 0.01268% 34, 527, 493= 0.1037% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
This student knows how to compute the rate and how to
This student knows how to compute the rate but does not express it in percent but did not provide clearly the correct
know how to express this in percent. (UNISTRUCTURAL) answer. (MULTISTRUCTURAL)
Student 2 Student 4
1, 408,058 = 1.27% 34, 527, 493= 10.37% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
Answer: 10.37% of the population of USA got infected of COVID-19 compared to the Answer: 10.37% of the population of USA got infected of
Philippines with only 1.27% COVID cases. This explains why USA is no.1 in the world in
terms of COVID cases while the Philippines is just ranked 24th.
This student knows how to compute the rate and how to express it in This student knows how to compute the rate and how to
percent to answer the problem. The student also gave more acceptable express it in percent to answer the problem. (RELATIONAL)
information than what is needed. (EXTENDED ABSTRACT)
Use of the SOLO Framework in Test Development in the New Normal 19
Dr. Marilyn Ubiña-Balagtas//December 2021
How will the rubric be designed using SOLO?
Adjectival
Levels of Quality Indicator SOLO Level
Description
The answer given is correct and beyond what was Extended Abstract
5 Excellent
asked for.
The answer given shows understanding of two or more Relational
4 Very Good processes in solving the problem and it directly answers
the problem.
The answer given shows understanding of two or more Multistructural
3 Good processes in solving the problem but does not directly
answer the problem.
The answer given shows understanding of one process Unistructural
2 Fair
that leads to answering the problem.
There is NO attempt to answer the problem or there is Prestructural
1 Poor
an answer but meaningless.
Dr. Marilyn Ubiña-Balagtas 20
December 2021
How are responses scored based on SOLO?
Levels of Adjectival Example 1 Example 2
Description
Indicator SOLO Level Code of Scoring of Scoring
Quality
5 Excellent The answer given is correct and beyond Extended 4 4 Points 10 Points
what was asked for. Abstract
4 Very Good The answer given shows understanding of 3 3 Points 8 Points
two or more processes in solving the Relational
problem and directly answers the problem.
3 Good The answer given shows understanding of 2 2 Points 6 Points
two or more processes in solving the
problem but does not directly answer the
Multistructural
problem.
2 Fair The answer given shows understanding of 1 1 Point 3 Points
one process that leads to answering the Unistructural
problem.
1 Poor There is NO attempt to answer the problem 0 0 Point 0 Point
or there is an answer but meaningless.
Prestructural
December 2021
How are responses scored based on SOLO?
Student 1 Student 3
1, 408,058 = 0.01268% 34, 527, 493= 0.1037% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
This student knows how to compute the rate but does not know how to percent but did not provide the clearly the correct answer.
express this in percent. (UNISTRUCTURAL) – 3 Points (MULTISTRUCTURAL) – 6 Points
Student 2 Student 4
1, 408,058 = 1.27% 34, 527, 493= 10.37% 1, 408,058 = 1.27% 34, 527, 493= 10.37%
111,025,935 332,929,050 111,025,935 332,929,050
Answer: 10.37% of the population of USA got infected of COVID-19 compared to Answer: 10.37% of the population of USA got infected of
the Philippines with only 1.27% COVID cases. This explains why USA is no.1 in COVID-19 compared to the Philippines with only 1.27%
the world in terms of COVID cases while the Philippines is just ranked 24th. COVID cases.
This student knows how to compute the rate and how to express it in percent This student knows how to compute the rate and how to express it in
and how to answer the problem. The student also gave more information than
percent and how to answer the problem. (RELATIONAL) – 8 Points
what is needed. (EXTENDED ABSTRACT) – 10 Points
How do we transform the
constructed-response
item into a
multiple-choice format?

23
December 2021
How do we transform the
Study the table below. What percent of the population of the USA got infected constructed-response
of COVID-19 compared
with that of the Philippines? item into a
multiple-choice format?
As of June 30, 2021
A. 10.37% of the population of USA got

infected of COVID-19 compared to the
Philippines with only 1.27%.
B. 0.1037% of the population of USA got

Philippines with only 0.01268%.
C. 0.1037 of the population of USA got

Philippines with only 0.01268.
D. USA has 9.1% more COVID cases than

the Philippines.

24
December 2021
SOLO
As of June 30, 2021 Options CODE
SOLO Levels
10.37% of the population of Relational

USA got infected of COVID-19
A compared to the Philippines
3
with only 1.27%.
0.1037% of the population of
B. compared to the Philippines
1
with only 0.01268%.
Unistructural
0.1037 of the population of
C. compared to the Philippines
1
with only 0.01268.
USA has 9.1% more COVID Multistructural

D. cases than the Philippines. 2

25
December 2021
How do we communicate test results using the SOLO Framework?
Study the given graph. What percent of the population of the USA got infected of COVID-19 compared
Options Explanation/
SOLO
Feedback
As of June 30, 2021 10.37% of the population of You know how to
compute the rate and
A compared to the Philippines
3 how to express it in
percent to answer the
with only 1.27%. problem.
0.1037% of the population of
B. compared to the Philippines
1
You know how to
with only 0.01268%. compute the rate, but
0.1037 of the population of you do not know how to
express this in percent.
C. compared to the Philippines
1
with only 0.01268.
D. USA has 9.1% more COVID You know how to
2 compute the rate and
cases than the Philippines. how to express it in
percent, but you
misunderstood the
problem as it does not
require subtraction.
26
December 2021
SESSION TARGET
Topics to Discuss
Development

27
December 2021
How do we combine SOLO with RBT to make
assessment multidimensional?
Revised Bloom’s Taxonomy
(Krathwohl & Anderson, 2001)
Dimensions Remember Understand Apply Analyze Evaluate Create

5 9 13 17 21
least student-centered
Factual & authentic
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
4 8 12 16 20
most student-
Metacognitive centered &
authentic

December 2021 28
Cognitive Behaviors in the Revised Bloom’s Taxonomy
Retrieving, recognizing, and recalling relevant knowledge from
Remember:
long-term memory.
If you aim for Constructing meaning from oral, written, and graphic messages
authentic Understand: through interpreting, exemplifying, classifying, inferring,
assessment, comparing and explaining.
which of these Carrying out or using a procedure through executing, or
Apply:
levels should implementing.
you target? Breaking material into constituent parts, determining how the
Analyze: parts relate to one another and to an overall structure or
purpose through differentiating, organizing, and attributing.
Making judgments based on criteria and standards through
Evaluate:
checking and critiquing.
Putting elements together to form a coherent or functional
Create: whole; reorganizing elements into a new pattern or structure
through generating, planning, or producing.

December 2021 29
Knowledge Dimensions in the
If you aim Revised Bloom’s Taxonomy
for
authentic
assessme
Knowledge Description
nt, which Dimension
of these
levels
Factual Terminologies, specific details and elements
should you Conceptual Knowledge of classifications, principles ,
target? generalization, theories, models, and structures
Procedural Knowledge of subject-specific skills and
algorithms/procedures
Metacognitive Self-knowledge, strategic knowledge, contextual
and conditional knowledge, knowledge of a
cognitive task
December 2021 30
Which is the most learner-centered and
authentic cell in this matrix?

Factual 1 5 9 13 17 21
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 8 12 16 20 24

December 2021 31

5 9 13 17 21
Factual & authentic
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
4 8 12 16 20
most student-
authentic

December 2021 32

5 9 13 17 21
Factual & authentic
Conceptual 2 6 10 14 18 22
1. What is 9 in 4 +
Procedural 3 7
5 = 9?
11 15 19 23
4 8 12 16 20
most student-
authentic

December 2021 33

5 9 13 17 21
Factual & authentic
Conceptual 2 6 10 14 18 22
Procedural 3 7 11 15 19 23
Metacognitive 4 2. What
8 is addition?
12 16 20
most student-
centered &
authentic

December 2021 34

5 9 13 17 21
Factual & authentic
Conceptual 2 6 10 14 18 22
Procedural 3 3.
7 What is the
11 1 st
15 19 23
step in adding
4 8 12 16 20
most student-
Metacognitive numbers? centered &
authentic

December 2021 35

5 9 13 17 21
Factual & authentic
Conceptual 2 64. Maria bought

104 ripe 14 18 22
Procedural 3 7and 5 unripe 11
mangoes. 15 19 23
How many ripe
4 8mangoes did12 16 20
most student-
Metacognitive she buy? centered &
authentic

December 2021 36

5 is the sum9of 13 17Show 21
Factual & authentic 11. What 23.
4 and 5? how to add
Conceptual 2 6 10 14 18number.
any 22
Procedural 3 7 11 15 19 23
4 8 12 16 20
most student-
authentic
15. How does the
3. What is the first sum of 4 and 5
7. How will you show
step when adding compare with the 19. Is the sum of 4 and
addition of 4 and 5? sum of 6 and 3?
numbers? 5 10? 37

4 ripe5and 5 916.Maria bought
13 4 ripe and 17 21
Factual 12.Maria bought
& authentic 5
unripe mangoes. How many unripe mangoes today. She
Conceptual 2 did she buy?6
mangoes 10bought 10 mangoes
14 yesterday. 18 22
How many mangoes did she buy
Procedural 3 7 11in all? 15 19 23
4 8 12 16 20
most student-
8.Maria bought 4 authentic
ripe and 5 unripe 20.Maria bought 4 ripe and
4.Maria bought 4 ripe and 5 5 unripe mangoes. She 24. You need to buy items
mangoes. Which
unripe mangoes. How many bought 10 mangoes in all.
is more, ripe or Do you agree? Explain. at the store. What are these
ripe mangoes did she buy? unripe mangoes? and their total? 38
Let’s see more
examples of how to
increase the
authenticity of items
guided by the RBT.

December 2021 39
Which is more authentic and targets higher thinking
skill? Explain your answer.
1. What is 15 + 5?
A. 10
B. 15
C. 20
2. During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10
B. Php15
C. Php 20
December 2021 40
1. What is 15 + 5?
Add – Apply - Procedural
A. 10
B. 15
C. 20
2. During their morning break, Romalyn bought pancit canton worth Php15 and
a glass of coconut juice worth Php5. How much did Romalyn spend for her
snacks?
A. Php 10 Add/Solve
B. Php15 Apply/Analyze
C. Php 20 Procedural/Metacognitive
December 2021 41
1. During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A. Php 10
B. Php15
C. Php 20
2. During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A. Php 10
B. Php15
C. Php 20

December 2021 42
1. During their morning break, Romalyn bought pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did Romalyn spend for her snacks?
A.Php 10 Add/Solve
C.Php 20 Procedural/Metacognitive
2. During your morning break, you bought a pancit canton worth Php15 and a glass of
coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10 Add/Solve
B. Php15 Apply/Analyze Personal/
C. Php20 Procedural/Metacognitive authentic

December 2021 43
1. During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10
B. Php15
C. Php20
2. During your morning break, you bought ______ worth Php____ and
_______worth Php_____. How much did you spend for your snacks?

December 2021 44
1. During your morning break, you bought a pancit canton worth Php15 and a
glass of coconut juice worth Php5. How much did you spend for your snacks?
A.Php 10 Add/Solve
Personal/
authentic
Procedural/Metacognitive
C. Php20
Add/Solve/Create
Apply/Analyze/Create Personal/
Procedural/Metacognitive authentic

December 2021 45
2. Construct your own word problem involving addition of whole numbers.

December 2021 46
Add/Solve/Create
Apply/Analyze/Create
1. During your morning break, you bought ______ Procedural/Metacognitive
worth Php____ and _______worth Php_____.
Personal/
How much did you spend for your snacks? Authentic/Restricted
2. Construct your own word problem involving

addition of whole numbers. Add/Solve/Create
Personal/
Apply/Analyze/Create Authentic/Extended
Procedural/Metacognitive

December 2021 47
Why combine
the RBT and SOLO
in test development?

48
December 2021
Why combine the SOLO Framework
and the RBT?
v Combining the SOLO Framework and the RBT makes assessment
multidimensional.
v Multidimensional assessment of learning is an approach of looking at
different facets of learning to identify the strengths and weaknesses of the
learner to better inform instruction and better describe achievement.
v Assessment is multidimensional if it satisfies any of the following:
1. It examines two or more dimensions in one’s learning (e.g., Revised Bloom’s
Taxonomy, PISA).
2. It examines both cognitive and non-cognitive dimensions of learning (e.g.,
TIMSS)
December 2021 49
Why do we need to practice
multidimensional assessment?
v Multidimensional assessment of learning is needed in the K to

12 program as the curriculum intends to develop holistically
the learners and equip them with 21st century skills to make
them locally and globally competitive.
v The curriculum framework of the different areas in basic and
higher education shows multidimensionality, hence assessment
should be multidimensional as well.

December 2021 50
Example of the Multidimensional Curriculum Framework
The curriculum framework shows multidimensionality; hence assessment should be multidimensional as well.

December 2021 51
Example of the Multidimensional Curriculum Framework
The curriculum framework shows multidimensionality;

hence assessment should be multidimensional as well.
December 2021 52
What are the models of
Some models of multidimensional assessment of learning that
are practiced in the Philippines are as follows:
Classroom
1. Revised Bloom’s Taxonomy
2. Revised Bloom’s Taxonomy combined with the SOLO
Framework
3. Combination of Different Assessment Approaches that
Appropriately Assess Different Learning Outcomes
December 2021 53
Some models of multidimensional assessment of learning that are
practiced in the Philippines are as follows:
National
1. Assessment of Competencies and Excellence of Students
(ACES) of Rex Institute for Student Excellence, Inc.
2. Basic Education Exit Assessment (BEEA)Framework

December 2021 54
Assessment of Competencies and Excellence of Students (ACES)
from Rex Institute for Student Excellence, Inc. (RISE)

December 2021 55

December 2021 56

December 2021
21st Century Skills SHS Exit Assessment Competencies

• Language and Communication
• Wika at Komunikasyon
• Humanities
• Media and Information Literacy
• Mathematics
v • Science
• Social Science and Personal Development Performance
Content • Philosophy Standards
Knowledge
Source: Balagtas et.al (2020). Analysis of the Basic Education Exit Assessment (BEEA) for SHS
Exit Assessment for Senior High School
Students: Input to the K to12 Reform

Some models of multidimensional assessment of learning that are

practiced in the Philippines are as follows:
International
1. Program for International Student Assessment (PISA)
2. Trends in International Mathematics and Science Survey (TIMSS)
3. Southeast Asia Primary Learning Metric (SEA-PLM)

December 2021 59
PISA 2015/2018 Scientific Literacy

Assessment Framework le dge
w
for Cognitive Dimension of Kno
th
Competencies De p
Knowledge

December 2021
CONTEXTS
• Personal
PISA 2015/2018
• Local/National Scientific
• Global
Literacy
COMPETENCIES
Assessment
• Explaining Item Framework
phenomena
scientifically
• Evaluating and
designing scientific
inquiry COGNITIVE
KNOWLEDGE • Interpreting data DEMAND
• Content and evidence • Low
• Procedural scientifically • Medium
• Epistemic • High

December 2021
PISA 2021 Framework

for Mathematics
Literacy
Figure #: Mathematics Framework of the PISA 2021

December 2021 62

December 2021 63

December 2021 64

December 2021 65
How do we combine
the RBT and SOLO

66
December 2021
How do we combine the RBT and SOLO
Objective: Identify the different cognitive processes in the Revised Bloom’s
Taxonomy
Item
1. What cognitive process in the RBT is involved when a student is required to
organize data into various categories?
A. analyze • What is the answer to the?
item?
B. apply • What cognitive process ?
and knowledge dimension
C. classify does the item represent?
?
• What levels of
D. understand understanding based on
?
the SOLO Framework do
Use of the SOLO Framework in Test Development in the New Normal the choices represent?
December 2021 67
Taxonomy
Item Factual - Remember
A. analyze • What is the answer to the?
item?
B. apply • What cognitive process
?
and knowledge dimension
C. classify does the item represent?
?
• What levels of
D. understand understanding based on
?
68
December 2021
Taxonomy
A. analyze ?
B.
Do the options in the MC item
apply ?
have the same
C. classify ?
degree of correctness?
D. understand ?

69
December 2021
Taxonomy
A. analyze ?
How will you describe the
B. apply offered responses ?
C. classify in terms of SOLO? ?
D. understand ?

70
December 2021
Taxonomy
A. analyze Relational
B. apply Uni-structural
C. classify Uni-structural
D. understand Uni-structural

71
December 2021
Taxonomy
B. apply How will you represent Unistructural

pre-structural in this
C. classify set of responses?
Unistructural
D. understand Unistructural

72
December 2021
Taxonomy
B. apply How will you represent Unistructural

pre-structural in this
C. classify set of responses?
Unistructural
D. characterize Prestructural

73
December 2021
Let’s have another
example of how to combine
RBT and SOLO
in test development.

74
December 2021
Objective: Identify the steps in developing summative tests
Example:
Which is the first step in constructing an Achievement Test ?
A. Examine the instructional objectives • What is the answer to the
B. Prepare a table of specifications item?
• What cognitive process
C. Construct test items and knowledge dimension
does the item represent?
D. Teach the students • What levels of
understanding based on
December 2021 75
Objective: Identify the steps in developing summative tests
Example: Procedural – Remember

Which is the first step in constructing an Achievement Test ?
A. Examine the instructional objectives Relational
B. Prepare a table of specifications Multistructural
C. Construct test items Unistructural
D. Teach the students Prestructural

December 2021 76
Objective: Use assessment results in making decisions about teaching and learning
Situation:
After teaching the day’s lesson, you gave your students a test. The result shows that almost all of
the students got a low score. Which of the following should you do?
A. Give a parallel test.
• What is the answer to the
B Teach the lesson again in the same way. item?
C. Count the frequency of errors to know what to explain again. and knowledge dimension
D. Determine the items students found difficult that would require does the item represent?
• What levels of
further explanation. understanding based on
December 2021 77
Situation: Metacognitive - Apply

A. Give a parallel test. Uni-structural
B Teach the lesson again in the same way. Uni-structural
C. Count the frequency of errors to know what to explain again. Multi-structural
D. Determine the items students found difficult that would require Relational
further explanation.
December 2021 78
How do we score test items using the SOLO Framework?
Situation: Metacognitive - Apply
A. Review its pre-requisite lesson. Uni-structural 1 point Somewhat
Proficient
B. Teach the lesson again in the same way. Uni-structural 1 point Somewhat
Proficient
C Count the frequency of errors to know what Multi-structural 2 points Nearly Proficient
to explain again.
D Determine the items students found difficult Relational 3 points Proficient
that would require further explanation.

79
December 2021
Let’s have another
example of how to combine
RBT and SOLO
in test development.

80
December 2021
Objective: Justify the appropriateness of an assessment method given the

objective of the lesson (Constructed-Response Format).
Situation:
You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old
telephone directories. Considering the congruency between the intent and the
assessment method used, did you do what is right? Justify your answer.

December 2021 81

objective of the lesson (Constructed-Response Format).
Situation: • What is the answer to the
You want to find out if your students can make a new item?
material out of recyclable materials. You gave an essay and knowledge dimension
test requiring them to explain what to do out of old does the item represent?
• What levels of
telephone directories. Considering the congruency understanding based on
between the intent and the assessment method used, the SOLO Framework
did you do what is right? Justify your answer. could the answers
represent?
82
December 2021

objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation: • What is the answer to the
You want to find out if your students can make a new item?
material out of recyclable materials. You gave an essay and knowledge dimension
test requiring them to explain what to do out of old does the item represent?
• What levels of
telephone directories. Considering the congruency understanding based on
between the intent and the assessment method used, the SOLO Framework
did you do what is right? Justify your answer. could the answers
represent?
83
December 2021

objective of the lesson (Constructed-Response Format). Evaluate-Metacognitive
Situation: Sample Answers Given
You want to find out if your students can
make a new material out of recyclable • Example 1:
materials. You gave an essay test requiring
them to explain what to do out of old • Example 2:
telephone directories. Considering the
congruency between the intent and the • Example 3:
assessment method used, did you do what
is right? Justify your answer. • Example 4:

84
December 2021
Objective: Justify the appropriateness of an assessment method given the objective of the lesson
(Constructed-Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an
essay test requiring them to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
Scoring Rubric
1 Poor You gave no response or showed no effort to respond.
2 Fair You gave a response but with no justification.
3 Good You gave a response with a justification that is partially acceptable or not applicable in the given
situation.
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation.
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not
only in the given situation.
December 2021 85
Objective: Justify the appropriateness of an assessment method given the objective of the lesson (Constructed-
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable
materials. You gave an essay test requiring them to explain what to do out of old telephone
directories. Considering the congruency between the intent and the assessment method used,
did you do what is right? Justify your answer.
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural
3 Good You gave a response with a justification that is partially acceptable or not applicable Multi-structural
in the given situation.
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the Relational
given situation.
5 Excellent You gave a response with a very convincing justification that is acceptable in most Extended
situations and not only in the given situation. Abstract
December 2021
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
1 Poor You gave no response or showed no effort to respond. Pre-structural
2 Fair You gave a response but with no justification. Uni-structural

3 Good You gave a response with a justification that is partially acceptable or not applicable in the given situation. Multi-structural
4 Very Good You gave a response with a justification that is totally acceptable or applicable in the given situation. Relational
5 Excellent You gave a response with a very convincing justification that is acceptable in most situations and not only in Extended Abstract
the given situation.
Sample Answers Given

• Example 1:
• Example 2:
• Example 3: Dr. Marilyn Ubiña-Balagtas 87
December 2021
How do we score test items using the SOLO Framework?
Response Format).
Situation: You want to find out if your students can make a new material out of recyclable materials. You gave an essay test
requiring them to explain what to do out of old telephone directories. Considering the congruency between the intent and the
Meta-cognitive - Evaluate
1 Poor You gave no response or showed no effort to respond. Pre-structural 0 Point Not Proficient
2 Fair You gave a response but with no justification. Uni-structural 2 Points Somewhat Proficient
3 Good You gave a response with a justification that is Multi-structural 5 Points Nearly Proficient
partially acceptable or not applicable in the given
situation.
4 Very Good You gave a response with a justification that is totally Relational 8 Points Proficient
acceptable or applicable in the given situation.
5 Excellent You gave a response with a very convincing Extended 10 Points Highly Proficient
justification that is acceptable in most situations and Abstract
not only in the given situation.
88
December 2021
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.

December 2021 89
Situation: You want to find out if your students can make a new material out of
recyclable materials. So you gave an essay test requiring students to explain what to do • What is the answer
out of old telephone directories. Considering the congruency between the intent and to the item?
• What cognitive
the assessment method used, did you do what is right? Justify your answer.
process and
A. Yes, the use of essay test is just appropriate to what is being measured. knowledge
dimension does the
B. No, because it is difficult to check the students’ responses to the task using
item represent?
an essay test. • What levels of
C. No, it is more appropriate to observe them while they are creating understanding based
something new out of recyclable materials on the SOLO
Framework could the
D. Yes, asking the students to explain what to do with the material is already choices represent?
enough to gauge what they are able to do.
December 2021 90
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave
an essay test requiring students to explain what to do out of old telephone directories. Considering the
congruency between the intent and the assessment method used, did you do what is right? Justify your answer.
Meta-cognitive - Evaluate
A. Yes, the use of essay test is just appropriate to what is being measured.
B. No, because it is difficult to check the students’ responses to the task using an essay test.
C. No, it is more appropriate to observe them while they are creating something new out of recyclable
materials
D. Yes, asking the students to explain what to do with the material is already enough to gauge what
they are able to do.
December 2021 91
Situation: Meta-cognitive - Evaluate
Situation: You want to find out if your students can make a new material out of recyclable materials. So you gave an
essay test requiring students to explain what to do out of old telephone directories. Considering the congruency
between the intent and the assessment method used, did you do what is right? Justify your answer.
A. Yes, the use of essay test is just appropriate to what is being measured. Uni-structural
(1 point)
B. No, because it is difficult to check the students’ responses to the task using an essay Multi-structural
test. (2 points)
C. No, it is more appropriate to observe them while they are creating something new Relational (3 Points)
out of recyclable materials
D. Yes, asking the students to explain what to do with the material is already enough to Uni-structural
gauge what they are able to do. (1 Point)
Dr. Marilyn Ubiña-Balagtas//December 2021 92
Some Rules in Using SOLO for Test Development
1. Construction of Choices: Provide responses that the students will

likely give if the format of the test is constructed-response.
2. Scoring of the Responses: Score based on the degree of correctness
of the responses and level of cognitive demand. Make sure that there
is only one correct and best answer in every multiple-choice item
unless the item format is constructed-response.
3. Interpretation and Feedback to Responses: Explain sufficiently what
makes the response correct/incorrect so that students will learn from
the feedback provided. Indicate the proficiency/competence level of
the learner from the kind of response given.

93
December 2021
Let’s look at
examples of questioning
in PISA.

94
December 2021
Let’s look at a sample test in PISA Math.
Source: PISA Released

Items in Mathematics,
December 2006, OECD

Dr. Marilyn Ubiña-Balagtas Ans: D
95
December 2021
Let’s look at a sample test in PISA Math.
How do we apply SOLO in explaining
students’ responses to the item?
SOLO Score/
Choices
Level Code
A. 6 mg Unistructural 1
B. 12 mg Unistructural 1
C. 26 mg Unistructural 1
D. 32 mg Relational 3
Source: PISA Released Items in Mathematics, December 2006, OECD

96
December 2021

SOLO Score/ Explanation/

Choices
Level Code Feedback
A. 6 mg Unistructural 1 You gave one correct
B. 12 mg Unistructural 1 idea that is based on
the given information.
C. 20 mg Unistructural 1
D. 32 mg Relational 3 You interpreted
correctly the graph
that led you to
giving the right
answer to the
problem.
Source: PISA Released Items in Mathematics, December 2006, OECD

97
December 2021
Let’s look
at a
sample
test item
in PISA
Reading.

Items in Reading,
December 2006, OECD

Dr. Marilyn Ubiña-Balagtas Ans:
December 2021
98 A
Let’s look at a sample test item in PISA Reading.
SOLO Score/
Choices Level Code
A. About two meters R 3
B. About 15 meters U 1
C. About 50 meters P 0
D. It has disappeared U 1
completely.
E. The information is P 0
not provided.
Source: PISA Released Items in Reading, December 2006, OECD

December 2021
99 A

Choices Level Code Feedback
A. About two R 3 You interpreted
meters correctly the graph that
led you to giving the
right answer to the
problem.
B. About 15 U 1 You gave one correct

meters idea that is based on
the information given.
C. About 50 P 0 You gave a response
meters that has no basis at all.
D. It has U 1 You gave one correct

disappeared idea that is based on
completely. the information given.
Source: PISA Released Items in Reading, December 2006, OECD E. The information P 0 You gave a response
is not provided. that has no basis at all.
December 2021
100 A
Let’s look
at a
sample
test item
in PISA
Science.

Items in Science,
December 2006, OECD

December 2021
101 A
Let’s look at a sample test item
in PISA Science. How do we apply SOLO in explaining
SOLO Score/
Choices Level Code
A. The Earth rotates on

R 3
its axis.
B. The Sun rotates on
its axis. U 1
C. The Earth’s axis is

tilted. M 2
D. The Earth revolves

around the Sun M 2
Source: PISA Released Items in
Science, December 2006, OECD

December 2021
102 A

Choices Level
Code Feedback
A. The Earth R 3 You were able to give the right

explanation why daylight and
rotates on its darkness occur on Earth.
axis.
B. The Sun rotates U 1 You know that the sun rotates

on its axis but this does not
on its axis. explain day and night on Earth.
C. The Earth’s axis M 2 You know that the Earth’s axis

is tilted which was explained in
is tilted. the information provided but it
does not explain the
occurrence of day and night on
Earth.
Source: PISA Released Items in The Earth You know that the Earth
Science, December 2006, OECD
D. M 2 revolves around the Sun which
revolves around is the source of light but it
Use of the SOLO Framework in Test Development in the New Normal the Sun does not fully explain the
occurrence of day andAns:night as
103 A
that is due to Earth’s rotation
December 2021 on its axis.
SESSION TARGET
Topics to Discuss
Development

104
December 2021
Inst’l Time Level of Behavior, Format, No.,& Placement of Items and
Content Competencies (hrs) # of Items % of items the Knowledge Dimension
R U Ap An E C
1.Method of Examine the 3 10 20% I. 1 I. 2 I. 1 1. 3 I. 2 II. 39-
Assessment good and bad
practices in #1 (F) #6-7 #15-16 #17-19 #36-38 40
assessing (C) (M) (M) (M) (M)
student’s
learning
Etc.
Total No. of Items 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum 4 pts - maximum 5 pts - maximum
for each item for each item for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
Total Number 190 points 45 Points 120 Points 25 Points
of Points
Balagtas’ Model for Test Development with RBT and SOLO Combined
December 2021 105
Level of Behavior, Item Format, No.,& Placement
# of Items
Content Inst’l Time % of items KD R U Ap An E C
F I. 1
#1
3 10 20% C I .2 I. 1
1.Methods of #6-7 #16
Assessment P II. 2
I. 3 #17
to 19 #39-40
M I. 2
#36-37
2. etc.
Total 15 50 100% 5 10 20 10 3 2
Scoring 3 pts - maximum 4 pts - maximum 5 pts - maximum
for each item for each item for each item
SOLO Codes P-0; U-1; M-2; R-3 P-0; U-1; M-2; R-4 P-0; U-1; M-3; R-5
Total Number of Points 190 points 45 Points 120 Points 25 Points
106
December 2021
Prepare a Grade Level Achievement Test for the Division
Things to Do
1. Prepare a Table of Specifications (ToS) for an
Achievement Test of a grade level. The number
of subject areas taught in the grade level will be
the same number of ToS to be prepared.
2. Use the suggested template in the preparation
of the ToS of an Achievement Test for a grade.
Dr. Marilyn Balagtas' Model TOS Template for Test Development with RBT and
SOLO Combined
https://tinyurl.com/pmz4hzyj
3. Exchange ToS for the validation guided by a

rubric.
107
December 2021
Suggested Template for the Table of Specifications

Post-Test
https://forms.office.com/r/bNmHPNGhm3

109
December 2021
What is your take-away
from the session?

110
December 2021
“You cannot manage
what you cannot measure”
Peter Drucker
“If you cannot measure,

you can’t improve”
Gray Mac Kenzie

December 2021
No one is perfect and so is
an assessment tool.
Reduce its imperfection
through constant revision.
Marilyn U. Balagtas
December 2021

December 2021
About the Speaker
DR. MARILYN UBIÑA-BALAGTAS is a University Professor at the
Philippine Normal University teaching courses on educational assessment,
evaluation, and research from undergraduate to doctoral level in face-to-face and
online modality. She has been a Dean of PNU College of Flexible Learning and
ePNU. She finished her Doctor of Philosophy with specialization in Educational
Research and Evaluation at the University of the Philippines. She has been a regional
finalist in Metrobank Search for Outstanding Teacher in the Tertiary Level and has
been awarded a post-doctoral fellowship for Leadership in Research by the University
of New England, Australia for her projects as the Inaugural Director of the
Philippine National Research Center for Teacher Quality (RCTQ) that has led the
development of the Philippine Professional Standards for Teachers (PPST). She is
the founding President of the PATEF-United Professionals for the Development and
Advancement of Teacher Education (UPDATE) and currently the President of the
Philippine Educational Measurement and Evaluation Association (PEMEA), Inc.
She is a consultant in educational assessment of a testing company in the Philippines.
She is a researcher, trainor, and a writer in areas of assessment and teacher education.

December 2021 113
Thank you.
When interested to use this material,
please cite as:
Balagtas, M. U. (2021). Use of the SOLO
Framework in Test Development in the
New Normal. Philippine Normal
University, Manila
MARILYN UBIÑA-BALAGTAS, Ph.D.
University Professor
College of Graduate Studies and Teacher Education Research
Philippine Normal University
President, Philippine Educational Measurement and Evaluation
Association. Inc. (PEMEA)
[email protected]

December 2021
How to construct written tests?
(For more details, please refer to this book.)

December 2021 115
For details on performance-based and portfolio
assessments and grading, please refer to this book.

December 2021 116
For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:

December 2021 117
For inquiries about Assessment in Learning 1 and 2 Textbooks,
please get in touch with given contact details below:

December 2021 118

12-6&9-21 (Handout) Final-MUBalagtas-DepEd Bulacan-SOLO

Uploaded by

Copyright:

Available Formats

12-6&9-21 (Handout) Final-MUBalagtas-DepEd Bulacan-SOLO

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

12-6&9-21 (Handout) Final-MUBalagtas-DepEd Bulacan-SOLO

Uploaded by

Copyright:

Available Formats

What is the SOLO framework?

What is the SOLO framework?

How can the SOLO framework be used in test construction?

How can the SOLO framework be used in test construction?

Use of the SOLO Framework

in Test Development in the

MARILYN UBIÑA-BALAGTAS, Ph.D.

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

v It is a framework for understanding with five levels,

Use of the SOLO Framework in Test Development in the New Normal

Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631

Use of the SOLO Framework in Test Development in the New Normal

Source of Illustration: Doug Belshaw - Own work, CC0, https://commons.wikimedia.org/w/index.php?curid=60807631

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

v Use SOLO in structuring responses to questions in a

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

As of June 30, 2021

Use of the SOLO Framework in Test Development in the New Normal

Student 2 USA Student 4

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

A. 10.37% of the population of USA got

B. 0.1037% of the population of USA got

C. 0.1037 of the population of USA got

D. USA has 9.1% more COVID cases than

Use of the SOLO Framework in Test Development in the New Normal

10.37% of the population of Relational

USA has 9.1% more COVID Multistructural

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Conceptual 2 64. Maria bought

Use of the SOLO Framework in Test Development in the New Normal

Dimensions Remember Understand Apply Analyze Evaluate Create

Dimensions Remember Understand Apply Analyze Evaluate Create

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal

Use of the SOLO Framework in Test Development in the New Normal