Nancy Arnold, Washington Ron Cammaert, Riverside Publishing Dan Wiener, Massachusetts Ed Roeber, Measured Progress Rachel Quenemoen, NCEO

What the heck does proficiency
mean for students with significant

cognitive disabilities?
Nancy Arnold, Washington

Ron Cammaert, Riverside Publishing
Dan Wiener, Massachusetts
NCEO
Ed Roeber, Measured Progress

Rachel Quenemoen, NCEO
National Center on Educational Outcomes

“…would allow States to use a
documented and validated standards-
setting process to define academic
achievement standards for students
with the most significant cognitive
disabilities, as defined in proposed Sec.
200.1(d)(2), who take an alternate
NCEO
assessment.”
March 20, 2003 NPRM for Title I

Many states have completed a careful
standards-setting process for their
alternate assessment, just as they have
done for their general assessments –
these efforts are needed to produce
alternate achievement standards
Examples shown in NCEO reports

(see Synthesis Reports 42, 47, 48,
NCEO
plus one more on its way)

“These [alternate achievement]
standards must be aligned with the
State's academic content standards
and reflect professional judgment of
the highest learning standards
possible for those students.”
NCEO
March 20, 2003 NPRM for Title I

WHO are the students who participate
NCEO
in alternate assessment aligned to

alternate achievement standards?
What do we measure? What
does good learning look like for
this small group of children?
What is “achievement?”
NCEO

A brief history of educational
goals for these students
• Developmental approaches
• Functional approaches
• Academic approaches – access to the general

curriculum, standards-based content, grade level
NCEO
contexts
* Diane Browder, 2001

Development of Alternate Assessments
Quenemoen, Rigney, & Thurlow, 2002
1. Careful stakeholder and policymaker development
of desired student outcomes for the population, reflecting
the best understanding of research and practice,
thoughtfully aligned to same content expected for all
students.
• 2. Careful development, testing, and refinement of
assessment methods.
• 3. Scoring of evidence of content aligned student work,
according to professionally accepted standards, against
criteria that reflect best understanding from research
and practice.
•
NCEO
4. Standard-setting process to allow use of results in

reporting and accountability systems.
• 5. Continuous improvement of the assessment process.

Alternate
Assessments are
works in progress
Alignment to content standards varies:
• Reading and math skills in context of grade level
curriculum contexts
• Reading and math skills in functional contexts
• Reading and math skills in isolation
NCEO
• Weak linkages to reading and math

All of these exist within current state approaches
Alternate Assessment Strategies
Thompson & Thurlow, 2001
IEP Analysis 3
Performance event 4
Combination 8
Checklist 9
NCEO
Evidence/Portfolio 24
0 5 10 15 20 25
Number of States
[2 states undecided]
Case Studies: Common Criteria
Quenemoen, Thompson, & Thurlow, 2003
• 1. Content Standards Linkage.

• 2. Independence.
• 3. Generalization.
• 4. Appropriateness.
• 5. IEP Linkage.
• 6. Performance. level of skill or mastery and
multiple settings; progress and
appropriateness; accuracy, mastery,
NCEO
progress, independence, multiple settings,

multiple occasions, or multiple purpose.

Validity of Alternate Assessments*
Face Validity – Are scoring procedures consistent with
important best practice indicators in the lives of
students with significant disabilities?
Concurrent Validity – Do scores correlate with other

measures of students achievement and indices
of quality programming at the school level?
Predictive Validity – How well do scores predict

NCEO
post-school success?
*Concepts developed by Harold Kleinert, U of KY

Development of Alternate Assessments
Quenemoen, Rigney, & Thurlow, 2002
1. Careful stakeholder and policymaker development
of desired student outcomes for the population, reflecting
the best understanding of research and practice,
thoughtfully aligned to same content expected for all
students.
• 2. Careful development, testing, and refinement of
assessment methods.
• 3. Scoring of evidence of content aligned student work,
according to professionally accepted standards, against
criteria that reflect best understanding from research
and practice.
•
NCEO
4. Standard-setting process to allow use of results in

reporting and accountability systems.
• 5. Continuous improvement of the assessment process.

Copies of the papers cited
and presented are at:
http://education.umn.edu/nceo
or Search for NCEO
Alternate Assessment Topic,
NCEO
Resources
What (the Heck) Does
“Proficient” Mean?
Standard Setting on the
WAAS
Nancy Arnold
Washington Department of Public Instruction
NCEO
CCSSO - San Antonio, TX

June 2003

Steps Taken to Determine Standard
Setting Process
• Review of NCEO Synthesis Reports
• Detailed Study of Standard Setting for Alternate

Assessment in Other States
• Review of Relevant Literature

– Standards for Educational and Psychological Testing
(AERA, APA, NCME)
– Setting Performance Standards: Concepts, Methods,
NCEO
and Perspectives (Cizek)
• Validation of Process with National Technical Advisory

Committee and Advisory Panel
Standard Setting Methodology
• Selection of panelists
– Teachers, parents and administrators
– Stratified sample
– Portfolio experts and novices
• Selection of standard setting materials

– Performance descriptors
– Scoring patterns
– Exemplar portfolios
NCEO
• Orientation and training of panelists

Standard Setting Methodology
(continued)
• Determining alternate achievement
performance descriptors
• Set cut scores in three rounds

– using scoring patterns
– revise cut scores using exemplar portfolios
– finalize cut scores using impact data
NCEO
• Evaluate standard setting process


Nancy Arnold, Washington Ron Cammaert, Riverside Publishing Dan Wiener, Massachusetts Ed Roeber, Measured Progress Rachel Quenemoen, NCEO

Uploaded by

Copyright:

Available Formats

Nancy Arnold, Washington Ron Cammaert, Riverside Publishing Dan Wiener, Massachusetts Ed Roeber, Measured Progress Rachel Quenemoen, NCEO

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nancy Arnold, Washington Ron Cammaert, Riverside Publishing Dan Wiener, Massachusetts Ed Roeber, Measured Progress Rachel Quenemoen, NCEO

Uploaded by

Copyright:

Available Formats

What the heck does proficiency

mean for students with significant

Nancy Arnold, Washington

Ed Roeber, Measured Progress

National Center on Educational Outcomes

National Center on Educational Outcomes

Examples shown in NCEO reports

plus one more on its way)

National Center on Educational Outcomes

March 20, 2003 NPRM for Title I

National Center on Educational Outcomes

in alternate assessment aligned to

National Center on Educational Outcomes

• Academic approaches – access to the general

National Center on Educational Outcomes

4. Standard-setting process to allow use of results in

National Center on Educational Outcomes

• Weak linkages to reading and math

• 1. Content Standards Linkage.

progress, independence, multiple settings,

National Center on Educational Outcomes

Concurrent Validity – Do scores correlate with other

Predictive Validity – How well do scores predict

National Center on Educational Outcomes

4. Standard-setting process to allow use of results in

National Center on Educational Outcomes

CCSSO - San Antonio, TX

National Center on Educational Outcomes

• Detailed Study of Standard Setting for Alternate

• Review of Relevant Literature

and Perspectives (Cizek)

• Validation of Process with National Technical Advisory

• Selection of standard setting materials

• Orientation and training of panelists

• Set cut scores in three rounds

• Evaluate standard setting process

You might also like