Criteria of Measurement Quality
Criteria of Measurement Quality
Criteria of Measurement Quality
• We began with assertion that social scientists can measure anything that
exists.
• In the context of a survey, we would expect that the question “Did you
attend religious services last week?” would have higher reliability than the
question “About how many times have you attended religious services in
your life?” This is not to be confused with validity.
RELIABILITY
• Reliability is when a particular technique, applied repeatedly to the Same object,
yields the same result each time.
• E.g.. Research on health hazard appraisal (HHA) the risks associated with
various background and lifestyle factors, making it possible for physicians to
counsel their patients appropriately.
• An ovary present in the first study was missing in the second study
• One subject was 55 years old in the first study and 50 years old three months later
• Test-retest revealed that this data-collection method was not especially reliable.
• If the two sets of items classify people differently, you most likely have a
problem of reliability in your measure of the variable.
USING ESTABLISHED MEASURES
• To ensure reliability in getting information, we can use measures that have
proved their reliability in previous research
• The heavy use of measures, though, does not guarantee their reliability.
• For example, the Scholastic Assessment Tests (SATs) and the Minnesota Multiphasic
Personality Inventory (MMPI) have been accepted as estb. standards in their
respective domains for decades. In recent years, though, they’ve needed fundamental
overhauling to reflect changes in society, eliminating outdated topics and gender bias
in wording.
RELIABILITY OF RESEARCH
WORKERS
• Measurement unreliability may also be generated by research workers,
interviewers and coders. There are ways to check on reliability in such cases.
• Even total reliability, Doesn’t ensure that our measures actually measure, What
we think they measure.
validity
• A term describing a measure that accurately reflects the concept it is
intended to measure.
• Validity means that we are actually measuring what we say we are
measuring.
• A measure of social class should measure social class, not political orientations.
• Your IQ would seem a more valid measure of your intelligence than the number of
hours you spend in the library.
• We may agree to its relative validity on the basis of
• Face validity
• Criterion-related validity
• Construct validity
• Content validity
Validity
14
criterion-related validity
• Tests of construct validity, then, can offer a weight of evidence that your
measure either does or doesn’t tap the quality you want it to measure,
without providing definitive proof.
17
Tension between Reliability and Validity
• A tension often arises between the criteria of reliability and validity, forcing a
trade-off between the two.
• Two quite different approaches to social research: quantitative, nomothetic,
structured techniques and qualitative, idiographic methods such as field
research and historical studies on the other. The former methods tend to be
more reliable, the latter more valid.
• If no clear agreement on how to measure a concept, measure it several
different ways.
• If the concept has several dimensions, measure them all.
• Measure concepts in ways that help us understand utility and the world
around us.
The Ethics of Measurement