An Exploratory Investigation of Rasch Model Residuals

4 , DOCUMENT RESPEE . ED 206 652 . 21 410,542 . . AUTHOR TITLE' 'e", PUB DAT ItOTE I , Ludlow, Larry H. An Exploratory Investigation of Raschtodel k.. Residuals. - Apr 81 " 37p.; Paper presented at the Annual Meeting of the ,Amhrican Edgcational Research Association (65th, Los Angeles, CA, April 13-17, 1981). t EDRS PRICK DESCRIPTORS IDENTIFIERS EF01/PCO2 Plus Pdstage. , .*Goodness of ,Fit:, *Latent Trait Theory; *Models; Statistical indlysis *Data Interpretation; Rasch Eodel; *Residuals . (Statistics) .1 , . . 'ABSTRACT Residual patterns should, be studied in order to. understand when and why data deviate from a model. This paper illustrates some techaques fot.explbring residual patterns resulting from the difference betveem observed and expected scores as predicted by a Rasch model for polychotdmous data. The` score residual divided. by its" "stand d deviation is"cilled a standardized residual. A varietfof p ots of residuals are presented to illustrate residual pattern inte retatioh. A systematic analysis of.residuals can offer the investiga or a decision, facilitation, technique, not found in conventional ummarr fit statistics. (BW) -.. 4,4 4 .4r ti 4 % 10- A F *********************************************************************** ReprodUctions supplied EDRS are the best that can be made .* A from the original: document. 41*********************!*************e***************It*****************14 .US. WARTA'S/CT OF EDUCAT101t NATIONAL INSTITUTE OF EDUCATION EDUCATIONAL RESOURCES INTOMAAOON CENTER OR/CI XTes decree Ms Dom npooducoO m mond ken Ni oral a lir' meamon ortmtog rt O Miner Wore hem Me mom to nprow NerotecUon ;NMI) Poona. nee or opeNes soled n the downed Do NN Afton*/ morimm officel ME "mom a Poky I. AN EXPLORATORY INVESTIGATION OF RASOH MODEL RESIDUALS PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY L Larry H. Ludlow 1,-4,41.aw. 'MESA Department of Education University of Chicago TO THE EDUCATIONAL RESOURCES INFORMATION CENTER (MCI" % and _Rehabilitative Engineering Research_ and Development Center Hines Veterans Administration Hines, Illinois ...... et . 6. % . . . Paper presented at the American Educational. Research Association Annual Convention, Los Angeles, Cal., April 12-17, 1981 ( I 1 A 1.% g . tai ,/ Pge 1 _Models are convenient expregsions of how we think things ought When we Use models to understand experience, however, it to be.. is ine vitable that less than a perlect'explaffation results.' This need not mean. the end.of the model.' By '$tu,ing the residuals *, from our expectations we can learn from, and come to, a better understanding of our experiences. The importance of models.And leading' to better togain insptcting residuals better understood data has clearly deboytrated in (pole analysis of variance among others, Anscombe and Tukey . Likewise, (1966). on the staggs(Joreskog,1979). \tf. deviate from 'residuals -in of a model. (Mead, the For example, when But these the best 'why data th4'Rasch model is and for people statipti,c; are across items to i snce these statis- to sample size andvit.est length there is consj troversy regarding the magnitude-to be regarded as misfi ting s Although these summary statistics have,demonstrated their use. fulngss,-the,analysis of model, Ot must be carried further, I and Smith fi 01, V S compute fit, often' inadequate . as Draper k however, when and analyses,may locate the source of model departure. tics.are sensitive testing " items across people 1975). model at the investigatibn order to understand dFchotomous data statistics foi and Smith The .psychometric literature, . applied to literature by, and Draper as comprehensive an effort patterns in residual been the analysis of covariance structures depends 'interpretion does not reveal (1963) information airge the display.of residual patterns Just 4n 4 0 Page 2 4 . addition to , ../ applied in summary statistics' so may the same' approach:be . ,. I our work. This paper illustrates some (techniques . , . . . ' . . c found useful for exploring residual patterns xesultirig from the , difference between observed and expected scow as predicted by a Rasch model for polychotomous data developed byGeoff °Masters (Masters, (980) Masters and Wright, 101). * s We can derive three alternative forms with which to compute a residual from the model. The observed minus expected score yields N ' 116 what we call a snore residual, variance is called a logit The score residual divided by its The score residual divided residual. 4 by its standard deviation Fs called a standardized residual. Our choice Of the residuhl to 'inspect was based on hmijell it piovided useful information. The score residual was rejected because floor and ceiling effects restricted the variation in the extremes. This led to our formulation which proved to be sensitiie to perhaps too much so. paring logit and ,tables and logit Variations in'the extrem but The same pattern-s could be found when com- , standardized. pictures in of the results ,but the order to handle adjustment the magnitude to of logit . residuals proved to hem`wrihgonyenience. resi'dualp manifest 'a familiar Since the 'standardized, interprelatisp metric, 4 of their , patterns becarde the easiest of the choices. I In Sloane's (1981) I discbssAllin the'sumnary.approaoh to residu- als Suggestecfsome misfititin4 Posjtive-fits'result ghen \ , . d , r I. ,p. ... e .. Page q_ . .. . more able children score ldwer than expected. (yielding negative / residuals) and less able children score higher than. expected '., e. 4 . . . . residuals). (yielding positive Negative fits result . .- when alke N. . . . . . chilften do better . than expected (positive residuals) geld " ,.. . less , . able children do worse .than expected (negative residuals). But . these summary statistics 'do not tell us exactly who has 1,scored. t , . ! unusually on which items, nor how widespread the problem is. . . ". ( 4 r , The first picture created to study this was a . plot oY stated:- . . ardized residuals against.child abilities (Figure 1). contains the Original sample of 500 children. This plot 40. From*left to right N. the abilities increase. . . ,. . Xesiepresent 10 or more 0141dren witt a . .' . given ability of residuals spread of points shows the. . , . . ... Each coil= and residual, for the children with that ability: For 1 instance,, the leftmoit column shows the The picture we least able child. model is a random pattern with a 14 residuals expect for data for the . fitting the mein near zero and a standard deviation near one for each vertical array of residuals. . .. _ , . 1 .-What we see in.Fi9ure 1 indicates soflething else. . A rectangle . marking off plus end minus 3 standard deviations was added to the picture to highlight the asymmetry of the distribution in the 4th The quadrant. points outside the , . large negatfve residuals. . area are able_ children_ with . It- is recognized that very Able chil- . . dren will appear unusual only when they miss . . f , when they ducceed on . . relatively hard items, 1 f . hence , / only appear unusual easy items, Likewise, the least able children will :,large negative re/siduals. ' , ., , 1 . . a V . . a" Page 4 1021dieg large positve residuals. of finding, out'who he able The task, then, 1 becomes one which easy'items,were children are, 'missed, and whether a valid explanation, other than one of chance . . . . ccurence, can be suggested for the large negative residuals. . One way to see what item residual matrix is happening is and to scan,it,for to print.the patterns. person by This matrix, can be constructed according to various sorting schemes depending upon what aspect the billestigator wishes to h'ghlight. It may 4 . ,,,aaso be built . according to group membership. indicate unusual behaviors S eh individuals or,grouDs of had on single items or blocks of items. a matrix may people have Exayiples of this type of 'matrix can be'found in Wright and Stone (1979). Figure 2 shows-such a matrix. In this example the columns ace items sorted by their difficulty.and the rows are children sorted by their ability.' Both sorts are descending. may be collected in the margins. . Various summaries The entries in the matrix are A standardize0 truncated residuals. Any residual with an absolute value equal to Or greater than 9 was set equal to 9. "i statistics, ho wever, residuals. data are were computed from real.valued Standardiied In these matrices; a criterion suffic iently focused whpn are brought out. This allows for the to greet); The summary must, be set so that person by item interactions matrix tOrbe simple enough This criterion was set.sUch'that only negative residu- als equal ;o or less than--3.0 and only children who had at least , - one suoh redidmal ate displayed. This matrix concentrates on \ 11 I . 'Page 5 surprising mistakes. In the SUBJ. ID field in ,the first 2 columns are the age of column is the child's sex. left margin of the matrix, the the child in months and the third '12" for girls). (. "1. ", for boys', The t' Plight most margin lathe child's logit ability. Inspection of the ID field suggests that boys and 'girls Are spread equally throughout -the ability range. to be some age differentiation, however. .4 . There does appear The older children gen0 erally have the greater abilities Aile the younger children are .primarily .in the lower ability range. The fact that older child dren perform better "bn a developpenial instrument is hardly sur: f prising. Of interest to us is whether the failures4Ln relatively e sy items can be explained by a characteristic ofthe item that 1 q,.to a related group o ,Fbildren haying had uqeijrcted trouble with it. r - In order to Aimpli y this what appeared direction. - to be 'an age . demonstration rte chose fo emphasiie factor operating in,an un expected 1 First controlled for sex by looking-only at boys and then selected jhe 56 oldest and.42._youngest ones. . -" .1 , Figu re 3 is the plot of the 98 boy abilities by theirres%du. Ills. This plot corresponds with that of,Figure 1. though, tire 3, At. this point, 'we do /hot know the age 'of the most ableschildren in Fig- nor what,items are involvAd, nor if there'is even a dif- Page 6 . ference in the abilities distributed among the two age groups. Figure 4 presenti 98 boys. In the the residual matrix for the ID field the third column is the age is less than 47 months, are the age and the first 2 columns grqup'classification.("l" if the age age is "2" if ,the The criterion was set months). reduced, set of greater than 59 for negative residuals 'ecival to 7 or less than-2.0. This matrix highlights those more able chit -. 4 dren whohave done worse than expected. This concludis our discussion of . search techniqu es for under-_ \ . standing the summary statistics used by Sloane. ',These results do . not completely explain her misfitting items becaLse the gilp are %. . . not included as part of,our older and younger boy dichotomy. . . . point shyld be cle4r, however, that the misfit. in Sloane's ori. . ginal soXution can Inspection of the The . be explored / through a residual matrices is one . analysis. residual way of understanding 1 why an item has misfit the model. . 1 No4 we look a.t '4he subisareple of '98 boys to see if the varie- d' ble definition has remained the same-for the two age groups. -intent' is to. demonstrate a process that may the question is asked "Have these Our be applied_ whenever groups of people performeA th same on the instrument?". Figure 5'is an ability frequency distribution chap. The youn , est boys are on the upper map, the older boys on the lower. 4 8 , The r le p .. J AJ A t 4 . numbers indicate the . 4 Page 7 ' number of people located at . the same pos.- - Double digit numbers are read vert,icalli,.e.g..in the sec: tiipn. , ond map , the 1 above the,8 refers tos 18 people with an ability near 3.Q. 'The M and S refer to the respective means and standard deviations. Figure'S shows how much more able the than the younger. Figures 6 and 7 show_ the ability plots for the younger and older boys, no negative abilities for the older t ive side of th'e plot from'Figure 7. confirms that the er boys are residual respectively. ,,Theretmere boys so we removed the negaA compdripon with Figure 3 large negative residuals are due.primarily td the older boys,. We npw turn to the:investigation of the items . Figure 8 is d plot of reWuals again.st item difficulties. . The items range from easiest at the left to hardest at"the right: 7 The tectangle marks off plus and minus 3 standard deviations. .111. The 3rd quadrant shows thdt the easier items have the large newt- tiv residuals. This-picture confims'what we learned from Figures 1 .1* 3 and 4. The information in the sorted matrices of Figure's 2-and 4 is _..," _convenient when the sample in, the matrix dpes.not exceed 100. that point three pagesare yequired.to present the picture. . motivates us to seek another way :of At / . This Alb presenting the- residual / . information for large samples or when group c6m0BriSpn'are to be . , , made. . A useful way to compare tributions over 1 . , . groups in terms of.resfdual.dis- individualitems is shown in Figures 9 and 10. In Figure 9, for ITEM8, we see the residual .dastribution for each I tt. Page;8 I group of boys on resi'dudl units. a line extending from -5 to +5 standardized The inteders on the line represent the number of boys who had,a given residual value. Again the numbers are read vertically and theM and S represent the group means and standard, deviations. 'of thes .residuals. Thistypeof ma 'allows a detailed examination between .group perf rmances on individual combine items that people Also, , we can re similar and build these maps fqr groups of. across item behavior Items.. of within and Not pnly has occur ed but we can we find where can identify surprising which children and involed. _ , . A useful summary f all'the,item maps is contained in Table 1. ti The Table contains he group means and standard each itdm's residualr This table may be used in i deviations for a variety-of , . ways. 'We could take the difference between group means and look , 1 at the items with the largest differe;cesr We could look at each i item where th'ere was a difference in the sign of the mean. . this example; Por - we chue:tO cqncentrate on the 20 column. of means and standartikdeviations, in particular, the negtive means. When data fit the model, c residuals are ability and difficulty free and differences in summary statistics should be attributable 401' to chance. We have seen in these data, however, a combination of . high ability . . re$iduals. e, and low' ditficulty that qince'the olddr boys'are results in high negative g enerally the more able we generally . . : 1 . p. 0 P. 4 ,/ , at . . . expect them to have higher . means and emaller.standard devriatiens / . Y ein'the Page 9 p , . residuals relative to the . younger .. ones. . , . . I The negative V means for the older boys under column.2 represent instancleswhere . ,..... $ the mean performance by (the older boys was vorse,in terms of K. expected behavior than Se 'this instrument surprisinrio see that expected . from the vri designed to measere some of the more able, congRderably worse than expected. . .. .. . younger ones. development it is older boys ISerforming J This is not to say their per- ,forrianile was bad- just that it was not as good as was expected. We need?co knpw explained tin chance if the surprising residual means can be . or whether something more fundamental is at stake. 'From inspecting, the maps for ITEM an FEM14, in Figure 9, is evident that rt outliers (indicated by ,asterisks) 'Skewed the means of the older boys. These children may aiso b found in Figure 4 under the two items in question. the probrem down items are to two outliers oh each item, functioning as intended but furtlier if we cared to because dre n-who had Arising have. After tracing we conclude the we could take bhe matter we have identified.specific chil- trpuble. p Looking at -the re idual diitributions ITEMT1, in Pigur-1(),, we see that in each case distribution is bimodal with a gisoup dard deviation below the mean. ITEM? and the older boy of children about one' sten. . not surprising. for,ITeM6, . Those with/positive residuals are Those with the negative' residuals suggest a pa(- tern of deficiency or erroneous observation and we may ask if -the S ,,\N 43% t negative residuals are from larger, the same children. From a matrix of residuals wAth a criterion set at -1 we found that, facts ,dome of the same children tions of two middle lead large residuals on combina- of the three items range of Apparently the problem the same older boys. and these children were distribution for the ability in with ITEM6 and ITEM7 in the. the older boys. it due to a few of Since the median for these items is consid. erably to the right of the respective. means / items are functioning as intended. we suggest these It would be reasonable, how- F ever, to monitor these items in future applications. Such an interpretation is not possible, howev.er, for ITEM11 in Figure 10. tion is Here the median approaches the mean but the:distribu- still bimodal. Who made the mistakes on this Interestingly, it was the most able of the olds -r boys. item? '7e deter- . mine this by looking again at Figure 4 and, noticing the largest residuals beloftg to boys in the,highest ability leve,l. "Xhe prob- lem with ITEM11 may be due to the older boys who did Oot confusion on.the part know to who of some Of in front he of instruction. was dreected. - Where doe's this leave us? 14e.cotild be aAkea why we did not . items_ppparitely calibrate the for the two 'groups of plot the item diffiCulties against boys. and one another with error bands. ,.1. This was done in Figure 11...to undersc re why it is not necessary and can be misleading. therpOduals loting One x,reason for 1 . from a total . -., sample calibration.was to show that the tame group . yx . . . rr II 1 . 6 Page 11 . . . 4 . . differences(are, seen as when separate calibrations are done.. The, -. . .. . . first thing ig to check, i n-Figure 11 ; is wheiher the same items thSt . the residual analysis tagged aspeculiar also stand out in,the, ' . plot. We .see they, do in' fact eitipei. lie oUtside...d.r close to the . . . 3 standard error bands' oortoboration. .important pOini, however, is not ttk If'we saw, only this plot we might conclude there . ' , was a veiable . :. definition problem in thiseidata, items meant, one thing. to theolder one where the boys and another, to the youni',.. I ge r. But this is 40 . .not the- case as we,have, seen by exp.loring and , ,. 4 - , xplail]dng.the'residual patterns. .certain items are indicates That 'why.' ,, . if oneis interested in compaits of item' v , difficulties, 7r. 4 test the difference between I& & 4 peEuliar but do:11001frot explain The same argument holds true putingt,-statistcs to 4. s The. plq in.Figure 11 merely The residual analysis not only identifiei the.same. * 0 judicious investigation why there was items Kit nay explain' with a pecOliariOF and shows that, except for ITE ' 1 it is not neces- ''.' sarilf a problem, of 'Ibem conttruction., In conclusion, .....,, we argue that a systematic analysis of residu- als Offers the investigator a decision facilitation technique not found ... P . . -. . may be used / to understand individual people, - # k.. groups of people The pr.pcess . . . , , summary fit satstics. .il the, c'pnventional ,4 or groups of items. Such individual items, an understanding of . . residual pptterns may prove useful as . a means of addressing , ssues of 'item bias'ow'quessinge, and "discrimination'. _ ti , N" - .i-1 -.= . .4-, Pr, a s ,13 / 4 DS Page `12 . . .; References ., , Adscombe,FJ. and Tukey,S.W. 'The examination and analysis of I residualp," Tichnolme,trics,5,341-1613 (1963). ' $ . Drape N..R. and Sinjth,H. Applied Regression Analysis. John Wiley . n onF,New:Yark,, 1966. ' . . Joreskog,K.G. and Sorbom,D. Advances in Factor Analysks.and b r Structural Equation Methods. Abt Books, Cambridge, Mash. 1979. Masters,G.H. A Rasch model for rating scales!' Unpublished doctoral II dissertation, University' of Chicago, 1980. . Maste'rp,G.H. and yright,BD Rating Scale Adtlysis. Chicago:MEgA . Press, 1981.. , _ - . . . Mead,R.J. Analysis of.fit'to thslla sch model. doctoral: . . 1 dissertation, University of Chicago; 1975. APINK Sloane,K.D.."An application of tie partial credit model," Paper, presented at the American Educational Research Association 4, Annual Convention, Los Angeles', Ca., Aril, 1981. Wri9ht,B.D. and Stone,M.H. Best Test Design. bhicago:MESA Press, f J..- Oh .. 1979:- -- r .1 , A 14 r- .; . to A 0. ft 1 P 0.. .. r 4 ... ".. ,...- a , . . .., . . a IR I AN EXPLORA.TOR* INVESTIGATION .OF RASCH MODEL RESIDUALS 4 ... _ ... . 4.. .. 1 LARK 1 \ / H. LUDLOW MESA DEPARTMENT OF EDUCATI9N UNIVERSITY OF CHICAGO 4 '4, AND . REHABILITATIVE ENGINEERING RESEARCH AND DEVELOPMENT CENTER HINES VETERANS ADMINISTRATION HINES, ILLINOIS . * 4 . . _ I . , .. L. . PAPER ohtsENTED AT THE AMERICAN EDUCATIONAL RESEARCH ASSOCIATION ANNUAL CONVENTION. LOS ANGELES. CAL.. APRIL 1217. MI (TABLES AND FIGURES) ft / . z s PM i I r e. ad t $4 .4 t # .3. DIAL: FINE WITORCONCEPI : ALL TODDLERS INCLUDED: STD RESIDUALS PLOT OF ABILITIES VS RESIDUALS' 4 . A UNEXPECTED SUCCESS I. THE RECTANGLE IS CENTERED ON THE MEAN ABILITY FOR THE 500 CHILDREN THE MEAN (M). STANDARD DEVIATION (5). TEST LENGTH (L) ANR SAMPLE SIZE (N) ARE GIVEN BELOW THE PLOT 2 1 21 11 1 11 1 1 1 4531143 -14. 113133X5 7 2248353XX8X XX 811237XXX4XXX X X 2 4 33X561XXXXXXX )(XX X 112 918XXXXXXXXXX.XXX X X 1 1 5 X97X8XXXXXX XXX X X 14-112-428XXXXXX6-XX-XkX-X -X 349 72XXXXXXXXXXX XX '4 1 4. I r 5 3 32182X42XXXXXXXX X X 332 5t2XX153XXXXX XX 113 21244X585XXXX XX 1. 2 3212722 XX3 X4X 21,221 4642 6X 9X 1 1 332331333 8 X56 1 1111 26X242 961 1 111 23 16 86 1 2 3 1113113 12 6 235 Z 3 $ 112 1 1 X X X ) X 6 9 7 I 8 x, 1 1 3 9 2 2 1 4 1 1 3 10 1 1 2 1 UNEXPECTED MISTAKES 1 1 4 44 . $401.94 HIGHER Nu 500 Sm1.26 LUDLOW.8MESA PSYCHOMETRIC LAGORATO Y. UNIVERSITY OF CHICAGO FIGURE 1 . . 7,00 ABILITY LOWER 40 ry -0 .00 ,-.7.00 ..- 11 4, 61'11 4911 . 4 1 2 tip* 5 7, . 4 4 4 3 5011 5711 4 5411 4421 4421 4211 8 3 3 t 571t 5513 ...21 . 3 2e 4 4 ". , 4811 3 6121 5511 5 . -02 0,1 3 3 0.1* 3 4 .' ., .3 > 1 5 1 3 1 1 3 , i .. 3 : " :4,-: '' 4.2 : ' 'kV... '''' ' ' 7. .., ' ,-.,,,,.,,_ s 0) .,I: )., . 6 1 1 a 16 , - .e-.,", ". .7. , % .-_, .. * . ' 4 1 3 00 00 S 1 1 3 1 t 1 1 L3 7 4 3 3* %ay -0 1 - 8 $-0! 1 4 1. 6 1 10 06 02 4 , w.: . . . ..-. -..P ""t ',S, ' , ..,...p. .. I '1 1 . 4 , , 2 " _. .. . 0 1 - .V L. ..1 .1 :' .% 5,t'f /..4: . . , \ 7 0 0 2 6 .s. -'1-4-H41' 4 k. 1 1 22 "2 0 0 4 -' 4 . :''.... . ki. , 1 0 0,'S 0 3-0'4-0; .,, ' 5-1 8 . .. 2;'13,-4r3110.12-'6:71 '. 1 1 -00 35 . ,, S. .., is * ., 4....; .. 1., ts. --'1` 4521 0 - 7 1 0 3 5311 1 1 1 5511 4821 4 1 . 6 -0'0 2 6 0 1 1.6 1 4 -0 0 1.03 - 1 4 5- 5221 4021 4021 5421 2 1 -0.2 33.* . 23 1621 1 1 1 3 5523 5111 1.8 8 8 8 8 8 8 8 8 8 1 4 20 1 o 5921 20 20 20 20 1 S 1' 3, 4411 20 20 1 4 6 %21 --- 22 2.2 22 1 3 I. 22 1 4 5121 5321 5921 5721 5711 5511 '' 1 1 4 s 592 1 ' t 115 0 4d21 3013 22 22 22 22 1 3- 5911 5421 442t 5613 . -0 2 2 1 -0 22 -0 2 2 3 7 -0 -01 27 02 2 -0 2 8 -0 0 .1 8 -0 3 1*. 0 0 1.4 x02 52 -0 2 2 4 -0 24 -0 2 2 4 -0 0 1 5 -0 0 2 4 -04 6 1 -0 3 -0 13 -0 3 3 1 -00 8 2-, t° -0 -0 0 1'7 . 11 I .. : .2 2 .. ..... 4 . . t 4. 0 L,Ht. LUDLOW. MkSA.P5YCNOMETRIC LABORATORY. UNIVERSITY'OrCHICAGO I FIGURE 2 1 1 I 8 al .! AGEBOYS 047 OR GT 59 MONTHS STD RESIDUALS DIAL. FINE MOTOR. CONCEPT PLOT OF ABILITIES VS RESIDUALS 00 P 1 PVP 1 1 1 1 12.122 2 1211111 1 22 1 11 3 $3 411 143411225 1 X 1 1 1 2 23112 GX34334 6X8 X 514247653358X ixX x X 1 2 21,27392578X XXX X X 14-94x-x X 1-221177794 3 4, 32316X9X 2221 5X 423 289X332 5 6 X 3 3 2 1t 3811322212 56 6 112 273312116 4X 1 1 311 311 3 1 11X 4 1 13 11 211.1 241 2 2 R r 0,06 E I U A L" 11 1 1 2 11.12 II 1 1 . 1 1 1 2 1 4 1 4 ti 121 2 1 1 1 1 2 ti 4 4 a Ir 1 -7.00 LOWER I HIGHER ABILITY M.1.0 22 7.00 - 0.004,, St.81 1.0 14 Nor OW. VESA PSYCHOMETRILATORY. UNIVERSITY OF CHICAGO L.H. LUOLON. FIGURE is MATRIX OF SORTED TRUNCATED RESIDUALS FOR THE SUB-SAMPLE OF 98 BOYS COLUMNS ARE ITEMS SORTED BY DIFFICULTY ' ROWS ARE PEOPLE SORTEO BY ABILITY RESIDUALS(OBSERVED-EXPECTED ) SCORES " ONLY NEGATIVE RESIDUALS LE-2 ARE SHOWN .ts Iism Ntmcs MS MEAN %cod ' 6421 6021 6422 6021 6321 6021 6221 6121 6421 6021 6421 6221 6021 6421 6021 6221 #5 #4 NG #7 #11 #10 #3 #2 #13 #9 #14 012 #8 -0 2 0.1 6 2 3 3 '1 2 NEGATIVE RESIDUALS FROM 2 '1.2 4.1 0 0 0.0 0 7 0.7 0 5 4.1 0.5 M. 00 2 6 3 3 421. 6121 4 2 2 2 , 4111 2 4213 3813 4211 2 1 -0 1 1 4 -00 1 1 At 2 f,S 1.3 48.1 . 4 7 u 1 8 6 1 4 .1 1 0 9 1 1 1.0 , 1.8 1.0 1 4 0.6 -04 1.3 1.4 -0.2 f 1 -0.1 2 2.5 2 5 2 2 2.2 08 1 A 2 9 2 9 0 9 0 6 0 6 0'5 0'4 -0.1 -0.2 -0 9 ,-0 2 -0.0 4. 2 2.1 3.1 2 9 2 9 1 0 0.0 2 "41.11 2511 3711 3811 3511 0.7 33 0.0" 1.1 -0.1 2 1 3.3 3 3 1 -0.2 -0.3 0.1 -0,1 -0.0 -0.1 4 1. 33 -0.Q 0.0 2 4 1 0 8 0-9 -0.0. 1 2 0.1 0.8 -0.1 .1 7 2 4 1 3 717 15 4 3 3 3 3 1 -0 0 0 0 2 4 4 4 -00 -02 7 2 6121 3711 6121 0 2 9 Ng 5 1.2 0:1 0 1 THE 98 :BOYS 2 .2 -6121 4111 6121 #1 -ti ssi 2 L.H. LUDLOW. MESA PSYCHOMETRIC LABORATORY. UN IVERSITY OF CHICAGO FIGURE A N.) '24 I a ) -FREQUENCY DISTRIBUTION OF , a. 7 e BOYS YOUNGER THAN (7 MONTHS No Mu 0 30" SDI 0 86 . - 98 PEOPLE.ABILITY MEASURES IN 2,GROUPS 42 .v -. .. 1 1 2 21 12 13 1 42462'/11 a -5 . -3 -4 . . BOYS OLDER THAN 58 MONTHS Mo 2 93 No SD. 0 76 -2 -1 1 3 1 S 14 SI. -.). . / . ^4 3 2 1 a Ot. Loom 56 --1 1 -4 -5 -2 -3. ' -1 0 1 2 2 1 4 S 7 8 M--r----S 3 2 1 5' 9 --4 IOGITS 4. 7 4. ti 11 I s. C / 4. L.H, LUOLOW, MESA PSYCHOMETRIC LABORATORY, UNIVERSITY OF CHICAGO - 26 .44 .11 25 FIGURE 5 'r 4 4 ,/< o . t.. a , . . PLOTS DF ABILITIES VS STANDARD1ZEO RESIDUALS- 41. ... . . . . BOYS LESS THAN 47 MONTHS OLD `N 9. BOYS GREATER THAN 59.MONTHS OLD 7 00 7 00. 1 1 1 F 1 4 1 1 0;00 1 4 221 1 In. 1 22 141 3 1 R ' 12 122 1 1 2 a 1 411 1434111 2 23112 6X311 51424765312 2 212739224 13 5 3 2 5'2 X 5 3 4 32316X9X411 .3 .423 289X311 A lot L 1 1.31 1 2111 1 11 1121 5X Z1 4 6 X, III 46 6 -111S 4X S ' I D U A 1 1 11 1 1 11X 4 1' 2 210.2 1 +1 1 11. 1 X SitC0 1 311.2' 241 2 1 '1112 X 2387 3 2 11 38113221 2.5 1 1 112 273311 51, ur 3231 4X8 8XX X 1387 XXX X R 1.-221177794-2-.3.. re 1 X 12 1 f 3 1 2 4 1 t -7.00 -7.00 7.00 .HIGHER -7.00 LOWER MO.3 S.86 Lo 14 *No 421 2 .1 1 7 00 HIGHER 0.00 LOWER 142 93'c 5.76 Li, 14 , No 56 L.11.LUDLOW4 MESA PSYCHOMETRIC LABORATORY, UNIVERSITY OF CHICAGO FIGURE 7 28, -f 1 OIAL. PINE MOTOR, CONCEPT. AGEBOYS LT 47 OR GT 59.MONTH5"57ENESIDUALS -PLOT OF DIFFICULTIE1 VS RESIDUALS ... 7.00 4. $ 11, 0 1 2 1 ) 2 1 1 4 2 1 11 1 1 2 2 X X . 0.00 E 3 2 I 0 2 1 U 1 2 A 1 1 , 1 1 2 , 2 22 1 1 22 3 5 8 6 8 8 7 X 9X X 255 4 8 984 X X 09 3XX X 6 X XXX X X X X7 2 """ 3- X3X -X-6- -5-584.X 5 4X3 34 9 XX X 8 XX 3 7 X35 64 8 16 8 3 212 2 X. X 22 3 324 6X 2 4 7 1 72 6 2 111 2 11 2 13 5 JP 1 221 2 2 2 5 4 x - -X 1 4 3 X X 2 5 2 4 1 1 3 1 2 1 2 1 1 UNEXPECTED DIFFICULTY .67.00 + 1- + .3.50 EASIER Gi -0.00 DIFFICULTY . ..r\.,... Le 14 . go 1. 3.50 'HARDER 98 . MESA PSYCHOMETRIC LABORATORY. UNIVERSITY OF CHICAGO .. 4 . '. FIGURE 8 29 0 4 4 'STANDARDIZED RESIDUAL FREQUENCY DISTRIBUTIONS e ITEMS BOYS YOUNGER THAN 47 MONTHS NSD. 0 80 Me 0.01 42. C . . t , 12' 1 n . -1' -4 -3 -2 -3 _ 1111451221 12211144 2 M S I S , -1 0 1 1 2 4- 3 STDZD 11 BOYS OLDER THAN 59 MONTHS SD. 1.14 Me -0.09 N 1 56. 12 1 M n---S -4 -5 -3 , -2 I 98221 if i . ... La'"' it -1 - i 2 4 3 . STDZD . (.. MP V .. S 0 , . V ITiM14 1' -`..I. BOYS YOUNGER THAN 47 MONTHS Me ,-0.18 No 1.00 SDP' 42 1 2 1 2 1 -4 -5 BOYS OLDER THAN 59 MONTHS SON 1.39 No Mu -0.20 1 1 -2 -3 ,-4 1121 11 4 11421 1 a 5 0 -1 1 2 3 4 STDZD 2 3 4 STDZO 56. 111 4 .1. 1 -'S -5 123314 M S -2 -3 1 841 4f 5 14 0 / .. . .0. . Ms I . . L.H. LUDLOW. MESA PSYCHOMETRIC LABORATORY, UNIVERSITY OF CHICAGO RE 9 I I I. . 4- if .. L. e 1 31 a- r , ITEMS NGER THAN 47 MONTHS BOYS 1. 4i 0.13 SD *1.06 No 42 4 1)1 14534642 1 111 2 1 t 1 S -2 -3 -4, -1 _., 3 2 1 4 STDZD 4 STD20 4 STUD 4 STUD BDY1. OLDER THAN 59 MONTHS * Me0 07 SO 0 80 .56 No 1 17 1 3 .1 1 2 1191611 1 a 5 -3 -1 2 dm 3 2 1 . 6 mok ITEm7 goys YOUNGAR THAN 47 kONTHS Mm 0 11 No 50- 0 94 42 22 314 42322121 2 1113 321 -3 -4 BDYS OLDER THAN 59 MONTHS Mm LO D7 50- 0 pg 0 -1 -2 2 1 - 3 4111 N 56 c 1 1 6 2 -4 -5 2 -/ -3 . 1 3 3 1 1173 34 1 5 - -1 0 5 2 t 3 . 0 ITEM11 BOYS YOUNGER THAN 47 MONTHS Mo 0.41 SO' x1.03 N 42. . :451 111 ., 13 1 1 1 o%.,1 -5 -4 ,BOYS OLDER THAN 59 MONTHS No -D.28 SO' 1.00 . . -3 N -2 M-- --coo . 2 1 3 4 STDZD 4 STOZO °* 1 . . -PP 5 Asp 4 3 -, 0 -1 56. . . 2 12 31 1 4 2 1315231 S A. 33 1221 M Co S -3 -4 -1 -2 8 5 84 1 1 11 S 0 .i : 2) 4' 3 ' E LH LUDLOW. MESA PSYCHOMETRIC LABORATORY. UNIVERSITY OF CHICAGO 32 .33 FIGURE 10 -Ur $ 4' A 1 a \it 4 , ..... . , t LT 47 OR GT 59 MONTHS;STO RESIDUALS OIAL: FINt NOT04.(CONCEPP.AGiv I RESIDUAL SUMMARY TABLE' ' ITEMS LISTED IN SEQUENCE, GROUPS ACCORDING TO ID CODE OLDER BOYS YOUNGER BOYS ITEM NAME #1 #2 #3 44 45 06 #7 r8 49 ,1Q oil #12 413 414 0 -1.81 -0.19 -0,15 1,27 1.47 1.13 0 97 -0.33 -0.67 0.26 0,46 -1,51 -0,20 -0.71 I MN -0,02 40.22 0.05 -0.49 -0 34 0.13 0.11 0.01 0.00 -0,04 0.41 0,05 0.06 -0.18 5 . 2 1 MN SD 1.14 0.85 0.96 0.70 0.84 1.06 0.94 0,80 1.04 0.85 1.03 1.12 0,92 1.00 SD 0 06 0.50 0 13 0.72 0 77 0.80 -0.07 0.89 -4.09 1.14 0 06 0.62 0.03 1 01 -0.28 1,00 0,10 0,57 0.15 1,03 -0.20 1.29 0'.13 0,30 0.26 0.29 0 23 -0.01 THIS TABLE CAN FOCUS ON, AMONG OTHERS, 1) ABSOLUTE VALUE MEAN DIFFERENCES. '2) ITEMS WITH REVERSED SIGNS IN THE MEANS. 3) ITEMS WHERE-THE OLDER BOYS MEAN IS LESS IRAN THE YOUNGER BOYS MEAN . ti ' L,H. LUDLOW. MESA PSYCHOMETRIC'LABORATORY, UNIVERSITY OF CHICAGO TABLE.' F .1/ . 35 4 11. PLOT OF TWO ITEM CALIBRATIONS 4.00-3 H A . D E R HARDER FOR THE YOUNGER BOYS 46 06 Y #5 #4 07 THE UNDERSCORED ITEMS ARE THE SAM, UNEXPECTED ITEMS SETH IN TAW 1 13, a F -#8 0.00 #2 1 11104 #11 U #9 #13 #14 L T Y 1 HARDER FOR 'THE OLDER BOYS 012 E A 5 I ' E R; 4 -4.00 r -4.00 EASIER VI - 0. 00 A DIFFICULTY . MEAN A= 0.00 0.13 MEAN 13= SA 1.28 See 0.99 , ' 4,00 HARDER R0.677 N 12 GROUP A: ITEMS FOR BOYS GREATER THAN t9 MONTHS OLD GROUP 8; ITEMS FOR BOYS LESS THAN 47 MONTHS OLD 36 4 I THE CONFIDENCE INTERVAL REPRESENTS 3. STANDARD ERRORS L.H. LUDLOW, MESA PSYCHOMETRIC LABORATORY. UNIVERSITY OF CHICAGO FIGURE It 4 37 Po