25 Comparison Sequential Test for Mean Times Between Failures

25 Comparison Sequential Test for Mean Times Between Failures Yefim Haim Michlin1 and Genady Grabarnik2 1Technion - Israel Institute of Technology 2St' Johns University Israel USA 1. Introduction The present study deals with the planning methodology of tests in which the parameters of two exponentially-distributed random variables are compared. The largest application field of such tests is reliability checking of electronics equipment. They are highly cost-intensive, and the requirements as to their resolution capability become stricter all the time. Hence the topicality and importance of an optimal plan permitting decisions at a given risk level on the basis of a minimal sample size. Such comparison tests are required for example in assessing the desirability of replacing a “basic” object whose reliability is unknown, by a “new” one; or when the influence of test conditions on the results has to be eliminated. This is the case when an electronics manufacturing process is transferred to another site and the product undergoes accelerated testing. Recently, equipment and methods were developed for accelerated product testing through continuous observation of a population of copies and replacement of failed objects without interrupting the test. For such a procedure, the sequential approach is a feasible and efficacious solution with substantial shortening – on the average – of the test duration (see e.g. Chandramouli et al. 1998; Chien et al. 2007). In these circumstances there is high uncertainty in the acceleration factor, with the same effect on the estimated reliability parameters of the product. This drawback can be remedied by recourse to comparison testing. The latter serves also for reliability matching in objects of the same design and different origins, or a redesigned product versus its earlier counterpart, or different products with the same function (see e.g. Chien & Yang, 2007; Kececioglu, 2002). The exponential nature of the Time Between Failures (TBF) of repairable objects, or the time to failure of non-repairable ones – is noted in the extensive literature on the reliability of electronic equipment (Kececioglu, 2002; Chandramouli et al, 1998; Drenick, 1960; Sr-332, 2001; MIL-HDBK-781A, 1996). For brevity, the TBF acronym is used in the sequel for both these notations. Mace (1974, Sec. 6.12) proposed, for this purpose, the so-called fixed sample size test with the number of failures of each object fixed in advance – which is highly inconvenient from the practical viewpoint. For example, when the "basic" object has "accumulated" the 454 Modern Approaches To Quality Control specified number of failures, one has to wait until the "new" one has done the same, and if the latter is substantially more reliable, the waiting time may be very long. The international standard IEC 61650 (1997) deals with two constant failure rates, which is equivalent to the problem just described. However, this standard, which forms part of an international system of techniques for reliability data analysis, does not refer to the planning aspect of the tests. A solution to our problem was outlined in (Michlin & Grabarnik, 2007), where it was converted into binomial form, for which Wald's sequential probability ratio test (SPRT) is suitable (Wald, 1947, chap. 5). Wald and Wolfowitz (1948) also proved that this test is the most efficacious at two points of its characteristic, but it has one drawback – the sample size up to a decision can be many times larger than the average. This is usually remedied by resorting to truncation (see e.g. Wald, 1947; Siegmund, 1985). A methodology is available for exact determination of the characteristics of such a truncated test with known decision boundaries. It was proposed by Barnard (1946) and developed by Aroian (1968). It served as basis for an algorithm and computer programmes (Michlin et al. 2007, 2009) used in examining its properties. Hare we consider the inverse problem – determination of the test boundaries from specified characteristics. In the absence of analytical dependences between the boundary parameters and characteristics, the search is hampered by the following circumstances:  The number of parameter-value combinations may be very large.  While shortening of the step makes for more combinations, it cannot be guaranteed that combinations with optimal characteristics are not missed.  The standard optimum-search programmes are unsuitable for some of the discrete data of the type in question. The theme of this chapter is the planning methodology for comparison truncated SPRT's. Formulae derived on its basis are presented for calculation of the test boundary parameters. The rest of the chapter is organised as follows: In Section 2 is given a description of the test and its conversion to SPRT form. In Section 3 are described the quality indices for a truncated test and criteria for the optimal test search. In Section 4 are discussed the discrete nature of the test boundaries and its characteristics; a search algorithm is presented for the oblique boundaries. Section 5 describes the planning methodology, and approximative dependences are presented for calculation of the boundary parameters. Section 6 deals with planning of group tests. Section 7 presents a planning example and applications. Section 8 – the conclusion. 2. Description of test and its SPRT presentation 2.1 Description of test procedure in time domain. Checked hypothesis In the proposed test two objects are compared – one “basic” (subscript “b”) and the other “new” (subscript “n”). In the course of such tests, the “null” hypothesis is checked, that the ratio of the mean TBF (MTBF) of these objects exceeds or equals a prescribed value Φ0, versus the alternative of it being smaller than the latter. The compared objects work concurrently (Figure 1). When one of them fails, it is immediately repaired or replaced. The unfailed object is not replaced but allowed to continue working until it fails in turn (in which case it is neither replaced nor repaired), or until the test terminates. A situation may occur in which there has been no failure in one object and it kept working throughout the Comparison Sequential Test for Mean Times Between Failures 455 whole test, as against several failures in the other object. The total work times T are equal for both objects. Fig. 1. Scheme of test course (Upward marks – failures of basic item; downward marks – those of new item; T – time, common to both systems) (Michlin et al., 2011). The probability density of the TBF for each of the compared objects has the form: fTBF (t)=(1/ )*exp(-t/ ) where is the MTBF for the "new" ( n) and “basic” ( b) objects respectively. At each failure, a decision is taken – continuing the test versus stopping and accepting the null hypothesis, or rejecting it in favour of the alternative (Michlin & Migdali, 2002; Michlin & Grabarnik, 2007): H0 :    0 H1 :    0 where  Pa   0   1     Pa  1       n /b (1) (2) and are the probabilities of I- and II-type errors; in the sequel, their target values will be denoted by the subscript "tg", and their actual values – by the subscript "real". Pa(Φ) is the probability of acceptance of H0, which is the Operating Characteristic (OC) of the test; 1  0 / d (3) d>1 being the discrimination ratio. Mace (1974 , Sec. 6.12) presents the following estimate ̂ for Φ, obtained with the aid of the maximum likelihood function (for the proof, see Kapur & Lamberson 1977, Sec. 10.C):   (T / r ) (T / r )  n n b b where rn and rb – the accumulated number of failures over times Tn and Tb. As in this test Tn=Tb=T, we have:   r /r  b n (4) Figure 2 shows an example of the test field. In the course of the test, it can reside at a point of this field characterised by an integer number of failures of each of the objects. When one of them fails, the test “jumps” to a neighbouring point located above (failure of “n”) or to the right (failure of “b”). With the test course thus described, shifts from point to point occur only on failures in one of the objects, i.e. the time factor is eliminated from the analysis. When the Accept boundary is crossed, the test stops at the Accept Decision Point (ADT), 456 Modern Approaches To Quality Control and when its Reject counterpart is crossed – at the RDP. The boundaries consist of two parallel oblique straight lines (accept line (AL) and reject line (RL)) and the truncation lines parallel to the coordinate axes and intersecting at the Truncation Apex (TA). ADP RDP Truncation Apex TA rnMax 20 rbMax New item number of failures (rn) Accept Line Reject Line Centerline 10 0 0 10 20 30 40 50 60 Basic item number of failures (rb) Fig. 2. Truncated test field for Φ0=4.3, d=2, 2011). real=0.098, real=0.099, RASN =9.2%, (Michlin et al., 2.2 Binomial presentation of test and SPRT solution For all points of the test field, the probability of the next failure occurring in the new object (i.e. of a step upwards) is constant and given by the following expression (for proof see Michlin & Grabarnik, 2007): PR ( )  1  1    (5) A binomial SPRT is available for such a test (Wald, 1947, chap. 5), whose oblique boundaries are: Accept line (AL): rb= rn/s+h’b (6) Reject line (RL): rn= rb·s+hn (7) where s is their slope, uniquely determined by the SPRT theory depending on , , Φ0, d (Wald,1947; Michlin & Grabarnik, 2007), and given by: where s   ln q  ln q  ln d  (8) q   1  0   d  0  (9)   The absolute terms of (6) and (7) are given by: ha  ln  * 1   *   ln q  ln d  (10)   457 Comparison Sequential Test for Mean Times Between Failures hn  ln 1   * *  ln q  ln d  hb'  ha / s (11) (12) The expressions (10)-(12) have one drawback: the parameters * and * are unknown. Their dependence on 0, , Φ0, d, and on the TA coordinates is available only in the form of the limits between which the parameters lie (Michlin et al., 2009). Still, these limits suffice for determining – from the above expressions – corresponding search limits for h'b and hn. A search methodology, within these limits, for exact values ensuring the target characteristics – is, basically, the goal of this work. 2.3 Calculation of test characteristics acc. to given boundaries The probability of hitting a given point of the test is given by (Barnard, 1946; Michlin & Grabarnik, 2007): Prb , rn ( )  Prb ,rn  1 (  )  PR     Prb  1,rn ( )  1  PR     (13) while that of hitting the given ADP is: PADP  rn ,    Prb  1,rn     1  PR     and that for the given RDP is: PRDP  rb ,    Prb , rn  1     PR    (14) (15) Pa(Φ) is the sum of all the probabilities PADP  rn ,   of hitting all ADP, hence the actual values of and , namely real and real , are given by:  real  1  Pa   0  ;  real  Pa   1  (16) The Average Sample Number (ASN) of a truncated test is calculated as: ASN      r TAb b 0 rb  rnRDP (rb ) PRDP  rb ,    TA r  rbADP (rn ) PADP  rn ,   r 0 n n n (17) where rnRDP (rb ) is the rn-coordinate of the RDP with given rb. The Average Test Duration (ATD) for each object is: ATD      b  ASN     1  1   (18) 3. Comparative characteristics and optimality of test In this Section the optimality criteria for the test, on which the comparison- and selection algorithm is based, are substantiated, and the problems of the study are clarified. In (Michlin & Grabarnik, 2007) were presented three optimality criteria which can be calculated for the specified boundaries: 458  Modern Approaches To Quality Control Closeness of the test OC to the prescribed one. For given d, the measure of this closeness is RD:    RD    real   tg  tg     real  tg      2  tg   2 (19) with real and real as per (16). The degree of truncation, which characterises the maximum test duration whose measure can be, for example, the sum of the TA coordinates. The efficacy of the test according to Wald (1947) and to Eisenberg & Ghosh (1991), as the measure of which RASN was adopted (Michlin et al., 2009) – the relative excess of the function ASN(Φ) of the truncated test over ASNnTR(Φ), its non-truncated counterpart which can be taken as ideal: RASN   5  ASN i 1     i   ASN nTr   i   i 1 ASN nTr   i  5 (20) where Φi - values of Φ in geometric progression: 0   d i4 for i = 1…5 (21) ASN(Φ) – calculated as per the recursive formulae (17), (13…15) ASNnTr(Φ) – calculated by Wald’s formulae (1947, chap. 3) obtained for a non-truncated test of the type in question: ASN nTr      1       Pa   ln B   1  Pa    ln A  1      ln  1  0   d  0   ln d where     d  1   0    d   0   d  0    Pa    A  1    1  0   (22)    A  B  A   1   real   real B   real  1   real  (23) (24) (25) (26) – an auxiliary parameter calculated by (23) for Φ values as per the progression (21). The choice criterion for the optimal test is: min(TAn+TAb) (27) (minRd at given TA)&(Rd< Rd0)&(RASN< RASN0) (28) subject to: 459 Comparison Sequential Test for Mean Times Between Failures where Rd0 and RASN0 – threshold values of Rd and RASN. The TA of such a test called Optimal TA (OTA). Section 5 presents approximative formulae for determination of those OTA coordinates which permit reduction of the search field to 2 to 6 points. A particular problem in this context is: for a given TA, find h’b and hn (eqs. (6), (7)) ensuring min Rd. 4. Discreteness of test boundaries and their search at given TA This Section deals with the interrelationships between the boundary parameters of the test on the one hand, and the characteristics of the test itself (namely, real and real) and those of its quality (introduced in the preceding Section, Rd and RASN) – on the other. These interrelationships lack analytical expression and are further complicated by the discreteness of the test. Thus one had to make do with typical examples of their behaviour in the vicinity of the optimum. With this behaviour clarified, an efficacious search algorithm could be developed for the optimum in the discrete space in question. Clarity of the picture is essential both for the developer of the planning methodology and for the practitioner planning the binomial test in the field. 11 rnMax 10 TA 9 rbMax New system number of failures, rn 4.1 Discreteness of test boundaries As the slope s of the oblique test boundaries, described by eqs. (6) and (7), is unrelated to and (see eq. (8)), the search for them under the min Rd stipulation reduced to finding the absolute terms in the describing equation, namely the intercepts h’b and hn on the coordinate axes (Figure 3). 8 7 Continue zone 6 25 4 AL 3 hn RL ADP 2 RDP 1 Shifted AL Shifted RL 0 0 1 h'b 1 2 3 4 5 6 Basic system number of failures, rb Fig. 3. Test Plane (Michlin & Grabarnik, 2010). 1 – Example of interval of h’b values over which the test ADP's do not change. 2 – Ditto for hn and RDP. 460 Modern Approaches To Quality Control Stopping of the test occurs not on the decision lines, but at points with integer coordinates, with ADP to the right of the AL and RDP above the RL. If the AL is shifted from an initial position (solid line in Figure 3) to the right, the test characteristics remain unchanged until it crosses an ADP, which in turn is then shifted in the same direction by one failure. The AL positions at these crossings are shown as dot-dashed lines, and its shifts are marked with arrows. Projecting the termini of these arrows, parallel to the AL, on the rb axis, we obtain the values of h’b at which the changes occur. An analogous process takes place when the RL is shifted upwards. The intervals of h’b and hn over which the test characteristics remain unchanged are marked in Figure 3 by the circled numbers 1 and 2 respectively. When the AL is shifted to the right (h’b increased) Pa(Φ) is reduced, i.e. real increases and real decreases. When the RL is shifted upwards, the effects are interchanged. These relationships are monotonic and stepwise, and differ in that change of h’b is reflected more strongly in real and more weakly in real. With hn the pattern is reversed. 4.2 Basic dependences between oblique boundaries and test characteristics In (Michlin et al., 2009, 2011; Michlin & Kaplunov, 2007) were found the limits within which * and * of the optimal tests should be sought. These limits can also serve for determining the search limits of h’b and hn, as per (10) – (12). Figure 4 shows an example of the above, with the limits for h’b and hn calculated, according to the data of (Michlin et al., 2009), for d=2, Φ0=1, tg= tg=0.1, TAb=27, TAn=38, RASN≤12%. In the figure, the points mark the centres of rectangles within which the characteristics remain unchanged. The resulting picture is fairly regular, even though the spacings of the columns and rows are variable. In space, the Rd points form a cone-shaped surface. Fig. 4. Contours of RASN (dashed lines) and RD (solid lines) vs. h’b and hn. (Michlin & Grabarnik, 2010). The dots mark the centres of rectangles within which the test characteristics do not change. 1 – 4 are the corner points at which the test characteristics are calculated in the search for the optimum (Subsection 4.3, stage ‹1.1›). Comparison Sequential Test for Mean Times Between Failures 461 The figure also contains the contours (isopleths) of Rd (solid lines) and RASN (dashed lines), given as percentages. In macro the Rd contours can be described as oval-shaped, whereas in micro they are quite uneven, so that derivatives calculated from a small set of points would show large jumps, which would hamper the search for the minimum Rd . It is seen that in the vicinity of that minimum, RASN≈11%. Figure 5 shows two projections representing real and real, calculated according to the coordinates of Figure 4, so that to each point of the latter corresponds one of real and real. These points form intersecting almost-plane surfaces. In the upper figure the coordinate axes are oriented so that the intersection zone ( real - real) is perpendicular to the page; in the lower figure. the orientation is such that the rows of real points reduce in projection to a single point – in other words, they form parallel or almost-parallel lines. Fig. 5. Two projections of real and real “planes”. (Michlin & Grabarnik, 2010). 462 Modern Approaches To Quality Control Figure 6 shows analogous projections for RASN, and we again have an almost-plane surface, monotonic and uneven in micro. The provided examples show that the described patterns characterise the dependences of real, real and RASN on h’b and hn within the limits determined in Subsection 5.3 (Michlin et al., 2009, 2011; Michlin & Kaplunov, 2007). Over small intervals these dependences are stepwise, and the lines through the step midpoints are uneven as well. Fig. 6. Two projections of RASN “plane”. (Michlin & Grabarnik, 2010). 4.3 Search algorithm for oblique test boundaries Standard search programmes for minima (such as those in Matlab) operate poorly, or not at all, with discrete data of the type in question. Availability of known regularities in the behaviour of the functions real, real, RASN, RD makes it possible to construct a fast and efficacious algorithm. These known regularities are: The values of h'b and hn at which the test characteristics change.  The limits of h'b and hn, yielding tests with the specified characteristics.  Almost-plane monotonic dependences of real, real and RASN within the above limits,  stepwise and unstable but also monotonic in narrower intervals. Comparison Sequential Test for Mean Times Between Failures 463  Stronger dependence of real on hn than on h'b; the reverse – for real. In expanded form, the search algorithm for min Rd consists in the following: 1st stage. ‹1.1› Calculation of the test characteristics at the four vertices of a rectangle (Figure 4) whose coordinates are obtained from the relationships presented in Subsection 5.3. ‹1.2› Approximation of real(h'b, hn) and real(h'b, hn) as planes, and determination of the first estimate h'b1, hn1 yielding min RD (point 5, Figure 7). Checking for RD ≤ RD0. If satisfied, stopping of search. Fig. 7. Example of search scheme for min(RD). (Michlin & Grabarnik, 2010). 5 – 11 are points of test characteristics calculation. 2nd stage. Determination of point 6 – from Re-checking for RD ≤ RD0. real5, real5 and the slopes of the -, -planes as per ‹1.2›. 3rd stage. Alternating advance parallel to the h'b- and hn-axes. In view of the discreteness and complexity of the RD function, the search for its minimum was reduced to one for the points h'b and hn where Δ and Δ change sign: Δ = real – tg; Δ = real – tg This problem is easier to solve, as both Δ and Δ are monotonic functions of h'b and hn. The search can be stopped at every step, subject to RD ≤ RD0. ‹3.1› If at point 6 (‹2› above) Δ 6 > Δ 6, a path parallel to the hn-axis is taken in uniform steps Δhn, until Δ changes its sign (points 6,7,8 on Figure 7), Δhn= Δ 6/a3, where a3 is the 464 Modern Approaches To Quality Control coefficient in the equation of the -plane as per ‹1.2›. Beyond that point, the root Δ (hn) is searched for by the modified Regula Falsi method (point 9). (The term "root" refers here to one of a pair of adjoining points at which the function changes its sign and has the smaller absolute value). The special feature of this procedure is accounting for the discreteness of the solution. ‹3.2› At the point of the root Δ , a right-angled turn is executed and a path parallel to the h'b-axis is taken, searching for the Δ root (point 10). ‹3.3› The alternating procedure is continued until a situation is reached where two consecutive turns involve only movement to an adjoining point. This point 10 corresponds to min(RD). If in ‹3.1› Δ 6 < Δ 6, we begin from ‹3.2›. 4.4 Efficacy of algorithm With a view to assessing the efficacy of the proposed algorithm, a search for the h’b and hn values yielding min Rd was conducted with the aid of a Matlab programme which realised this algorithm, and alternatively with the Matlab fminsearch command, with the same function WAS (Michlin & Grabarnik, 2007) referred to in both cases. This function determines the test characteristics according to its specified boundaries. The run covered different tests with RASN0=5 and 10%. The calculation results are shown in Figure 8. Fig. 8. Comparative efficacy of proposed algorithm. (Michlin & Grabarnik, 2010). 1 – fminsearch (Matlab) found min RD or stopped close to it; 2 – fminsearch failed to find RD; A, B, C – short, medium and long tests, respectively. In it, the abscissa axis represents the product TAb*TAn, which we term “density factor of test states”. The higher the latter, the denser the disposition of the test points in the search zone (see Figure 4), the smaller the changes in the test characteristics from point to point, and the closer the search to one over a continuous smooth surface. A small value of the product is associated with a short test, due to be completed at small sample size and moderate computation times for the characteristics; a large value – with long tests, completed on the average at large sample sizes and long computation times. 465 Comparison Sequential Test for Mean Times Between Failures The ordinate axis represents the “acceleration factor of calculation”, which is the ratio of references to the WAS-function by fminsearch and the proposed algorithm respectively. The larger the ratio, the faster the algorithm compared with the standard Matlab function. The diagram shows that at low densities (short tests, zone A) fminsearch fails to find min Rd. In zone C (long tests) the command finds it or stops close to it, but with 3 to 6 times more references to WAS. In zone B (medium tests) the minimum is either not found, or found with 2.5 to 5 times more references to WAS. By contrast, the programme based on the proposed algorithm found the minimum in all cases. Accordingly, for the present task – searching for the optimum in a discrete space – the proposed algorithm accomplishes it much faster than the Matlab standard fminsearch command, thus saving computation time in long tests. Moreover, it guarantees a solution – a critical aspect in short tests, where fminsearch usually fails to find one. 5. Estimates for boundary parameters 5.1 Search methodology for optimal test boundaries In (Michlin & Grabarnik, 2007) it was established that for Φ0=1 and = , the OTA lie on the centreline (which runs through the origin parallel to the AL/RL), so that rn  s  rb (29) This was checked for different Φ0. With given tg= tg, d, and RD≤1%, a search was conducted for three location zones of the TA – namely, with RASN≤5%, 5%<RASN≤10%, and RASN>10%, the last-named being restricted by the above requirement on RD, i.e. achievability of tg and tg. A typical example of such zones for tg= tg=0.05, d=2, and Φ0=1, 2, 3 is shown in Figure 9. The fan-shaped zones have their apices on the corresponding centrelines. These apices are the OTA locations, as with the imposed limits satisfied they are closest to the origin (heaviest truncation). In these circumstances the search zone is narrowed, the location problem being converted from two- to one-dimensional. To study the relationships between the sought boundary parameters (TA, *, *) and the specified test characteristics (Φ0, d, tg= tg, RASN max), a search was run over a large population of optimal tests with the characteristics given in the Table below. Lower Upper limit limit Φ0 0.3 5 d 1.5 5 = 0.05 0.25 tg tg RASN max 5% 10% Table 1. Regions of characteristics covered by search Number of levels 9 12 5 2 5.2 Search results for OTA and their curve fitting The dots in Figure 10a mark the OTA for tg= tg=0.05, RASN≈10%, and wide intervals of d and Φ0. Figure 10b is a zoom on the domain in 3a representing the "short" tests, namely those with small ASN and – correspondingly – low TA coordinates. It is seen that all curves smooth out as the distance from the origin increases (the tests become longer), the reason 466 Modern Approaches To Quality Control Truncation Apex rn coordinate ≡ rnMax 95 1 Φ0 = 1 2 85 3 75 4 Φ0 = 2 65 Φ0 = 3 55 45 35 25 35 45 55 65 75 85 95 Truncation Apex rb coordinate ≡ rbMax Fig. 9. TA zone boundaries for three 0 values and three RASN zones (Michlin et al., 2011): 1 = Boundary beyond which RD≤1% is unachievable at any RASN; 2 = Boundary for RASN≤10%; 3 = Boundary for RASN≤5%; 4 = Centreline. Remark 1. 0=1 subgraph: OTA for each RASN zone circled. Remark 2. For this figure: d=2, tg= tg=0.05, RD =1%. being the weakening influence of discreteness of the test characteristics (Michlin et al., 2009). Φ0=0.3 60 (a) Basic system max number of failures, r bMax 20 15 d=2.54 10 d=3.18 5 Fig. 10. (a) OTA locations for different d and 0, and for short test zone. (Michlin et al., 2011). 60 55 50 45 40 35 d=5 0 30 350 300 d=1.65 250 200 d=2 d=1.77 150 100 d=5 50 d=3 0 25 25 d=1.5 d=1.54 50 30 20 100 Φ0=3 Φ0=4 Φ0=5 35 15 Φ0=1.5 Φ0=2 150 40 5 Φ0=1 45 10 200 50 0 Φ0=0.7 New system max number of failures, r nMax Φ0=0.5 250 (b) 55 300 0 New system max number of failures, r nMax 350 Basic system max number of failures , r bMax tg= tg=0.05, RASN≈10%. (b) Zoom on 467 Comparison Sequential Test for Mean Times Between Failures The Φ0-isopleths in the figures are broken radial lines, whereas their d-counterparts are symmetrical about the rn= rb line and approximate neatly to a hyperbola:  rn  rb    k  d   q  x , RASN   where k  d   exp  5.58  d  1   q  x , RASN    1  rb 1 1 4    1 1 ; 1  1  1.10 ln x  0.41RASN  1.03RASN ln x  20 (30) (31) (32) 40 OTA rnMax New system number of failures, x – common target value for and , x= tg= tg; RASN – in relative units rather than in percent. The formulae indicate that the approximate curves differ only in the scale factor k(d), common to both axes – it remains the same for any pair (x, RASN). As the formulae do not contain Φ0, the OTA is searched for through its required adherence to the centreline, whose expression (29) is uniquely determined by d and Φ0. Accordingly, the sought OTA is the integer point closest to the intersection of the curve (30) and the centreline (29) (Figure 11). 20 0 Hyperbola by (30) Centerline by (29) 20 40 60 80 100 Basic system number of failures, rbMax Fig. 11. Determination of OTA. (Michlin et al., 2011). The coefficients in (31) and (32) were found through the requirement of minimal root mean square error (RMSE) – the difference between the OTA's found as per eqs. (29) and (30). For the data in the Table, RMSE=0.88, indicating high estimation accuracy for such a broad domain. 5.3 Estimates for α* and β* As already mentioned, the problem of finding the oblique boundaries reduces to that of finding * and *. This Subsection presents regressional dependences of the latter on the test characteristics Φ0, d, x= tg= tg, and RASN max, as well as their counterparts for the upper and lower limits ( U and  L , U and  L ) of these parameters. These dependences, determined on the basis of the total data on optimal tests with the characteristics in the Table, were sought in the form: 468 Modern Approaches To Quality Control  M  c  x ;  M  c   x. (33) The Matlab tool for stepwise regression yielded the coefficients for the above: c  1.10  0.021  ln x   0.0081 02  0.036 0 d  1.07 RASN ln x 2 (34) where RMSE=0.061 and R2=0.83, the latter being the coefficient of determination, and c   1.09  0.096 ln x  0.14 d  0.018 0 d  1.11R ASN ln x (35) with RMSE=0.069 and R2=0.80. The limit formulae read U     1  c B  M  L  U        1  c B  M  L  where c B  0.045  0.14 ln d  0.031ln x c  B  0.059  0.16ln d  0.048ln x (36) (37) (38) (39) and such that all * and * obtained for the Table are included.  Figure 12 shows example dependences for the regressional value  M and the upper and lower limits, versus x= tg for Φ0=3 and d=1.5, 3. Also included are the actual values of *. (The graphs for * are analogues). The bounded zone becomes narrower as d and tg   decrease. It is seen that at low d,  M and  M can serve as the calculation values without undue deviation of real and real from their targets. The search methodology for * and * of the optimal test, described in detail in Section 4, is based on knowledge of the limits (36), (37), which is one of the reasons for its high efficacy. 5.4 Accuracy assessment of proposed planning The accuracy of the proposed planning, using eqs. (30) – (32) and (33) – (35) – was assessed by applying them in calculating the test boundaries for all characteristic values in the Table. This was followed by calculation of real, real and RASN for these tests and their deviation from the targets. The RMSE's of real and real decrease with decreasing d and RASN. For d≤2 they do not exceed 3 to 4% of the target value and for large d they reach 8 and 10% at RASN=5 and 10% respectively. In the former case this is very satisfactory accuracy, while in the latter case it may become necessary to find more accurate values of the boundary parameters – for which the methodology outlined in Section 4 is recommended, using eqs. (30) – (32) and (34) – (39) for the search limits. 469 Comparison Sequential Test for Mean Times Between Failures 0.25 0.20 α* 0.15 alSt  * for d=1.5 alStL *L alStU  *U alStM  *M Bisector 0.10 0.05 (a) 0.00 0.00 0.05 0.10 0.15 0.20 0.25 αtg=βtg 0.35 0.30 0.25 α* 0.20 alSt * for d=3 alStL *L alStU *U  *M alStM Bisector 0.15 0.10 0.05 (b) 0.00 0.00 0.05 0.10 0.15 0.20 0.25 αtg=βtg Fig. 12. Actual *, regressional dependence, and upper and lower search limits. RASN=5%. (a) d=1.5. (b) d=3. (Michlin et al., 2011). 6. Group tests In this case the items are compared groupwise, which makes for economy in the time to a decision. The items of the respective subgroups, Nb and Nn in number, are drawn at random from their respective populations with exponential TBF's, and tested simultaneously. On failing, they are immediately replaced or repaired – just as in the two-item tests. The subgroup can be treated as a single item with an N-times shorter MTBF (Epstein & Sobel, 1955). The planning procedure remains the same, except that Φ in the calculations is replaced by Φg: 470 Modern Approaches To Quality Control  g    Nb Nn (40) Thus when Nb=Nn= N, the test boundaries remain as in the two-item case, except that the test duration is also N times shorter (see (18)). When Nb≠Nn, it is recommended to check the efficacy of larger groups, e.g. in terms of a shorter average test duration ATDg(Φ). By (18) and (40) we obtain:     1  1 g  ATDg  g   b N b   ASN g  g (41) where ASNg(Φg) is the ASN of the group test as per (17) or (22), except for Φg replacing Φ of (40). The planning example covers also the problem of choice of Nb and Nn, while ensuring min ATDg and satisfying additional essential test-planning conditions. 7. Example of test planning A large organisation operates a correspondingly large body of mobile electronic apparatus whose MTBF is substantially shortened under the stressful exploitation conditions. The manufacturer offers to modify this equipment, thereby significantly improving its resistance to external impacts, albeit at increased weight and cost. In a fast laboratory test the modified (hereinafter "new") apparatus exhibits high reliability, but so does the original ("basic") one. Accordingly, it is decided to check the MTBF increase under field conditions on an experimental batch. The requirements regarding the test OC are established as follows. If the MTBF of the new product is 5 times that of the basic (Φ=5), replacement is beneficial; at Φ=2.5 it does no harm; but at Φ=1.5 it is unacceptable. These findings follow from the OCnTr of a non-truncated SPRT with ==0.1, d=2, Φ0=5 (Figure 13), constructed as per (23) – (24). The apparatus are operated in sets of 28 items, so that conditions within a set are practically uniform. Each set comprises both new and basic items, so as to offset the influence of fluctuating conditions. A "failure" in this context is defined as any event that necessitates repair or re-tuning of the item, with enforced idleness for more than 20 seconds. The failed item is either treated in situ – or replaced by a spare, repaired and stored with the spares. Thus the size of the operative set remains 28. The assignment is – planning a truncated test with the proportions of new and basic items in the test group chosen so as to ensure a minimal ATD. Below is the planning procedure: a. As the OC's are practically the same for truncated and non-truncated tests when their Φ0, d, real and real coincide (Michlin & Grabarnik, 2007) – we chose the initial parameters given above: Φ0=5, d=2, tg=tg=0.1 (42) =10% (43) and specified RASN max Thus the test has an ASN and ATD close to that of the non-truncated SPRT, and at the same time its maximal duration is heavily restricted, a fact of practical importance for the organisation. Comparison Sequential Test for Mean Times Between Failures b. 471 Eqs. (41) and (18) yielded the approximate dependences of ATDg(Φ)/ b on Nn for different Φ, given Nn+ Nb=28. A minimum was found at Nn≈18. Figure 14 shows examples of these dependences at Φ= Φ0 and Φ= Φ1, which are seem to be almost flat over a wide interval around the minimum, and Nn=15 was chosen accordingly. With this choice, ATDg(Φ) only slightly exceeds the minimum, while the number of new items is lower, with the attendant saving in preparing the experimental batch. By (40) we have  0 g   0  13 / 15  4 1 3 (44) The values of ASNg(Φ) and ATD(Φ), obtained by (41) and (18) with allowance for (44) – confirmed the practicability of the test. c. Eq. (8) yielded s=0.330. Simultaneous solution of (29) and (30) yielded, after roundingoff, the TA coordinates: rbMax=66, rnMax=22. Eqs. (33) through (35) yielded *=0.0909, *=0.074, which in turn, by (11) and (12), yielded h'b=4.804, hn=4.453. The decision boundaries for a test planed on the basis of these parameters are shown in Figure 2. Figure 13 shows the exact values of the functions OC(Φ) and ASN(Φ) as per eqs. (13) – (18), which in turn yield the test's real characteristics: Φ0=5, d=2, real =0.098, real =0.099, RASN=9.2%, in very close agreement with the given (42) and (43) – evidence of the high accuracy of eqs. (30) – (35). Fig. 13. OC and ASN of truncated group test and of non-truncated theoretical (subscript nTr) test; normalised expected duration of group test ATDg()/b for 0=5, 0g=41/3, d=2, real=0.098, real=0.099, rbMax=66, rnMax=22. (Michlin et al., 2011). 472 Modern Approaches To Quality Control The OC(Φ) of the planned test (Figure 13) practically coincides with that of the nontruncated test OCnTr(Φ) with the same real and real. The ASN of the former is higher than that of the latter, in accordance with RASN=9.2%. The diagram also shows the estimate for the normalised ATD, i.e. the ratio ATDg(Φ)/ b. Assuming ˆb  10 hr , the time requirement of the test should be reasonable. In practice, it ended with acceptance of the null hypothesis in 16 hr, following the twenty-first failure in the basic subgroup, by which time a total of 2 failures in the new subgroup had been observed. ATDg ( )/ b 6 5 ATDg(Φ0 ) ATDg(F0) 4 ATDg(F1) ATD g(Φ 1 ) 3 2 1 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 New devices number, Nn Fig. 14. Normalised expected group test duration vs. number of new devices, for =0, and =1. (Michlin et al., 2011). 8. Conclusion The example in Section 7 demonstrated the potential of the proposed planning methodology for a truncated discrete SPRT. An innovative feature in it are the test-quality characteristics RASN and RD – which represent, respectively, increase of the ASN on truncation and closeness of the test OC to the non-truncated one. This innovation permitted comparison of different SPRT and automatisation of the optimum-choice process. It was found that over a large domain about the solution, the RASN and boundary parameters are linked monotonically and almost linearly. This implies sound choice of this characteristic and simplifies the planning. An efficacious search algorithm was developed for the optimal test boundaries, incorporating the obtained interrelationships. The findings can be summed up as follows: A truncated SPRT was studied with a view to checking the hypothesis on the ratio of  the MTBF of two objects with exponential distribution of TBF. Comparison Sequential Test for Mean Times Between Failures         473 It was established that the basic test characteristics real, real, RASN depend monotonically on the absolute terms in the equations of the oblique test boundaries. At the search limits for these absolute terms, determined in Section 5, these dependences are almost plane. real and real change stepwise with the smooth changes in the absolute terms of the oblique boundaries; expressions are derived for the minimal intervals of these terms, over which real and real remain unchanged. These and other established regularities yielded an efficacious algorithm and programme for determining the optimal location of the test boundaries. The found links between the input and output characteristics of the test, and the fastworking algorithm for its planning, permit improvement of the planning methodology and its extension to all binomial truncated SPRT. On the basis of the above body of information, regressional relationships were derived for determining the TA coordinates and oblique-boundary parameters of the optimal tests. Also derived were formulae for the limits of the latter parameters. These are very close at low d and RASN and draw apart as the characteristics increase; the reason being increasing influence of the test's discreteness. The regressional relationships and boundary-parameter limits permit quick determination of these boundaries for the optimal test with specified characteristics. The methodology is also applicable in group tests, with the attendant time economy; moreover, it permits optimisation of the respective group sizes. A planning and implementation example of this test is presented. 9. Acknowledgements The authors are indebted to Mr. E. Goldberg for editorial assistance, and to MSc students of the "Quality Assurance and Reliability" Division of the Technion: Mrs. E. Leshchenko, and Messrs. Y. Dayan, D. Grinberg, Y. Shai and V. Kaplunov, who participated in different stages of this project. The project was supported by the Israel Ministry of Absorption and the Planning and Budgeting Committee of the Israel Council for Higher Education. 10. Acronyms ADP AL ASN ATD MTBF OC≡ Pa() OTA RDP RL RMSE SPRT accept decision point accept line average sample number average test duration mean TBF operating characteristic truncation apex of the optimal test reject decision point reject line root mean square error sequential probability ratio test 474 TA TBF WAS Modern Approaches To Quality Control truncation apex time between failures or time to failure program name 11. Notations exact value of ASN for a truncated test, obtained recursively (17) ASN calculated via an analytical formula (22) for a non-truncated test ATD function for given  with the appropriate subscripts, coefficients in the approximative equations discrimination ratio d= Φ0/ Φ1 absolute terms of Accept, and Reject oblique boundaries, respectively h’b, hn Nb, Nn item numbers of "basic" and "new" subgroups in group test acceptance probability of H0 at given Φ Pa()≡OC PADP(rn,), PRDP(rb,) probabilities of reaching the given points ADP, RDP probability of new system failing next during test PR() system number of failures observed up to time T rb, rn rbADP (rn ) rb-coordinates of ADP for given rn ASN() ASNnTr() ATD() c rnRDP (rb ) rn-coordinates of RDP for given rb R2 RD Rd0 and RASN0 s T TAb, TAn x , real, real tg, tg *, * coefficient of determination relative excess of the ASN of the truncated test over its non-truncated counterpart relative deviation real and real from their targets threshold values of Rd and RASN slope of oblique boundaries current test time rb- and rn-coordinates of TA, respectively common target value for and , x= tg= tg probabilities of I- and II-type errors in test exact real values of  and  computed for prescribed stopping boundaries target values of , parameters determining the constant terms of initial boundary lines , b, n   n /b MTBF, same for the basic system true MTBF ratio Φ0 Φ1 Φg Φ value for which the null hypothesis is rejected with probability Φ value for which the null hypothesis is rejected with probability 1Φ for group test RASN   , U ,  L ,  M , U , L regressional value, upper and lower search limits of * and * M b, and for the new system n respectively 12. References Aroian L. A. (1968). Sequential analysis-direct method. Technometrics. Vol. 10, pp. 125132. Comparison Sequential Test for Mean Times Between Failures 475 Barnard, G. A. (1946). Sequential test in industrial statistics, Journal of the Royal Statistical Society. Suppl., Vol. 8, pp. 1-21. Chandramouli, R.; Vijaykrishnan N. & Ranganathan, N. (1998). Sequential Tests for Integrated-Circuit Failures, IEEE Transactions on Reliability. Vol. 47, No. 4, pp. 463– 471. Chien, W. T. K. & Yang, S. F. (2007). A New Method to Determine the Reliability Comparability for Products, Components, and Systems in Reliability Testing. IEEE Transactions on Reliability, Vol. 56, No. 1, pp. 69–76. Drenick, R. F. (1960). The failure law of complex equipment. The Journal of the Society for Industrial Applications of Mathematics, Vol. 8, No. 4, pp. 680-689. Eisenberg, B., & Ghosh, B. K. (1991). The sequential probability ratio test. In: Handbook of Sequential Analysis, B. K. Ghosh, Sen P.K (Ed.), pp. 47-66, Marcel Dekker, NY. Epstein, B. & Sobel, M. (1955). Sequential life test in the exponential case. The Annals of Mathematical Statistics, Vol. 26, pp. 82-93. IEC 61650 (1997) Reliability Data Analysis Techniques – Procedures for Comparison of Two Constant Failure Rates and Two Constant Failure (Event) Intensities. Kapur, K. C. & Lamberson, L. R. (1977). Reliability in Engineering Design. Wiley, NY, pp. 342363. Kececioglu, D. (1993). Reliability & Life Testing: Handbook. Vol. 1, Prentice Hall, NJ, pp. 133156. Mace, A. E. (1974). Sample Size Determination. Robert E. Krieger Pub. Co., NY, pp. 110114. Michlin, Y. H. & Grabarnik, G. (2007). Sequential testing for comparison of the mean time between failures for two systems. IEEE Transactions on Reliability, Vol. 56, No. 2, pp. 321-331. Michlin, Y. H.; Grabarnik, G., & Leshchenko, L. (2009). Comparison of the mean time between failures for two systems under short tests. IEEE Transactions on Reliability, Vol. 58, No. 4, pp. 589-596. Michlin, Y. H. & Grabarnik, G. (2010). Search boundaries of truncated discrete sequential test. Journal of Applied Statistics. Vol. 37, No. 05, pp. 707-724. Michlin, Y. H.; Ingman, D. & Dayan, Y. (2011). Sequential test for arbitrary ratio of mean times between failures. Int. J. of Operations Research and Information Systems, Vol. 2, No. 1, pp. 66-81. Michlin, Y. H. & Kaplunov, V. (2007). Optimal truncation of comparison reliability tests under unequal types I and II error probabilities, Proceedings of the 9th Conference of Israel Society for Quality, Tel-Aviv, Nov. 2007, 6 pp. Michlin, Y. H. & Migdali, R. (2004). Test duration in choice of helicopter maintenance policy. Reliability Engineering & System Safety, Vol. 86, No. 3, pp. 317-321. MIL-HDBK-781A (1996). Reliability test methods, plans, and environments for engineering, development, qualification, and production. US DOD, pp. 32-42. Siegmund, D. (1985). Sequential Analysis: Tests and Confidence Intervals, Springer, NY, pp. 34-63. 476 Modern Approaches To Quality Control Sr-332. (2001). Reliability prediction procedure for electronic equipment., Telcordia Technologies Inc.,. Red Bank, NJ, Section 2.4. Wald, A. (1947). Sequential Analysis, John Wiley & Sons, NY. Wald, A. & Wolfowitz, J. (1948). Optimum character of the sequential probability ratio test. The Annals of Mathematical Statistics, Vol. 19, No. 3, pp. 326-339.