Criteria for evaluation of proposed protozoan detection methods

Introduction

complex and difficult to perform, yielding highly variable results, demonstrating differential response Currently, the only US EPA approved protozoan in a variety of water matrices, and being costly method is the 'ICR Protozoan Method for Detecting (Clancy et al., 1997;Schaefer, 1997). Alternatë Giardia cysts and Cryptosporidium oocysts in water methods have been proposed and preliminarily testby a fluorescent antibody procedure ' (Fout et al., ed. 1996)

under the Information Collection Rule (ICR)

The intensive effort in development of methods (61 Federal Register 24354). The ICR protozoan and method components has resulted in a number of method usually underestimates the levels and occur-procedures that have not been tested in a consistent rence of Giardia sp. and Cryptosporidium sp. More-manner. Because many of these methods have not over, false positives and false negatives have been been validated in a multi-laboratory, collaborative found during testing in a number of laboratories. The format, there is an acute need for a procedure for ICR method has also been criticized for being evaluating the effectiveness of proposed methods. Great differences exist in the procedures used for testing of proposed methods. There are significant *Corresponding author. Tel.: 1 1-513-5697192; fax: 1 1-513variations in the water matrices, the concentration of 5697117.Ë -mail address: [email protected] (H.D. Alan Lindquist) protozoan cysts, and / or oocysts, the method for 0167-7012 / 99 / $ -see front matter © 1999 Elsevier Science B. V. All rights reserved. PII: S0167-7012( 99 )00039-1 enumeration of these protozoan spikes, the quality of sponses to a variable, and that good scientific the spike material, and even the descriptions of the judgement should be used when assigning appromethods that have been provided by developers. As a priate scores to methods that fall outside the parameresult, it is very difficult to compare the data that ters of the response set for any criterion. have been developed.

A logical set of criteria must be established to conduct preliminary evaluation of proposed methods 2.1. Requirements for data generation to measure to determine if further testing or investigation is the statistical performance warranted. This type of framework is proposed here, allowing the expeditious screening of methods. It is A variety of statistical measures are required to designed to be robust enough to allow comparison of evaluate each method. They must be reported on the widely divergent methods. The framework includes a basis of having conducted the entire method from table, to be filled in by evaluators, of important sampling to detection and identification to have characteristics of the proposed methods. The evalua-validity or usefulness for comparison. All parameters tion criteria also comprise a set of necessary bench-in the statistical performance of the method must be marks for method and technique developers to based on at least ten spiked samples and one blank accomplish before consideration for further testing.

sample prepared in accordance with the following Knowledge of this framework will allow method guidelines. developers to collect the relevant data once a method All spiked samples and blanks are to be prepared has been developed.

using reagent grade water which is either distilled The technical criteria for evaluation of the per-deionized or double distilled water. This water formance of each method is based on the use of a should have a resistivity of 18 milliohms or greater, defined water matrix with defined cyst or oocyst and be substantially free from bacterial contaminapopulations, at a specified spiking level, and a tion by filtration through a 0.2-micron pore sized standardized method of spike enumeration. All of filter. Sterility of this water must be checked at these criteria are reasonably obtainable. Although C.

regular intervals by culture. The sole exception to the parvum and G. lamblia are specifically addressed in use of reagent grade water is the case wherein the the criteria proposed here, it should be possible to absence of particulates is an interference to the use these criteria to evaluate proposed methods for correct operation of the proposed method. In this other protozoa. Adaptation of these criteria for other case, chemically defined particulates may be added protozoa will have to include species specific param-at a defined rate to allow the method to work. eters for preparation of inocula for spiking experi-Spike material must be obtained from a traceable ments, and may require modification of other criteria source of laboratory animal-derived cysts andä s well.

oocysts. A traceable source of oocysts should be reasonably available to all qualified researchers, should maintain documentation of the species of 2. Methods animal from which the parasites were initially isolated, the species of laboratory animals within which A set of criteria defining significant technical the parasites have been passed, and the methods used parameters and characteristics to be used as evalua-in propagation of the parasite. Parasite spike material tion measures for proposed methods was developed should be purified to a nearly homogeneous populato determine if further testing of proposed methods tion of monodispersed parasites with a final processshould be undertaken. These parameters were sepa-ing method of cesium chloride purification (Arrated into criteria with each criterion being assigned rowood and Donaldson, 1996) or equivalent. Cysts a number of response levels designed to be inclusive of G. lamblia must be less than 3 weeks old and of the possible set of responses. It is understood that oocysts of C. parvum must be less than 6 months it may not be possible to account for every contin-old. While these organisms should not be preserved, gency when designing a comprehensive set of re-they should be stored in a medium designed to retard the growth of bacteria, for example, reagent grade analyzing the data, in an attempt to operate the test water containing antibiotics.

in a blind fashion. In the event that the spike for anÿ All protozoan cysts or oocysts must be enumerated sample is prepared by an individual who is the by hemocytometer count, or by another method analyst for that sample, the fact that the test was not demonstrated to give results of equal or greater conducted in a blind fashion must be reported. precision and accuracy than hemocytometer count-This procedure is not designed for the specific ing. Enumeration by hemocytometer refers to materi-testing of method components. If method compoal being counted using a bright line hemocytometer nents are to be tested, they should be included in a designed for phase optics. After the hemocytometer complete method. An example of this would be to chamber is filled with an appropriate volume and the include a new processing step into Method 1622 hemocytometer placed on the microscope stage, it (EPA-821-R-97-023, 1997), using one of the recmust be allowed to settle for 2 min, then protozoa ommended filtration devices, the new method comare counted using phase contrast optics at no less ponent for processing, and one of the All of the technical parameters must be adhered tö and not more than 120, cysts and oocysts must be in testing to develop statistical parameters. Failure to counted per hemocytometer platform. If cysts and adhere to these guidelines will result in a score of 0 oocysts are not monodispersed, then the cyst or for all technical statistical parameters. oocyst preparation must be treated to provide for monodispersion (i.e. by addition of 0.01% (v / v) 2.2.1.1. Percent recovery Tween 20 and vortexing for 2 min) and recounted.

The percent recovery is the percent of the initial The average of six hemocytometer platforms counted spike dose recovered at the end of a method trial. In in this way is taken and used to calculate the dilution order for percent recovery to be valid, it must be required to prepare a concentration of ten cysts noted as a method percent recovery in which thë and / or oocysts per 1 liter. A suitable portion is then spike was added to a water sample, and the entire added to the sample of reagent grade water. The final method performed, from initial concentration to final spike dose is calculated to be ten cysts and / or sample enumeration. A spiked sample that is used tö oocysts per liter, and the minimum sample volume test the efficacy of a method component may provide should be 10 l, although larger volumes may be a component percent recovery, but not a method tested if the method design is intended for use with percent recovery. larger volume applications. Spiked water samples must be used within 12 h of enumeration of the spike Not reported or 0%: 0 material.

, 25%: 1 Well-documented raw data should be available for 25-# 50%: 2 inspection at the time a method is presented. If this 50-# 75%: 3 type of data is not available, the evaluation of 75-100%: 4 statistical parameters will not be possible.

In addition, for every ten samples analyzed by the 2.2.1.2. Limit of detectionc omplete method, at least one blank must be in-

The number of cysts and oocysts that may be cluded. Blanks may be used for statistical signifi-reliably detected per unit volume reported as number cance only if at least ten blanks are used. Care must cysts and oocysts per liter by species is the limit of be taken to intersperse blanks and positive samples.

detection. If possible, all samples and the blank should be prepared by an investigator other than the individual Not reported or . 10 000 / l: 0 10 000-. 1000: 1 of certain test types that confirmation may be easier 1000-. 100: 2 or more difficult. This difference may change or be 100-. 10: 3 magnified as the matrix becomes more complex. , 10: 4 Many proposed methods will not have addressed the matrix effect or ability to confirm presumptive 2.2.1.3. Precision objects. When only known positive material is added, Precision relates to repeatability, and is the percent the specificity is the ratio of confirmed to presumpcoefficient of variation (standard deviation as a tive objects as a percent. percent of mean) for the method when performed in its entirety using spiked samples.

Not reported, or 0%: 0 , 25%: 1 Not reported, or $ 100%:

. False positive rate

The false positive rate is the apparent detection of 2.2.1.4. Lower 95% confidence limitc ysts and oocysts in known blank samples. Since, in This confidence limit represents the minimum some samples, less than 100% of the sample volumë number of cysts or oocysts that will be detected in 95 may routinely be analyzed, this is generally a of 100 trials if the method were to be performed presence / absence criterion. This is reported as the using the conditions specified in this document. The number of positives detected in blank trials randomly limit is calculated from the sample mean using the interspersed with positive trials. central limit theorem and is a single tailed test derived from the percent recovery data. An example Not reported, or blank samples not performed, calculation of this would be: the average recoveryor false positives $ 20%: 0 [1.645 3 (the standard deviation of the recovery / the False positives present but fewer than ten blank square root of the sample size)].

samples were analyzed: 1 No false positives present, but fewer than ten Not reported, or # 0: 0 blank samples analyzed: 2 . 0-25: 1 More than ten blanks analyzed, and percent of . 25-50: 2 false positives . 5%, and , 20%: 3 . 50-75: 3 More than ten blanks analyzed and percent of plified by the ASTM format (D 2777-96, 1996). If All of the cysts and oocysts in an inoculated sample possible all statistical measures should be derived of laboratory grade distilled water should in fact be from collaborative testing. In the earliest stages of cysts and oocysts. Therefore specificity should be a testing, there should be at least confirmation of measure of the ability to confirm the identity of cysts methods and results by independent laboratories. or oocysts. For example, in the ICR method, it may not be possible to confirm by differential interference contrast (DIC) all objects that react with the fluores-No independent laboratory testing undertaken: 0 cent antibody and are therefore only presumed Independent laboratory confirmation underway: 1 positive. This presumptive nature may confound Independent laboratory confirmation undertaken analysis of the results of the method. It is the nature successfully: 2 reference method for demonstrating viability is paradiamidino-2-phenyl-indole dihydrochloride (DAPI) site proliferation via infecting susceptible animals staining in Method 1622 is also intended as ar esulting in cyst or oocyst production, or death of the confirmatory technique. All antibody based methods host due to parasite proliferation, or detection of reported to date have shown some level of cross infective forms that have proliferated in the approreactivity with other species. Because of the difficulpriate tissue on necropsy. Surrogate viability inty of confirming protozoan presumptive identificadicators, are tests that have been tested alongside the tion, and the lack of specificity of antibody based reference method and have been found to be corremethods, any proposed complete method must inlated with the reference method. Most surrogate clude some mechanism of confirmation of taxonomic viability indicators, may be found to deviate from status, beyond antibody recognition. Any new, prothis correlation under certain specific conditions. If posed method of taxonomic confirmation other than the method is not designed to give viability inenhanced contrast microscopy should be validated formation, then this column should be left blank. against DIC microscopy in a multi-laboratory study.

The method to identify an organism to genus is Viability data indicated by a surrogate viability not confirmed by multi-laboratory study, or it indicator that has not been validated against the does not include a method to confirm the reference standard (animal infectivity model): 0 identification of the organism, or the method Surrogate viability indicator tested against one does not detect all strains of the target species defined set of conditions and poorly correlated known to be infective to humans: 0 to the reference standard, or found to be poorly Identification to genus of the organism(s) is by correlated to the reference standard under connon-selective methods: 1 ditions expected in the type of sample analyzed: 1

Organisms of interest are identified to genus Surrogate viability indicator data validated with a selective test (that selectively identifies against reference standard, under one defined target organisms of interest from background set of conditions: 2 particles), and an independent confirmatory test: 2 Surrogate viability indicator data validated Two independent testing mechanisms, at least against reference standard using numerous conone is valid to species level identification, and ditions, correlating well with the reference at least one is a selective test: 3 standard in several of these conditions: 3

Surrogate viability indicator data validated filter sample at sampling location, or at the laboraagainst reference standard, 1:1 correlation found tory. Other special limitations on sampling should be in all of a broad range of conditions tested, or described here. method includes the reference test: 4 Not described, or described in general terms, or 2.2.4. Method description described with no test data presented, or well The description of the method must be thorough described but would not allow testing in a enough to allow evaluation of the completeness, suitable range of water types / volumes for availability and practicality of the method. practical use: 0

Not completely described, some test data pre-2.2.4.1. General description sented, full range of parameters unknown: 1 The general description should be sufficient to allow a scientist or technician, unfamiliar with the Well described would allow water testing in an method to perform the method and achieve results acceptable range of conditions: 2 comparable to those reported by the developers. The

Tested by multiple laboratories, demonstrated to format used is to be determined by the method be effective with either source or drinking developer, but should include: scope and application, water: 3 summary of method, definitions, interferences, safety, equipment and supplies, reagents and standards,

Tested in a collaborative multi-laboratory study sample collection, preservation and storage, quality demonstrating that sampling of both source and control, calibration and standardization procedure, drinking water yields similar statistical recovery data analysis and calculations, method performance, results: 4 pollution prevention, waste management, references, tables, diagrams, flowcharts, and validation data. The description should serve as a complete, self con-2.2.4.3. Managerial criteria tained instruction set for evaluating a source or This element is intended to encompass the range drinking water sample, from sampling through analyof managerial concerns that may arise during the sis, interpretation and reporting of results. Failure to implementation of methods, either in a collaborative include any of these steps will result in a finding that multi-laboratory effort, or during actual implementathe method description is incomplete.

tion of a method. These managerial considerations should be addressed within the methods description, Incomplete, not given: 0 however, since these considerations may exceed the Well described but not complete: 1 requirements of the methods description format Apparently complete enough to allow indepenchosen by the developer, they may be included in a dent testing: 2 separate document. This information is to include a Complete, tested by one independent laboratory: 3 full description of: personnel required for method Complete, tested by independent laboratories, performance including skill levels as well as training and it is written in a format recognized by the requirements for the personnel performing the meth-U.S. Environmental Protection Agency, or by od broken down by technique where appropriate, a an international consensus methods validation list of equipment and supply providers and whether organization (such as the American Society for these providers meet industry quality assurance Testing and Materials or AOAC International): 4 standards, and the open market availability for any critical or proprietary components.

Sampling

The section on sampling must include information If none of this information is given or it is not as to the range of sample volumes within which the documented: 0 method will perform as described including sample holding times and preservation techniques, ability to If this information is partially complete, or any supporting documentation is only partially numerical score should have relevance to both the available: 1 suitability of the method for use and for the state of preparedness of the method at the time of scoring. If all points are addressed, but some of the The higher the numerical score, the better the information is given in non-quantitative fashion, likelihood that the method is prepared to a sufficient or includes general or anecdotal evidence: 2 state of readiness to be tested against the current If all points are covered, but there is no reference method. evidence of independent confirmation of the information, or some of the information: 3 2.4. Case study If all points are thoroughly addressed, and

In this paper, data from trials of Method 1622: supported by independent confirmation, and all Cryptosporidium in water by filtration / IMS / FA critical method components are readily available (Clancy Environmental Consultants, Inc., 1997) and on the open market: 4 the description of this method (EPA-821-R-97-023, 1997) were critically evaluated to categorize this 2.2.5. Likely candidates for future research procedure with respect to the criteria presented here.

Certain methods or method components may not

Method 1622 is actually a menu of options designed be available for immediate use, but might be likely to be flexible enough to encompass a variety of candidates for future use. These may include methconditions. These data were not specifically generods wherein the current equipment / supply sources ated in response to this criteria set, but use of these are limited in some way, but may become available data will serve to demonstrate how data collected in the future, or methods providing particular promduring the development of a method can be utilized ise but still lacking critical elements that would allow for this criteria set. The report from which the a decision for further evaluation to be made. These statistical parameters were derived tested several of candidates should be denoted by a non-numeric the options available within the method. Each of symbol. This denotation indicates that the method is these is treated as a separate method for statistical unsuitable for use as an alternative to existing analysis. method / methods at this time.

Because the nature of the report from which the methods data was extracted was not to address the 2.3. Method of scoring criteria set in question, the data set was examined twice. The first was a strict evaluation of the data set The evaluation of proposed methods should be according to the criteria, the second was less restriccarried out without preference or prejudices. This tive, for demonstration purposes only. In practice, can be best accomplished by keeping the identity of using the criteria set to evaluate trials that have not the proposer blind to the evaluator. It is necessary met the minimum requirements for statistical evaluathat the basis for evaluation be predetermined, and tion will lead to incorrect results, but this step was defined as strictly as possible. A separate evaluation taken in this paper for demonstration purposes. sheet should accompany each evaluation to explain Method identifier 1 reports the results of a statistithe rationale for the evaluation. The evaluation cal analysis of the method using the criteria set out in should allow for proposed methods to be sent back to this document. Method identifiers 2, 3 and 4 are the investigator for correction, allowing the inves-given for demonstration purposes, showing what the tigator sufficient information to determine whether results would have been if there had been at least ten the method development effort should be continued trials of each method, and the trials had been or redirected.

conducted at a inoculation density of 10 oocysts per Any score of 0 on any item results in the method liter. Method identifier 2 reports using the Gelman being automatically returned to the developer(s) for filter cartridge method as described in Method 1622.

correction. All methods that have scored in each

Method identifier 3 reports the results of data from category are then compared for numerical score. The use of the flat plate membrane. Method identifier 4 Table 1 a Table to be filled in by evaluators, containing evaluation results for the method identifiers 1-4 as referenced in the text MID SPER CT DATA MDSC FUT TOT %R DL PR CL SP FP OR VI GD SM IN ES MC 1 0 0 0 0 0 0 3 3 4 2 2 4 1 * 1 9 2 3 0 4 3 0 2 3 3 4 2 2 4 1 * 3 1 3 4 0 4 4 0 2 3 3 4 2 2 4 1 * 3 3 4 3 0 4 3 0 2 3 3 4 2 2 4 1 * 3 1 Max 56 a MID, method identifier; SPER, statistical performance; %R, percent recovery; DL, limit of detection; PR, precision; CL, 95% confidence limit; SP, specificity; FP, false positive rate; CT, collaborative testing; DATA, nature of data generated; OR, organism; VI, viability; MDSC, method description; GD, general description; SM, sampling; IN, interferences; ES, equipment and supplies; MC, managerial criteria; FUT, likely candidates for future; * indicates a method that is a likely candidate for future research; TOT, total points. reports the results of trials of the vortex flow to the developers for further data collection activity filtration (VFF) method, using either gravity or that may lead to the method being evaluated at a positive pressure feed and a variety of spiking rates.

later time. The score for the detection limit criterion for Method 1622 is 0 due to the lack of supporting data.

Results

For all method data collected, oocyst stock preparations were enumerated by a well slide IFA pro-Application of the criteria to Method 1622 is cedure, demonstrated by the report authors in a represented in Table 1. A summary table of the separate document to provide equal or less variability values used to determine the statistical parameters than hemocytometer counting, in the report authors' are given in Table 2. However, none of the methods laboratory. Blank controls were analyzed with each within Method 1622 was conducted ten times with experiment. None of these controls were ever posian inoculation density of 10 oocysts per liter. This tive. led to the score in method identifier 1 to be '0' for all

The three methods were compared again assuming statistical measures. This method should be returned that they had met the conditions for analysis and that the statistical parameters would have remained con-objectively evaluate method performance. This stant if the conditions for analysis required by the criteria set includes both technical and managerial evaluation criteria had been followed. In practice the components. These criteria are designed to evaluate conditions for analysis should never be neglected in entire methods. Individuals wishing to have specific this way, as this will lead to incorrect conclusions.

components of a method evaluated under these The vortex flow filtration was conducted under criteria must include those components in the context two conditions, using either gravity feed, or pressure of a complete method. This is an implicit recognition feed for the water sample, with three levels of of the fact that any method component, however spiking. The ability to accumulate this data from promising, does not exist independently of the entire trials with slightly different operational parameters as method in which it is used. allowed by the method description, demonstrates the The criteria selected are not comprehensive. Howflexibility of this criteria set. Judgment must however ever, given the state of protozoan methods that have be used to ensure that the data from radically been proposed, these criteria represent a minimum different iterations of the same test is not accumu-standard required for a method to receive further lated as this may bias the analysis.

consideration. One example of a criterion that was The increments for the response range for the purposely not included in this criteria set is cost. percent recovery criterion may appear to be so large This criterion was eliminated because cost is, in as to be unusable, but the three options comprising theory, related to supply and demand and a favorable Method 1622 demonstrated that even closely related rating with this criteria set may then increase the methods can be differentiated using this broad scale.

demand for proprietary components of the method, The precision criterion in this case is not sufficient thus affecting cost, and perhaps resulting in a to distinguish between the three methods analyzed in lowering of the rating of the method. This type of the case study. The specificity was reported as zero feedback loop would be detrimental to the applicabecause this information was not reported for these tion of the criteria. method trials. There were no false positives detected,

As the number of trials of a method increases all but only one blank analyzed with each trial.

of the statistical measures of a method should A multi-laboratory collaborative test is underway approach the true measure for the method. Thus for this method, and should provide more and higher there may be some advantage for a method developer quality data for further analysis of this method. The to conduct numerous iterations of a method to data type indicators, indicate that the method is achieve optimum precision. This is a good practice, currently implemented for a single species with some as it will help to eliminate those unlikely occurrences type of confirmation of that species identification.

wherein the first few trial runs of the method might There is no information on viability provided by the provide data uncharacteristic of the method. method, and so this criterion is left blank.

The lower 95% confidence limit parameter is The management criterion is the same for all of related to the concept of sensitivity. The sensitivity the iterations of Method 1622 because the same of a test can be determined only by a series of tests information is reported for each. Method 1622, and at or near the limit of detection to determine the the data examined to generate this document, has lower 95% confidence limit at this level. The lower only limited information as to the skills and person-bound on the limit of sensitivity is one organism in a nel requirements for method performance.

given volume of water. Without performing the actual experiments on samples inoculated with onë oocyst, it is not prudent to extend data from a concentration range to cover the extreme lower limit 4. Discussion of concentration. However, it is also important to know, at a given concentration of organisms and There are two independent sets of conclusions to volume of water, what percentage recovery is to be be drawn from the results presented in this paper.

expected. The lower 95% confidence limit calculates First, it is possible to establish a set of criteria to this value. This value begins to address the issue of sensitivity, and may be thought of as a sensitivity In addition to allowing for comparison of methmeasure in cases of methods wherein the minimum ods, this criteria set illustrates a paradigm for method detection limit is ten organisms per liter.

development. Previously, methods developers have, Specificity was not reported from the data collec-on occasion, developed a technique and substituted tion effort used for this case study. This demon-this into the ICR method, to replace a component of strates the necessity for understanding the type and that method and lead to a marginal improvement in ranges of criteria that will be used to evaluate a data the ICR method. This criteria set makes clear that an set. It is possible that specificity was actually de-entire method may be developed, using whatever termined in the course of the data collection, but was techniques or methods components are available or not reported. Failure to know that this would be a efficacious. The intended result of this is that methcritical parameter may have resulted in this infor-ods developers having developed a technique that mation not being reported in the case study.

could improve detection but perhaps without re-The false positive rate for this data set also sources to develop an entire method, should evolve suffered from the lack of sufficient data being their technique into a full method by selecting from collected to satisfy reporting requirements. If more commercially available method components, the blanks had been analyzed, it is possible that these ones that lead to optimal recovery. The criteria are methods could have scored higher for this parameter.

designed to be broad enough to evaluate methods Eventually, in practice, a false positive may be used that are entirely novel. to indicate a contamination event, and a well written

The burden on regulators and granting agencies quality control procedure would require a re-evalua-should be reduced if they apply these criteria to tion of samples taken during the period of time when proposals for development, testing or application of the false positive or false negative on a quality particular methods. The criteria may be used as a control sample occurred.

preliminary screen when wholly developed methods The effect of physical, biological, and chemical are to be compared. They may be used as an variations in the water matrices and parasite strain endpoint if preliminary development of a method is differences are not evaluated by this criteria set, but the desired end of a project. Or they may be used as left for future testing. These are crucially important a yardstick to compare methods used to study the technical parameters, however, when attempting to occurrence of protozoans in water. rapidly select from a large number of potential It is not possible to include in any single documethods, it is prudent to select standards that are ment, or set of criteria, all of the nuances of the reasonably obtainable by existing proven tech-application of those criteria. Evaluation of methods nologies.

is ultimately an exercise in decision making, which These criteria are not intended to resolve all the involves factors other than a strictly technical evaluaissues in methods selection. For example, it may be tion of the methods. These criteria are intended only possible that further testing uncovers that one meth-to help ensure that each method has an opportunity od that works well in reagent water fails in water for a scientifically sound evaluation of its respective containing particulates which are rich in clay, while technical merit. another method working well in both reagent and

The second conclusion to be drawn from these clay laden water fails in water with iron rich results is that although the data in existence for sediment. In cases like these, further testing should Method 1622 indicates that the method is promising, lead to a more rational basis for administrative it is incomplete, and more data must be gathered decisions. These decisions may include seeking a before judging it satisfactory. Critically important are method that is applicable in all situations. Alter-data on the ability to confirm organisms detected, natively the decision may be made to require that the and data on personnel and skills required to accommethod to be used in any given situation be chosen plish the method. It is also necessary to perform each on the basis of certain critical parameters of the method at least ten times with a inoculation rate of water matrix to be tested.

ten oocysts per liter.