Method Evaluation

Investment evaluation methods play an important role in today's competitive manufacturing environment. Shrinking profit margins and diversification require careful analysis of investments and decisions regarding these investments are crucial for the survival of the manufacturing industry. Both economic evaluation criterion and strategic criteria such as flexibility, quality improvement, which are not quantitative in nature, are considered for evaluation. Much has been written about the deficiencies of traditional models for justifying advanced manufacturing systems. The use of fuzzy set theory allows incorporating unquantifiable, incomplete and partially known information into the decision model. In this paper, an analytic hierarchical process (AHP) based on fuzzy numbers multi-attribute method is proposed for the evaluation and justification of an advanced manufacturing system. Finally, an example of machine tool selection is used to illustrate and validate the proposed approach.

Ursus arctos is a common complaint in parts of the Abruzzo region (Italy) where extensive grazing is practised. The reliability of current damage evaluation methods was found to be weak in relation to identification between canid attacks (wolf or dog) as an ultimate cause of livestock death. An enquiry into the characteristics of farming units damaged by predation showed how the problem of predation could not be viewed in isolation from the socio-economic context in which it occurred. From a sample of communes concerned over predation, only 4.1% of 563 claimants (controlling 25 farming management units) could be considered as 'chronically' affected by predation (> two attacks per year) and took up over a third of the claims. The majority claimed less than one attack per year but 36.1% of these held sheep, cattle or equids for allegedly non-commercial reasons. There is evidence for the expansion of wolves in ltaly." the availability of a variety of wild ungulates has been identified as a strong factor in diverting predator pressure from domestic livestock. This may hold true if the management methods of 'at risk' livestock categories are closely monitored and reappraised where necessary. Moreover, economic incentives such as compensation should be aimed primarily at owners most likely to enter into conflict with wildlife, owing to their economic reliance upon traditional husbandry methods.

Although the greenhouse effect is by many considered as one of the most serious environmental problems, several economic studies of the greenhouse effect, most notably Nordhaus's DICE model, suggest that it is optimal to allow the emissions of greenhouse gases (GHG) to increase by a factor of three over the next century. Other studies have found that substantial reductions can be justified on economic grounds. This paper explores into the reasons for these differences and identifies four (partly overlapping) crucial issues that have to be dealt with when analysing the economics of the greenhouse effect: low-probability but catastrophic events; cost evaluation methods; the choice of discount rate; the choice of decision criterion. The paper shows that (i) these aspects are crucial for the policy conclusions drawn from models of the economics of climate change, and that (ii) ethical choices have to be made for each of these issues. This fact needs wider recognition since economics is very often perceived as a value neutral tool that can be used to provide policy makers with "optimal" policies.

We develop a statistical method that simultaneously estimates daily survival rate and observer effect. We used Monte Carlo simulation to (1) evaluate the performance of the model, (2) compare model performance with models that ignore observer effects, and (3) evaluate methods of choosing between competing models of survival. When observer effects were absent, all models produced unbiased estimates of daily survival rate. In the presence of observer effects, however, models that ignore these effects underestimated daily survival rate. In such cases, estimates of nesting success were strongly affected even when observer effects were relatively small. In contrast, estimates of daily survival rate and nesting success produced by the model that considers observer effects consistently had little bias. However, estimates of daily survival rate from this model were less precise than those from the simpler model. Objective criteria for choosing between competing models did not perform well with sample sizes of 150 to 600 because subtle but important observer effects are difficult to detect. Likelihood-ratio tests had low power for rejecting the null hypothesis of no observer effect over a wide range of levels of observer effect and with sample sizes of 150 to 600. Estimates of daily survival rate from models selected based on Akaike's Information Criterion (AIC) had higher bias than estimates from the model that estimates observer effect when observer effect was present. Estimates from AIC-selected models had lower mean squared error than estimates from the model that estimates observer effect when observer effects were small, but the pattern reversed as effects increased. We recommend that researchers estimate observer effects using the more complex model when observer effects are possible and decide whether to use estimates of daily survival from the simpler or more complex model based on analysis results and simulation or analytic results for relevant sample sizes, daily survival rates, and observer effects. To illustrate use of the analytical techniques, we analyzed field data from Dusky Flycatcher (Empidonax oberholseri) nests monitored during the nestling stage. The observer effect was estimated to be 1.003 (95% CI 0.866 to 1.162); thus, point estimates of daily survival were very similar from the simpler (0.971; 95% CI 0.957 to 0.985) and more complex model (0.970; 95% CI 0.925 to 1.000). In this case, analysis results and simulation results indicate that the simpler model is adequate and provides an estimate of daily survival rate with small potential bias and increased precision compared with an estimate from the more complex model.

This article presents a new approach to the investigation of resident views of neighborhood using Geographic Information Systems (GIS). GIS, an interactive mapping and analysis tool, allows multiple layers of information about a given place to be represented simultaneously, thus exposing the interactions among layers, and allows the conception of neighborhood to be represented in greater complexity. The results of

The effectiveness of information retrieval technology in electronic discovery (e-discovery) has become the subject of judicial rulings and practitioner controversy. The scale and nature of e-discovery tasks, however, has pushed traditional information retrieval evaluation approaches to their limits. This paper reviews the legal and operational context of e-discovery and the approaches to evaluating search technology that have evolved in the research community. It then describes a multi-year effort carried out as part of the Text Retrieval Conference to develop evaluation methods for responsive review tasks in e-discovery. This work has led to new approaches to measuring effectiveness in both batch and interactive frameworks, large data sets, and some surprising results for the recall and precision of Boolean and statistical information retrieval methods. The paper concludes by offering some thoughts about future research in both the legal and technical communities toward the goal of reliable, effective use of information retrieval in e-discovery.

The purpose of this study was to determine whether low-intensity pulsed ultrasound (LIPUS) could enhance the regeneration of myofibers and shorten the healing time in injured muscle. NIH C2C12 cells, a wellknown myoblastic cell line, are subclones derived from the mouse myoblast cell line established from normal adult C3H mouse leg muscle. The cells differentiate rapidly and produce extensive contracting myotubes expressing characteristic muscle proteins. We exposed C2C12 cells to LIPUS therapy using the EXOGEN 20001 system ultrasound apparatus (Exogen Inc., Piscataway, NJ, USA) with a total treatment of 20 min every 24 h. At intervals of 2, 4, 6 and 8 days, cell growth was measured by the increase in cell number and western blot analysis of myogenin and actin. Forty mice (C57BL10J1/1) were divided into five groups of eight animals each and used in the published laceration injury model. The gastrocnemius muscle of the left leg was lacerated in all the animals. The control group (sham ultrasound) did not undergo LIPUS therapy. The ultrasound 7-, 14-, 21-and 28-day groups (only changing the number of days during which the ultrasound was applied to the injured muscle) were treated with LIPUS (20 min/day) for 7, 14, 21 and 28 consecutive days, respectively. All animals were sacrificed at 4 weeks after the injury. Evaluation methods included muscle regeneration and muscle contractile properties. LIPUS therapy produced a significantly higher proliferative rate and cell number at days 6 and 8 (p , 0.05). Densitometric evaluation revealed an increase in myogenin and actin proteins in cells treated with LIPUS in the 4-, 6-and 8-day groups. The regeneration of myofibers, fast-twitch and tetanus of LIPUS-treated muscles (21 and 28 days) was significantly greater relative to control muscles. There was no major strength difference between the normal noninjured muscle and the group treated with LIPUS for 28 days. In conclusion, this was the first experimental study to show that LIPUS therapy is able to enhance the regeneration of myofibers with better physiologic performance in injured mice muscles after laceration, especially prior to postoperative week 4. Findings of this study demonstrate a scientific basis for future clinical trials and establish an indication for LIPUS in enhancing muscle healing after laceration injury. (

We introduce a framework for evaluating human detectors that considers the practical application of a detector on a full image using multi-size sliding window scanning. We produce DET (Detection Error Tradeoff) curves relating miss detection rate and false alarm rate computed by deploying the detector on cropped windows as well as whole images, using in the later either image resize or feature resize. Plots for cascade classifiers are generated based on confidence scores instead of varying the number of layers. To assess a method's overall performance on a given test, we use the ALMR (Average Log Miss Rate) as an aggregate performance score. To analyze the significance of the obtained results, we conduct 10-fold cross validation experiments. We applied our evaluation framework to two state of the art cascade-based detectors on the standard INRIA Person dataset, as well as a local dataset of near infrared images. We used our evaluation framework to study the differences between the two detectors on the two datasets with different evaluation methods. Our results show the utility of our framework. They also suggest that the descriptors used to represent features and the training window size are more important in predicting the detection performance than the nature of the imaging process, and that the choice between resizing images of features can have serious consequences.

Health expectancy indices combine the mortality and morbidity experience of a population into a single composite indicator. This paper summarizes and evaluates methods for the calculation of health expectancies and presents trends in the expectation of life with disability and handicap in Australia from 1981 to 1993. Unlike other countries for which recent health expectancy time series are available, Australian results indicate that the expectation of years with disability has increased for both males and females. Possible explanations for this are examined.

Our objectives were to evaluate elution and bait plant methods to detect infectious tobamoviruses in forest soils in New York State. Soils were collected from two forest sites: Whiteface Mountain (WF) and Heiberg Forest (HF). The effectiveness of four buffers to elute tomato mosaic tobamovirus (ToMV) from organic and mineral fractions of WF soil amended with ToMV was tested, and virus content was assessed by enzyme-linked immunosorbent assay (ELISA). The effectiveness of Chenopodium quinoa (Willd.) bait plants to detect the virus also was tested. Both methods then were utilized to detect tobamoviruses in 11 WF and 2 HF soil samples. A phosphate buffer (100 mM, pH 7.0) eluted more ToMV from soil than the other buffers tested. Mineral soil bound more virus than organic soil. Virus recoveries from virus-amended organic and mineral soils were 3 and 10%, respectively, and the detection sensitivity was 10 to 20 ng/g of soil. Roots of bait plants grown in all virus-amended soils tested pos...

With the rise of Web-based applications, it is both important and feasible for human-computer interaction practitioners to measure a product's user experience. While quantifying user attitudes at a small scale has been heavily studied, in this industry case study, we detail best Happiness Tracking Surveys (HaTS) for collecting attitudinal data at a large scale directly in the product and over time. This method was developed at Google to track attitudes and open-ended feedback over time, and to characterize products' user bases. This case study of HaTS goes beyond the design of the questionnaire to also suggest best practices for appropriate sampling, invitation techniques, and its data analysis. HaTS has been deployed successfully across dozens of Google's products to measure progress towards product goals and to inform product decisions; its sensitivity to product changes has been demonstrated widely. We are confident that teams in other organizations will be able to embrace HaTS as well, and, if necessary, adapt it for their unique needs.

We present a new evaluation methodology to better evaluate video processing performance. Recent evaluation methods [10], [9], [11] depend heavily on the benchmark dataset. The result may be different if we change the testing video sequences. The difference is mainly due to the video sequence content which usually includes many video processing problems (illumination changes, weak contrast etc.) at different difficulty levels. Hence it is difficult to extrapolate the evaluation result on new sequences.

Although the greenhouse effect is by many considered as one of the most serious envi- ronmental problems, several economic studies of the greenhouse effect, most notably Nordhaus's DICE model, suggest that it is optimal to allow the emissions of greenhouse gases (GHG) to increase by a factor of three over the next century. Other studies have found that substantial reductions

In this paper a novel method was presented for evaluating the fabric pills using digital image processing techniques. This work provides a novel technique for detecting pills and also measuring their heights, surfaces and volumes. Surely, measuring the intensity of defects by human vision is an inaccurate method for quality control; as a result, this problem became a motivation for employing digital image processing techniques for detection of defects of fabric surface. In the former works, the systems were just limited to measuring of the surface of defects, but in the presented method the height and the volume of defects were also measured, which leads to a more accurate quality control. An algorithm was developed to first, find pills and then measure their average intensity by using three criteria of height, surface and volume. The results showed a meaningful relation between the number of rotations and the quality of pilled fabrics.

This paper presents the structure and activitities of the recently established Competence Center in Speech and Language Technology in Saarbrücken. The objectives of the Competence Center are to provide a comprehensive information service about speech and language technologies, including live demonstrations of the most important language technology (LT) systems, and to advance the state of the art in the evaluation of LT systems for real-world applications. The Competence Center comprises the following components:

Development of techniques for evaluation of the tissues structure is one of the main tasks in medical diagnostics. At present time the ultrasound imaging technique is widely used for qualitative evaluation of tissue structure. Tissue evaluation methods are using A, B, M scans of tissues. Significant part of valuable information is lost by using common diagnostic methods. New technique for tumours tissue structure evaluation using ultrasound radiofrequency signals (RF) is presented in this paper. Echospectral methods allow get additional information about tissue structure, gives more information about treatment process. The mean spectral frequency, the width of power spectrum and effective value of correlation function are used for tumours tissue characterization during brachytherapy treatment.

The evaluation of instructional products is widely recognized as an essential activity of any teaching-learning process. Instructional products, such as online courses, training programs, tutorials, instructional modules, course management systems, and web-based study guides, are required to be evaluated during their development, usage or at the end of the learning process. This kind of evaluation is not a simple process. It demands the use of resources, such as people, tools, methods and time, which must be properly planned and controlled to ensure the efficiency and effectiveness of the process. We describe, in this paper, an evaluation method for guiding educators, teachers, and instructional designers to evaluate different types of instructional products. The proposed method intends to solve problems that were identified in the existing evaluation methods, such as the lack of generality, adaptability and manageability. This method is general, complete, technique independent, well-structured and adaptable. Its design was based on principles and concepts taken from Method Engineering and Software Engineering. It is composed of a process model, a product model, and a team model. These three components are properly integrated in order to effectively guide an evaluation team during the process of evaluating instructional products. Its main contribution is in providing a general methodological framework that can be easily adapted to evaluate a variety of instructional products.

Receiver operator characteristic (ROC) analysis is an emerging automated target recognition system performance assessment tool. The ROC metric, area under the curve (AUC), is a universally accepted measure of classifying accuracy. In the presented approach, the detection algorithm output, i.e., a response plane (RP), must consist of grayscale values wherein a maximum value (e.g. 255) corresponds to highest probability of target locations. AUC computation involves the comparison of the RP and the ground truth to classify RP pixels as true positives (TP), true negatives (TN), false positives (FP), or false negatives (FN). Ideally, the background and all objects other than targets are TN. Historically, evaluation methods have excluded the background, and only a few spoof objects likely to be considered as a hit by detection algorithms were a priori demarcated as TN. This can potentially exaggerate the algorithm"s performance. Here, a new ROC approach has been developed that divides the entire image into mutually exclusive target (TP) and background (TN) grid squares with adjustable size. Based on the overlap of the thresholded RP with the TP and TN grids, the FN and FP fractions are computed. Variation of the grid square size can bias the ROC results by artificially altering specificity, so an assessment of relative performance under a constant grid square size is adopted in our approach. A pilot study was performed to assess the method"s ability to capture RP changes under three different detection algorithm parameter settings on ten images with different backgrounds and target orientations. An ANOVA-based comparison of the AUCs for the three settings showed a significant difference (p<0.001) at 95% confidence interval.