Characterising uncertainty in plume dispersion models

9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes 4.10 CHARACTERIZING UNCERTAINTY IN PLUME DISPERSION MODELS John S. Irwin*1 and Steven R. Hanna2 1 NOAA Atmospheric Sciences Modeling Division, RTP, NC 27711 USA 2 Hanna Consultants, Kennebunkport, ME, 04046 USA INTRODUCTION In a discussion on predictability, H. Tennekes stated that , “Because the atmosphere is a chaotic system in which all forecasts are divergent, particularly as far as the smaller scales of motion are concerned, we must insist that no forecast is complete without a preceding assessment of forecast skill. In the same spirit, no observation is complete without an appropriately sampled estimate of the variance of the properties observed, and no model calculation is complete without a calculation of the variance of the calculation.”(see Hooke et al., 1990). In the current paper, we provide quantitative estimates of some of the major sources of uncertainty in plume dispersion modeling (e.g., variance of the model predictions) and then provide a preliminary assessment of their effects. The specific sources of uncertainty that are investigated are stochastic effects in the crosswind concentration profile, plume dispersion parameters, plume rise, and transport wind direction and speed. DISCUSSION Crosswind concentration profile variations The widely-used Gaussian approximation for characterizing the crosswind distribution of mass of a dispersing plume as it is carried downwind provides a smoothed viewed of what is really seen in the world. Irwin and Lee (1996) analyzed the Prairie Grass data, as well as additional tracer data from the Kincaid power plant, which had a 183-m stack with a typical buoyant plume rise on the order of 200 m. They concluded that the scatter in the concentration values about the ensemble average Gaussian lateral profile can be characterized for both experimental data sets as having a log-normal distribution with a geometric standard deviation (GeoSD) on the order of 2. The SCIPUFF model (Sykes et al., 1998) is one widely-used plume model that explicitly solves for the fluctuations in concentration internal to the plume as described above. Typically, the relative fluctuation (standard deviation divided by the mean) is simulated to be about 2 on the plume centerline, and is larger towards the edges of the plume. The SCIPUFF estimates of uncertainties are consistent with what has been independently found by the authors, as discussed in the previous paragraph. Note, SCIPUFF simulates additional sources of uncertainty which we have not investigated (e.g., uncertainty due to mesoscale wind fluctuations using inputs of wind speed variance and Lagrangian mesoscale integral scale, and uncertainty if the plume is located far from the wind observation site). Dispersion parameter uncertainty Irwin (1984) calculated the bias in the dispersion parameter estimates, and observed that the bias varied from one site to the next, and also calculated the random errors about the systematic bias at each site. To further explore these uncertainties, an analysis was conducted of the field experiments from 26 different sites listed and discussed in Irwin (1983). Nine of these sites involved elevated releases and the other 17 sites involved near-surface releases. The data were divided into four (4) groups: 1) elevated vertical dispersion values (5 sites), 2) * On assignment to the Office of Air Quality Planning and Standards, U.S. Environmental Protection Agency - 287 - 9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes elevated lateral dispersion values (4 sites), 3) near-surface vertical dispersion values (6 sites), and 4) near-surface lateral dispersion values (11 sites). For each experiment within each of the four groups we: 1) computed the average and geometric mean of ratio P/O, where P is the predicted and O is the observed dispersion, and 2) computed the standard deviation and geometric standard deviation of P/O ratio values. For the current analysis, Model 3 as described in Irwin (1983) was used for the predictions. Table 1 summarizes the results obtained from the analysis described. A log-normal distribution was seen to be a reasonable characterization for all of the random error distributions, even though a normal distribution is seen to be indicated at 10 experiment sites (see notations in Table 1). We looked to see if a large bias in the P/O ratio correlated with larger scatter in the random errors as reflected in the geometric standard deviation values, but such was not seen. We looked to see if the bias or geometric standard deviations in the vertical and lateral dispersion were correlated but such was not seen. If we assume that the random biases and random errors come from independent log-normal distributions, we can model the uncertainty by expressing the dispersion parameters as σ y ,z = by ,z ⋅ ry ,z ⋅ σ yo,z , where the subscripts y and z respectively refer to the lateral and vertical dispersion, b and r are random bias and error factors, σ yo,z is the model’s estimate of the dispersion, and σ y ,z is the observed dispersion, including the effects of uncertainty. We can use the Table 1 results to characterize the distributions of b and r. We can characterize the 26 biases as a log-normal distribution with a GeoSD of 1.35 (e.g., b y , z ), and we can characterize the 26 GeoSD values by their average, 1.51 (e.g., ry , z ,). Note, a log-normal distribution with a GeoSD of 1.50 means 90 %of the values are within a factor of 2. Hanna’s (2002) informal expert elicitation concerning uncertainties in σy and σz and his reanalysis of Draxler’s (1984) observations of σy and σz in many field experiments confirm the uncertainty magnitudes suggested above. Six developers of widely-used Gaussian dispersion models were asked to estimate the uncertainties, expressed as 90 % confidence bounds, in σy and σz. The six experts agreed that the uncertainties in both components were about a factor of two to three for hourly-averaged observations. Draxler (1984) provides reviews of field observations of dispersion from many sites. The figures in Draxler’s (1984) chapter were analyzed by Hanna (2002) to determine the range of the scatter of the points in the σy and σz plots. For the 18 figures with relevant data, there is seen to be a consistent factor of about 10 (i.e., an order of magnitude) range of variation in the plotted points from a best-fit line across all the figures. That is, for a given median σy or σz estimate (as from a best-fit line) at a given downwind distance or travel time and for a given stability, the observed σy or σz values cover an order of magnitude range. This corresponds to an uncertainty of about a factor of 3. A range of a factor of five would encompass nearly all of the observations, so we recommend that “random” σy or σz values should be no more than a factor of five away from the median value. The probabilities of values inside the factor of five range can be increased by the area of the CDF curve outside of the factor of five range. Furthermore, as stated above, the 90% range (i.e., from the 5th to the 95th percentiles) is approximately a factor of two. It is seen that the independent assessments of uncertainties of σy and σz summarized in Table 1 and by Hanna (2002) produced similar estimates of uncertainty (about ± factor of two); however, since both studies used similar field data sets, this result is expected. - 288 - 9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes Plume rise uncertainty Analysis of Figures 4 and 5 of Erbrink (1994), suggest the P/O plume rise ratio values can be characterized as having a log-normal distribution with a GeoSD of 2.10. Use of vertical profiles of the wind and temperature at 30 meter intervals reduces the uncertainty in the P/O plume rise ratios as characterized by a GeoSD from 2.10 to 1.48. Table 5.1 of Briggs (1969) list 22 plume rise estimates from 17 sites. Analysis of these estimates suggest that the distribution of P/O plume rise ratios can be characterized by log-normal distribution with a GeoSD of 1.34. Combining the results from Erbrink (1994) and Briggs (1969) suggest the uncertainty in the plume rise can be characterized as a random bias that follows a log-normal distribution with a GeoSD of order 1.34, and the random error that follows a log-normal distribution with a GeoSD of order 2.0. Transport wind direction and speed uncertainty There is indirect evidence through the comparison of observations with predictions from mesoscale meteorological modeling of what the uncertainties might be in characterizing the transport wind direction and speed. Hanna and Yang (2001) compared winds predicted for a nine-day period by the RAMS and MM5 meteorological models for the 12-km OTAG grid in the eastern US. For these comparisons, the root mean squared error (rmse) was less than 1.9 m/s for wind speed and on the order of 60 degrees for wind direction. They also compared MM5 predictions for a 4-day period for the 4-km grid in the central California SARMAP domain where four dimensional data assimilation (FDDA) was employed. For these comparisons, the rmse was 2.5 m/s for the wind speed and 66 degrees for the wind direction. These results are consistent with comparison of MM5 10-m predictions made for the contiguous US for 2001 with observations from 785 sites for the period June 21 – September 21, 2001, where the average rmse over all 785 sites was determined to be 1.7 m/s for wind speed and 64 degrees for the wind direction. From analysis of the MM5 comparisons for 2001, it was determined that for the wind speed standard deviation of the site to site biases was 0.74 m/s and the average over the 785 sites of the standard deviations of the residuals was 1.5 m/s. For the wind direction, the standard deviation of the site to site biases was 16 degrees and the average of the standard deviations of the residuals was 61 degrees. For the following list of experiments there was sufficient information to directly compare the transport wind direction (as indicated by a wind direction sensor near the release) with the actual transport directions (as indicated by the location of the center of mass of the tracer at each downwind arc): Project Prairie Grass, Green Glow, Hanford 30, Hanford 64, Hanford 67. The Hanford 64 involved releases at 26 m, and the Hanford 67 involved releases at 2 m, 26 m and 56 m. The wind sensor transport direction uncertainties could be characterized as a random overall bias for all releases during an experiment which was superimposed upon a random error that varied from one release to the next. The random bias followed a normal distribution with a standard deviation of about 4 degrees, and the random error followed a normal distribution with a standard deviation of about 2 degrees. These uncertainties in transport direction are substantially less for the tracer experiments than were determined in the MM5 comparisons. So we might be tempted to conclude that the uncertainties determined for these tracer experiments represents a lower bound on the wind speed and transport uncertainties, whereas the RAMS and MM5 comparisons provide evidence that much larger uncertainties are possible. Since these tracer experiments generally involved releases near the ground over simple terrain and on-site research-grade wind data were available, then these would be expected to represent minima. When the Kincaid tracer data are subjected to the same analysis, then a standard deviation of 20 degrees is found. However, the Kincaid plume - 289 - 9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes was from a tall stack and plume rise was substantial, so the wind observation from a tower at a height of about 100 m was not representative of the wind at a height of several hundred meters, where the plume was located. Combining the results from the studies discussed above suggest the uncertainty in the wind speed and wind directions can be envisioned as having a random bias that varies from one site to the next, and a random error at each site about the site bias. The random bias in wind speed is envisioned as following a normal distribution with a standard deviation of order 0.5 m/s, and the random bias in the wind direction is envisioned as following a normal distribution with a standard deviation of order 10 degrees. The random error in wind speed and direction can be simulated as errors in the North-South and East-West components which are envisioned to follow a normal distribution with a standard deviation of order 1.0 m/s for each component. Propagation of uncertainties Monte Carlo (MC) simulations were conducted to assess the sensitivity of the maximum concentration to random biases and errors. Using the Gaussian plume formula for the centerline concentration downwind from an elevated point source, with no mixing depth (ie unbounded above), we assumed the following values: median wind speed = 3 m/s, median σ y = σ z = 150 m, median plume rise = 25 m, and stack height = 25 m. The random bias and random error were specified in the dispersion parameters, plume rise and wind speed as outlined in the preceding paragraphs (but we forced the wind direction to never change). One-hundred “years” having 8760 hours each were simulated, and the resulting MC concentrations were divided by the median concentration (ie, concentration obtained with no errors). The results are summarized in Table 2, and the "averaging" times are determined as block averages (ie, they do not overlap). The resulting uncertainty distribution in the maximum concentration was seen to follow a log-normal distribution which can be summarized by the geometric standard deviation. This synthetic numerical experiment allows us to view in particular the impact of random bias and random errors as averaging time increases in a world where the dispersive conditions (on average) never change. CONCLUSIONS It is concluded that normal error distributions can be used to characterize wind speed and wind direction uncertainties, whereas log-normal error distributions can be used for the rest of the sources of uncertainty. The total uncertainty in the model inputs and formulations is viewed as a combination of random biases and random errors. The random errors (about some bias) have the greatest impact on hourly concentration values, while random bias errors have the greatest impact on the long-term concentration values. Left for future investigation is the problem of developing a means for propagating uncertainty in the winds that develops dynamically consistent wind fields. ACKNOWLEDGEMENTS AND DISCLAIMER Steven Hanna’s research has been supported by the U.S. Defense Threat Reduction Agency, the American Petroleum Institute and EPRI. The United States Environmental Protection Agency through its Office of Research and Development partially funded and collaborated in the research described here under Interagency Agreement DW 13-93999001-0 with the Department of Commerce. It has been subjected to Agency review and approved for presentation and publication. - 290 - 9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes Table 1. Summary of comparison of Model 3 ( Irwin, 1983) predictions of vertical and lateral dispersion parameters with field data from 26 sites. GeoSD is the geometric standard deviation. Elevated Lateral Dispersion Sites Elevated Vertical Dispersion Sites Experiment Site Number Bias GeoSD Experiment Site Number Bias GeoSD Hanford(67)-56m 68 1.00 1.52 Agesta 24 1.09 1.39 Hanford(67)-26m 210 0.81 1.51 Karlsruhe 87 1.03 2.04* NRTS 96 0.92 1.23 Porton 9 0.93 1.46 Karlsruhe 39 0.61 2.37* Hanford 19 1.46 1.49 NRTS 96 1.49 1.36* Hanford 57 0.90 1.19 Suffield 104 1.08 1.43 Near-Surface Lateral Dispersion Sites Near-Surface Vertical Dispersion Sites Experiment Site Number Bias GeoSD Experiment Sites Number Bias Mt. Iron 161 2.07 1.92* GeoSD NRTS-B 44 0.91 1.24 NRTS-B 42 0.57 1.90 NRTS-A 84 1.24 1.39 NRTS-A 46 1.26 2.32 Hanford 30 117 0.90 1.40* Prairie Grass 203 1.24 1.63* Green Glow 86 0.82 1.51 Round Hill I 52 1.03 1.40* Prairie Grass 315 1.12 1.68* Dry Gulch 266 1.72 1.62 Values with * denote cases where a Normal Ocean Breeze 172 1.78 1.44* distribution best characterizes the random Round Hill II 30 1.38 1.29 errors, but for which, we also found a logRound Hill I 72 1.13 1.41 normal distribution fits nearly as well. Handord(67)-2m 104 1.00 1.49* Table 2. The geometric standard deviation in maximum surface concentration values derived from Monte Carlo simulation of 100 “years” with constant average dispersion and wind conditions. The speed was 3 m/s, stack height was 25 m, plume rise was 25 m, vertical and lateral dispersion was 150 m, and no uncertainties were allowed in the wind direction. Averaging Time Random Errors Random Biases Random Errors and Biases 1-hr 2.59 2.07 3.36 3-hr 1.80 1.79 2.04 24-hr 1.26 1.62 1.56 30-days 1.05 1.59 1.46 Annual 1.02 1.59 1.46 REFERENCES American Society for Testing and Materials, (2000): Standard Guide for Statistical Evaluation of Atmospheric Dispersion Model Performance (D 6589), (Available at http://www.astm.org), 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428, 17 pages. Briggs, G.A., (1969): Plume Rise. TID-25075. U.S. Atomic Energy Commission, Available from the National Technical Information Service, U.S. Department of commerce, Springfield, VA 22151, 85 pages. Draxler, R.R., (1984): Diffusion and Transport Experiments. Chapter 9 in Atmospheric Science and Power Production (D. Randerson, ed.), DOE/TIC-27601, U.S. Dept. of Energy. NTIS, Springfield, VA 22161. 367-422. Erbrink, H.J., (1994): Plume rise in different atmospheres: a practical scheme and some comparisons with LIDAR measurements. Atmos. Environ. (28)28:3625-3636. - 291 - 9th Int. Conf. on Harmonisation within Atmospheric Dispersion Modelling for Regulatory Purposes Hanna, S.R. (2002): Meteorological Modeling in MACCS2. Final Report P047, Hanna Consultants, 7 Crescent Ave, Kennebunkport, ME 04046, Prepared for the U.S. Nuclear Regulatory Commission, Rockville, MD 20852, 23 pages Hanna, S.R. and R. Yang, (2001): Evaluations of mesoscale model’s simulations of nearsurface winds, temperature gradients, and mixing depths. J. Appl. Meteor. (40):10951104. Hooke, W.H., D.L. Albritton, R.E. Carbone, T. Gal-Chen, C.W. Kreitzberg, C.G. Little, and H. Tennekes, 1990: Report of the critical review panel – lower troposphere profiling symposium needs and technologies. Bulletin of the American Meteorological Association. Vol 71(5):680-690. Irwin, J.S., 1983: Estimating plume dispersion - a comparison of several sigma schemes. J. of Climate and Applied Meteorology. Vol. 22(1):92-114. Irwin, J.S. 1984: Site-to-site variation in performance of dispersion parameter estimation schemes. Air Pollution Modeling and Its Application III, C. De Wispelaere (ed.), Plenum Pub. Co. NY, pg. 605-617. Irwin, J.S. and R.F. Lee, 1996: Comparative evaluation of two air quality models: withinregime evaluation statistic. Fourth Workshop on Harmonization Within Atmospheric Dispersion Modelling For Regulatory Purposes, Oostende, Belgium, May, 1996, International Journal of Environment and Pollution, 8 (Nos. 3-6):346-355. Sykes, R.I., S.F. Parker, D.S. Henn, C.P. Cerasoli and L.P. Santos, (1998): PC-SCIPUFF version 1.2PD, technical Documentation, ARAP Rept 718, Titan, Princeton, NJ, 172 pages. - 292 -