Understanding the Sims-Cogley-Nason Approach in A Finite Sample

Munich Personal RePEc Archive Understanding the Sims-Cogley-Nason Approach in A Finite Sample Liu, Lin and Hussain, Syed University of Rochester, Lahore University of Management Sciences 10 July 2013 Online at https://mpra.ub.uni-muenchen.de/53118/ MPRA Paper No. 53118, posted 22 Jan 2014 15:50 UTC Understanding the Sims-Cogley-Nason Approach in A Finite Sample∗ Syed M. Hussain † Lin Liu ‡ July 2013 Abstract Kehoe (2007) advocates that in evaluating an economic model, the Sims-CogleyNason (SCN) approach should be adopted in which empirical impulse responses are compared to those obtained from the identical structural VAR run on model generated data of the same length as actual observations. This paper examines, using Monte Carlo simulation, finite sample properties of the SCN approach. Throughout the paper, we use the simple textbook New-Keynesian model as data generating process, and focus on effects of the identified monetary shocks, derived by structural VAR with shortrun identification assumption. We find that when the model violates the identifying restriction and monetary shocks are misidentified, the SCN approach has poor small sample performance. We show that: 1) The estimated impulse responses are biased and uninformative; 2) The parameter estimates derived by matching impulse responses are biased and with large mean square error. Ironically, the very reason calling for the SCN approach - misidentification, is also the cause for its poor finite sample performance. JEL Classification: C32; C51; E5. Key Words: Sims-Cogley-Nason Approach, Finite Sample Property, Structural VAR, New-Keynesian Model, Monetary Policy Shocks. 1 Introduction In evaluating an economic model with structural vector autoregression (SVAR), common approach - comparing empirical impulse response functions (IRFs) to those directly derived ∗ Acknowledgements to be added. Department of Economics, Lahore University of Management Sciences, Lahore, Pakistan, Email: [email protected] ‡ Department of Economics, Harkness Hall, University of Rochester, Rochester, NY 14627, USA, Email: [email protected] † 1 from the model (theoretical IRFs), is often thought to be problematic. One of the reasons is that the identifying assumption used for the empirical data may not actually be satisfied by the theoretical model. To resolve the issue, Kehoe (2007) proposes the Sims-CogleyNason (SCN) approach, which compares empirical IRFs to those obtained from running an identical SVAR on model generated data of the same length as the actual data.1 In the event that the model violates the identifying restriction, the estimated IRFs, which may not have meaningful economic interpretations, can be taken as moments like correlation etc, and used for matching with the empirical counterparts, as long as the empirical observations and model generated data are treated identically. Thus, the SCN approach is thought to be immune from the misidentification issue. To be useful in practice, however, the SCN approach must have good small sampling properties. In this paper, we examine, by Monte Carlo simulation, finite sample properties of the SCN approach. We apply it to two types of models: one satisfying the identification assumption and the other violating it. It turns out that finite sample properties of the SCN approach critically depends on whether the fundamental economic shocks are correctly identified or not. We find that, ironically, the very reason for calling the SCN approach misidentification, is also the cause for its poor small sample performance. Throughout the paper, we use the simple textbook New-Keynesian model (henceforth, the standard model) as the data generating process, and focus on the dynamical effects of the identified monetary shocks, derived by applying the structural VAR with short-run assumption.2 It assumes that monetary policy shocks do not have contemporaneous effect on the economy. In the standard model, however, they do affect the economy in the current period, and hence violate the identification. The reason for choosing this model is that as shown by Carlstrom et al. (2009), we can derive the VAR representation of the model dynamics analytically and obtain the true IRFs of the identified shocks, which can be used as benchmark for evaluating the estimated IRFs. For comparison, we also include the NewKeynesian model with time delay following Christiano et al. (2005) (henceforth, the CEE model), in which monetary policy shocks affect the economy with one period lag. The model satisfies the identifying restriction, and the monetary shocks are correctly identified. In this paper, we examine finite sample properties of the SCN approach when the data generating processes are the standard model – violating the identification, and the CEE model – satisfying the identification. We focus on two estimators to evaluate the small sampling properties of the SCN approach. One is the estimated IRFs to the identified shocks, derived by applying SVAR with short-run assumption on model generated data. The other is the estimated model parameters, obtained by matching the impulse responses to their 1 This approach is advocated by Sims (1989) and applied by Cogley and Nason (1995). The short-run identifying assumption is most commonly used in monetary economics for estimating effects of monetary shocks, for example in Christiano et al. (1999). 2 2 empirical counterparts, also known as the simulated method of moments. For samples of the size commonly found in economic applications, we find that: 1) In the standard model, the average of the estimated IRFs are substantially different from the true IRFs, whereas in the case of the CEE model, the two are virtually indistinguishable. Moreover, in the standard model the wide ranges of either confidence intervals or probability intervals suggest the estimated IRFs are not very informative. This is further confirmed by examining the proportion of the estimated IRFs, which share the same sign as the true IRFs. In contrast, in the CEE model the bands are narrow enough at the initial periods to provide useful model implications; 2) In the standard model, parameter estimators are biased and with large mean square error, while for the CEE model the estimators closely center around the true value with much smaller mean square error. Therefore, when the model violates the identifying restriction and the shocks are misidentified, the SCN approach has poor finite sample performance. The reason is that the identified shocks are a weighted combination of all exogenous shocks present in the model, namely monetary policy shocks, productivity shocks and price markup shocks. Because the CEE model satisfies the identification, the estimated IRFs of the identified shocks are primarily dominated by the effects of monetary policy shocks, and hence subject to sampling uncertainties from them. In contrast, in the case of the standard model, due to the incorrect identifying assumption, the estimated IRFs of the identified shocks are affected by the effects of all exogenous shocks, and thus exposed to sampling uncertainties from productivity shocks and price markup shocks, in addition to monetary policy shocks. Thus, with a small sample size the additional influences impose tremendous bias and uncertainty for the estimators. Several sensitivity analysis are conducted in the paper. These analysis show that first, the results are sensitive to relative importance of monetary policy shocks in accounting for the fluctuation. Interestingly, sampling uncertainties in the CEE model decrease as monetary policy shocks become more important, while the opposite happens in the standard model. Second, the results change significantly when measurement error is added to the standard model, whereas the results do not change for the CEE model. And third, sampling uncertainties do not fade away as number of simulations increase. Given a minimal set of identifying assumptions, SVAR allows us to estimate the dynamic effects of economic shocks and assess the empirical plausibility of the models. Recent development has been focusing on whether the estimated IRFs from the model generated data can replicate the theoretical IRFs, or whether the common approach is valid or not. Christiano et al. (2007) find that the standard SVAR procedures reliably uncover and identify the dynamic effects of shocks to the economy, in particular when the short-run assumption is used. Chari et al. (2008) show that SVAR with long-run restriction is not useful to guide the development of business cycle models, unless technology shocks account for virtually all 3 of the fluctuations in output. Despite of the disagreements, all these papers are conditioning on the fact that the model satisfies the identifying assumption. However, as pointed out by Kehoe (2007), majority of economic models do not satisfy the identifying restrictions, and suggest that a preferable alternative to the common approach is the SCN approach. On purely logical grounds, the SCN approach is superior to the common approach as the empirical data and model generated data are treated symmetrically, thereby avoiding the problems of the common approach. So the question we ask in this paper is: in practice can the SCN approach be better used to evaluate models, or are the IRFs estimated by the SCN approach good moments to look at? We also view this paper as an extension of Christiano et al. (2007) by considering a broader range of models. Our results confirm their findings that SVAR can reasonably uncover effects of economic shocks when the model satisfies the identification. In addition, we examine the sample sample properties of SVAR when the model violates the identification. The paper is organized as follows: Section 2 describes the models and the true IRFs; Section 3 presents the results on finite sample properties of the SCN approach; Section 4 provides the explanation; Section 5 conducts sensitivity analysis and Section 6 concludes. 2 The Model Throughout the paper, we use the standard New-Keynesian model and focus on effects of the identified monetary shocks, obtained by SVAR with short-run assumption. The reason for choosing this model is that VAR representation of the model dynamics can be derived analytically, so we could compute the true IRFs of the identified monetary shocks (see Carlstrom et al., 2009). The model consists of a representative household with preferences over consumption and leisure. The utility function is assumed to be separable in the two arguments. σ is the coefficient of risk-aversion and η is the inverse of Frisch labor supply elasticity. Production function is linear in labor which is the only input, and subject to a productivity shock ǫat . We denote the output gap by yt . Firms follow the Calvo pricing and lagged inflation indexation. The price setting is subject to a mark-up shock ǫπt . Central bank uses interest rate Rt as monetary policy instrument and adopts the Taylor rule with inertial given by τi . The responding coefficient for the inflation πt and output gap yt are τπ and τy , respectively. ǫR t is the monetary policy shock. All the shocks are orthogonal to each other and follow AR(1) process given as: ǫit = ρi ǫit−1 + νti , i = {a, π, R}, 4 where νti are i.i.d and follow normal distribution with mean zero and standard deviation σi . The three equations summarizing the model are given as: σ(1 + η) (ρa − 1)ǫat σ+η πt (1 + β) = βEΩt πt+1 + πt−1 + κyt + ǫπt (2) Rt = τi Rt−1 + (1 − τi )(τπ πt + τy yt ) + ǫR t (3) Rt − EΩt πt+1 = σ(EΩt yt+1 − yt ) + (1) where β is the discount factor, and κ is the slope of the Phillips curve. Equation (1) is a log-linearized IS curve derived from the household’s maximization problem. Equation (2) is the Phillip’s curve derived from the Calvo-style staggered price setting behavior. Equation (3) is the Taylor rule. Let zt = {yt , πt , Rt } and ǫt = {ǫat , ǫπt , ǫR t }. We consider two variants of the model, which differ in the information sets Ωt . One is the standard model, in which households and firms make their decisions after observing all the realization of shocks, that is, Ωt = {zt−1−k , ǫt−k , k ≥ 0}. The other is the CEE model, in which households and firms make their decisions after the realization of productivity and mark-up shocks, but before the current period monetary policy shocks, that is Ωt = {zt−1−k , ǫat−k , ǫπt−k , ǫR t−1−k , k ≥ 0}. Clearly, the standard model violates the identifying assumption as monetary policy shocks have contemporaneous effects on output and inflation, while the CEE model does satisfy the identifying restriction. The parameter values chosen for the model largely follow from the literature (see Smets and Wouters, 2007), shown in Table 1. The method of undetermined coefficients (see Christiano, 2002) is used to solve the models, and for more details refer to the appendix. To avoid from any confusion, few terminologies are clarified here. There are two kinds of monetary shocks. • One is the monetary policy shock ǫR t that appears in the Taylor rule, which we call the exogenous monetary shocks. • The other is the shocks identified by short-run assumption, which we call the identified monetary shocks. There are three types of IRFs. • The theoretical IRFs give the effects of the exogenous monetary shocks. They are derived directly from the model, and only require specifications of monetary policy shocks. They are most often seen in the literature, for example Christiano et al. (2005). • The true IRFs describe the effects of the identified monetary shocks. Since they are 5 Table 1. Parameter Values Used In the Models Parameter Explanation Value β κ σ η τi τπ τy ρa ρπ ρR σa σπ σR Discount Factor Slope of the Phillips Curve Intertemporal Elasticity of Substitution Frisch Elasticity of Labor Supply Taylor Rule - Inertial Coefficient Taylor Rule - Inflation Coefficient Taylor rule - Output Coefficient Auto-correlation of Productivity Shocks Auto-correlation of Price Markup Shocks Auto-correlation of Monetary Policy Shocks Std Error of Productivity Shocks Std Error of Price Markup Shocks Std Error of Monetary Policy Shocks 0.99 0.12 1 1 0.95 1.5 0.5 0.95 0.9 0.5 0.0040 0.0040 0.0015 obtained from the analytical VAR representation of the model dynamics, they are immune from the random sampling uncertainties. • The estimated IRFs also describe the effects of the identified monetary shocks. They are estimated by applying SVAR to the model generated data, and require full specifications of all shock processes in the model.3 Since the length of the simulated data sets are given by the length of empirical observations, the simulated IRFs suffer from finite sample problems. 2.1 The True IRFs Figure 1 shows the true IRFs of the identified monetary shocks, along with the theoretical IRFs for both models. All the responses are normalized so that the initial rise in interest rate is 25 basis points. For the CEE model shown in Panel A, the true IRFs almost coincide with the theoretical ones. This is not surprising since the model satisfies the identifying assumption, the identified monetary shocks are the same as the exogenous ones and we can 3 We generate a model simulated data set with length 180. The first 200 periods are discarded. Then we estimate the IRFs by running SVAR on the simulated data set using Choleski identification. We repeat this exercise N times which gives us N IRFs. We take the average of these N IRFs, and gives the average of the estimated IRFs. 6 correctly uncover the effects of the exogenous monetary shocks.4 For the standard model shown in Panel B, the true IRFs are substantially differently from the theoretical ones. In particular, the responses of both output and inflation are noticeably muted as compared to the theoretical counterparts. The reason is that the standard model violates the identifying assumption. As a result, the identified monetary shocks are different from the exogenous monetary shocks. In fact, the identified shocks are a weighted combination of all three exogenous shocks present in the model as shown by Carlstrom et al. (2009). Thus, the true IRFs are given by the weighted combinations of the impulse responses from all three shocks.5 Since the true IRFs of the identified shocks are derived analytically, they are not subject to sampling uncertainties, which provide us a criterion for evaluating finite sample properties of the estimated IRFs. 3 Finite Sample Properties of the SCN approach In this section, we examine finite sample performance of the SCN approach, with focusing on two estimators. One is the estimated IRFs of the identified shocks, derived by applying SVAR with short-run assumption on model generated data. The other is the estimated model parameters, obtained by matching the impulse responses, also known as the simulated method of moments. 3.1 The Estimated IRFs We generate 500 simulated data sets with length equal to 180 periods each. For each data set, we apply SVAR with short-run assumption to estimate the IRFs. The variables are arranged in the order – output, inflation and interest rate. So we obtain 500 estimated IRFs. First, in Figure 2 we examine the mean estimated IRFs, which are the average of all the estimated impulse responses. Panel A shows that in the CEE model the mean estimated IRFs and the true IRFs are very close to each other. In contrast, in Panel B for the standard model the mean responses are markedly different from the true ones. Both output and prices 4 Strictly speaking, monetary policy shocks are not correctly identified in the CEE model either. The reason is that the VAR representation of the model dynamics is infinite ordered while in practice SVAR is always with finite number of lags. This truncation bias could lead to misidentification of monetary policy shocks. However, we show that the resulting bias in this case is negligible as opposed to Chari et al. (2008) where the truncation bias from long-run restriction is not trivial. Moreover, for the CEE model, it is not possible to derive the true IRFs analytically since it does not admit an analytical VAR representation. To calculate the true IRFs, we apply SVAR on the simulated data with length of 10, 000 observations. 5 For a better account on how to derive the true IRFs and what is the difference between the true IRFs and the theoretical IRFs, please refer to Carlstrom et al. (2009). We do not elaborate here as it is not the focus of our paper. 7 initially rise instead of falling, which suggests that the price puzzle could be just a result of small sample bias. Moreover, the mean estimated response are relatively larger than the true responses in the long run. Clearly, they contain a substantial amount of bias. Second, in order to show the amount of sampling uncertainties associated with the estimated IRFs, in Figure 3 we look at both sample probability intervals and confidence intervals. Probability intervals are those estimated IRFs among the 500 IRFs that are two standard deviations away from the mean. They describe the extent of uncertainties associated with random realization of the shocks in the model. For each data set, we derive 95 percentage confidence intervals along with the estimated IRFs,6 and the average of all these intervals are the confidence bands presented in Figure 3. We can see that for both cases the confidence bands and probability intervals are very similar. This confirms the findings in Christiano et al. (2007) that confidence intervals correctly reveal the amount of sampling uncertainties contained in probability intervals. However, we find that in the CEE model the bands are very narrow at the initial few periods, suggesting that the drop in both output and inflation and the rise in interest rate are significant. In contrast, in the standard model the bands are too wide to provide any useful inference. In other words, they support a broad range of empirical results, and are not very informative. Thirdly, we look at sign count for each horizon of the estimated impulse responses, shown in Figure 4. It gives the proportion of the estimated impulse responses sharing the same sign as the true responses. As pointed out in Christiano et al. (2007), too much sampling uncertainty implied by SVAR does not necessarily mean the approach can not be used for evaluating competing models. They give an example to show that we could look at sign count instead. Therefore, here we examine the sign count for both models. In the standard model, for output and inflation only at around 50 percent of horizons of the estimated impulse responses have the same sign as the true ones. This is true for all the time periods, except the first period since by identification the responses are zero. In the case of CEE model at least 95 percent of the estimated impulse responses have the correct sign for the first four periods, that is, when the responses are statistically significant. This result further confirms that when the model violates the identification, the estimated IRFs are uninformative. 3.2 The Estimated Model Parameters The SCN approach is often used to estimate model parameters by matching the empirical IRFs with the mean estimated IRFs. It is called simulated method of moments. In this section, we examine the biasness and mean square error (MSE) of the estimators. We choose the auto-correlation of monetary policy shocks, ρR , as the parameter to be 6 We use Monte Carlo simulation to derive the confidence intervals for each simulated data set. For details, please refer to Christiano et al. (1999). 8 Table 2. The Statistics for the Parameter Estimators The CEE Model The Standard Model Mean 0.491 0.750 Median 0.502 0.835 MSE 0.012 0.120 Pr(ρˆR < ρR ) 0.491 0.160 Notes: This table shows the summary of estimates of the parameter, ρR , through simulated method of moments. The true value is ρR = 0.5. In the case of CEE model, the estimates are closely centered around the true value. In the case of the standard model, the estimates substantially biased upwards. The median is greater than the mean which indicates a heavier right tail in the probability density function of the estimates. See also Figure 5. estimated. The reason is that the persistence of monetary policy shocks matter for the estimated impulse responses of both models, and hence it can be identified. The true parameter value is chosen to be 0.5. To proceed, with the true persistence we simulate one data set from which we estimate the impulse responses, denoted as Λ̂. We treat Λ̂ as the empirical IRFs, and ρR is then estimated by solving the following minimization problem: ρ̂R = arg min (Λ̂ − Λ(ρR |Θ))′ Σ−1 (Λ̂ − Λ(ρR |Θ)). ρR Θ consists of those model parameters whose values are already given and not to be estimated. This includes all parameters except ρR . Λ(ρR |Θ) is the mean estimated IRFs. Σ is the efficient weighting matrix. The diagonal terms are given by the variances of the coefficients of IRFs Λ̂, and the off-diagonal terms are all zero. Then parameters ρ̂R is estimated by minimizing the distance between Λ̂ and Λ(ρR |Θ). We repeat this procedure for 500 times, and obtain a series of estimators. And this is done for both models. Table 2 reports the summary statistics of these estimators.7 The first row shows the estimation results of the CEE model. The mean of all the estimators is 0.491 and the median is 0.502, suggesting the estimation is neither biased nor skewed. The second row for the standard model gives remarkably different results. The mean of all the estimators is 0.75, which is above the true parameter value. The median is 0.835 so that the estimation has a heavier right tail. The MSE of the estimators are about 10 times larger than the one in the CEE model. Figure 5 plots the probability density functions for the parameter estimators. Clearly, the estimators of the CEE model center around the true parameter value, while those of the standard model are biased with larger standard deviation. Therefore, when a model violates the identification, the estimator is biased and with large MSE. 7 While calculating these summary statistics, we drop those values that lies in the estimation boundary. 9 4 The Explanation So far, we have shown that when the model violates the identifying assumption, the SCN approach has poor finite sample properties. Here, we explore its underlying reasons. To explore the reasons for the poor small sample performance of the SCN approach, we first provide the link between an economic model and its VAR representation. The state space form of the model equilibrium is given as: st = F1 st−1 + F2 ǫt zt = F3 st−1 + F4 ǫt , where st is a vector of possibly unobserved state variables, zt is a vector of variables observed by the economists, and ǫt is the vector of exogenous shocks.8 For the simple models we have here, st = zt = {yt , πt , Rt } and ǫt = {ǫat , ǫπt , ǫR t }. The 3 × 3 matrix Fi (i = 1, 2, 3 and 4) are dictated by the model specifications and parameter values. As shown in Fernandez-Villaverde et al. (2007), the model admits the following VAR representation: zt = H1 zt−1 + H2 zt−2 + · · · + Hj zt−j + · · · + F4 ǫt , (4) where Hj = F3 [F1 − F2 F4−1 F3 ]j−1 F2 F4−1 . That is, the current value zt is the sum of all the past values and the current period shocks. In practice, however, the structural VAR we estimate is: zt = B1 zt−1 + B2 zt−2 + · · · + B4 zt−4 + B0 ǫ̃t , (5) where B1 · · · B4 are 3 × 3 matrix to be estimated and B0 ǫ̃t is the estimation residual. It is different from equation (4) in that: 1) only finite number of lagged variables are included in the estimation; 2) in order to uncover effects of monetary shocks, matrix B0 is assumed to be lower-triangular, while matrix F4 may or may not be lower-triangular depending on the model specifications. Thus, the identified monetary shocks are not the same as the exogenous monetary shocks. In fact, they are a function of all exogenous shocks and all the past lagged variables that are not included in the estimation. To further investigate how the misidentification affects finite sample properties of the SCN approach, we run the following regression. We regress the identified monetary shocks, 8 Note we intentionally choose the number of exogenous shocks to be equal to the number of observables, because if there are more shocks than observables we can never possibly recover the true monetary shocks. 10 9 denoted by ǫ̃R t , on all the exogenous shocks: ǫ̃R = γ1 ǫat + γ2 ǫπt + γ3 ǫR t t + ̟t . We repeat the regression for all the simulated data sets. Table 4 gives the average of all these estimates, along with 95 percentage confidence intervals, which are given by the 2.5th percentile and 97.5th percentile of all the estimates. The true value of the estimated coefficients are presented as well. Since the identified monetary shocks are the weighted combination of three exogenous shocks, the impulse responses of the identified shocks are the weighted combination of the effects of these shocks, where the weights are given by those estimates in Table 4. So making use of the weights and the theoretical IRFs of the exogenous shocks, in Figure 6 we provide the contribution of each exogenous shock to the impulse responses of the identified monetary shocks. All the IRFs are normalized so that the sum of initial rise of interest rate caused by all exogenous shocks is 25 basis points. Recall that the only difference between the two models is that the CEE model satisfies the identifying assumption and the standard model does not. For the CEE model, the SCN approach should uncover the exogenous monetary shocks. That is, the identified monetary should only consist of the exogenous monetary shocks. So true coefficient on the productivity and price markup shocks should be 0 and the one on exogenous monetary shocks should be 1. Due to the inclusion of finite lags and small sample size, the estimates are not exactly the same as the true value, but very close. As a result, in Panel A of Figure 6 the impulse responses from both productivity shocks and price markup shocks are negligible. The IRFs of the identified monetary shocks are entirely dominated by the responses from the exogenous monetary shocks, and thus subject mainly to sampling uncertainties from random realization of the exogenous monetary shocks. In contrast, for the standard model the identifying assumption (B0 is lower-triangular matrix) is not consistent with its VAR representation (F4 is not lower-triangular matrix). This leads the identified monetary shocks to be contaminated by shocks other than exogenous monetary shocks. As a result, shown in Panel B of Figure 6 the responses of all three shocks are all very important for shaping the overall IRFs of the identified shocks. This brings the added uncertainty, since the estimated IRFs are subject to sampling uncertainties from productivity shocks and price markup shocks, in addition to exogenous monetary shocks. Hence, the SCN approach has poor small sample performance when applied to the model violating the identification. 9 In the estimation, we include a constant and also control for the lagged variables from the previous two years to previous three years (zt−5 · · · zt−12 ). 11 5 Sensitivity Analysis In this section, we test sensitivity of the estimated results by varying size of monetary policy shocks, amount of measurement error included, and number of simulations used in the estimation. 5.1 Size of Monetary Shocks One of the results in Christiano et al. (2007) is that sampling uncertainties reduce substantially when the fundamental economics shocks, which we intend to identify, become more important in accounting for the fluctuation. This leads us to wonder how the estimated results change for both models, as we vary the relative importance of the exogenous monetary policy shocks. To do so, we experiment with varying the size of monetary shocks. Recall that in the baseline case, standard error of monetary shocks is 0.15%. Table 3 shows that in both models as we increase the size of monetary shocks, they become more and more important in explaining the fluctuation. For example, in the baseline CEE model, monetary policy shocks explain 36.9 percent of output fluctuations, compared with 62.2 percent when we double the size of monetary shocks. In Figure 7, the upper panel shows that the mean of the estimated IRFs in the CEE model are almost unaffected by changing in the size of monetary shocks, and the bottom panel shows that the probability intervals shrink as monetary policy shocks become more important, suggesting sampling uncertainties reduce significantly.10 This confirms the finding in Christiano et al. (2007). As a result, MSE of the estimated parameter values decrease significantly, shown in Panel A of Table 5. For the standard model, shown in Figure 8, we find that changing in shock sizes does not qualitatively affect the mean of the estimated IRFs, except that output and inflation are more volatile in the first few periods. Interestingly, we find that remarkably different from the CEE model, the probability intervals for output and inflation expand as the shock size increase. It suggests sampling uncertainties rise as the result of increase in shock size. In Panel A of Table 5, MSE of the estimated parameter values initially increases and then decreases. 5.2 Measurement Error In applying the SCN approach, measurement error is added to the model simulated data. This is often done to avoid the singularity problem in the estimation. Here, we examine 10 We only report the results on probability intervals here, because confidence intervals behave very similar to them. 12 Table 3. The Contribution of Monetary Policy Shocks to the Fluctuations The CEE Model σR = 0.0015 σR = 0.0030 σR = 0.0045 Output 36.9 62.2 76.6 Inflation 27.7 49.9 65.3 The Standard Model Interest Rate 14.3 27.8 39.8 Output 37.5 62.9 77.1 Inflation 28.9 51.7 67.0 Interest Rate 7.2 14.3 21.2 Notes: This table reports the fraction of fluctuations in output, inflation, and interest rate that monetary shocks explain. Following Christiano et al. (2007), it is calculated as the ratio of two variances: the numerator is the variance of the variable with only monetary policy shocks and the denominator is the variance of the variable with all three exogenous shocks. All statistics are averages of the ratios, based on 500 simulations of 180 observations for each model. The results are for various sizes of the monetary shocks. The table shows that as the size of the shock - represented by the standard error of the monetary shock process - increases, it explains a greater fraction of fluctuations in both the standard and the CEE model. whether the estimated results are sensitive to this seemingly naive procedure. To do so we follow Chari et al. (2005), a measurement error vector ξt is added to the variable zt : z̃t = zt + ξt , with E(ξt ξt′ ) = Γ. Γ is the variance-covariance matrix of ξt . Its diagonal terms are given by a constant times the variance of the corresponding element of zt , and the off-diagonal terms are zero. In the baseline case, the constant is 0, that is, no measurement error. We experiment with increasing the constant to 0.0001 and 0.0004. Since the measurement error added is very small, it does not affect the overall fluctuations and the relative importance of monetary policy shocks. Figure 9 shows that for the CEE model, the estimated IRFs and probability intervals do not exhibit any noticeable change. The sampling uncertainties added due to measurement error have no impact on the CEE model. Thus, the estimated parameter values in Panel B of Table 5 are roughly similar for different value of measurement error. However, the results are remarkable different for the standard model, shown in Figure 10. The mean of the estimated impulse responses become more pronounced as size of measurement error increases. At the same time, the probability intervals become much narrower, suggesting sampling uncertainties decrease. In Panel B of Table 5, we find that the estimated parameter values are closer to the true value and less skewed, and MSE declines, as amount of measurement error added rises. 13 5.3 Number of Simulations Is it possible to improve small sample properties of the SCN approach by increasing number of simulation used in the estimation? We find that the answer is no for both models. In the baseline case, number of simulations we use is N = 500. Here, we increase it to N = 1000. Figure 11 shows that in the case of the CEE model, the estimated impulse response and probability intervals do not change as number of simulations is varied. In the standard model, shown in Figure 12, the mean of the estimated IRFs are different for various number of simulations, but the probability intervals are very close to each other. It suggests that sampling uncertainty can not go away with a larger number of simulations. Table 5 Panel C shows that the parameter estimators for both models are not affected much, when number of simulations is varied. To sum up, we find that: 1) When monetary policy shocks become more important in accounting for the fluctuation, sampling uncertainties in the CEE model decrease, while they increase in the standard model; 2)The estimated results in the CEE model are not sensitive to the amount of measurement error added, whereas sampling uncertainties drop substantially in the standard model; 3) The sampling uncertainties in both models do not fade away as number of simulation increases. 6 Conclusion To evaluate theoretical models, the SCN approach is recommended because it avoids the problems of common approach, by treating empirical observations and model generated data symmetrically. This approach, however, in practice suffers from finite sample problem when the model violates the identifying assumption. So ironically, the very reason for adopting the SCN approach is also the cause for its poor small sample performance. We are not arguing against the usage of the SCN approach. We think that it strictly dominates the common approach. However, we think while using this approach, a researcher should be more careful in interpreting the results. In particular, the large sampling uncertainties associated with the approach make it more difficult to decide whether the model is relevant or not. Moreover, measurement error should be added with caution since it may actually become the driving force of the results. In addition, instead of using the impulse responses as the moments, we could look at other moments, like standard deviation and correlation of simulated data etc.. For example, even though the estimated impulse responses are not statistically significant in the standard model, the correlation between output and interest rate is significantly negative, lying in between −0.73 and −0.22. We derive the results in the paper by using only the simple New-Keynesian model. It is 14 possible some of the results may change with different models. But we think finite sample properties of the SCN approach when applying to more complex economic models with more fundamental economic shocks can hardly be any better, since the results of this simple model already contain too much sampling uncertainties. Moreover, we do not look at longrun identifying assumption in this paper. But as suggested by Christiano et al. (2007) and Chari et al. (2008), we think finite sample properties of the SCN approach with long-run assumption can only be even worse, compared with the short-run assumption. References Carlstrom, Charles T., Timothy S. Fuerst, and Matthias Paustian, “Monetary policy shocks, Choleski identification, and DNK models,” Journal of Monetary Economics, October 2009, 56 (7), 1014–1021. Chari, Varadarajan, Patrick Kehoe, and Ellen R. McGrattan, “A critique of structural VARs using real business cycle theory,” Working Papers 631, Federal Reserve Bank of Minneapolis 2005. , , and , “Are structural VARs with long-run restrictions useful in developing business cycle theory?,” Journal of Monetary Economics, 2008, 55 (8), 1337 – 1352. Christiano, Lawrence J, “Solving Dynamic Equilibrium Models by a Method of Undetermined Coefficients,” Computational Economics, October 2002, 20 (1-2), 21–55. Christiano, Lawrence J., Martin Eichenbaum, and Charles L. Evans, “Monetary policy shocks: What have we learned and to what end?,” Handbook of Macroeconomics, 1999, 1, 65–148. , , and , “Nominal Rigidities and the Dynamic Effects of a Shock to Monetary Policy,” Journal of Political Economy, February 2005, 113 (1), 1–45. , , and Robert Vigfusson, “Assessing Structural VARs,” in “NBER Macroeconomics Annual 2006, Volume 21” NBER Chapters, National Bureau of Economic Research, Inc, January 2007, pp. 1–106. Cogley, Timothy and James M. Nason, “Output Dynamics in Real-Business-Cycle Models,” American Economic Review, 1995, 85 (3), 492–511. Fernandez-Villaverde, Jesus, Juan F. Rubio-Ramirez, Thomas J. Sargent, and Mark W. Watson, “Monetary policy shocks: What have we learned and to what end?,” American Economic Review, June 2007, 97 (3), 1021–1026. 15 Kehoe, Patrick J., “Assessing Structural VARs: Comment,” in “NBER Macroeconomics Annual 2006, Volume 21” NBER Chapters, National Bureau of Economic Research, Inc, January 2007, pp. 1–106. Sims, Christopher A., “Models and Their Uses,” American Journal of Agricultural Economics, 1989, 71 (2), 489–494. Smets, Frank and Rafael Wouters, “Shocks and Frictions in US Business Cycles: A Aayesian DSGE Approach,” American Economic Review, June 2007, 97 (3), 586–606. 16 Appendix: Solving the Model Ignoring the expectation operator, equations (1), (2), and (3) can be rewritten in matrix notation as α0 Zt+1 + α1 Zt + α2 Zt−1 + β0 St+1 + β1 St = 0 (6) where Zt is the vector of state variables and St is the vector of exogenous disturbances.         −σ 0 −1 σ 1 0 0 0 0 yt         Zt =  π t  , α 0 =  0 β 0  , α 1 =  κ −(1 + β) 0  , α2 = 0 1 0  (1 − τi )τy (1 − τi )τπ −1 Rt 0 0 0 0 0 τi We consider two variants of the model presented above. In the first case - the standard model, all shocks are realized before decisions are made. Thus this specification allows for contemporaneous effect of interest rate shocks on both output and inflation. In this case, the β matrices are given by    σ(1+η)    (ρ − 1) 0 0 0 0 0 ǫat a σ+η       St =  ǫπt  , β0 = 0 0 0 , β1 =  0 1 0 0 0 0 ǫR 0 0 1 t In the second case - the CEE model, interest rate shocks only have contemporaneous effects on interest rate but not on output and inflation. This specification of the model satisfies the identification scheme (namely the Choleski decomposition) used in empirical estimation of VAR. In this case, the shock structure of the model can be represented as   ǫat  π     σ(1+η)   ǫt   R (ρa − 1) 0 0 0 0 0 0 0 0 0 0 0 σ+η  ǫt       St =  0 1 0 0 0 0 ǫa  , β0 = 0 0 0 0 0 0 , β1 =   t−1  0 0 0 0 0 0 0 0 1 0 0 0 π  ǫt−1  ǫR t−1 In the case of the standard model, the decision rule for the state variables can be written as Zt = AZt−1 + BSt (7) where Zt is the vector of state variables as defined above and St is a 3 × 1 vector of contemporaneous exogenous shocks. A and B are 3 × 3 coefficient matrices, entries of whom depend on the structural parameters of the model. In the case of the CEE model, the decision rule 17 is given by Zt = A′ Zt−1 + B ′ St−1 (8) where Zt is as defined above. St in this case is a 6 × 1 vector that includes the contemporaneous as well as lagged values of exogenous disturbances. A′ is a 3 × 3 and B ′ is a 3 × 6 coefficient matrix. The model is solved using the method of undetermined coefficient, see Christiano (2002). 18 Table 4. The Estimated Weights of Three Exogenous Shocks Panel A: The CEE Model True Weight (γ) Estimated Weight (γ̂) Productivity Shocks (γ1 ) Price Markup Shocks (γ2 ) Monetary Shocks (γ3 ) 0 -0.004 [-0.155 0.141] 0 0.004 [-0.131 0.139] 1 0.944 [0.843 1.043] Panel B: The Standard Model 19 True Weight (γ) Estimated Weight (γ̂) Productivity Shocks (γ1 ) Price Markup Shocks (γ2 ) Monetary Shocks (γ3 ) -0.999 -1.059 [-1.213 -0.939] 0.002 -0.005 [-0.186 0.146] 0.027 0.082 [-0.085 0.251] Notes: This table reports the estimated weights of all three exogenous shocks in contributing to the identified monetary shocks. All the estimated weights are the average of the estimates, based on 500 simulations of 180 observations for each model. Moreover, we give the 95 percentage confidence intervals. Along with the estimated weights, we also report the true weights. In both panels, the estimated weights are very close to the true weights, which fall into the confidence intervals. Panel A shows that for the CEE model, the identified shocks mostly consist of the exogenous monetary policy shocks. Panel B shows that for the standard model, the estimated and true weights of the productivity shocks are significantly different from zero thereby indicating a substantial presence of the productivity shock in the identified monetary shocks. See also Figure 6 Table 5. Parameter Estimators For the Sensitivity Analysis Panel A: Varying Size of Monetary Shocks The CEE Model σR = 0.0015 σR = 0.0030 σR = 0.0045 Mean 0.491 0.495 0.499 Median 0.502 0.501 0.501 MSE 0.012 0.004 0.002 Pr(ρˆR < ρR ) 0.491 0.496 0.494 The Standard Model Mean 0.750 0.744 0.742 Median 0.835 0.875 0.875 MSE 0.120 0.124 0.118 Pr(ρˆR < ρR ) 0.160 0.168 0.154 Panel B: Varying Measurement Error The CEE Model 20 Error = 0 Error = 0.0001 Error = 0.0004 Mean 0.491 0.493 0.493 Median 0.502 0.499 0.500 MSE 0.012 0.011 0.011 Pr(ρˆR < ρR ) 0.491 0.505 0.505 The Standard Model Mean 0.750 0.614 0.532 Median 0.835 0.715 0.602 MSE 0.120 0.078 0.060 Pr(ρˆR < ρR ) 0.160 0.290 0.376 Panel C: Varying Number of Simulations The CEE Model N = 500 N = 1000 Mean 0.491 0.493 Median 0.502 0.506 MSE 0.012 0.012 Pr(ρˆR < ρR ) 0.491 0.483 The Standard Model Mean 0.750 0.770 Median 0.835 0.855 MSE 0.120 0.130 Pr(ρˆR < ρR ) 0.160 0.149 Notes: This table reports the statistics of the parameter estimators in the standard and the CEE model. It describes how the statistics change when various changes are made to both model. Parameter estimation was carried out by minimizing the distance between model generated estimated IRFs and the IRF generated by using true value of the parameter. Panel A shows that varying the size of the monetary shocks has negligible effect on parameter estimation. Panel B shows that adding measurement error to the simulated data sets substantially reduces the amount of sampling uncertainties in the standard model where the estimation remains largely unaffected in the CEE model. Panel C shows that varying the number of simulations does not affect the estimation results too much. Figure 1. The True IRFs of the Identified Monetary Shocks – Both Models 0 -0.5 Inflation (%) -1.5 -2 -2.5 0 5 Periods -0.2 0.15 -0.4 -0.6 -1.2 10 21 2 Inflation (%) -2 -4 -6 0 5 Periods 5 Periods 10 -0.1 Panel B: The Standard Model 0 0.4 0.3 -1 -1.5 -3 0 5 Periods 10 The Theoretical IRFs The True IRFs 0.5 -0.5 -2.5 0 0 10 -2 -8 0.1 0.05 -0.05 0.5 0 Output (%) 0.2 -1 -3.5 -10 0 -0.8 -3 0.25 Interest Rate (%) Output (%) -1 -4 Panel A: The CEE Model 0.2 Interest Rate (%) 0.5 0.2 0.1 0 -0.1 0 5 Periods 10 -0.2 0 5 Periods 10 Notes: This figure shows the true and theoretical IRFs from the standard and CEE models. Theoretical IRFs are derived directly from the model by solving it while true IRFs are obtained from the analytical VAR representation of the model. Panel A shows that for the CEE model, the two types of IRFs are virtually indistinguishable. Panel B shows that the true IRFs are quite different from the theoretical IRFs in the case of the standard model. The true IRFs of output and inflation are muted in comparison to the theoretical IRFs. Figure 2. The Mean Estimated IRFs of the Identified Monetary Shocks – Both Models Inflation (%) 0 Output (%) Panel A: The CEE Model 0.2 -1 -2 0.2 -0.2 0.15 -0.4 -0.6 -0.8 -3 -1 0 5 Periods -1.2 10 22 0.3 0.2 0.1 Inflation (%) Output (%) 5 Periods -0.1 -0.2 -0.3 Panel B: The Standard Model 0 5 Periods 10 5 Periods 10 0 5 Periods 10 0.3 0 -0.1 -0.4 0 0.35 0.25 0.2 0.15 0.1 -0.3 -0.5 0 0.4 0.2 -0.2 -0.4 0.05 -0.1 10 0.1 0 -0.6 0 0.3 The Mean Estimated IRFs The True IRFs 0.1 -0.05 Interest Rate (%) -4 0.25 0 Interest Rate (%) 1 0.05 0 5 Periods 10 0 Notes: This figure shows the mean estimated and true IRFs for the standard and the CEE model. The estimated IRFs refer to the IRFs computed by generating data sets of the same length as the empirical data and applying SVAR to these model generated data sets. Panel A shows that the mean estimated and true IRFs are very close for the case of CEE model. Panel B shows that the two types of IRFs are significantly different in the case of the standard model. In particular, price puzzle - prices moving in the same direction as the movement of interest rate - arises in the case of standard model. Output initially rises instead of falling as in the true IRF and then drops by much more. Figure 3. The Confidence Intervals and Probability Intervals of the Estimated IRFs – Both Models 2 1 1 0.5 -1 -2 -4 -5 0.2 0 -0.5 -1 The Mean Estimated IRFs Confidence Interval -1.5 Probability Interval -3 0 5 Periods -2 10 23 20 0.1 0 -0.1 -0.2 -0.3 0 5 Periods 10 -0.4 Panel B: The Standard Model 15 15 0.3 Interest Rate (%) Inflation (%) Output (%) 0 Panel A: The CEE Model 0 5 Periods 10 0 5 Periods 10 5 10 0 -5 5 Interest Rate (%) 5 Inflation (%) Output (%) 10 0 -5 0 -10 -10 -15 -20 0 5 Periods 10 -15 0 5 Periods 10 -5 Notes: This figure shows the confidence and probability intervals for the standard and the CEE models along with the mean estimated IRFs. Confidence intervals are the mean of all the confidence intervals estimated for each individual estimated IRF. Probability intervals are those estimated IRFs that are two standard deviations away from the mean. Panel A shows that for the case of CEE model, the two types of intervals are narrow in the first few periods allowing us to confidently state about the behavior of output and inflation. Panel B shows that for the standard model, the confidence and probability intervals are too large to provide with any useful inference. Figure 4. The Sign Count of the Estimated IRFs – Both Models 1.1 1.1 1 1 1 0.9 0.9 0.9 0.8 0.8 0.8 Output Inflation Interest Rate 1.1 24 0.7 0.7 0.6 0.6 0.6 0.5 0.5 0.5 0.4 0 5 Periods 10 0.4 0 5 Periods 10 The CEE Model The Standard Model 0.7 0.4 0 5 Periods 10 Notes: This figure shows the sign count for the standard and the CEE models. Sign count refers to the proportion of the time each coefficient of the estimated IRF shares the same sign as the corresponding coefficient in the true IRF. Figure 5. The Probability Density Function of the Parameter Estimators – Both Models 4 3.5 The CEE Model The Standard Model 25 Probability Density Function 3 2.5 2 1.5 1 0.5 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 The Estimated Parameter: Auto-correlation of Monetary Policy Shocks 0.8 0.9 1 Notes: This figure shows the probability density function of the estimates of the parameter for the standard and the CEE model. The parameter estimated was the auto-correlation coefficient in the monetary shock process. Parameter estimation was carried out by minimizing the distance between model generated estimated IRFs and the IRF generated by using true value of the parameter. The dashed line shows that for the CEE model, the parameter estimates are distributed around the true parameter value of 0.5. The solid line shows that the parameter estimates for the standard model are centered away from the true value. Figure 6. The Contribution of the Exogenous Shocks to the Estimated IRFs of the Identified Monetary Shocks – Both Models 0 -0.5 Inflation (%) -1.5 -2 -2.5 0 5 Periods 0.15 -0.4 -0.6 -1.2 10 26 4 0.1 0.05 0 -0.05 0 5 Periods -0.1 10 Panel B: The Standard Model 1.5 3 Inflation (%) 1 0 -1 -2 -4 0 5 Periods 10 5 Periods 10 Productivity Shocks Price Markup Shocks Monetary Shcoks 0.5 0.5 0 -0.5 -3 0 0.6 1 2 Output (%) -0.2 -1 -3.5 -5 0.2 -0.8 -3 0.25 0 Interest Rate (%) Output (%) -1 -4 Panel A: The CEE model 0.2 Interest Rate (%) 0.5 0.4 0.3 0.2 0.1 -1 0 -1.5 -0.1 0 5 Periods 10 0 5 Periods 10 Notes: This figure shows the contributions of each of the three shocks in the model to the identified monetary shock. Thus the impulse response to the identified monetary shock is the weighted combination of the impulse response to each of the individual shocks. Panel A shows that since in the CEE model, the short run identification assumption is satisfied, the monetary shock is correctly identified and the contributions from the other two shocks are negligible. Panel B shows that in the standard model, when the identification assumption is not satisfied, productivity and mark-up shocks have non-trivial contributions in the identified monetary shocks thereby increasing the sampling uncertainty. Figure 7. The Estimated IRFs Varying Size of Monetary Shocks – the CEE Model Inflation (%) Output (%) 0 -1 -2 0.2 0.25 0 0.2 -0.2 0.15 Interest Rate (%) 1 -0.4 -0.6 -0.8 -3 -4 -1 0 5 Periods -1.2 10 -0.05 0 5 Periods -0.1 10 1 0.5 0.2 Inflation (%) -0.5 -1 -3 -1.5 -4 -5 0 Interest Rate (%) 27 Output (%) 0 0.3 -2 0 5 Periods 10 -2 R = 0.0045 0.05 1 -1 R = 0.0030 0.1 2 0 R = 0.0015 0 5 Periods 10 0 5 Periods 10 0.1 0 -0.1 -0.2 -0.3 0 5 Periods 10 -0.4 Notes: This figure reports the estimated IRFs of the identified monetary shocks for different values of the size of the monetary shocks in the CEE model. The top panels describes the mean of estimated IRFs, which show that varying shocks sizes has virtually no effect on the estimated impulse responses. The bottom panels depict the probability intervals, which shrink as the size of monetary shocks increases. It indicates that sampling uncertainty drops as monetary policy shocks become more important in accounting for the fluctuations. 0.3 0.4 0.4 0.2 0.35 0.2 0.1 0 -0.1 -0.4 -0.2 -0.6 -0.3 0 5 Periods -0.4 10 28 40 20 30 15 20 10 10 5 0 -10 -30 -15 0 5 Periods 10 -20 0.2 R = 0.0015 0.15 R = 0.0030 R = 0.0045 0.05 0 5 Periods 0 10 0 5 Periods 10 0 5 Periods 10 6 4 -5 -10 0.25 0.1 0 -20 -40 Interest Rate (%) -0.2 0.3 Interest Rate (%) 0 -0.8 Output (%) Inflation (%) 0.6 Inflation (%) Output (%) Figure 8. The Estimated IRFs Varying Size of Monetary Shocks – the Standard Model 2 0 -2 -4 0 5 Periods 10 -6 Notes: This figure reports the estimated IRFs of the identified monetary shocks for different values of the size of the monetary shocks in the standard model. The top panels describes the mean of estimated IRFs, which show that varying shocks sizes has little effect on the estimated impulse responses. In particular, the responses of output and price become more volatile in the initial few periods. The bottom panels depict the probability intervals, which expand as the size of monetary shocks increases. It indicates that sampling uncertainty increases as monetary policy shocks become more important in accounting for the fluctuations. Figure 9. The Estimated IRFs Varying Measurement Error – the CEE Model Inflation (%) Output (%) 0 -1 -2 0.2 0.25 0 0.2 -0.2 0.15 Interest Rate (%) 1 -0.4 -0.6 -0.8 -3 -4 -1 0 5 Periods -1.2 10 0 -0.05 0 5 Periods -0.1 10 1 0.3 1 0.5 0.2 Inflation (%) -1 -2 -0.5 -1 -3 -1.5 -4 -5 0 Interest Rate (%) 29 Output (%) 0.05 2 0 0 5 Periods 10 -2 error = 0 error = 0.0001 error = 0.0004 0.1 0 5 Periods 10 0 5 Periods 10 0.1 0 -0.1 -0.2 -0.3 0 5 Periods 10 -0.4 Notes: This figure shows the effect of adding measurement error to the model-generated data sets in the CEE model. The top panels show that the mean of the estimated IRFs does not change as measurement error is added to the data sets. The bottom panels show that the width of the probability internals are also immune to measurement error being added to the model-generated data sets. Figure 10. The Estimated IRFs Varying Measurement Error – the Standard Model 0.5 0.6 0.45 0 0.4 0.35 -0.5 -1 Interest Rate (%) Inflation (%) Output (%) 0.4 0.2 0 -1.5 -0.2 -2 -0.4 0.3 0.25 0.2 0.15 error = 0 error = 0.0001 error = 0.0004 0.1 0.05 0 5 Periods 10 30 20 0 5 Periods 0 10 15 15 0 5 Periods 10 0 5 Periods 10 5 10 0 -5 5 Interest Rate (%) 5 Inflation (%) Output (%) 10 0 -5 0 -10 -10 -15 -20 0 5 Periods 10 -15 0 5 Periods 10 -5 Notes: This figure shows the effect of adding measurement error the model-generated data sets in the standard model. The top panels show that the mean of the estimated IRFs increases dramatically as the size of measurement error is increased. Output falls by 3 times more when measurement error is added as compared to the case of no measurement error. Inflation also responds by more when measurement error is added. Panel B shows that the width of probability intervals decrease as the size of measurement error is increased, indicating the sampling uncertainty drops when measurement error increases. Figure 11. The Estimated IRFs Varying Number of Simulations – the CEE Model Inflation (%) Output (%) 0 -1 -2 0.2 0.25 0 0.2 -0.2 0.15 Interest Rate (%) 1 -0.4 -0.6 -0.8 -3 -4 -1 0 5 Periods -1.2 10 0.05 0 -0.05 0 5 Periods -0.1 10 1 0.3 1 0.5 0.2 Inflation (%) -1 -2 -0.5 -1 -3 -1.5 -4 -5 0 Interest Rate (%) 31 Output (%) 0.1 2 0 0 5 Periods 10 -2 N = 500 N = 1000 0 5 Periods 10 0 5 Periods 10 0.1 0 -0.1 -0.2 -0.3 0 5 Periods 10 -0.4 Notes: This figure shows the effect of increasing the number of simulations for the CEE model. The top panels describe the mean of the estimated IRFs and the bottom panels are the probability intervals. They show that in the case of the CEE model, increasing the number of simulations has no effect on either the estimated IRFs or their probability intervals. Figure 12. The Estimated IRFs Varying Number of Simulations – the Standard Model 0.3 0.2 0.1 0.35 0.3 -0.1 -0.2 -0.3 Interest Rate (%) Inflation (%) Output (%) 0.4 0.2 0.1 0 0 -0.1 -0.2 -0.4 0 5 Periods -0.4 10 32 20 0.25 0.2 0.15 0.05 0 5 Periods 0 10 15 15 N = 500 N = 1000 0.1 -0.3 -0.5 -0.6 0.3 0 5 Periods 10 0 5 Periods 10 5 10 0 -5 5 Interest Rate (%) 5 Inflation (%) Output (%) 10 0 -5 0 -10 -10 -15 -20 0 5 Periods 10 -15 0 5 Periods 10 -5 Notes: This figure shows the effect of increasing the number of simulations for the standard model. The top panels describe the mean of the estimated IRFs and the bottom panels are the probability intervals. The top panels show that as number of simulations increase, the impulse responses of both output and inflation show small quantitative changes. The bottom panels show that the width of the probability intervals do not change with the number of simulations increasing, indicating the sampling uncertainty does not fade away as we increase the number of simulations.