All Questions
Tagged with confidence-interval hypothesis-testing
347 questions
1
vote
2
answers
65
views
Opposite results between one-side confidence interval and one-side hypothesis test for one proportion each other?
I'm trying an example in PennState STAT 415 (Example 9-2)
Method 1: a right-tailed hypothesis test
In short, the observed sample proportion is $\hat{p}=\dfrac{104}{590}=0.176$. The null hypothesis ...
- 1,075
1
vote
1
answer
42
views
Multiple regression confidence intervals for slope coefficient of a covariate
When constructing such a confidence interval and evaluating whether it contains 0 (for the t test to see if the covariate is of relevance in the regression) are the other variables being controlled or ...
- 183
1
vote
2
answers
87
views
Hypothesis Testing vs Confidence Intervals for sample size calculation
Suppose we're auditing a low-risk group historically showing a mean tax evasion of \$7,883 with a standard deviation of \$27,274 upon auditing. I aim to determine if the average evasion is close to ...
- 157
2
votes
0
answers
56
views
How to perform data-driven choice of sample size
Say I want to compare the result of 2 regression model. One is the reference model and the second one is the new model, which I want to know if it performs better. Each model is evaluated with the ...
- 171
0
votes
0
answers
20
views
regression models comparison with paired t-test but inflated variance
I have 2 dependant dataset D1 and D2 that I used to train and evaluate a regression model within a nested cross-validation that ...
- 171
1
vote
1
answer
63
views
Determining statistical significance
I am trying to determine the statistical significance of my results at a 95 percent confidence level. I have the coefficients and the standard errors. I wanted to check if the following statement is ...
4
votes
1
answer
140
views
Calculation of effect size, statistical power, and confidence interval for different statistical tests
Introduction.
I am studying the number of ants which go to their ant nest, every minute, from different areas around the ant nest. I also know an approximate distance that those ants travel to reach ...
- 444
1
vote
1
answer
36
views
How do we deal with confidence intervals containing values that are consistent with both the null and alternative hypothesis?
For instance, assume that when testing for a significant difference in means of some measure between a control and a treatment group, I observe a difference in means of 0.5, suggesting the treatment ...
- 85
4
votes
2
answers
365
views
confidence interval and rejection
Assume I have a data sample $X_{1}, \dots X_{n}$ from a normal distribution. Then, we consider the following hypothesis
$$
H_0: \quad \mu = \mu_{0}
$$
and alternative
$$
H_1: \quad \mu \neq \mu_{0}
$$
...
- 668
0
votes
0
answers
116
views
How to implement an Anderson-Rubin test and calculate confidence intervals for an iv model set up with plm in r?
I have run an IV model using the plm-package in r. The underlying data is panel data.
My instrument is "weak", i.e. the F-statistic lies below 10.
How can I implement an Anderson-Rubin test ...
- 345
0
votes
0
answers
17
views
Statistical test for variability among more than one group
I am very new to statistics and still learning. Below are the funnel graphs which shows the effect/percentage of polypharmacy(patients with two or more drugs) against practice/hospital population size....
- 141
3
votes
1
answer
47
views
Can/should a scientific paper simultaneously perform one-sided hypothesis testing and a two-sided confidence interval?
Let's say you wish to perform a one-sided hypothesis test in which you want to reject $H_0: \Delta \leq 0$ in favor of $H_1: \Delta >0$. You select this hypothesis test specifically because you are ...
0
votes
0
answers
25
views
What is the meaning of confidence intervals when NOT using sample data and instead have data on the whole population? [duplicate]
I'm trying to understand why you would need confidence intervals or compute statistical tests (for comparing proportions) when your data is NOT from a sample and you have data on the whole population.
...
- 77
0
votes
0
answers
49
views
Appropriate Trend Analysis Test for Small Sample Size
Note: I have read Finding an appropriate trend test but unfortunately this post does not apply for me
Suppose I have a small sample of data for 2 numeric variables $T$ and $Y$ where $T$ represents ...
- 215
0
votes
1
answer
50
views
Confidence interval for superiority/inferiority tests
I am looking into superiority/non-inferiority tests and my understanding is that the null hypothesis for a superiority test is
$$
H_{0}: \epsilon \leq \delta
$$
where $\delta \geq 0$ and $\epsilon$ ...
- 241
6
votes
0
answers
69
views
What do One-Sided Confidence Ellipses Look Like?
To make things concrete, take a simple linear model:
$$E[y \vert x_1, x_2] = \alpha + \beta_1 x_1 +\beta_2 x_2$$
In a one-sided hypothesis test, like $\beta_1 \ge k$ vs. $\beta_1 \lt k$, the ...
- 38.3k
2
votes
0
answers
21
views
How to read and use equations for confidence interval from sequential testing
in the book "Group Sequential and Confirmatory Adaptive Designs in Clinical Trials" by Wasmer and Brannath - they explain on p.85 how to calculate a confidence interval.
They consider a ...
- 241
0
votes
0
answers
22
views
Bootstrapped Confidence Intervals controlling for FWER?
I'm currently using the bootci package in MATLAB to compute bootstrapped confidence intervals. I have data for two variables and I'm interested in comparing differences in means of these variables. ...
- 1
1
vote
0
answers
28
views
Reasonable hypothesis testing from confidence interval oracles
Assume that we have two scalar quantities $X$ and $Y$, and a confidence interval oracle $CI$ for each of them, so that $CI_X(\alpha)$ and $CI_Y(\alpha)$ will produce level-$\alpha$ confidence ...
- 301
1
vote
1
answer
107
views
Estimating maximum effect size and Skillings-Mack test
I'm working with data that fit the Skillings-Mack test: unbalanced incomplete block design. Or Friedman-test-like with missing data.
In case of negative test result I'd like to estimate the maximum ...
- 190
0
votes
0
answers
17
views
Mean-parameterizable models that have invariant concentration functions, but that aren't translation-invariant?
Definitions: Sorry for the ad hoc terminology -- comments or answers that provide pointers to standard terminology would be much appreciated. For simplicity I'd like to restrict discussion to real-...
1
vote
0
answers
67
views
Can we derive type 2 error probability from confidence interval?
If we can compute confidence intervals for an estimate $\hat{\theta}$, then we can also compute the p value for an associated hypothesis $\theta=\theta_0$, which is the probability of a type 1 error (...
- 1,176
14
votes
5
answers
1k
views
Compare 90th percentiles of two samples (confidence interval, test)
I have a dataset of ambulance response times before and after a quality improvement change. I'm looking to see if there is a difference in response times before and after the change. Specifically, I'm ...
- 141
2
votes
1
answer
118
views
Cochran-Mantel-Haenszel Test confidence interval
I'm looking at a collection of very similar dichotomous outcome studies, and in order to assess the relative risk of an intervention and account for study variability, I believe I should use the ...
- 323
4
votes
1
answer
146
views
Confidence interval on ratio of estimates for exponential random variables
Given exponential random variable X, the MLE for the scale parameter is $\hat{\beta_x} = \bar{x}$, and the confidence interval for that estimate is:
$$\frac{2n\bar{x}}{\chi^2_{\frac{\alpha}{2},2n}} &...
- 1,176
1
vote
1
answer
104
views
Difference between F-test and confidence intervals on variance estimates
Given n samples from a normally-distributed variable X, we estimate variance as $s^2=\frac{1}{n-1}\sum{(x_i - \bar{x})^2}$. We can also get a confidence interval for such a variance estimate as:
$$...
- 1,176
3
votes
0
answers
75
views
Different conclusion from confidence interval and NHST
I would like to ask. I gave to students an example where they test :
$H_0: \pi=0.65$
$Ha: \pi>0.65$
$\hat{p}= 0.76$
$n=50$
Students have to calculate NHST and also confidence interval for ...
- 93
2
votes
0
answers
102
views
Why don't deep learning-based computer vision studies utilize statistical tests for comparisons? [closed]
Many readers familiar with scientific articles in the field of deep learning-based computer vision might have observed a common practice: the absence of statistical significance tests for comparing ...
- 53
2
votes
0
answers
386
views
83.4% confidence intervals for parameter estimates other than means?
I've seen some discussion of using a 83.4% confidence interval to compare two parameter estimates, so that non-overlapping confidence intervals correspond to a statistical difference between these ...
- 11.6k
2
votes
1
answer
490
views
Confidence Intervals of ROC Curve's AUCs overlap but delong test is significant?
I am using ci.auc in the pROC library to calculate AUC's confidence intervals and roc.test to calculate delong test.
When I run the following:
...
- 21
1
vote
1
answer
100
views
Understanding the significance level of a confidence interval
I generated 1000 confidence intervals with 95% significance level and I am testing H0: μ = 0 vs H1: μ != 0.
What means if 97.5% of my confidence intervals have the 0 included? It should be exactly 95% ...
- 11
2
votes
1
answer
55
views
When failing to reject a null hypothesis can we also calculate the probability of observing the treatment assuming the alternative is true?
Say we have a null hypothesis, $H_0$ that there is no difference in the proportions between two arms of an experiment. You power the test to alpha and beta levels such that your $H_1$ MDE is 1% ...
8
votes
2
answers
715
views
Infinite upper confidence interval using Fisher exact test in R
I am having trouble interpreting the upper confidence interval for the odds ratio (OR) after executing Fisher's exact test in R.
I have the following contingency table which contains the response ...
- 81
1
vote
0
answers
13
views
Confidence interval for the mean time to failure
Suppose I have a physical experiment which involves repeatedly testing objects until one fails. I have a mathematical model for the mean time to failure and I would like to carry out an experiment to ...
- 2,077
8
votes
2
answers
263
views
T-test on entire set of data vs. 2 t-tests on 2 halves of the same set of data
I have a set of data samples which have been gathered from an electronic system. The system is in one of two states, which is selected at random before the data sample is collected. For simplicity, ...
1
vote
0
answers
46
views
Confused by p-values and hypothesis testing [duplicate]
I'm (re)studying hypothesis testing from the book All of Statistics by Larry Wasserman (2005, Chapter 10, link), because when I studied the concept in one of my university courses I didn't understand ...
- 43
0
votes
1
answer
85
views
How can I find a confidence intervall for $\theta$?
We have given a sample $X_1,...,X_n$ which is distributed with respect to $f(x;\theta)=e^{\theta-x} \Bbb{1}_{\theta\leq x<\infty}$. We want to use the statistic $T:=\min\{X_1,...,X_n\}$ to find a ...
- 117
0
votes
0
answers
21
views
A correct understanding of z-score confidence intervals with improper, informative priors [duplicate]
Please check that my understanding of hypothesis testing, confidence intervals, and their relation to the prior on population mean $\mu$ is correct.
Let $X_i\sim N(\mu, \sigma^2)$ be IID samples for $...
- 473
2
votes
1
answer
122
views
Likelihood Ratio vs Modified Frequentist Approach (CLs)
I'm a physicist trying to finally get a hold on practical statistics for particle physics and am having problem with the following -- I apologize for the lack of formality below.
Suppose the number of ...
- 121
0
votes
1
answer
334
views
How to test if there's a statistically significant difference between two counts, based on relative percentile?
How to test if there's a statistically significant difference between two counts?
I'm struggling to test if two ratios on the same row of my dataset are significantly different from each other. I need ...
0
votes
0
answers
50
views
What kind of formal guarantee does a confidence interval provide after an observation?
According to my understanding, $C(X)$ is a random variable defined as follows:
Let $\mathcal{P}$ be a family of distributions (defined by user). Let $\alpha>0$. Let $\theta$ be some parameter of a ...
- 211
0
votes
1
answer
75
views
Bootstrap P-value and confidence intervals with more than two samples
I have been trying to find a simple way to use the bootstrap for a hypothesis test that involves more than two samples. The motivation for using the bootstrap is for the usual reasons: the test ...
1
vote
1
answer
150
views
Confidence interval for transformed parameter
I am working on confidence intervals for transformed parameters in dose-response log-logistic model.
For simplicity, let's assume 2 parameter regression model with normal errors, where $\theta=(a,b)$ ...
- 11
6
votes
4
answers
730
views
Why do the bootstrap calculated p-value and the confidence intervals seem to contradict each other [R code, specific example]?
Below you'll find the data and the corresponding R code used to perform the bootstrap hypothesis test to compare the ratio of the means of two samples and additionally, to estimate the 95% confidence ...
- 144
1
vote
1
answer
151
views
Hypothesis testing using samples with different measurement errors/intervals
Are there generalizations of common hypothesis tests (e.g. t-test, mann-whitney) that can take into account different confidences in the sample measurements?
For example, if I have two sets of ...
- 111
2
votes
0
answers
47
views
If a confidence interval and test error are both expressions of a margin of error, why do we need both?
Let's say:
We have a population consisting of $N$ units, of which we take a sample of $n$ observations. For simplicity, this will be a bivariate dataset $(X, Y)$, whose true population is best ...
- 605
0
votes
0
answers
17
views
Does a distribution of difference-in-means, be considered as distribution of means?
I have mean sales data of a product, pre & post an event, for a list of countries.
i.e I have 2 sales value for a product per country.
My aim is to see if there ...
- 317
2
votes
2
answers
228
views
Intuition for inverting one sided hypothesis tests
Is there an intuitive way to remember if we get an upper bound or lower bound for a parameter when inverting a one-sided hypothesis test? In Casella & Berger, it is written:
If we have $H_0: \mu = ...
0
votes
0
answers
100
views
Is it useful to find a $\chi^2$ pivotal quantity?
My statistics class makes a big deal out of the following fact: for any random sample $X_1, \ldots, X_n$ with continuous, invertible cdf $F(x;\theta)$,
$$-2\sum_i\ln F(X_i;\theta) \sim \chi^2(2n).$$
...
- 251
2
votes
1
answer
670
views
How does Clopper and Pearson interval compute
Suppose we have $10$ observations and we run $20$ trials. A two-sided binomial test with $H_0:p=0.4$ from R I get is
...
- 207