Exact Multinomial Test
Uwe Menzelvisibility
…
description
8 pages
link
1 file
Description The package provides functions to carry out a Goodness-of-fit test for discrete multivariate data. It is tested if a given observation is likely to have occurred under the assumption of an ab-initio model. A p-value can be calculated using different distance measures between observed and expected frequencies. A Monte Carlo method is provided to make the package capable of solving high-dimensional problems.
Related papers
Package 'emt' Title Exact Multinomial Test: Goodness-of-fit Test for Discrete Multivariate Data
2013
Description The package provides functions to carry out a Goodness-of-fit test for discrete multivariate data. It is tested if a given observation is likely to have occurred under the assumption of an ab-initio model. A p-value can be calculated using different distance measures between observed and expected frequencies. A Monte Carlo method is provided to make the package capable of solving high-dimensional problems.
Goodness-of-fit testing using components based on marginal frequencies of multinomial data
British Journal of Mathematical and Statistical Psychology, 2008
The goodness-of-fit test based on Pearson's chi-squared statistic is sometimes considered to be an omnibus test that gives little guidance to the source of poor fit when the null hypothesis is rejected. It has also been recognized that the omnibus test can often be outperformed by focused or directional tests of lower order. In this paper, a test is considered for a model on a data table formed by the cross-classification of q dichotomous variables, and a score statistic on overlapping cells that correspond to the first-through qth-order marginal frequencies is presented. Then orthogonal components of the Pearson-Fisher statistic are defined on marginal frequencies. The orthogonal components may be used to form test statistics, and a log-linear version of an item response model is used to investigate the order and dilution of a test based on these components, as well as the projection of components onto the space of lowerorder marginals. The advantage of the components in terms of power and detection of the source of poor fit is demonstrated. Overcoming the adverse effects of sparseness provides another motive for using components based on marginal frequencies because an asymptotic chi-squared distribution will be more reliable for a statistic formed on overlapping cells if expected frequencies in the joint distribution are small.
Improved power in multinomial goodness-of-fit tests
Journal of the Royal Statistical Society: Series D (The Statistician), 2002
The Pearson's chi-square and the log likelihood ratio chi-square statistics are fundamental tools in goodnessof-fit testing. Cressie and Read (1984) constructed a general family of divergences which includes both statistics as special cases. This family is indexed by a single parameter, and divergences at either end of the scale are more powerful against alternatives of one type while being rather poor against the opposite type. Here we present several new goodness-of-fit testing procedures which have reasonably high power at both kinds of alternatives. Graphical studies illustrate the advantages of the new methods.
Estimate-based goodness-of-fit test for large sparse multinomial distributions
Computational Statistics & Data Analysis, 2009
The Pearson's chi-squared statistic (X 2 ) does not in general follow a chi-square distribution when it is used for goodness-of-fit testing for a multinomial distribution based on sparse contingency table data. We explore properties of . Goodness-of-fit tests for large sparse multinomial distributions. J. Amer. Statist. Assoc. 82 (398), 624-629] D 2 statistic and compare them with those of X 2 and compare the power of goodness-of-fit test among the tests using D 2 , X 2 , and the statistic (L r ) which is proposed by . Limited-and full-information estimation and goodness-of-fit testing in 2 n contingency tables: A unified framework. J. Amer. Statist. Assoc. 100 (471), 1009-1020] when the given contingency table is very sparse. We show that the variance of D 2 is not larger than the variance of X 2 under null hypotheses where all the cell probabilities are positive, that the distribution of D 2 becomes more skewed as the multinomial distribution becomes more asymmetric and sparse, and that, as for the L r statistic, the power of the goodness-of-fit testing depends on the models which are selected for the testing. A simulation experiment strongly recommends to use both D 2 and L r for goodness-of-fit testing with large sparse contingency table data.
A general family of limited information goodness-of-fit statistics for multinomial data
Maydeu-Olivares and Joe (J. Am. Stat. Assoc. 100:1009–1020, 2005; Psychometrika 71:713–732, 2006) introduced classes of chi-square tests for (sparse) multidimensional multinomial data based on loworder marginal proportions. Our extension provides general conditions under which quadratic forms in linear functions of cell residuals are asymptotically chi-square. The new statistics need not be based on margins, and can be used for one-dimensional multinomials. We also provide theory that explains why limited information statistics have good power, regardless of sparseness. We show how quadratic-form statistics can be constructed that are more powerful than X2 and yet, have approximate chi-square null distribution in finite samples with large models. Examples with models for truncated count data and binary item response data are used to illustrate the theory.
Investigating the power of goodness-of-fit tests for multinomial logistic regression
Communications in Statistics - Simulation and Computation, 2017
Goodness-of-fit tests are important to assess if the model fits the data. In this paper we investigate the Type I error and power of two goodness-of-fit tests for multinomial logistic regression via a simulation study. The GoF test using partitioning strategy (clustering) in the covariate space, was compared with another test, C g which was based on grouping of predicted probabilities. The power of both tests was investigated when the quadratic term or an interaction term were omitted from the model. The proposed test 2 *G p shows good Type I error and ample power except for models with highly skewed covariate distribution. The
A goodness-of-fit test in the multinomial legit model based on weighted squared residuals
Statistica Neerlandica, 1994
The asymptotic (normal) distribution of the sum of weighted squared residuals in the multinomial logit model is derived. The performance of a chi-squared test based on the asymptotic normality is discussed using some empirical examples.
Chi-Square Orthogonal Components for Assessing Goodness-of-fit of Multidimensional Multinomial Data
2011
It is common in the analysis of data to provide a goodness-of-fit test to assess the performance of a model. In the analysis of contingency tables, goodness-of-fit statistics are frequently employed when modeling social science, educational or psychological data ...
Multibridge: an R package to evaluate informed hypotheses in binomial and multinomial models
Behavior Research Methods, 2023
The multibridge R package allows a Bayesian evaluation of informed hypotheses H r applied to frequency data from an independent binomial or multinomial distribution. multibridge uses bridge sampling to efficiently compute Bayes factors for the following hypotheses concerning the latent category proportions θ: (a) hypotheses that postulate equality constraints (e.g., θ 1 = θ 2 = θ 3); (b) hypotheses that postulate inequality constraints (e.g., θ 1 < θ 2 < θ 3 or θ 1 > θ 2 > θ 3); (c) hypotheses that postulate combinations of inequality constraints and equality constraints (e.g., θ 1 < θ 2 = θ 3); and (d) hypotheses that postulate combinations of (a)-(c) (e.g., θ 1 < (θ 2 = θ 3), θ 4). Any informed hypothesis H r may be compared against the encompassing hypothesis H e that all category proportions vary freely, or against the null hypothesis H 0 that all category proportions are equal. multibridge facilitates the fast and accurate comparison of large models with many constraints and models for which relatively little posterior mass falls in the restricted parameter space. This paper describes the underlying methodology and illustrates the use of multibridge through fully reproducible examples.
A goodness-of-fit test for the multivariate Poisson distribution
2016
Bivariate count data arise in several different discipline s and the bivariate Poisson distribution is commonly used to model them. This paper proposes and studies a computationally convenient goodness-of-fit test for this distribution, which is based o n an empirical counterpart of a system of equations. The test is consistent against fixed alternative s. The null distribution of the test can be consistently approximated by a parametric bootstrap and by a weighted bootstrap. The goodness of these bootstrap estimators and the power for finite sample sizes are numerically studied. It is shown that the proposed test can be naturally extended to the multivariate Poisson distribution
Package ‘EMT’
July 2, 2014
Type Package
Title Exact Multinomial Test: Goodness-of-Fit Test for Discrete Multivariate data
Version 1.1
Date 2013-01-27
Author Uwe Menzel
Maintainer Uwe Menzel <[email protected]>
Description The package provides functions to carry out a
Goodness-of-fit test for discrete multivariate data. It is
tested if a given observation is likely to have occurred under
the assumption of an ab-initio model. A p-value can be
calculated using different distance measures between observed
and expected frequencies. A Monte Carlo method is provided to
make the package capable of solving high-dimensional problems.
License GPL
LazyLoad yes
Repository CRAN
Date/Publication 2013-01-29 10:28:59
NeedsCompilation no
R topics documented:
EMT-package .
EMT-internal .
multinomial.test
plotMultinom .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Index
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
2
2
3
6
8
1
2
EMT-internal
EMT-package
Exact Multinomial Test: Goodness-of-Fit Test for Discrete Multivariate data
Description
The package provides functions to carry out a Goodness-of-fit test for discrete multivariate data.
It is tested if a given observation is likely to have occurred under the assumption of an ab-initio
model. A p-value can be calculated using different distance measures between observed and expected frequencies. A Monte Carlo method is provided to make the package capable of solving
high-dimensional problems. The main user functions are multinomial.test and plotMultinom.
Details
Package:
Type:
Version:
Date:
License:
CCP
Package
0.1
2009-12-14
GPL
Author(s)
Uwe Menzel
Maintainer: Uwe Menzel <[email protected]>
EMT-internal
Internal functions for the EMT package
Description
Internal functions for the EMT package
Usage
ExactMultinomialTest(observed, prob, size, groups, numEvents)
ExactMultinomialTestChisquare(observed, prob, size, groups, numEvents)
MonteCarloMultinomialTest(observed, prob, size, groups, numEvents, ntrial, atOnce)
MonteCarloMultinomialTestChisquare(observed, prob, size, groups, numEvents, ntrial, atOnce)
chisqStat(observed,expected)
findVectors(groups,size)
3
multinomial.test
Arguments
observed
prob
expected
size
groups
numEvents
ntrial
atOnce
vector describing the observation: contains the observed numbers of items in
each category.
vector describing the model: contains the hypothetical probabilities corresponding to each category.
vector containing the expected numbers of items in each category under the
assumption that the model is valid.
sample size, sum of the components of the vector observed.
number of categories in the experiment.
number of possible outcomes of the experiment.
number of simulated samples in the Monte Carlo approach.
a parameter of more technical nature. Determines how much memory is used
for big arrays.
Details
These functions are not intended to be called by the user.
multinomial.test
Exact Multinomial Test: Goodness-of-Fit Test for Discrete Multivariate data
Description
This function runs a Goodness-of-fit test for discrete multivariate data. It is tested if a given observation is likely to have occurred under the assumption of an ab-initio model. A p-value can be
calculated using different distance measures between observed and expected frequencies. A Monte
Carlo method is provided to make the function capable of solving high-dimensional problems.
Usage
multinomial.test(observed, prob, useChisq = FALSE, MonteCarlo = FALSE, ntrial = 100000, atOnce = 10000
Arguments
observed
prob
useChisq
MonteCarlo
ntrial
atOnce
vector describing the observation: contains the observed numbers of items in
each category.
vector describing the model: contains the hypothetical probabilities corresponding to each category.
if TRUE, Pearson’s chisquare is used as a distance measure between observed and
expected frequencies.
if TRUE, the Monte Carlo approach is used.
number of simulated samples in the Monte Carlo approach.
a parameter of more technical nature. Determines how much memory is used
for big arrays.
4
multinomial.test
Details
The Exact Multinomial Test is a Goodness-of-fit test for discrete multivariate data. It is tested if
a given observation is likely to have occurred under the assumption of an ab-initio model. In the
experimental setup belonging to the test, n items fall into k categories with certain probabilities
(sample size n with k categories). The observation, described by the vector observed, indicates
how many items have been observed in each category. The model, described by the vector prob,
assigns to each category the hypothetical probability that an item falls into it. Now, if the observation is unlikely to have occurred under the assumption of the model, it is advisible to regard
the model as not valid. The p-value estimates how likely the observation is, given the model.
In particular, low p-values suggest that the model is not valid. The default approach used by
multinomial.test obtains the p-values by calculating the exact probabilities of all possible outcomes given n and k, using the multinomial probability distribution function dmultinom provided
by R. Then, by default, the p-value is obtained by summing the probabilities of all outcomes which
are less likely than the observed outcome (or equally likely as the observed outcome), i.e. by
summing all p(i) <= p(observed) (distance measure based on probabilities). Alternatively, the
p-value can be obtained by summing the probabilities of all outcomes connected with a chisquare
no smaller than the chisquare connected with the actual observation (distance measure based on
chisquare). The latter is triggered by setting useChisq = TRUE. Having a sample of size n in an
experiment with k categories, the number of distinct possible outcomes is the binomial coefficient
choose(n+k-1,k-1). This number grows rapidly with increasing parameters n and k. If the parameters grow too big, numerical calculation might fail because of time or memory limitations. In
this case, usage of the Monte Carlo approach provided by multinomial.test is suggested. The
Monte Carlo approach, activated by setting MonteCarlo = TRUE, simulates withdrawal of ntrial
samples of size n from the hypothetical distribution specified by the vector prob. The default value
for ntrial is 100000 but might be incremented for big n and k. The advantage of the Monte Carlo
approach is that memory requirements and running time are essentially determined by ntrial but
not by n or k. By default, the p-value is then obtained by summing the relative frequencies of occurrence of unusual outcomes, i.e. of outcomes occurring less frequently than the observed one (or
equally frequent as the observed one). Alternatively, as above, Pearson’s chisquare can be used as a
distance measure by setting useChisq = TRUE. The parameter atOnce is of more technical nature,
with a default value of 1000000. This value should be decremented for computers with low memory
to avoid overflow, and can be incremented for large-CPU computers to speed up calculations. The
parameter is only effective for Monte Carlo calculations.
Value
id
textual description of the method used.
size
sample size n, equals the sum of the components of the vector observed.
groups
number of categories k in the experiment, equals the number of components of
the vector observed.
stat
textual description of the distance measure used.
allProb
vector containing the probabilities (rel. frequencies for the Monte Carlo approach) of all possible outcomes (might be huge for big n and k).
ntrial
number of trials if the Monte Carlo approach was used, NULL otherwise.
p.value
the calculated p-value rounded to four significant digits.
multinomial.test
5
Note
For two categories (k = 2), the test is called Exact Binomial Test.
Author(s)
Uwe Menzel <[email protected]>
References
H. Bayo Lawal (2003) Categorical data analysis with SAS and SPSS applications, Volume 1, Chapter 3 ISBN: 978-0-8058-4605-8
Read, T. R. C. and Cressie, N. A. C. (1988). Goodness-of-fit statistics for discrete multivariate data.
Springer, New York.
See Also
The Multinomial Distribution: dmultinom
Examples
## Load the EMT package:
library(EMT)
## Input data for a three-dimensional case:
observed <- c(5,2,1)
# observed data: 5 items in category one, 2 items in category two, 1 item in category three
prob <- c(0.25, 0.5, 0.25) # model: hypothetical probability that an item falls into category one, two, or three
## Calculate p-value using default options:
out <- multinomial.test(observed, prob)
# p.value = 0.0767
## Plot the probabilities for each event:
plotMultinom(out)
## Calculate p-value for the same input using Pearson's chisquare as a distance measure:
out <- multinomial.test(observed, prob, useChisq = TRUE)
# p.value = 0.0596 ; not the same!
## Test the hypothesis that all sides of a dice pop up with the same probability (a 6-dimesional problem):
pdice = 1/6
prob <- c(pdice, pdice, pdice, pdice, pdice, pdice) # the model, determined by the hypothetical probabilities
observed <- c(4, 5, 2, 7, 0, 1) # the observation consisting of 19 throws ( = sample size)
out <- multinomial.test(observed, prob)
# p.value = 0.0357 ; better get another dice, this one seems to be biased
plotMultinom(out, showmax = 10000)
6
plotMultinom
# the same problem using a Monte Carlo approach:
# we have about 40.000 outcomes and choose at least 400000 trials (probably to be increased):
out <- multinomial.test(observed, prob, MonteCarlo = TRUE, ntrial = 400000)
# p.value = 0.0343 ; takes a few minutes on a laptop with 2 GB memory, 1.5 GHz speed
plotMultinom(out, showmax = 5000)
plotMultinom
Plot the Probability distribution fot the Exact Multinomial Test
Description
This function takes the results of multinomial.test as input and plots the calculated probability
distribution.
Usage
plotMultinom(listMultinom, showmax = 50)
Arguments
listMultinom
a list created by running the function multinomial.test.
showmax
maximum number of bars to show in the plot (to avoid long tails).
Details
The function multinomial.test creates an output list that is used as input in plotMultinom to
depict some results. If the default approach was used, the figure shows the exact probabilities of all
possible outcomes of the experiment. If the Monte Carlo approach was used, the relative frequencies
of the outcomes are shown as occurred during the simulated withdrawals. The probabilities/relative
frequencies are shown in descending order from the left to the right.
Value
The first argument (listMultinom) is returned without modification.
Note
For better visibility, the parameter showmax excludes very long right tails from the plot. However,
the default value of showmax should be incremented to get a significant plot if the number of possible
outcomes is big.
Author(s)
Uwe Menzel <[email protected]>
plotMultinom
See Also
The Multinomial Distribution: multinomial.test
Examples
## Load the EMT package:
library(EMT)
## input and calculation of p-values:
observed <- c(5,2,1)
prob <- c(0.25, 0.5, 0.25)
out <- multinomial.test(observed, prob)
## Plot the probability distribution:
plotMultinom(out)
plotMultinom(out, showmax = 30) # suppress part of the tail in the plot
7
Index
∗Topic htest
EMT-package, 2
multinomial.test, 3
plotMultinom, 6
∗Topic multivariate
EMT-package, 2
multinomial.test, 3
plotMultinom, 6
chisqStat (EMT-internal), 2
dmultinom, 4, 5
EMT (EMT-package), 2
EMT-internal, 2
EMT-package, 2
ExactMultinomialTest (EMT-internal), 2
ExactMultinomialTestChisquare
(EMT-internal), 2
findVectors (EMT-internal), 2
MonteCarloMultinomialTest
(EMT-internal), 2
MonteCarloMultinomialTestChisquare
(EMT-internal), 2
multinomial.test, 3, 7
plotMultinom, 6
8
Related papers
Contemporary implications of the first-century counter-ethos of Jesus to the scripted universe of gender and health in John 4 & 9. A narrative-critical analysis
Stellenbosch University, 2015
Secure and efficient handover schemes for WiMAX over EPON networks
International Conference on Circuits, 2010
Applicability of the Teetered Bed Separator for Beneficiating Indian Iron Ore Fines: An Experimental Study
Journal of Mines, Metals and Fuels, 2023
The impact of online advertisements on the purchasing habits of Nigerian University Students: A Case Study of Adekunle Ajasin University Akungba Akoko, Ondo State
World Jounal of Advanced Research and Review , 2024
Flaches Land und 1000 Hügel. Der denkmalpflegerische Umgang mit Wurten zwischen Wangerland und Butjadingen.
J.F. Kegler (ed.), Ostfriesland / Niedersachsenweit. Festschrift für Rolf Bärenfänger. Abhandl. u. Vorträge Geschichte Ostfriesland 87, 2020
The Serial Commutator FFT
IEEE Transactions on Circuits and Systems II: Express Briefs, 2016
L'azienda coniugale
Famiglia e matrimonio. Regime patrimoniale, in Trattato di diritto di famiglia, I, 2° ed., diretto da Bonilini, 2022
DERECHO Y REVOLUCIÓN EN STUCKA1
Crítica jurídica y política en Nuestra América, 17, 47 (CLACSO), 2023
Prácticas comunicativas en danza un estudio de caso del Balcón de los Artistas
Prácticas comunicativas en danza: un estudio de caso del Balcón de los Artistas como plataforma de expresión artística y creación de narrativas corporales entre niños y jóvenes de la ciudad de Medellín (Colombia), 2024
Реконструкція інженерних споруд та мереж, ландшафту на основі технології лазерного сканування
Sučasnì tehnologìï ta metodi rozrahunkìv u budìvnictvì, 2023
Optimization of the fermentation time and bacteria cell concentration in the starter culture for cyanide acid removal from wild cassava (Manihot glaziovii)
MATEC Web of Conferences, 2018
A tightly clustered hepatitis E virus genotype 1a is associated with endemic and outbreak infections in Bangladesh
PLOS ONE, 2021
Functional metagenomic landscape of polluted river reveals potential genes involved in degradation of xenobiotic pollutants
Environmental Research, 2021
Synthesis of a plane wave in the near field with a planar phased array
IEEE Antennas and Propagation Society International Symposium. Digest. Held in conjunction with: USNC/CNC/URSI North American Radio Sci. Meeting (Cat. No.03CH37450)
Brain-Targeting Form of Docosahexaenoic Acid for Experimental Stroke Treatment: MRI Evaluation and Anti-Oxidant Impact
Current Neurovascular Research, 2011
A study on ideal elements in ordered Gamma-semirings
Discussiones Mathematicae - General Algebra and Applications
Gait Deviations in Children With Osteogenesis Imperfecta Type I
Journal of pediatric orthopedics, 2017
Aportación al conocimiento de la bioecología de la raya fluvio-estuarina Potamotrygon sp. (Myliobatiformes, Potamotrygonidae) en el delta del Orinoco y golfo de Paria, Venezuela
Memoria De La Fundacion La Salle De Ciencias Naturales, 2007