Questions tagged [normalizing-flow]
Random variate representations based on repeated one-to-one transforms of a standard random variable which produce generative models and closed-form densities.
31 questions
1
vote
0
answers
9
views
Understanding the Definition of Conditional Optimal Transport Path
In this tutorial about flow matching, the authors define a probability path interpolating between $p_0 = N(0, I)$ and a target distribution $q$ as follows (Equation 2.2 of the paper):
$$p_t(x) = \int{...
- 11
2
votes
1
answer
45
views
Posterior estimation using VAE
Using normalizing flows, we can model model's posteriors $p(\theta|D)$, by feeding Gaussian noise $z$ to the NF (parametrized with $\phi$), using the output of the NF $\theta$ as model parameters, and ...
- 1,381
1
vote
0
answers
18
views
Can normalizing flows approximate bounded distributions in deep learning?
I’m exploring the use of normalizing flows in deep learning for generative modeling and I have a specific requirement: my target distributions are bounded (for example, between 0 and 1). I understand ...
0
votes
0
answers
56
views
How to train a continuous-time normalizing flow model?
I'm confused on how we can actually train a continuous-time normalizing flow model. There are two use cases for the discrete-time (original) normalizing flows, and I've tried to outline how I would do ...
- 473
0
votes
0
answers
67
views
How can we use ReLU activation in a Normalizing Flow model? More generally, is differentiable almost everywhere enough for a normalizing flow?
In some works, e.g., enter link description here
normalizing flow models are considered with ReLU activation.
For example, using a planar flow, $f = f_n \circ ... \circ f_1$, and each $f_i$ has the ...
- 473
1
vote
0
answers
18
views
Help finding Bayesian model with multi-modal posterior [closed]
Background:
There is a paper (link) that concerns combining MCMC methods with a normalizing flow (a type of generative model). The basic idea is that the normalizing flow helps propose samples, which ...
- 31
1
vote
0
answers
27
views
At what circumstances will the difficulty for the tasks of density evaluation and sampling be different?
In this tutorial video of normalizing flow, the presenter mentioned that for the original autoregressive flow, the density evaluation is fast and the sampling is slow. In contrast, for the inverse ...
- 141
5
votes
0
answers
186
views
What is the difference between copulas and normalizing flows?
The goal of normalizing flows is to produce arbitrarily complex probability-distributions from a simple distribution (usually the Normal distribution) through learning an invertible transform.
Copulas ...
- 61
1
vote
0
answers
58
views
Choice of base distribution in normalizing flows for product distribution
I'm currently trying to implement a normalizing flow (NF) to efficiently sample from a product distribution $p=fg$. Contrary to most examples I've come across, I actually know how to sample ...
- 131
0
votes
1
answer
102
views
Normalizing Flows Invertibility
I am currently reading up on RealNVP, which has the following transformations according Lilian Weng:
$$
\begin{aligned}
\mathbf{y}_{1:d} &= \mathbf{x}_{1:d} \\
\mathbf{y}_{d+1:D} &= \mathbf{x}...
- 19
0
votes
0
answers
87
views
Normalizing Flow Penalization
I am looking to train a normalizing flow, specifically a Masked Autoregressive Flow model. However, this model leads to high variance on lower dimensional, less complex data. I am using a neural ...
- 19
4
votes
2
answers
1k
views
Normalizing Flows KL divergence equivalency
This question is related to the normalizing flows concept in machine learning.
Let $X \sim P_X$ and $U \sim P_U$ be, respectively, the distribution of the data and a base distribution (e.g. an ...
- 141
3
votes
1
answer
166
views
Which parameters are updated in VAE with normalizing flow?
I've been reading this article about implementing a VAE with normalizing flows. What it's not clear to me, is which parameters are actually optimized using this approach. Should I only optimize the ...
- 659
0
votes
1
answer
70
views
Planar flows implementation to approximate Gamma distribution
I've been trying to implement in order to approximate Gamma distribution but the problem I've been encountering is that the hyperbolic tangent activation function that I used gives negative values, ...
5
votes
1
answer
322
views
loss function that penalizes empirical CDF
I have been doing literature review of generative models. From what I gather, there are likelihood based generative models that model the likelihood and use it as objective function to learn the ...
- 245
2
votes
1
answer
297
views
Issues with GAN and VAE models
I'm reading this amazing paper on Normalizing Flows https://arxiv.org/pdf/1908.09257.pdf but one sentence kind of bothers me:
GANs and VAEs have demonstrated impressive performance results on
...
1
vote
0
answers
134
views
Neural Networks with Tractable Integral [closed]
I'm looking for neural networks $n_\theta(x)$ which integrate to a constant $\int_{[0,1]^d} n(x)\; dx=c\in\mathbb{R}$. Notably, the constant should be the same for any weights $\theta$ for which $n_\...
4
votes
2
answers
433
views
Why aren't Normalizing Flows suitable for Discrete Distributions?
I am currently trying to understand why normalizing flows are not applicable to discrete distributions (a quick primer on NF can be found here).
The assumptions on the transformation f between the ...
- 219
0
votes
1
answer
121
views
Proof for expressive power of flow-based models: Normalizing Flows for Probabilistic Modeling and Inference by Papamakarios et al
I've been reading the great summary work on Normalizing Flows "Normalizing Flows for Probabilistic Modeling and Inference" by Papamakarios et al..
A few questions regarding a proof came up ...
0
votes
0
answers
112
views
Inverse Neural Networks
Suppose there is a series of transformation applied to the random variable $z_0$ such that
$$
z_M = f_{\theta_{M}} \circ f_{\theta_{M-1}} \circ \ldots \circ f_{\theta_{1}}(z_0) =: f_{\theta}(z_0).
$$
...
- 125
5
votes
1
answer
2k
views
Normalizing flows as a generalization of variational autoencoders?
Normalizing flows are often introduced as a way of restricting the rigid priors that are placed on the latent variables in Variational Autoencoders.
For example, from the Pyro docs:
In standard ...
- 299
3
votes
1
answer
2k
views
Normalizing flow training
I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong.
There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
- 145
2
votes
0
answers
94
views
Normalizing Flows notations and change of variable formula
Maybe it is a silly question, but I am wondering why in some papers the function $f$ is the mapping from $x$ to $z$ and in some others it is the other way around. Typically in this review we go from $...
- 237
1
vote
1
answer
888
views
What is multi-scale architecture?
When I was reading the paper "Density Estimation using Real NVP", I have found the term "multi scale architecture".
We implement a multi-scale architecture using a squeezing ...
- 309
4
votes
1
answer
1k
views
Inference in Normalizing Flow model: NICE(non linear independent components estimation)
I was recently reading Y. Bengio's paper on NICE (https://arxiv.org/abs/1410.8516). In the paper, authors have taken a view that a good representation involves easy learning of the data distribution. ...
- 181
4
votes
1
answer
842
views
Vector-Jacobian Product Computational Cost
The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
- 43
2
votes
1
answer
297
views
Confused by the latent variables in normalizing flow theory
I'm looking through the paper on variationl inference in normalizing flow and have difficulties with understanding some ideas.
I know there are latent variables $\mathbf{z}_i$ and observed variables $\...
- 197
1
vote
0
answers
40
views
Can we ignore the generation side of the method described in density estimation using Real NVP?
First appologies if my question is stupid. I am studying the paper "Density estimation using real NVP" by Dinh, Sohl-Dickstein and Bengio. link
The paper presented a nice idea that the generation ...
- 11
7
votes
2
answers
3k
views
Planar Flow in Normalizing Flows
While I've read "Variational Inference with Normalizing Flows" (abstract), I don't understand about an intuition of Planar Flow.
The author defined Planar Flow as below
Let $\boldsymbol{w} \in \...
- 271
6
votes
2
answers
5k
views
Difference between invertible NN and flow-based NN
After a little bit of reading on these two terms, I have the impression they are used for the same thing. So is there actually a difference between these two concepts, and if so, how are they ...
- 563
7
votes
1
answer
2k
views
What are the advantages of normalizing flow over VAEs with deep latent gaussian models for inference?
I am reading the normalizing flow paper and am a bit confused.
It seems that being able to model complex (correlated?) posterior is one of the advantages of the proposed approach (Section 2.3, last ...
- 115