Questions tagged [differentiable]
a function whose derivative exists at all points of its domain.
5 questions
0
votes
0
answers
23
views
Gradient flow through sampled tokens when training RNNs (but without teacher forcing)
Suppose we want to train an autoregressive generative language model based on a recurrent neural network (RNN) architecture without teacher forcing:
At each timestep, the RNN takes an input token $x_t$...
- 101
0
votes
0
answers
67
views
How can we use ReLU activation in a Normalizing Flow model? More generally, is differentiable almost everywhere enough for a normalizing flow?
In some works, e.g., enter link description here
normalizing flow models are considered with ReLU activation.
For example, using a planar flow, $f = f_n \circ ... \circ f_1$, and each $f_i$ has the ...
- 473
2
votes
0
answers
73
views
Asymptotic for LAD - Problem 16, p.84 Van der Vaart
From Van der Vaart's Asymptotics Statistics, we have the derivation of the asymptotics for the least square regression (Example 5.27). Now, the problem 16 of the same section regards the asymptotics ...
- 339
1
vote
1
answer
213
views
Differentiability in Generative Adversarial Networks
I've got some questions about the differentiability condition of GAN's, i.e. both G and D need to be differentiable wrt. their inputs and the parameters describing them. It's of more mathematical ...
- 11
5
votes
2
answers
412
views
Poisson log-likelihood is concave but not Lipschitz-continuous?
According to He et al. (2016), the log-likelihood of Poisson models,
$$
L (\beta) = \sum_{i} - \log(x_{i}^{\mathsf{T}}\beta) + y_{i}x_{i}^{\mathsf{T}}\beta - \log y_{i}!
$$
for a random variable, $y_{...
- 1,506