Is the density of PixelCNN normalized?

Question

PixelCNN++ constructs a model distribution $p(x)$ over images $x\in\mathbb{R}^{n\times n}$ as a product of conditional distributions over pixels

$$p(x)=p(x_1,...,x_{n^2})=\prod_{i=1}^{n^2} p(x_i| x_1,...,x_{i-1})$$

In particular, we choose to model $p(x_i| x_1,...,x_{i-1})$ as a logistic distribution $L(\mu_i, s_i)$ where $\mu_i, s_i = neural\_net_\theta(x_1,...,x_{i-1})$ for parameters $\theta$. The authors claim that $p(x)$ is a probability density distribution, i.e.

$$\int_{\mathbb{R}^{n\times n}} p(x) \; dx= \int_{\mathbb{R}^{n\times n}} \prod_{i=1}^{n^2} L( neural\_net_\theta(x_1,...,x_{i-1})) \; dx=1$$

Q1. How can we guarantee that this integral is 1 for any neural network?

Q2. Is it true for any continuous function?

Answer 1

The answer to both questions follows basic probability definitions. Consider a discrete distribution over $\mathbf{x}=(x_1,x_2)\in\{0,1\}^2$. The equation then becomes

$$ \begin{align} \sum_{ \mathbf{x} \in \{0,1\}^2 } p(\mathbf{x}) &= \sum_{ x_1 \in \{0,1\} }\sum_{x_2\in\{0,1\}} p(x_1)\cdot p(x_2|x_1) \\ &=\sum_{ x_1 \in \{0,1\} }p(x_1)\cdot \sum_{x_2\in\{0,1\}} p(x_2|x_1) \\ &=\sum_{ x_1 \in \{0,1\} }p(x_1) \cdot 1 \\ &=1 \end{align}$$

So the answer to both questions

A1. Expand the equation using basic definitions of probability. Start with simpler example as done above for clarity.

A2. As long as the continues function allows us to use the above proof we are fine.