Normalization condition with a neural network

[kelly0303]

Hello! I have some data points generated from an unknown distribution (say a 1D Gaussian for example) and I want to build a neural network able to approximate the underlaying distribution i.e. for any given $x$ as input to the neural network, I want the output to be as close as possible to the real $p(x)$, as given by the real (unknown distribution). I have in my loss function so far this: $$L = -\sum_i \log(p(x_i))$$ where the sum is over a minibatch. This loss, when minimized, should come close to the real distribution. However, I need to ensure that the predicted function is normalized i.e. $$\int_{-\infty}^{+\infty} p(x)dx = 1$$ otherwise $p(x)=1$ would minimize the loss function the way it is now. So I need my overall loss function to be something like this $$L = -\sum_i \log(p(x_i)) + |\int_{-\infty}^{+\infty} p(x)dx - 1|$$ How can I numerically impose the normalization condition such that to efficiently compute the loss during the training of the neural network? Thank you!

 

[sysprog]

What is the structure of your contemplated neural net? Will it dynamically adapt its structural complexity based on the input? Do you have any code or pseudocode that you could post?

 

[sysprog]

I stumbled across something on scanning for objects in 2D frames which I found interesting and which may be worth looking at for you: http://digitalassets.lib.berkeley.edu/ietfec/ucb/text/2007_682.pdf

 

[atyy]

You could try imposing the normalization by bruteforce using standard normalization or softmax: