Restricted Boltzmann machine uniqueness

  • I
  • Thread starter Jufa
  • Start date
  • #1
81
10
I am dealing with restricted boltzmann machines to model distributuins in my final degree project and some question has come to my mind.
A restricted boltzmann machine with v visible binary neurons and h hidden neurons models a distribution in the following manner:

## f_i= e^{ \sum_k b[k] \sigma^i[k] + \sum_s \log(c[ s ] + e^{\sum_k w[ s ][ k ] b[ k ] })} ##
## Z = \sum_i f_i ##
## p_i = f_i/Z ##
Where b[ k ] and c[ s ] are, respectively, the k-th and the s-th bias of, again respectively, the visible and hidden layer.
w[ s ][k] is the component s, k of the weight matrix of the network.
"i" here refers to a certain binary vector with components ##\sigma^i[k]##.
My question is:
Given a certain restricted boltzmann machine (i.e. a certain set of biases and weights) that models a certain distribution ##p_i##, is it possible to find another configuration (i.e. a different set of parameters and weights) such that it gives the same distribution?
Thanks in advance.
 
Last edited by a moderator:

Answers and Replies

  • #2
81
10
If some moderator can explain why my text is strikethrough I would appreciate it.
 
  • #3
12,755
6,627
We are attempting to fix the problem. I got rid of the strike through text but now the latex Mathjax isn’t rendering. We moved to a new version of forum software with a new post editor and perhaps it has injected or requires some special way to enter math expressions.
 
  • #5
14,756
12,138
I think I have reset the corrected original version. Please refresh your browsers.

Hint: Do not use [B],[U],[S],[I]. If you do not "cheat" as I did here, the interpreter takes it for the command tag BEGIN bold / underline / strikeout / italic.
 
  • Like
Likes Jufa and jedishrfu
  • #6
81
10
Many thanks!
 
  • #7
TeethWhitener
Science Advisor
Gold Member
1,964
1,404
I am dealing with restricted boltzmann machines to model distributuins in my final degree project and some question has come to my mind.
A restricted boltzmann machine with v visible binary neurons and h hidden neurons models a distribution in the following manner:

## f_i= e^{ \sum_k b[k] \sigma^i[k] + \sum_s \log(c[ s ] + e^{\sum_k w[ s ][ k ] b[ k ] })} ##
## Z = \sum_i f_i ##
## p_i = f_i/Z ##
Where b[ k ] and c[ s ] are, respectively, the k-th and the s-th bias of, again respectively, the visible and hidden layer.
w[ s ][k] is the component s, k of the weight matrix of the network.
"i" here refers to a certain binary vector with components ##\sigma^i[k]##.
My question is:
Given a certain restricted boltzmann machine (i.e. a certain set of biases and weights) that models a certain distribution ##p_i##, is it possible to find another configuration (i.e. a different set of parameters and weights) such that it gives the same distribution?
Thanks in advance.
I’m not very familiar with the topic, but trivially, if you set ##c[ s]=0##, then you get the same ##f_i## by exchanging the ##w[ s][k]## and the ##\sigma^i[k]##. So for instance,
$$f_{i,1}=\exp{\left(\sum_k{b[k]\sigma_1^i[k]}+\sum_s{ \log{e^{\sum_k{w_1[ s][k]b[k]}}}}\right)}$$
$$f_{i,2}=\exp{\left(\sum_k{b[k]\sigma_2^i[k]}+\sum_s{ \log{e^{\sum_k{w_2[ s][k]b[k]}}}}\right)}$$
Then let ##\sigma_1^i[k]=w_2[ s][k]## and ##\sigma_2^i[k]=w_1[ s][k]##.
That gives you the same probability distributions but I don’t know if that’s allowed with restricted Boltzmann machines.
 
Last edited:

Related Threads on Restricted Boltzmann machine uniqueness

  • Last Post
Replies
10
Views
6K
  • Last Post
Replies
1
Views
2K
Replies
0
Views
2K
  • Last Post
Replies
2
Views
2K
  • Last Post
Replies
2
Views
1K
  • Last Post
Replies
11
Views
15K
  • Last Post
Replies
3
Views
549
Replies
2
Views
2K
M
  • Last Post
Replies
5
Views
800
Replies
6
Views
1K
Top