Starting PR to push DBN to Pylearn2. #1190
Conversation
|
|
||
| WRITEME | ||
| Sets up the inference procedure for the DBM. | ||
| If the number of hidden layers is one, then use UpDown. |
There was a problem hiding this comment.
This doc is more user-facing. The comment about which inference procedure would make more sense as a comment starting with '#' in the implementation.
|
OK, the tests are failing here but passing locally. I'll figure this out. |
| equilibrium. In practice, sampling the hidden units once is sufficient. | ||
| Mean field inference for RBMs is exactly solvable due to the | ||
| conditional independence of the hidden units. Following this, | ||
| inference should be composed of exactly one mean field update. |
There was a problem hiding this comment.
I'm not sure if this is completely accurate, as it depends on the assumption from which you're basing your MF parameterization, no?
There was a problem hiding this comment.
The parameterization can make the MF update more expensive but it can't make you need more than one update. This is assuming the model really is an RBM.
|
Ian could you take a look at the Travis build error... it's not happening locally, and honestly I don't know why bvmp tests are failing: this PR has nothing to do with the methods or classes implemented there. |
|
Question about DBM implementation in general: currently if you're training a generate model with the labels, the modeling layer is the last hidden layer of the DBM. However, this seems to add extra switches throughout to account for this. Wouldn't it be better to implement the label layer as another special layer (like visible_layer), or to even perhaps make the visible layers a list of layers? What if you had other observables as well? |
|
@rdevon These errors seem to have been introduced by a recent change in Theano, see Theano/Theano#2195. |
|
I restarted travis, the problem should have been fixed in Theano. |
|
Thanks! |
| Checks whether inference procedure is compatible with an RBM. | ||
|
|
||
| A restricted Boltzmann machine (RBM) is a deep Boltzmann machine (DBM) | ||
| with exactly one hidden layer. Exact inference on a RBM is possible |
There was a problem hiding this comment.
This isn't accurate. Exact inference over the hidden units given all the visible units of an RBM doesn't require any kind of sampling. In that context, exact inference is equivalent to mean field inference.
|
This needs a rebase now. When you rebase, go ahead and squash so we don't have too many separate commits. |
| """ | ||
| Is implemented as UpDown is RBM compatible. | ||
| """ | ||
| return |
There was a problem hiding this comment.
This returns None, should return a bool
Added RBM subclass of DBM for convenience. Add docs to DBM. Added chain initialization to DBM for some future changes to gradient estimation. Starting PR to push DBN to Pylearn2. Added RBM subclass of DBM for convenience. Add docs to DBM. Added chain initialization to DBM for some future changes to gradient estimation. fixed line formatting Made the docs numpydoc compliant. Modified inference procedure to have a method "is_rbm_compatible" that will raise a NotImplementedError if not appropriate for RBM. Removed assert for RBM and UpDown inference. Made the docs numpydoc compliant. Modified inference procedure to have a method "is_rbm_compatible" that will raise a NotImplementedError if not appropriate for RBM. Removed assert for RBM and UpDown inference.
|
OK, rebased and made changes. I think this will be the last PR to this branch, as I'm moving development to dbm_v2... dbn will follow it. After merge I will bring these changes to dbm_v2. |
|
@goodfeli You were reviewing this originally, any update? |
|
Got side-tracked. Hoping to pick this back up eventually but don't have time ATM. |
Added RBM subclass of DBM for convenience.
Add docs to DBM.
Added chain initialization to DBM for some future changes to gradient estimation.