Support pr inference in differentiable computation graph

[rtjoa]

Summary

We update autodiff to represent the log probabilities of Dist{Bools}s symbolically, to train arbitrary loss functions instead of just doing MLE.

In fact, we support "arbitrary interleavings" - computation dependent on log probabilities can be used as flip parameters, to create more symbolic log probabilities, etc.

We also refactor to shrink the interface where possible, clean up BDD differentiation, and add autodiff support for matrices and trig functions.

Main API changes

The core construct we add is the struct LogPr(::Dist{Bool}) <: ADNode.

• To compute an ADNode containing a LogPr, use compute_mixed rather than compute.
• To perform inference on a Dist containing a flip whose probability is dependent on a LogPr, use pr_mixed rather than pr.
• train!(::Valuation, loss::ADNode; epochs, learning_rate) updates a valuation (dict from Vars to values) to minimize loss by GD

We also add basic loss functions, mle_loss and kl_divergence.

Tests

New tests are added to test/autodiff_pr.