Bag prior weights

[amueller]

Hi!
I wanted to check in about the weights in the bag prior. It seems they are 0.961 for the MLP and 0.038 for the GP. Does that seem right? That's what I got after MCMC'ing out the double sampling.

[SamuelGabriel]

That can very well be. The GP prior is not very helpful for most datasets, and the MLP prior does include the SCM setting if I remember correctly, so that would check out. And what do you mean by MCMC'ing out? Shouldn't it just be the softmax over this line?

TabPFN/tabpfn/scripts/model_builder.py

Line 225 in 5805a9a

prior_bag_hyperparameters = {'prior_bag_get_batch': (get_batch_gp, get_batch_mlp)

i.e. softmax(2,1)[0] = .73?

[amueller]

Hm I don't think that line is used, because there's this:

TabPFN/tabpfn/priors/prior_bag.py

Line 15 in 5805a9a

prior_bag_priors_p = [1.0] + [hyperparameters[f'prior_bag_exp_weights_{i}'] for i in range(1, len(prior_bag_priors_get_batch))]

which overwrites the parameter in the chaining of get_batch. So you're first sampling a number x between 2 and 10 and then you're sampling from the softmax(1, x) That is what I did the sampling over (actually MC'ing, not MCMC'ing, sorry)