Deviance Information Criterion (DIC) in greta?

Hi All,

I’m working on fitting some logit regression models to a large dataset, and I’ll need to do model selection using DIC. It looks like DIC isn’t built in to greta, so I was wondering if anyone already has a solution.

I can write my own function to calculate it on the greta output, but I wanted to post here first to say some sort of built-in DIC function would be a new feature for greta, and to see if there is already a robust solution out there.

Thanks!
Vincent

Yes, it would be handy. Should be fairly straightforward to implement, too.

What do you think the interface should look like?

I think the user could have the option to provide the likelihood to use for calculating deviance in the form deviance_likelihood(y) = ... (but with a better function name), that defaults to the likelihood of the data. An option for the user to specify it will come in handy if an integrated likelihood should be used (for example for models where a latent state needs to be integrated out of the likelihood before calculating DIC). Then maybe a simple flag somewhere (maybe in mcmc(..., DIC = TRUE)?) to specify that DIC should be calculated?

Right, dealing with the variable focus issue of DIC for hierarchical models is a good point I hadn’t considered!

I’m not crazy about having a DIC argument to mcmc, since it’s not something that modifies how mcmc is done.

We could compute DIC from the draws object though (it actually stores all the ‘raw’ parameter values, which is how calculate()works), perhaps with an option to modify which bits of the model to look at (considering all distributions by default). So how about something like:

draws <- mcmc(m)
dic(draws)

or

dic(draws, focus = list(y))

where ‘focus’ is a list of greta arrays that have distributions assigned to them.