Is there a way to output tensorflow probability code from Greta?

#1

Is there a way to output tensorflow probability code from Greta?

2 Likes
#2

greta doesn’t create tensor-flow probability code internally so unfortunately it can’t output it.

I do wonder though @nick whether if greta was written today it would generate tf probability code.

#3

That’s sort of true. greta does produces tensorflow code (including some tensorflow probability functionality) internally, but not in a form that’s readable to humans. So you can build a model, and internally it will create a tensorflow function for the log density (if you dig around in the model object you can see this as dag$generate_log_prob_function()), with a transformed vector of parameters as its input. Though it’s not designed to be used in that way, and all you’ll see in the function is a load of meta-programming code, which isn’t helpful.

Having a one language write out human-readable and editable code in another language is an incredibly hard task, and out of greta’s scope. So I don’t think I’d attempt that if I were starting again now!

2 Likes
#4

Yeah, I was interpreting code to mean code that could be understood as a transliteration of the model which, as you say, greta doesn’t create.

While I agree it is very hard it is interesting to note that it is how rethinking, Turing (sort of) and Stan all work.

#5

Right, rethinking and stan both have a domain-specific language which is written in a single block, parsed and translated. greta works with R objects directly, and so uses metaprogramming instead of a parser. I guess that’s the difference.

I don’t know Turing and it’s unsurprisingly hard to google. Could you please share a link?

#6

In both cases though the probabilistic model is represented in some intermediate form (i.e. the dag in greta) which is then operated on. So I think in a sense the metaprogramming of greta and the parsing of Stan are doing the same thing.

And of course, this is the Turing I was referring too.

1 Like
#7

Hi Nick,

Thanks for your information.

All I plan to do was to replace a tf matrix with a tf sparse matrix and replace matrix multiply with sparse multiply in the meta code. I did not see Greta documents any sparse coding. I will try.

Cheers

Chen

#8

Ah right. To add new operations and objects, you need to create functions that wrap the op() constructor function and list the output dimension, operation name, and corresponding tensorflow operation. See here for an example with %*%.

Sparse greta arrays are not yet implemented. In part because tensorflow doesn’t have great support for its SparseTensor type. I’m about to add some more comments on this topic to this other forum thread on the topic: Sparse matrix format for count and binary data & setting dtype of tensors