CODEWITHSUNDEEP
python-3.xdeep-learningpytorch
Peter
Peter

Reputation: 117

Understanding deep Markov model code on pyro

I am going through an a deep Markov model code given on pyro's website:

https://pyro.ai/examples/dmm.html

I am really confused about combiner module that they've implemented. You an find combiner module on line 104 on this GitHub page:

https://github.com/pyro-ppl/pyro/blob/dev/examples/dmm/dmm.py

The article they are following is:

https://arxiv.org/abs/1609.09869

They explain combiner module in section 4 (Structured inference networks). I am really confused about why they are doing three linear transformations in the code starting from line 104 on GitHub. Aren't they just suppose to use RNN to produce the distribution or am I missing something? Insights would be appreciated.

Upvotes: 1

Views: 249

Answers (1)

David Marx
David Marx

Reputation: 8558

The Combiner module implementation corresponds to the formulas described at the bottom of page 4, "Combiner Function for Structured Approximations (for DKS)". mu_t is loc and sigma_t**2 is scale.

The RNN state is used to parameterize the distribution, but that distribution is parameterized by two variables. These variables are extracted from the RNN state via the transformations in question.

Upvotes: 1

Related Questions