PyMC4 "unrolled"

0_README.md

Status: WIP

Abstract

I'm trying to figure out how to convert PyMC4 from TF1 (1.13) and TFP 0.6.0 to TF2 (2.0.0-preview) and TFP 0.7.0. I haven't figured it out yet but for now this gist might serve as a handy "one pager" to PyMC4.

1_TF1_Test_Code.py

from tensorflow_probability import distributions
import tensorflow_probability as tfp
import tensorflow as tf
from scipy import stats


print("tf_version {}".format(tf.__version__))
print("tfp_version {}".format(tfp.__version__))


# Initializing nodes in TF1 Graph
class _RandomVariable:

    def __init__(self):

        self._distribution =  distributions.Normal(loc=0, scale=1)
        self._backend_tensor = None

    def as_tensor(self):
        self._backend_tensor = self._distribution.sample()
        return self._backend_tensor

    def log_prob(self):
        return self._distribution.log_prob(self)

# This stuff happens in a couple handoffs between context and rv usually
def _convert_rv_to_backend(d, dtype=None, name=None, as_ref=False):
    if isinstance(d, _RandomVariable):
       return d.as_tensor()
    return d

tf.register_tensor_conversion_function(
    _RandomVariable, conversion_func=_convert_rv_to_backend, priority=0
)


rv = _RandomVariable()


test_val = 0

if tf.__version__[0] == "1":
    log_p = rv.log_prob()
    sess = tf.Session()

    # With initialized tensor we can still pass in particular values with sess.run
    print(sess.run([log_p], feed_dict={rv._backend_tensor: test_val}))


if tf.__version__[0] == "2":
    # Due to eager execution I'm unsure how we pass in "correct" values before execution
    print(rv.log_prob())

# Scipy for reference
print(stats.norm.logpdf(x=0))

2_TF1_Output.md

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
If you depend on functionality not listed there, please file an issue.

tf_version 1.13.1
tfp_version 0.6.0
2019-03-21 08:58:10.650723: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-03-21 08:58:10.672225: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 4200000000 Hz
2019-03-21 08:58:10.672666: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x564aed2ac220 executing computations on platform Host. Devices:
2019-03-21 08:58:10.672678: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
-0.6931471805599453
[-0.6931472]

3_TF2_Output.md

tf_version 2.0.0-dev20190321
tfp_version 0.7.0-dev
2019-03-21 09:18:37.772084: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-03-21 09:18:37.800215: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 4200000000 Hz
2019-03-21 09:18:37.800696: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x562377c1e150 executing computations on platform Host. Devices:
2019-03-21 09:18:37.800708: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): <undefined>, <undefined>  
tf.Tensor(-1.8590579, shape=(), dtype=float32)  
-0.9189385332046727