A state space model distribution for pymc3

pymc3_statespace_model.ipynb

I used this code for my research. Check the package BSSPy for the work https://arxiv.org/pdf/1901.07469.pdf.

Just remember the caveats that if you use a mean-field VI then it does not preserve the Markov dependency. i.e. $N(x_t | x_t-1 )$
More elaborate discussions about mean-field and structured mean field here. http://www.ee.columbia.edu/~sfchang/course/svia-F03/papers/factorial-HMM-97.pdf

I think the full rank ADVI may preserve this dependency but can be slower.

Also, I would ask for some thoughts regarding the Tau initialization -

    A = mc.Normal('A', mu=np.eye(2), tau=1e-5, shape=(2,2))
    **Tau = mc.Gamma('tau', mu=100, sd=100)**
    
    X = StateSpaceModel('x', A=A, B=T.zeros((1,1)), u=T.zeros((x.shape[0],1)), tau=Tau, shape=(y.shape[0], 2))

It seems like a single random variable for the Precision term is being broadcasted. I would prefer to declare it as a covariance matrix instead. I am trying to put together a code with reparameterization trick and will push a new module soon.