DistributionsAPI.md

Univariate API

A new univariate distribution type should implement all of the following methods:

• Core constructors
  • MyDistribution{T}(args[...])
  • We need to clarify whether constructors should handle input validation or not. There are use cases in which people want to avoid input validation.
• params(d::MyDistribution{T})::Tuple: A tuple of the distribution's parameters in our canonical order.
• minimum(d::MyDistribution{T})::T: The lowest value in the support of MyDistribution.
• maximum(d::MyDistribution{T})::T: The highest value in the support of MyDistribution.
• location(d::MyDistribution{T})::T: The location of a location-scale family.
• scale(d::MyDistribution{T})::T: The scale of a location-scale family.
• partype(d::MyDistribution{T}): TBD
• mean(d::MyDistribution{T})::T: The theoretical mean of the distribution.
• median(d::MyDistribution)::T: The theoretical median of the distribution.
• mode(d::MyDistribution)::T: An arbitary theoretical mode of the distribution.
• modes(d::MyDistribution)::Tuple{T}: The theoretical modes of the distribution.
• var(d::MyDistribution)::T: The theoretical variance of the distribution.
• skewness(d::MyDistribution)::T: The theoretical skewness of the distribution.
• kurtosis(d::MyDistribution)::T: The theoretical kurtosis of the distribution.
• entropy(d::MyDistribution)::T: The theoretical entropy of the distribution.
• pdf(d::MyDistribution, x::Real)::promote_type(T, x): The PDF of the distribution evaluated at x.
• logpdf(d::MyDistribution, x::Real)::promote_type(T, x): : The log of the PDF of the distribution evaluated at x.
• cdf(d::MyDistribution, x::Real)::promote_type(T, x): The CDF of the distribution evaluated at x.
• ccdf(d::MyDistribution, x::Real)::promote_type(T, x): 1 minus the CDF of the distribution evaluated at x.
• mgf(d::MyDistribution, x::Real)::promote_type(T, x): The moment generating function of the distribution evaluated at x.
• cf(d::MyDistribution, x::Real)::promote_type(T, x): : The characteristic function of the distribution evaluated at x.
• rand(d::MyDistribution{T})::T: A random sample from the distribution.
• fit_mle(d::MyDistribution{T}, x)::MyDistribution{T}: An instance of the distribution fit by MLE to data.

We need to clarify whether constructors should handle input validation or not. There are use cases in which people want to avoid input validation.

Maybe a keyword argument in constructors that defaults to true, e.g. MyDistribution{T}(a, b, ...; validate=true)? That way if people really need to turn off input validation they can. If we do that, I'd sort of prefer that method to be undocumented, as I think we should encourage people to use correct, sensible inputs.

maximum(d::MyDistribution{T})::T: The highest value in the support of MyDistribution.

What do we do for discrete distributions defined on the positive integers with no upper bound, like the geometric distribution? Inf is a float, but all values in the support are integers.

params, partype, cf

Personally I'd prefer the long-form names, i.e. parameters, parametertype (assuming that's what partype is), and characteristic (or similar), but I don't feel too strongly about it.

This is awesome. Thanks so much for writing it out! This will be a huge step forward for the package.

partype(d::MyDistribution{T}): TBD

We just needed a convenient way to distinguish the type parameter T from the element type of the distribution, which remains Int64 for univariate discrete and Float64 for continuous. Perhaps ambiguously, these are also just Float64 for MvNormal, etc.

Obviously if we have this:

rand(d::MyDistribution{T})::T: A random sample from the distribution.

then eltype needs to return a T as well, and we could perhaps fold partype into it? Though I suppose that only works for univariates. I suppose one would like to distinguish between partype and eltype for multivariates:

partype(d::MvNormal{T})::T
eltype(d::MvNormal{T})::Vector{T}

fit_mle(d::MyDistribution{T}, x)::MyDistribution{T}: An instance of the distribution fit by MLE to data.

Yes. As I said, this is on the to-do list but not implemented yet. We should decide whether
fit_mle(d::MyDistribution{T}, x::Array{S})::MyDistribution{promote_type(T, S)}
is preferred, though.

Another point to consider:

Now that automatic type conversion of Integer arguments to Float64 is no longer happening in inner constructors, we had to make a choice about whether we should have

Normal(0, 1)::Normal{Int64}

Bottom line: in the interest of minimal surprise to users and feasible type stability (if you have to worry about T <: Integer, the implicit promotions from log, *, etc. mean that most methods can no longer return a T) we are deliberately upcasting Integer arguments to the appropriate Floatx. This, and the unofficial requirement that

Normal(0.0, 1.0f0)::Normal{Float64}

i.e., we prefer a single type parameter, does tend to increase the number of cases constructors must handle. In addition, distributions now all have methods for

convert(::Type{Normal{T}}, Normal)
convert(::Type{Normal{T}}, μ, σ)

I agree that this sort of extra plumbing, plus type stability, does increase the overhead to adding new distributions. It also makes the code harder for new people to dig into. As Andreas points out, it's a cost-benefit with automatic differentiation and whatever new tricks with Real subtypes we haven't thought of yet. If we believe that it's going to become increasingly rare that people will add lots of new distributions, and we document these requirements well, then maybe that's all positive on balance, since most of the work is nearly done. In addition, there are now tests that do check for the conversions above, and it's possible to add them for integer promotion in constructors. So the test suite will catch a lot of failures (at least for univariates).

So again, thanks for starting this discussion, it really does need to be documented.