diff --git a/dev/distributions/index.html b/dev/distributions/index.html index cf351676..b940ae6e 100644 --- a/dev/distributions/index.html +++ b/dev/distributions/index.html @@ -1,5 +1,462 @@ Distributions.jl integration · Bijectors
Edit on GitHub

Basic usage

Other than the logpdf_with_trans methods, the package also provides a more composable interface through the Bijector types. Consider for example the one from above with Beta(2, 2).

julia> using Random;
+
+
+
+
+
        Random.seed!(42);
 
 julia> using Bijectors;
@@ -21,3 +478,4 @@
 transform: Bijectors.Logit{Float64, Float64}(0.0, 1.0)
 )
julia> tdist isa UnivariateDistributiontrue

We can the then compute the logpdf for the resulting distribution:

julia> # Some example values
        x = rand(dist)0.2909654089284631
julia> y = tdist.transform(x)-0.890699923433023
julia> logpdf(tdist, y)-1.3650442380572652
+ diff --git a/dev/examples/index.html b/dev/examples/index.html index 991a5e3f..0099e44f 100644 --- a/dev/examples/index.html +++ b/dev/examples/index.html @@ -1,5 +1,462 @@ Examples · Bijectors
Edit on GitHub

Univariate ADVI example

But the real utility of TransformedDistribution becomes more apparent when using transformed(dist, b) for any bijector b. To get the transformed distribution corresponding to the Beta(2, 2), we called transformed(dist) before. This is simply an alias for transformed(dist, bijector(dist)). Remember bijector(dist) returns the constrained-to-constrained bijector for that particular Distribution. But we can of course construct a TransformedDistribution using different bijectors with the same dist. This is particularly useful in something called Automatic Differentiation Variational Inference (ADVI).[2] An important part of ADVI is to approximate a constrained distribution, e.g. Beta, as follows:

  1. Sample x from a Normal with parameters μ and σ, i.e. x ~ Normal(μ, σ).
  2. Transform x to y s.t. y ∈ support(Beta), with the transform being a differentiable bijection with a differentiable inverse (a "bijector")

This then defines a probability density with same support as Beta! Of course, it's unlikely that it will be the same density, but it's an approximation. Creating such a distribution becomes trivial with Bijector and TransformedDistribution:

julia> using StableRNGs: StableRNG
julia> rng = StableRNG(42);
julia> dist = Beta(2, 2)Beta{Float64}(α=2.0, β=2.0)
julia> b = bijector(dist)              # (0, 1) → ℝBijectors.Logit{Float64, Float64}(0.0, 1.0)
julia> b⁻¹ = inverse(b)                # ℝ → (0, 1)Inverse{Bijectors.Logit{Float64, Float64}}(Bijectors.Logit{Float64, Float64}(0.0, 1.0))
julia> td = transformed(Normal(), b⁻¹) # x ∼ 𝓝(0, 1) then b(x) ∈ (0, 1)UnivariateTransformed{Normal{Float64}, Inverse{Bijectors.Logit{Float64, Float64}}}(
+
+
+
+
+
 dist: Normal{Float64}(μ=0.0, σ=1.0)
 transform: Inverse{Bijectors.Logit{Float64, Float64}}(Bijectors.Logit{Float64, Float64}(0.0, 1.0))
 )
julia> x = rand(rng, td)                   # ∈ (0, 1)0.3384404850130036

It's worth noting that support(Beta) is the closed interval [0, 1], while the constrained-to-unconstrained bijection, Logit in this case, is only well-defined as a map (0, 1) → ℝ for the open interval (0, 1). This is of course not an implementation detail. ℝ is itself open, thus no continuous bijection exists from a closed interval to ℝ. But since the boundaries of a closed interval has what's known as measure zero, this doesn't end up affecting the resulting density with support on the entire real line. In practice, this means that

julia> td = transformed(Beta())UnivariateTransformed{Beta{Float64}, Bijectors.Logit{Float64, Float64}}(
@@ -76,3 +533,4 @@
  0.028232832692400064
julia> cov(samples; dims=2)   # ≈ I2×2 Matrix{Float64}:
   0.930775   -0.0621717
  -0.0621717   1.13257

We can easily create more complex flows by simply doing PlanarLayer(10) ∘ PlanarLayer(10) ∘ RadialLayer(10) and so on.

+ diff --git a/dev/index.html b/dev/index.html index 3812acfa..51a1e18f 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,5 +1,462 @@ Home · Bijectors
Edit on GitHub

Bijectors.jl

This package implements a set of functions for transforming constrained random variables (e.g. simplexes, intervals) to Euclidean space. The 3 main functions implemented in this package are the link, invlink and logpdf_with_trans for a number of distributions.

Bijectors.link — Function
link(d::Distribution, x)

Transforms the input x using the constrained-to-unconstrained bijector for distribution d.

See also: invlink.

Example

julia> using Bijectors
+
+
+
+
+
 
 julia> d = LogNormal()   # support is (0, Inf)
 LogNormal{Float64}(μ=0.0, σ=1.0)
@@ -38,3 +495,4 @@
 julia> # The difference between the two is due to the Jacobian
        logabsdetjac(bijector(LogNormal()), ℯ)
 -1
source

The distributions supported are:

  1. RealDistribution: Union{Cauchy, Gumbel, Laplace, Logistic, NoncentralT, Normal, NormalCanon, TDist},
  2. PositiveDistribution: Union{BetaPrime, Chi, Chisq, Erlang, Exponential, FDist, Frechet, Gamma, InverseGamma, InverseGaussian, Kolmogorov, LogNormal, NoncentralChisq, NoncentralF, Rayleigh, Weibull},
  3. UnitDistribution: Union{Beta, KSOneSided, NoncentralBeta},
  4. SimplexDistribution: Union{Dirichlet},
  5. PDMatDistribution: Union{InverseWishart, Wishart}, and
  6. TransformDistribution: Union{T, Truncated{T}} where T<:ContinuousUnivariateDistribution.

All exported names from the Distributions.jl package are reexported from Bijectors.

Bijectors.jl also provides a nice interface for working with these maps: composition, inversion, etc. The following table lists mathematical operations for a bijector and the corresponding code in Bijectors.jl.

OperationMethodAutomatic
b ↦ b⁻¹inverse(b)✓
(b₁, b₂) ↦ (b₁ ∘ b₂)b₁ ∘ b₂✓
(b₁, b₂) ↦ [b₁, b₂]stack(b₁, b₂)✓
x ↦ b(x)b(x)×
y ↦ b⁻¹(y)inverse(b)(y)×
x ↦ log｜det J(b, x)｜logabsdetjac(b, x)AD
x ↦ b(x), log｜det J(b, x)｜with_logabsdet_jacobian(b, x)✓
p ↦ q := b_* pq = transformed(p, b)✓
y ∼ qy = rand(q)✓
p ↦ b such that support(b_* p) = ℝᵈbijector(p)✓
(x ∼ p, b(x), log｜det J(b, x)｜, log q(y))forward(q)✓

In this table, b denotes a Bijector, J(b, x) denotes the Jacobian of b evaluated at x, b_* denotes the push-forward of p by b, and x ∼ p denotes x sampled from the distribution with density p.

The "Automatic" column in the table refers to whether or not you are required to implement the feature for a custom Bijector. "AD" refers to the fact that it can be implemented "automatically" using automatic differentiation.

+ diff --git a/dev/search/index.html b/dev/search/index.html index e167c897..3e28efef 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,460 @@ Search · Bijectors

Loading search...

    + + + + + + diff --git a/dev/transforms/index.html b/dev/transforms/index.html index fb0c9447..7ffa0449 100644 --- a/dev/transforms/index.html +++ b/dev/transforms/index.html @@ -1,5 +1,462 @@ Transforms · Bijectors
    Edit on GitHub

    Usage

    A very simple example of a "bijector"/diffeomorphism, i.e. a differentiable transformation with a differentiable inverse, is the exp function:

    • The inverse of exp is log.
    • The derivative of exp at an input x is simply exp(x), hence logabsdetjac is simply x.
    julia> using Bijectors
    julia> transform(exp, 1.0)2.718281828459045
    julia> logabsdetjac(exp, 1.0)1.0
    julia> with_logabsdet_jacobian(exp, 1.0)(2.718281828459045, 1.0)

    Some transformations are well-defined for different types of inputs, e.g. exp can also act elementwise on an N-dimensional Array{<:Real,N}. To specify that a transformation should act elementwise, we use the elementwise method:

    julia> x = ones(2, 2)2×2 Matrix{Float64}:
+
+
+
+
+
  1.0  1.0
  1.0  1.0
    julia> transform(elementwise(exp), x)2×2 Matrix{Float64}:
  2.71828  2.71828
@@ -122,3 +579,4 @@
 
 julia> (a = x.a, b = (x.a + x.c) * x.b, c = x.c)
 (a = 1.0, b = 8.0, c = 3.0)
    source
    + diff --git a/index.html b/index.html index 6a5afc30..3ac25969 100644 --- a/index.html +++ b/index.html @@ -1,2 +1,3 @@ +