Encodings

Multiresolution Hash Encoding

GridEncoding implements Multiresolution Hash Encoding which can be used to encode 3D points prior to MLP.

Instantiate ge = GridEncoding(Backend) with a supported backend. GridEncoding handles its parameters explicitly, meaning it is up to the user to initialize and maintain them with θ = NerfUtils.init(ge).

To encode input coordinates x pass them along with the parameters θ:

# Random 3D points on the same `Backend`.
x = adapt(Backend, rand(Float32, 3, N))
y = ge(x, θ)

Note, that inputs must be in [0, 1] range, otherwise the kernel might break. GridEncoding does not check input values.

Computing gradients

GridEncoding defines respective chain rules for its kernels using ChainRules.jl package, meaning it supports any AD that works with them. Following examples use Zygote.jl for that.

Compute gradients w.r.t. θ:

∇ = Zygote.gradient(θ) do θ # Passing θ explicitly to the `gradient` function.
    sum(ge(x, θ))
end

To compute gradients w.r.t. θ and input x you have to pass additional input argument IG:

∇ = Zygote.gradient(x, θ) do x, θ
    sum(ge(x, θ, IG))
end
∇[1] # Gradient w.r.t. x.
∇[2] # Gradient w.r.t. θ.

It is done this way to dispatch to a different kernel that in the forward pass precomputes necessary values for the backward pass to speed things up.

See GridEncoding docs for the description of each argument during instantiation as you might want to configure number of levels, their resolution and feature dimensions.

function GridEncoding(
    kab; n_dims::Int = 3, n_levels::Int = 16, scale::Float32 = 1.5f0,
    base_resolution::Int = 16, n_features::Int = 2, hashmap_size::Int = 19,
    align_corners::Bool = true, store_level_ids::Bool = false)