Renderer
Nerf.jl provides a simple renderer which can be used for visualizing trainer NeRF model.
using LinearAlgebra
using Nerf: Renderer, Camera
To create a renderer, pass camera that uses dataset's intrinsics, trainer's bounding box and cone:
camera = Camera(MMatrix{3, 4, Float32}(I), dataset.intrinsics)
renderer = Renderer(Backend, camera, trainer.bbox, trainer.cone)
Tiled renderer
To allow rendering high-resolution frames, Renderer
automatically switches to a tiled mode, once the number of pixels in the frames exceeds tile_size
parameter.
Controlling the size of the tile can be done with tile_size
keyword when creating Renderer
:
renderer = Renderer(
Backend, camera, trainer.bbox, trainer.cone;
tile_size=512 * 512) # 512x512 pixels in each tile.
Then, set the position of the camera (here we randomly select a pose from the dataset):
pose_idx = clamp(round(Int, rand() * length(dataset)), 1, length(dataset))
Nerf.set_projection!(camera, Nerf.get_pose(dataset, pose_idx)...)
And render whole frame in one go:
Nerf.render!(renderer, trainer.occupancy, trainer.bbox) do points, directions
model(points, directions)
end
Nerf.render!
accepts as first argument a function that will return RGBA
colors evaluated at each point & direction in points
& directions
.
Thus the rendered does not know anything about the NeRF model and can be used for generic ray marching visualizations.
Renderer
writes everything to its internal buffer. To save the rendering result, convert the buffer to an image and save it:
rgba_image = to_image(renderer.buffer)
rgb_image = RGB.(rgba_image)
save("image.png", rgb_image)