Architecture

Our initial approach was to have Lytro run as a standalone application that Unity could interface with. However, we found that option offers no real advantage while simultaneously increasing the complexity of the implementation. Unity natively supports plugins written in C++ and compiled into DLLs. It can load these at runtime and provides interfaces for working with plugin DLLs. For these reasons we chose to implement Lytro rendering via a plugin DLL.

Lytro is compiled to a DLL that is used as a native plugin by the Unity game engine. There is a Lytro Renderer script built in Unity that interfaces with the native plugin and is also responsible for compositing the color and depth information from Lytro into the Unity rendering loop.

In order to facilitate easy communication between Unity and Lytro we have created a UnityRenderingPlugin virtual class to be implemented by the Lytro programming team. It serves as a bridge between Unity and Lytro and removes the need to have domain knowledge on how Unity renders a frame.

Figure 1: Lytro sample data composited with Unity objects.

Figure 2: Lytro volume rendered as cube (left) or a sphere (right)

Inside of the Unity Editor, the Lytro Renderer renders a shape (usually a cube, but could be any shape) in the scene view which can be transformed like any other GameObject in the Unity Editor. This allows it to be positioned, scaled, and rotated to fit the scene it’s meant to be a part of.

UI/Flow/wireframes

final render in unity