We gave our interns the task of finding ways to combine virtual reality and green screen technologies. A key product in this field is LIV, a mixed reality engine that has both a SteamVR plugin for working alongside existing games or a simple Unity SDK for allowing mixed reality in your own creations.
LIV has support with all Unity-based virtual reality games, as well as improved and optimised support for select titles, and using the software is easy too. LIV is available on Steam and works alongside the built-in VR software, simply requiring only a VR headset, a green screen, and a camera. The interns found that webcams can work directly with the software, but the use of an external source – such as a 4K camcorder – required an additional capture card to take the live feed and use it for the mixed reality effect.
Using LIV was simple, all the interns had to do was calibrate a virtual camera to line up with the position and field of view of the camera in the real world. To begin with this seemed like a difficult task, but once everything was set up the calibration was easy, and immediately we could watch the players on the screen, appearing virtually in the digital world, swinging swords to a beat in Beat Saber – an SDK-enabled game – and fighting off gladiators in Gorn – a VR game made with Unity.
Initially, the interns used a webcam for filming the players, and after messing around with the chroma key settings and values, achieved a good level of green screening, albeit with occasional flickers of green from seams in the curtain. This effect was significantly improved with a camcorder over a webcam, which captured a higher frame rate with reduced motion blur, giving substantially better visuals.
A final step the interns took to improving the experience of the green screen, was to mount a Vive tracker onto the camera, which smoothly works alongside the LIV engine to allow the camera’s real-world movements to be mapped to virtual camera movements, meaning spectators could watch the player from any angle within the world.
Bristol VR Lab is enthusiastic about experimenting with mixed reality further and seeing what potential it can bring to enhancing user experiences in VR.