VR and Mixed Reality (XR) are an incredible feat of technology. With VR offering the ability to immerse users in completely Synthetic Environments, and XR allowing for the seamless fusion of virtual content with the real world, they offer an unprecedented potential for applications in training and simulation, unlike any other. But historically they’ve had a variety of drawbacks that have ultimately dissuaded prospective users from taking the step into the world of virtual training.
VR-2, VR-2 Pro and XR-1 Developer Edition
Here at Novatech, we’ve had first-hand experience of this when designing solutions for the simulation and training scene; there have been challenges with hardware capabilities, such as visual fidelity and controller versatility, as well as issues integrating hardware into simulators due to a lack of dedicated software. Among a whole foray of other obstacles, it’s safe to say we have found the notion of virtual training a difficult one to sell, despite its fantastic potential.
But that all changed when we attended DSET earlier this year, and met with MS&T's 2020 Industry Simulation & Training Award Winner for Most Innovative Product, Varjo. We tried out their XR-1 Mixed Reality headset and were, to say the least, astounded by the results. Impressed with the work they’ve been doing for the industry, we decided to sit down and have a chat with the Finnish VR and XR company, to really help demonstrate just how far this technology has come.
As Neil put it, “Varjo have created the world’s most advanced professional VR and Mixed Reality solution.” It’s certainly a bold statement, but he’s certainly not exaggerating either.
For a few years now, Varjo have been working with their partners across multiple industries in an effort to create industry-leading VR and XR capable headsets, fit for use in “real-life, professional workflows.” Part of this endeavour has meant striving for the ability to recreate extremely high-quality images in an equally high resolution, allowing for users to perceive every fine detail imaginable. And Varjo have achieved exactly that.
“All Varjo headsets… the VR-2, VR-2 Pro and XR-1; they all support ‘Human-eye Resolution.’ That’s delivered thanks to… our Bionic Display technology… and it provides up to 62 pixels per degree of resolution; it allows you to see everything in photo-realistic quality as if you had 20:20 vision… That means you can effortlessly read text, you can effortlessly read cockpit displays, heads up displays, see objectives and targets out the window – not just see them, but identify them – and you can do all of that at realistic visibility ranges… [There’s no] leaning in or squinting. In the Varjo [headsets] you see all of that in a natural way… as you would in real life.”
Varjo & Unity: Dynamic Human-eye Resolution Photogrammetry
Put simply, Varjo’s headsets are operating at close to four times the resolution of typical commercial headsets. To say that’s a significant difference would be a gross understatement.
This substantial improvement to visual fidelity raises another matter for the training and simulation scene too. Especially so for those operating within the Defence Sectors and Armed Services; whereas training and simulation was once occupied with functionality before fidelity, when it comes to VR and XR (at least with Varjo), the case in point here is that fidelity actually provides an additional layer of functionality in a number of ways.
The XR-1, for example, is "the world’s only video-passthrough Mixed Reality headset capable of operating at that ‘Human-eye’ resolution. In essence, what that allows you to do is combine the real-world with the virtual content and we can even track real-world objects or equipment within that virtual space… This enables trainees to experience tactile feedback by touching physical items that can be immersed in a virtual world for real hands-on training, to practise those motor skills and develop muscle memory that will translate into the actual scenario – so you feel the forces as you would in the real world.
“Mixed Reality really takes virtual training that step further [from VR] in terms of natural interaction… in terms of… that capability to use the real controls, the real aircraft or vehicle, or to use and hold the real equipment, whilst operating in a purely synthetic environment that can represent any scenario required.”
Varjo x Bohemia Interactive Simulations: Cost-Effective Mixed Reality Trainer
And it isn’t just about how great the display looks. Impressive as that ‘Human-eye’ resolution is for offering previously unseen levels of detail in the virtual world (pun intended), it also aids in other areas of training that you might not at first consider.
Neil explained that, “We know our headsets; we know they need to work for long stressful missions… [and] it helps beyond just that visual quality… with things like reducing eye strain… [allowing] you to stay in the headset much longer.
“What’s more, we have active vents and fans to get heat away from the head so it’s more comfortable, as well as preventing the headset from fogging up. We also have things like automatic IPD adjustments; often when people get into a VR or XR headset, you need to adjust the optics for each individual user… but in the Varjo… we have a built-in eye tracker that automatically adjusts the interpupillary distance to each individual user… No guess work, no estimation, no discomfort – we deal with all of that with the Varjo solution.”
“Many people think of us as a hardware company, but we’re actually a software company too; that overall solution is something we think is really key.
Varjo’s commitment to delivering an all-in-one, professional grade VR and XR solution doesn’t just stop at an improved optical experience, or the overall hardware experience, with everything that encompasses. Something Varjo have been doing slightly differently to other VR and XR companies is offering a unique focus on customer and client feedback, which helps inform their software, as well as their hardware. As an end user, this translates into a wealth of useful features and resources, all at your disposal to use as you see fit. Plus, with everything being designed and developed ‘under one roof,’ as well as close collaboration with other companies providing industry standard tools (think Bohemia Interactive with VBS4), the interaction between hardware and software becomes seamless, mitigating any technical hassle when it comes to getting a whole system up and running.
“Our software roadmap is driven entirely by the feedback we’re getting from working with our partners and customers in the industry today, such as Lockheed Martin, Saab AB and Flight Safety International. To give a specific example: Chroma Key, which is a capability we have with the Mixed Reality XR-1. It’s just like green screen technology… Whenever the headset sees a coloured screen it can replace it with virtual content – that feature came directly from aerospace customers using our headset and looking for a low-cost high-quality way of deploying training solutions.
“And that kind of set up gives you a lot of flexibility, portability – imagining the flight simulation scenario, you can literally pop a chair and say, a standard HOTAS, in front of that green screen; well now you’ve got your controls right there, but when you look up, you can see everything else as if it were really there – the environment, the cockpit and virtually, anything else.”
XR-1 real-time chroma key
Neil added, on the subject of that flexibility that, “my demo kit goes in an airline carry-on, and that includes the fabric Chroma Key screen… But the solution goes beyond just portability, it’s about the ease of integrating the Varjo headset and the software with physical simulators, with motion chairs, with motion platforms. Effectively, it’s entirely customisable and tailorable to the customer use case. You don’t have a fixed set of capital expenditure in a fixed location; no big dome room, or a massive, multi-million-pound projector set up. You can effectively have highly portable, highly versatile solutions that could be delivered in a van, or you could even go as far as being hand carried, directly to where that training needs to be deployed. Chroma Key is a perfect example of that.”
Why would you tie up a multi-million-pound asset, maybe even tens-of-millions, to run through procedural training that you could do in a completely immersive way in VR and Mixed Reality?
For Novatech too, having tailored solutions is a must have. Anyone with experience in the industry will know that there is no such thing as “one size fits all.” What Varjo are offering however, is a fantastic, commercially available product for VR and XR training and simulation. One that has unlimited potential for customisability at the user’s end. And that’s something we’re very onboard with. From our perspective, Varjo are offering an infinitely scalable solution to the variety of professional sectors utilising virtual training and more broadly speaking, virtualisation, be it small, creative start-ups or huge corporations, and especially so for the Defence Sector; there’s a huge value to steering clear of fixed capital expenditure, as well as virtualising training wherever it is, not only possible, but as equally effective and efficient (if not more so) than the real world equivalent – in particular, negating skills fade and offering rigorous, versatile part-task training, are two clear candidates for something as advanced as Varjo headsets.
“With VR and XR you can save time, manage capacity, deliver training to exactly where you need it when you need it. Obviously, there’s always going to be a point where you have to get out of the simulator and into the real thing. That’s a given. But with things like part-task training, there’s huge potential – think about aerospace again; why would you tie up a multi-million-pound asset, maybe even tens-of-millions, to run through procedural training that you could do in a completely immersive way in VR and Mixed Reality?
“And it goes beyond that too. Think about training effectiveness… with Varjo headsets, we support human performance assessment. Varjo devices track the user’s eye movements and pupil size, and then passes this information back to the host application to support assessment. In the simplest way, you can record the gaze tracking overlaid on the session… right down to the X, Y, Z co-ordinates of where an individual has been looking or dwelling, so that when you sit down after the session to see and assess how a trainee has performed, you’ve got all of that data available to you and integrated into the application… Of course, that’s in the simplest sense and it can go much further with biometrics… but in essence, following a VR and XR training session, you know exactly how the trainee acted, exactly how they reacted and how the stimulus had affected them… and you can see whether they followed the correct procedures and how they worked through the process.”
Astronaut Training in the Boeing Starliner Program – Unmodified Varjo VR-2 Capture
This is, ultimately, a complete game changer for the world of training and simulation. The ability to move hardware to wherever it’s needed, on almost any scale and for any budget, in any industry, was something almost unimaginable in the professional VR and XR workspace for the last 10+ years. In fact, possibly one of the biggest testimonies to the success of Varjo’s pursuits in the field comes from the likes of Boeing. Neil commented on their testimonies to the success of Varjo headsets which they used for their Starliner Astronaut Training Program, pointing out that, “it was only with the Varjo technology that they could initiate full mission training entirely in VR for the first time – they actually didn’t believe the technology was there yet… but with the Varjo technology they were able to read the crew displays from a normal position in the capsule – something Boeing said was mandatory if virtual training was ever going to be possible.”
Novatech supply custom, purpose-built hardware to some of the biggest names in the Security, Aviation, Defence and Marine industries.
If you have a project you'd like to discuss, please contact our dedicated Simulation and Training team using the form below, or call us on 02392 322500.
14 Jul 2021
We caught up with PNY's David Flood to discuss how the 4th industrial revolution GPU refreshes and other major tech developments are powering incredible and exciting transformation in the media and entertainment industry.