lower body poses in VR, personalized footwear & VR weapons training

---

1. Meta – predicting lower body poses in VR

Facebook is working on a way of predicting a user’s lower body poses when they’re in virtual reality.

When a user is wearing a VR headset and holding the two controllers, a user’s upper body pose can be estimated through a combination of: head position, head angle, wrist angle, and wrist position.

At present, most games and interactions in VR rely on a user’s upper body movements. This is why when you use the Oculus Quest 2, the avatars are chopped off below the waist.

However, Meta want to use the data from the upper-body poses to estimate lower body poses. They’ll do this by firstly building a library of body pose data. Using this, they’ll train a machine learning model to predict the lower body pose data. Then when a VR headset is in use, Meta will be able to use generative adversarial models to be able to “generate” a lower body pose.

Why is this interesting?

Well, firstly, legs would enable for more realistic and richer social interactions in VR. Secondly, interesting use cases could develop, such as dancing (without relying only on your hands) and running. While full body tracking is possible in some VR systems, they tend to rely on full body sensor suits – something that is impractical for the casual use-case.

2. Raytheon – training to use surface-to-air missiles in VR

Raytheon is a new company I’ll be tracking in Patent Drop. It operates in the aerospace and defence space, and is generally considered to be at the forefront of innovation in military tech.

This patent application that was filed last month is a good example of this.

In this filing, Raytheon reveal that they’re exploring using VR to train people on man portable air defense systems (‘MANPADS’). These are portable surface-to-air missiles that can be used to shoot at low flying aircraft. If you’ve been following the conversations around the West arming Ukrainian fighters, you may have heard about the ‘Javelin’, which is a surface-to-air missile that has been helping secure the airspace.

In theory, training people on these weapons in VR is useful. It could give people the familiarity and muscle memory of using these weapons before they arrive in a live battle environment.

However, there are few issues with training with these weapons in VR. Firstly, the weapons tend to be held on a person’s shoulder. However, the bulkiness of the VR headset can make it difficult for a user to get the correct head position. Secondly, when wearing a VR headset, it can be difficult to actually “see” the missile correctly, while simultaneously being in the immersive virtual environment.

Raytheon’s solution is to first give a user instant feedback when they’re holding the device correctly on their shoulder. This would be done by looking at the position of a person’s head relative to the training MANPADS device. Once the correct position is reached, the view in the VR headset will snap to a “aim down sights” position, where a user can then begin to aim the weapon at virtual targets.

This filing is interesting for a number of reasons. Firstly, it’s a reminder of the military training applications of VR. Secondly, this kind of immersive training that blends virtual environments with physical props, is a blurring of military and gaming applications. In the near-future, the best gamers will be the deadliest soldiers. Gaming will be a funnel for military recruitment.

3. Adidas – individualised footwear

Footwear tends to be mass produced with large quantities of identical products. This creates economies of scale for producers and means that the output is cost efficient for consumers.

However, this process doesn’t lend itself to people finding optimally fitting shoes. This problem is evident for athletes who need the exact right size footing for each foot. This can be costly and time consuming to find.

Adidas is looking at providing individualised footwear for more people, while also making it cheap and fast to produce.

It will work by firstly capturing input data from a user. This includes:

anthropometric data – e.g. body mass, foot length, etc

biomechanical data – e.g. a user’s playing style, motion analysis, ankle instability etc.

personal preferences – e.g. toe allowance, fit perception, traction

Adidas plans of capturing this data through treadmill analysis, 3D foot scans, pressure plates, sprint analysis, jump analysis and questionnaires.

All of this input data will then be converted into manufacturing instructions that enable the individualized footwear to be produced.

More interestingly, Adidas are exploring using AI to improve the algorithm for converting the input data into manufactured shoes. The filing mentions that they would train their model on user feedback data – using both qualitative feedback, as well as sensor data.

The next wave of production will be leveraging AI to create personalised products, in a way that’s as cost & time-efficient as mass-production.