Facebook’s VR interactions, Apple’s haptic touch bar, and Roblox’s game quality algo

1. Facebook – artificial reality triggered by physical object interactions

Facebook is looking towards creating an artificial reality system where content is presented based on what physical objects a user is interacting with.

For instance, as illustrated in the diagram above, a user sitting on a chair could be a trigger to launch an artificial reality driving game. Now when the user gets up off the seat, that could be a trigger for exiting or pausing the game. In this example, using physical objects as trigger points provides a seamless way of launching and exiting from an artificial reality experience.

The content that is launched from these physical objects could either be configured, or based on other contextual information. For example, imagine if Facebook had access to your calendar and could see that you have a conference call now. When you sit down at that specific time, a VR conference call could be launched instead of a driving game.

More creepily / interestingly, Facebook’s filing also describe presenting different types of artificial reality content based on a user’s biometric information. For example, if the VR headset determines that a user is exhibiting signs of stress, a calming artificial reality experience might be played.

Why is this interesting?

One of the recurring themes in Patent Drop is the idea of the physical and digital worlds merging into one. This latest filing shows Facebook, like Snap, is actively thinking about blurring the two worlds. Although the examples in this filing are relatively minor, things become exciting when you imagine a world where all of the physical objects in our home can be used to interact with the virtual world. For instance, imagine using your real-world baseball bat to swing a virtual baseball bat in one game, and as a sword in another.

That said, there are big questions as to whether we ought to trust Facebook with all of the data it could be capturing from our bodies and our physical environments.

2. Apple – haptic engine for the macbook touchbar

Apple looks like it’s going to make the touch bar on the Macbook provide haptic feedback.

For anyone who has a Macbook with the touch bar, you probably know how sub-par the user experience is. Besides accidentally tapping icons on the touch bar when typing, one other problem is that the experience feels extremely flat.

Keyboards naturally have haptic feedback – you touch a key, the key depresses, gives a mechanical sound, and then returns to its natural state. However, by having a digital touch bar without haptic feedback, Apple’s keyboard became a disjointed experience with the ‘haptic’ keys and the ‘flat’ touch bar.

One of the main constraints for building haptic feedback was the unusual aspect ratio of the touch bar. Essentially, using the conventional design methodology for creating haptic feedback would not have created a sufficient force for feeling haptic feedback.

So in this filing, Apple have created a new haptic engine that looks to provide feedback for the dimensions that are seen with the touch bar. If you’re interested in the intricate details of how it works, the patent link is in the title.

I’m including this filing because I haven’t seen any press around Apple making the touch bar have haptic feedback, so consider this an exclusive 🙂

3. Roblox – determining the quality of a game

Given Roblox’s IPO last week, I thought I’d share a new patent application that has just been made public.

One of the key pillars to Roblox’s success is the fact that users can develop games on the platform and make serious money.

As is the case with most user-generated platforms, there can be a huge variance in quality. In this instance, some of the games are higher quality, others lower quality, and it can be difficult for the platform to identify the quality without having players play it. Putting a low-quality game in front of a user can result in a bad experience and therefore impact core metrics such as engagement, revenue and retention.

So in this filing, Roblox look into trying to identify the quality of a game based on developer engagement metrics, before a user has had a chance to play with it.

It will work by Roblox measuring the number of actions performed by developers using the game development tool, and how many actions are performed during at least one time frame. In essence, the first proxies for time taken by the developers to make the game, and the second proxies for the complexity of the game design.

Games that have higher developer time are determined to be higher in quality, while games that have lower developer time are determined to be lower in quality. Moreover, games that have very low developer time may be malicious in nature.

To make the quality identification better, Roblox also describe weighting some developer actions more heavier than others – e.g. dragging and dropping an object from a toolbox could be worth more than changing the colour of the object. Moreover, ‘deletes’ and ‘undo actions’ may not be considered actions as these could be the results of mistakes or developers becoming familiar with the tool.

What’s important to note is that while the quality score based on developer time is important in deciding which games to push on the platform, it is unlikely to be the only indicator of quality. Once a game starts to be played by people and they leave reviews and ratings, these metrics are likely to have a higher impact on assessing the quality of the game.

For anyone who has worked on a UGC platform, this problem is a familiar one. It’s fascinating to see Roblox’s approach to trying to tackle it. The parallel on TikTok would be proxying video quality on how long someone spends making their video, and how many specific effects they use during any given second.

The key difference is that on TikTok, the cost of a low-quality video is quite small. The time investment into each video is low and the switching cost to a new video is extremely small – all you need to do is swipe. For Roblox, the time and cognitive investment of entering a new game, exploring it, and find out it’s bad, is very high. So hence why Roblox need to find a proxy for quality before relying on user feedback.