Google Creates Shoppable Images (plus more from Microsoft & Facebook)

Google’s shoppable images, Microsoft’s autocorrect for VR keyboards, and more

1. Google – shoppable images based on search

Last week, Google filed a patent application for creating shoppable images that merchants can use to create ads for their products.

Let’s suppose a user searches for “laundry room”. Google will retrieve an image that contains a list of products that are related to the search term. In this case, the image could include a washer, dryer, laundry bags etc. The image will then be displayed with shoppable advertisements attached to the relevant products. If a merchant is wanting to advertise their products for this search term, Google will select (or in the future, generate) an image that contains relevant products offered by the merchant, as well as any promotional messages.

Why is this interesting?

In the filing, Google mention that lots of people use the search engine for finding images of products they might be looking to buy. Currently, when you go into Google Images, there is no advertising component. Although there is Google Shop, Shop is only really relevant for people with high intentionality for making a purchasing decision. By creating shoppable images, Google could be looking to monetise the ‘discovery’ component that people currently use Google Images for.

Wherever there is a high volume of traffic, specific user intent, and a content ranking algorithm, there is the opportunity for advertising revenue to be generated. By opening Google Images to advertising, Google massively ramp up advertising capacity. And the truth is that Google’s monopoly on search means that advertising money will naturally flow here: whether it’s for brands wanting to capitalise on the new opportunities for being discovered, or for brands that need to defend themselves against competitors targeting their keywords (e.g. if Samsung targeted the search term “Apple”).

All in all, it looks like advertising is coming to Google Images.

2. Microsoft – autocorrect for VR keyboards

Since I started Patent Drop, one recurring theme that has popped up in VR patents is how to solve the problem of using keyboards in a VR world. The problem might seem trivial if we only associate VR with entertainment. But filings from Microsoft, Intel and Apple show that they are all seeing VR being deployed in an enterprise context, and hence virtual keyboards are essential for people getting work done using a familiar form factor.

In this latest filing, Microsoft is thinking about how to make typing work in VR. With physical keyboards, users get tactile feedback from tapping on a key. However, with virtual keyboards, typing speeds are much slower because users try to hit the right key in one slow deliberate action, check if the correct input was registered, and then move onto the next key.

Instead, Microsoft want to use a form of autocorrect to estimate what key the user was intending to hit, irrespective of whether the user actually makes contact with the correct key. They’ll do this by studying the ‘path’ of a user’s fingers as they type. Based on this path, the system will assign probabilities to the keys the user intended to hit. The intention is then to help users type more speedily when in VR.

If you’re interested in this topic further, check out the 3rd summary in #010 PATENT DROP where I look at Apple’s patent filing for a VR Keyboard.

3. Facebook – bone conduction audio for VR / AR headsets

Facebook is looking at how to solve the problem of cross-talk when using bone conduction audio systems.

For quick context, VR / AR headsets conventionally use headphones to transmit audio to users. The problem with this is that this shuts the user out from their real world environment, which could be dangerous. If we were to paint a dystopic nightmare scenario, imagine you’re busy playing in your VR world, while someone around you is choking on an olive stone. You can’t hear or see what’s happening because you’re wearing headphones while wearing your VR headset. Bad!

One solution to this problem is sending audio content to a user via bone or cartilage conduction. This works by vibrating bone and / or cartilage in a user’s head, which generates acoustic pressure waves that travel directly to the ear’s cochlea, bypassing the eardrum.

However, one issue with bone conduction systems is something called “crosstalk”. This is where sound emitted for one inner-ear is picked up by the other inner-ear, due to the audio vibrations travelling through a user’s cranium. This matters particularly for VR because cross-talk reduces the ability to sense sound direction, which is integral to fully immersing yourself in a 360-degree virtual world.

To mitigate this, Facebook describe adjusting the audio content so that it’s dampened (i.e. low amplitude) at regions of the bone conduction system where cross-talk might be picked up.

If there were two takeaways from this filing, it would be: 1) Facebook is looking at using bone conduction technology for their VR headsets; and 2) Facebook is actively thinking about how to maintain a sense of spatial audio with this technology.