Meta Enables 3D Conversations (plus more from Mastercard & Nvidia)

3D chats, pedestrian warnings & crypto card payments

---

1. Meta – 3D conversations in artificial reality

Meta is working on enabling 3D conversations to take place, where remote participants will be presented to you as if they’re face-to-face.

This will work by implementing a processing pipeline that captures the video of a user, reconstructs the image into a 3D visual, and then displays it to the user if they have a 3D displaying device. 3D devices include VR headsets and artificial reality glasses. In the filing, Meta claims to be able to do this with less than 100 m/s latency.

The motivation for doing this is to close the gap in experience of meeting irl vs meeting online. If Meta’s betting its future company success on the metaverse, creating an enterprise use-case for its hardware and its software (e.g. Horizons Workroom) will be key.

This problem is already getting the attention of other players. Cisco already has a prototype of creating 3D holograms of people for remote meetings.

The future of remote work is still up for grabs and being written. Following the pandemic, I think most people have rushed to setup a home in one of the following camps: remote only, IRL only, and hybrid. The truth is, we’re arriving at conclusions based on primitive technology. I suspect the user experience of remote work would’ve changed significantly by the time we have our next pandemic (lol).

2. Nvidia – assessing pedestrian crossing risk

An emerging theme appearing in making driving safer is creating pedestrian warning systems.

Ford is working on a system for autonomous vehicles to be able to communicate with nearby pedestrians, warning them if they’re in danger. It’s an interesting approach because it we move away from having a ‘single point of failure’ with self-driving systems, and creates a communication layer between the vehicle and the pedestrians.

Along the same theme, Nvidia is working on creating a system for vehicles that assesses the risk to pedestrians and warns them if they’re in danger.

Using cameras, LIDAR data and machine learning, vehicles will assess pedestrian risk data. For instance, if there’s an elderly person crossing the road, Nvidia might pick up that the pedestrian has a hearing aid and a walking stick. Nvidia would then calculate the trajectory of the pedestrian and assess if they’re in danger. If a certain risk threshold is met, the vehicle would then emit a beamformed audio signal at the direction of the elderly person, to ensure that the elderly person is aware of the risk.

According to the filing, Nvidia’s system would take into account a number of inputs for assessing risk, such as:

pedestrian’s head pose

body pose

clothing

any articles on a person (e.g. wearing headphones?)

gestures

weather conditions

time of day

For example, a child who’s on a smartphone, wearing headphones, looking down on their phone while crossing the road, would probably be at a high risk level, and would receive a warning if Nvidia’s system assessed that they’re on a collision path with a vehicle.

As vehicles increasingly become loaded with sensors, cameras and machine-learning models, it’s interesting to see what applications develop besides autonomous driving.

3. Mastercard – accepting crypto at point-of-sales terminals

There’s been a pretty high barrier to entry for most merchants at to accept crypto payments from customers. Firstly, they’ve had to acquire new point-of-sales terminals that can receive crypto payments. Secondly, they’ve also had to setup their own crypto wallets, which many merchants find difficult.

In this filing, Mastercard reveal that they’re wanting to enable merchants to take crypto payments using existing point-of-sales devices.

It will work by a customer presenting their card as usual to make a fiat payment. The issuer of the card would then assess whether a crypto-payment is possible. If it is, the issuer will initiate a crypto transaction with an exchange, where it will convert the crypto into the exact fiat required. The merchant will in turn receive fiat for the transaction, while the consumer would have spent crypto.

With Mastercard having announced a partnership with crypto-exchange Gemini in 2021, this looks like the next phase for crypto – people being able to make crypto-payments to buy things in their day-to-day life.