Sign up to uncover the latest in emerging technology.
Plus: Adobe’s AI beauty editor; Shopify is going Web 3.0
Happy Monday and welcome to Patent Drop!
Today’s Patent Drop is brought to you by Skyflow 一 Skyflow delivers security, compliance, and data residency via a simple yet powerful API. Hoping your sensitive data is secure isn’t viable – but neither is making it inaccessible to critical workflows. Rest easy with sensitive data isolated, protected, and governed with Skyflow — learn more here.
This Monday, we’re deep diving into Google’s plans to make your smart home even smarter, Adobe’s tech to make your skin picture perfect, and Shopify’s goal to go Web 3.0. Let’s take a look.
#1. Google’s (smarter) smart speaker
Talking to your smart speaker can be like arguing with a roommate who isn’t listening but is connected to the internet. Google is determined to fix your relationship.
The company filed a number of patent applications for tech aiming to make talking to your smart home a more conversationally fluid experience. First up: the company wants to patent tech enabling “natural conversations” using what it calls “soft endpointing.” Through natural language understanding, this tech can tell whether a user has merely paused in the middle of a sentence, or if they have finished speaking, allowing a user to take more time when giving a smart home a command. If they’ve paused, the system will fill the space with an utterance to show that it’s still listening, for example, saying “mhmm.”
“These turn-based dialog sessions, from a perspective of the user, may not be natural since they do not reflect how humans actually converse with one another,” Google said in its filing.
Next, Google wants to correct any misunderstandings. The company filed a patent for fixing “speech misrecognition” of spoken commands. Essentially, this tech aims to correct when a smart assistant mishears a user, such as if it mistakes one word for another which sounds phonetically similar, by allowing a user to quickly correct the smart assistant without starting an entirely new command.
In one example, a user asks the speaker “turn on the light in Red’s room,” and the assistant responds “Turning on the lights in the bedroom.” Rather than the user having to say “turn off the lights in the bedroom” and “turn on the lights in Red’s room,” a user can just say “I said Red’s room,” and the system will correct its mistake.
Along with making its AI’s conversation and comprehension skills better, the company seeks to patent tech to help its system with multitasking. Essentially, Google aims for its AI to “perform multiple tasks through an individual command,” by parsing out commands into multiple individual tasks. In one example, a user asks the system “please plan a date night.” The system then takes the actions of putting a date into the calendars of both the user and their significant other, selecting a suitable restaurant, and making a reservation.
Finally, the company wants to patent tech that activates through a “detected gesture and gaze.” Here’s how it works: This system will monitor vision data for a user gesturing or gazing in its direction, then wait for audio input, and will be able to more quickly respond to commands than if it were just listening for audio activation.
Adding in innovations like these, particularly the gaze and gesture detection, can make Google’s smart home devices far more accurate, according to Sara Maria Hasbun, founder of speech technology company Meridian Linguistics. The more accurate this tech is about when exactly it’s listening and monitoring, Hasbun said, “the safer users feel with respect to their privacy.”
When looking at Apple’s speech recognition tech as an example, “We all know the stories in which someone accidentally activates Siri and launches some kind of unwanted or embarrassing sequence like recording a conversation and sending it to their boss,” Hasbun said. “These horror stories, and general concerns about privacy, are probably among some of the main reasons why potential users are holding back from purchasing a voice assistant.”
Google seems to be working really hard to make sure that users feel heard. All of these innovations aim to reduce latency and improve the user experience. And as we’ve covered in Patent Drop before, making conversations between humans and AI more fluid – whether that be through context, voice differentiation or, like these innovations, simply making systems smarter – is an important step in making a user actually feel comfortable using a device.
#2. Adobe’s (better) beauty filter
Is it impossible to ignore that zit on your nose in your favorite vacation photo? Adobe’s got something for that.
The company is working on tech that automatically retouches people’s skin in photos, targeting face imperfections like skin blemishes and redness, using AI. This system utilizes neural networks to target and correct certain skin features by separating each image into different frequency layers, or layers of color information. Think of it as a better, more targeted version of a beauty filter.
“Conventional systems continue to suffer from a number of problems with regard to accuracy, efficiency, and flexibility of computing device operations, and in particular, with respect to skin retouching,” Adobe said in its filing. “For example, when performing image corrections, many conventional systems overcorrect digital images causing them to appear unnatural and unrealistic.”
While this would likely be implemented in Photoshop or Lightroom first and foremost, this feature could potentially show up in the Adobe Suite’s video editors as well.
While this stands to make a photo editor’s job a lot easier, there are a number of potential issues with making an advanced beauty editor so easily accessible.
For one, the use of beauty filters themselves is a hot-button topic: According to a survey by Consumer Reports, 59% reported that they found the consistent use of filters on social media “troubling,” as they can have detrimental effects on body image and self-esteem, especially among young women. Filters that are widely available to the average user can also perpetuate colorism, as they are often built with biases and don’t support a wide range of skin tones.
Secondly, since this tech works by training neural networks to make the edits it sees fit, Adobe must be careful about exactly how it’s training them. AI can often be trained using biased data, or simply created by people with implicit biases, whether it be intentional or unintentional.
If Adobe’s not careful, it could create a system that artificially lightens or darkens people’s skin or cuts out beauty marks it deems to be blemishes. Though AI can be made without biases, “the human being which writes the code cannot,” said Jeff Kagan, tech industry analyst and columnist.
“Bias is part of the AI world. AI networks are not perfect,” Kagan said. “They have the same strengths and weaknesses as the human who wrote the code. This is an important issue we must resolve as we move further into this rabbit hole.”
SPONSORED BY SKYFLOW
Protect Sensitive Data Without Compromise
If your company’s sensitive customer data 一 credit card data, SSNs, etc. 一 is stored alongside other data, like your office’s birthday spreadsheet, you might as well post it on a highway billboard. A breach of one user account could compromise this valuable PII, damaging customer trust in your brand.
Skyflow provides architectural solutions to data privacy by isolating, protecting and governing sensitive data, all via a simple but powerful API. Now, you can protect the privacy and security of sensitive data, take control of your payment processing, and easily meet data residency requirements with Skyflow Data Privacy Vault.
Remove Sensitive Data from the “Breach Zone”: Learn how to avoid the 5 Common Data Privacy Pitfalls and keep sensitive data out of the wrong hands. Skyflow’s zero trust access controls mean only authorized users have access, and only for authorized workflows.
Effective and Scalable Data Residency: Navigate the complexities of data residency laws with ease. Leverage privacy enhancing technologies like tokenization and polymorphic encryption to use sensitive data for critical workflows without compromising on data privacy.
Microsoft Word shortcuts on your desktop? Great. Data security shortcuts in your backend? Bad Idea.
Protect sensitive data from malicious actors to preserve customer trust, and avoid the headaches, fines, and penalties that afflict companies after privacy missteps with Skyflow Data Privacy Vault.
#3. Shopify’s metaverse
Shopify is going Web 3.0.
The company is seeking to patent methods for creating “multi-user augmented reality” content. To break it down, this tech allows users to use AR to see what a product would look like in their home, similar to how other AR shopping works. What differentiates this tech from other AR shopping experiences is that multiple users can look at the same item while physically in different places. Basically, it allows people to shop together… without actually being near one another.
This works in a few different ways: First, a user can allow a merchant or another shopper to see from their perspective an AR object in their own home, allowing the user to get other opinions about how said object looks. Second, users can use AR to shop for the same item in their own respective homes, but see each other’s perspectives if they so choose.
“Shared AR experiences can improve the communication and collaboration between multiple simultaneous users,” Shopify said in its filing.
Shopify itself has been in the AR space since 2018, showing how much of a value-ad this kind of experience can be to merchants and businesses. In a blog post from December, Shopify laid out its commitment to AR retail, saying “The world of augmented reality shopping may seem like futuristic sci-fi. But in reality, businesses are already using AR to sell products.”
“The AR market is still in its infancy, but it’s growing fast,” The blog post reads. “Early adopters of AR have, in recent years, been gamers and those interested in the “metaverse.” But AR isn’t only for them.”
Shopify is far from the first company to offer AR shopping – Just look at Amazon, Target, and even Walmart as examples. However, most AR experiences offered by retailers are only single users, which limits users to a solitary shopping experience.
Allowing for a multi-user experience, as this patent does, might make a buyer feel more confident in what they’re buying. Plus, integrating multi-user AR shopping into its product offerings might make a merchant pick Shopify over competitors like Squarespace, BigCommerce, or Amazon.
Another thing this patent reveals? Shopping will certainly be a major feature of metaverse technology as it continues to grow.
You thought we were done?
Apple wants your speaker to fit anywhere in your house. The company wants to patent “flexible speakers” which can mold to any point in your house. For example, do you want to put a speaker that fits in a specific nook, cranny, or corner? Apple’s invention can do that.
Snap wants you to charge on the go. The company filed a patent application for a carry case for “rechargeable eyewear devices,” essentially allowing you to charge your AR Spectacles from anywhere without the risk of scratching the lens.
Sony wants to make school fun. The company filed a patent application for “game-based lesson plans and learning” for a virtual environment corresponding to real-world actions. Think of it as a better version of Kahoot.
What else is new?
Snap is rolling out its own AI-powered chatbot called My AI. The feature, based on Open AI’s ChatGPT, will only be available to Snapchat+ subscribers.
Skydio, an autonomous drone startup, has raised $230 million to expand its U.S. factory presence 10-fold. The company is now valued at $2.2 billion.
Twitter cut another 50 employees in it’s latest slew of layoffs. Among them was Esther Crawford, head of Twitter payments and Twitter Blue.
Behind closed doors, the shifting tides between Wall Street and Washington work in secret, shaping the world of power. At the intersection of it all is Power Corridor, written by Leah McGrath Goodman. This 2x weekly newsletter delivers unparalleled insights and analysis into what’s happening in the world of power, who’s doing what and why – and how it affects us all. Sign up for early access here.