Internet browsing is a "lonely experience," says Microsoft. Copilot Vision is your AI companion that can visually process what you're browsing and help you out along the way.
On Thursday, Microsoft announced a preview version of Copilot Vision for Copilot Pro subscribers. Microsoft introduced Vision in October as a solution to the problem that users have to explain what they're seeing to Copilot.
Vision sits on the bottom of your Microsoft Edge browser and, when enabled, "it sees the page you're on, it reads along with you, and you can talk through the problem you're facing together."
Vision is one of the first features to debut in Copilot Labs, its testing ground for AI projects. Labs, which has a noticeably softer and warmer look is Microsoft's way of sharing "works-in-progress" geared towards helping users with everyday tasks.
One demo shows Copilot Vision helping a user find the right place to stay, another helps a user make sense of photos of their grandmother's handwritten recipes. It can also help you out in a Minecraft game or with holiday shopping by pointing out the right products on the page.
How Copilot Vision handles privacy
If Copilot Vision reminds you of Windows Recall, Microsoft's "privacy nightmare" tool that consistently takes screenshots of your computer screen, there are several key differences. First, Vision is opt-in, so you can choose if you want to use it. Second, as soon as you end the session, you Copilot conversation is deleted. That said, "only Copilot’s responses are logged to improve our safety systems," read the announcement.
Microsoft has also thought through some of the sticky issues of web browsing when it comes to paywalled and sensitive information. Vision only works with a select set of websites for now, and it is programmed to "to not provide responses based on paywalled content," according to its FAQs page.
Crucially, Microsoft says Copilot Vision does not store or use any publisher content to train its models. Microsoft also says Vision "honors website controls for usage of their content by Copilot," which presumably means sites with robots.txt code that block web crawlers. Mashable has reached out to Microsoft for further clarification of Vision's data controls and will update this story with a response.
Starting today, Copilot Vision is rolling out to a limited set of Copilot Pro subscribers. A Copilot Pro subscription costs $20 a month.
