Microsoft customers complain Copilot doesn’t work as well as ChatGPT. Microsoft says they’re not using it right.

Microsoft customers complain Copilot doesn’t work as well as ChatGPT. Microsoft says they’re not using it right.


copilot collage with open AI and microsoft ceo



Hannes P Albert/picture alliance via Getty Images;Chelsea Jia Feng/BI



  • A top complaint from Microsoft’s customers is that Copilot doesn’t perform as well as OpenAI’s ChatGPT.
  • Microsoft says customers aren’t using its new artificial-intelligence tools properly.
  • The company is paying a partner to produce videos to teach customers how to write better prompts.

One of the top customer complaints about Microsoft‘s Copilot is that it doesn’t seem to work as well as ChatGPT, according to employees with direct knowledge of customer feedback.

“Every time a customer starts using it, they start comparing it to ChatGPT and saying, ‘Aren’t you guys using the same technology?'” one of the people said.

ChatGPT, OpenAI‘s artificial-intelligence chatbot, has set relatively high expectations for customers who are now trying out Microsoft Copilot tools for the first time. The software giant’s efforts to meet those expectations is an important test as the AI industry attempts to switch users from free consumer chatbot offerings to something more valuable.

Microsoft is racing to add value to these AI tools before customers start asking whether they’re getting a proper return from the extra money they’re spending on this much-hyped technology.

Copilot vs. ChatGPT

Copilot for Microsoft 365 has reached the most customers so far after the company made it generally available in November. This is a version of the AI assistant that works alongside the company’s suite of business applications such as Word, Outlook, and Teams.

Feedback for the tool has been mixed to leaning positive so far, according to the Microsoft employees who spoke with BI.

There are the usual Microsoft problems: Some customers are using older versions of products such as the Outlook email service, and they expect whizbang AI Copilot capabilities to work with this somewhat aging software.

But Microsoft employees told BI the comparisons with ChatGPT kept coming up. These sources asked not to be identified discussing private matters.

The “work” version of the Copilot tool uses internal customer data to help provide automated support to employees working on tasks such as summarizing meetings.

It queries this sensitive information from sources including Microsoft’s SharePoint collaboration software. This sometimes means responses aren’t as quick or thorough as a free and open web-based chatbot such as ChatGPT that’s been trained on information from the entire internet.

Microsoft employees told BI that customers who were saying this Copilot tool didn’t compare favorably with ChatGPT just didn’t understand how the different products work.

This was confirmed by Jared Spataro, Microsoft’s corporate vice president of AI at work, in an interview arranged by the company.

Microsoft’s Copilot tools are built on what the company calls the Azure OpenAI model. This takes OpenAI’s top GPT models and adds additional capabilities.

These GPT models provide a broad foundation of knowledge. Copilots essentially sit on top of this and tap private customer data to provide bespoke support in specific work situations. The system naturally has more restrictions, according to Microsoft employees. For instance, it only temporarily accesses internal data and then deletes it after each query.

Teaching customers how to prompt

Another problem for Microsoft is that users are typically bad at writing prompts, the employees say. These are special instructions that get the most out of AI models and chatbots. There’s even a new job for this emerging skill, known as prompt engineering.

“It’s a copilot, not an autopilot, you have to work with it,” one Microsoft staffer told BI.

The answer an AI tool gives can only be as good as the question asked. “If you don’t ask the right question, it will still do its best to give you the right answer, and it can assume things,” this person added.

One of the employees said Microsoft had hired its partner BrainStorm to create training videos to help customers create better prompts for Copilot. Microsoft and BrainStorm declined to comment on this.

Some Microsoft customers have even created their own internal teams to help train staff on how to make the best use of these AI tools.

Spataro said Microsoft had put a lot of work into providing support for prompt engineering within the Copilot for Microsoft 365 product itself, such as with FAQs and prompt examples.

‘Work’ Copilot vs. ‘web’ Copilot

A source of customer confusion is that there’s a “work” version of Copilot for Microsoft 365 and a “web” version of the tool.

The web-based version generates similar outputs to ChatGPT and runs in a similar way. Meanwhile, the “work” version is the one that uses internal private customer data to provide more bespoke and specific responses.

For instance, a Microsoft customer may use the web version of Copilot to search publicly available information about a client. Then they could switch to the work version to find out what extra information is available about this client from internal corporate data.

Still, employees told BI that customers were getting confused about why the “work” version wasn’t giving responses as quickly or as thoroughly as the web-based Copilot.

Spataro said Microsoft had been taking measures to help customers understand how these different Copilot offerings operate. For instance, the company is introducing a toggle switch so customers can swap between the “web” and the “work” version of Copilot to help them understand which dataset — the web or their internal files stored within SharePoint — is being queried.

“Copilot for Microsoft 365 is unlike any other AI experience at work, with a deep understanding of your job and organization that combines top of the line AI models, the web, and your business data to enable new scenarios that directly impact the bottom line in a way that wasn’t possible before generative AI,” Spataro said in a statement.

Are you a Microsoft employee or someone else with insight to share?

Contact Ashley Stewart via email ([email protected]), or send a secure message from a non-work device via Signal (+1-425-344-8242).

Axel Springer, Business Insider’s parent company, has a global deal to allow OpenAI to train its models on its media brands’ reporting.

Axel Springer, Business Insider’s parent company, has a global deal to allow OpenAI to train its models on its media brands’ reporting.

Click Here To Read More

Related posts