By Tim Biggs
In 1997 Microsoft introduced the Office Assistant, which generally took the shape of an anthropomorphic paperclip, and users were not impressed. A quarter of a century later, it’s rolling out similar features leveraging the power of generative AI, and presumably hoping for a warmer reception.
The Copilot is a new feature for enterprise installations of Office 365, meaning it should be available to millions of workers around the world in the coming months. Like the paperclip (which these days is commonly referred to as Clippy), it understands many different document formats and wants to help users get started on their projects or refine their ideas. But Clippy never had access to the large language models of OpenAI’s GPT-4, or the huge amount of data each client company has in its Microsoft Graph. This is Clippy on steroids.
The Copilot in Office can draft a document for you, or get you ready for your next meeting.
In Word, the copilot could draft a job ad based on a short description of the role, or you could ask it to rewrite a paragraph to make it briefer, more active or more professional. It also has access to all your cloud files, so you could ask it to write a proposal based on notes from yesterday’s meeting, or summarise arguments from a series of documents.
It will also exist in Excel, PowerPoint, Outlook and more, serving a similar role. It could turn a long document in to a five-slide presentation complete with stock photos, or draft a “friendly reminder” email without you having to go through the trouble of remembering how to sound friendly.
At an event announcing the features, Microsoft 365 head Jared Spataro was keen to point out that, like all generative AI, it wasn’t infallible and should always be double-checked.
Copilot drafts a friendly-sounding email based on a quick prompt.
“Sometimes Copilot will get it right, other times it will be usefully wrong, giving you an idea that’s not perfect but still gives you a head start,” he said.
Microsoft has been on a spree of AI-powered product announcements recently, all leveraging a partnership with OpenAI, including adding GPT-powered chat to its Bing search engine. But Copilot isn’t quite the same as just asking Bing because it uses an organisation’s data in the cloud to tailor responses.
Copilot turns a file of notes into a Word document.
A Microsoft spokesperson said it did this without requiring deeper data permissions, and while inheriting all of an organisation’s existing security, compliance, and privacy policies.
“Through these new products and tools, users will be able to access and reference content they already have access to, from emails and chats to decks and documents. Existing rights controls are respected,” the spokesperson said.
“Copilot’s large language models are not trained on your tenant data or on your prompts. And importantly, with Copilot your data — including your prompts — stay within the compliance boundary.”
Microsoft also has an AI companion coming for Teams, called Business Chat, which sits in on meetings and can be called upon to make summaries or generate updates based on emails, documents, notes or other data.
But the AI features aren’t strictly for business, as Microsoft’s new announcements for image-generating AI aim squarely at consumers as the company continues to press its temporary advantage over search market leader Google.
The latest Bing features include a revised version of “Knowledge Cards” in search, which appear for certain topics and include pictures, summaries, graphs, timelines and more, looking like an AI-generated school report. There’s also a new feature called “Stories”, which look just like the feature of the same name in Instagram or Snapchat, but have an AI voice read AI-generated summaries of certain topics with pictures.
New AI-powered elements in Bing search results, including a Story at the top and Knowledge Cards down the right side.
Potentially more impactful is an integration of Bing Image Creator – which is based on a version of OpenAI’s powerful DALL-E 2 text-to-image system – into Bing Chat.
That means that in the same chat where a user is asking the Bing chatbot to give suggestions on a kitchen renovation, they could ask how that would look, and Bing would be able to create a believable image from scratch.
Yusuf Mehdi, Microsoft VP and consumer chief marketing officer, said applications ranged from “time travelling” by asking to see certain places in different times, to applying specific parameters for a creative project to get an idea of where to start.
Pictures generated by Bing Image Creator using various prompts.Credit: Tim Biggs / Bing Image Creator
“It’s helpful for inspiration. If someone wants to create a charcuterie board but they don’t know what a good one looks like ... or they’re planning a kids party want ideas for say astronauts in a field of sunflowers, what would that look like,” he said.
“It’s a faster way to get to exactly what I want versus what I did in the past, which is I had to go through Pinterest, or I tried to search and I wasn’t getting exactly what I wanted.”
Explained simply, DALL-E is a system that can understand a text prompt and link it to visual representations, thanks to a machine learning process trained on a huge number of photos and artworks found online. It’s far from perfect, however, with requests for photo-realistic images in particular frequently creating impossible or unsettling scenes.
Crafting a prompt for Bing Image Creator
Whereas something like Google Search requires a short, accurate query to be most effective, generative AI works best with lots of detail and specificity. To get the above images, we used:
- Close up painted portrait of a man’s face. Freckles, green eyes, trimmed beard.
- 90s heavy metal album cover with a chameleon.
- A portrait of a demon. Cubism.
- Mid-century modern living room with lots of plants.
- Sydney Harbour in 1952.
- A steaming mug of coffee but the steam is turning into an image of thought bubbles.
Anyone can currently access the Bing Image Creator at bing.com/images/create/, but you have to craft your prompt carefully on your own. The integration with Bing Chat means the AI can potentially use your plain language explanations to create a prompt that will result in images closer to what you want. Or you could easily repeat but modify a prompt by saying “that’s good but make it rounder and blue”.
The AI-powered Bing Chat itself was only introduced in February, and prompted concerns when it began acting erratically or inappropriately in certain situations, largely when pushed or abused by the users testing it. Partly because of that, the integration with Bing Image Creator is rolling out to users gradually.
The system has precautions to prevent abuse built in, but generative AI can be hard to predict.
“Obviously, we’re doing a lot of work with responsible AI and putting safeguards in to prevent the misuse of it for violent images, harmful images,” Mehdi said.
“We’re playing with watermarking a version of the Bing icon so that it’s very clear [these are AI-generated images] and we’re going to keep revising and get feedback.”
AI image generation has also come under attack from some artists, who say it plagiarises works found around the web, and disincentivises people commissioning from human artists.
Mehdi said DALL-E trained only on licensed and publicly available content, but that there would be a mechanism for concerned artists to ban Bing from fulfilling requests to crib their style. Microsoft is also hoping to strike commercial deals with artists that would compensate them when their work is used for these kinds of requests.
The new features come as Google has made an early version of its Bard chatbot available for the first time, to a limited number of users in the US and the UK. The search giant also has generative AI features ready to go for its Workspace, to counter Microsoft’s Copilot. Built with Google’s in-house language models, they will also be able to draft or refine entire documents based on natural language prompts.
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.