At the recent Microsoft Ignite event in Chicago, CEO Satya Nadella and his team told what I thought was a strong, cohesive story to pull together the complex strands of enterprise AI. The company still managed to make many separate announcements during the three-hour keynote — I’ll spare you the rundown of every single detail — but there was a unifying arc to it that I think positions Microsoft well as AI continues its move from hype generator to value generator in the enterprise.

Making Clear Strides To Achieve ROI For Enterprise AI

Let me reiterate the obvious: enterprise AI is really complex. That holds even more for a company the size of Microsoft, which came out of the gate early in 2023 with AI-driven search, quickly got into copilots (and launched Copilot) and is now helping to fundamentally change personal computing with the Copilot+ PC. That’s before we get to Microsoft Azure’s role as the second-largest cloud service provider in the world. So it only makes sense for the company to commit itself to innovating in AI in myriad ways.

As he kicked things off at the event, Nadella emphasized that the company’s AI efforts aren’t pursuing tech for the sake of tech, but to achieve real outcomes for business. In part this is an answer to the recent chorus of criticisms from Wall Street that the tech industry and its enterprise customers are overbuilding AI without having solid ROI to show for it. But from everything I can see, it also reflects Microsoft’s genuine intentions with AI: to empower people and companies to do more with multi-model (and multi-modal) AI, better reasoning and planning capabilities and richer context for greater accuracy and relevance.

Nadella did point out some impressive technical achievements: AI performance is doubling every six months, consumption of Azure’s AI models has also more than doubled in the last six months, Copilot is now twice as fast while delivering 3x higher user satisfaction and so on. But there was a louder drumbeat of business results, starting with an example in the opening video that described an AI use case worth $50 million annually to the customer. Later on, Nadella and other presenters went through countless AI use cases, from monitoring legal contracts to shortening customer-service calls to accelerating protein research by 5x.

While I absolutely agree with the shareholder perspective that any major area of investment must justify itself with meaningful returns for the business, I’m also seeing more and more real-world examples of quantifiable — and often large — ROI on AI investments reported by disciplined, no-nonsense companies including Microsoft, Lenovo, HPE, Adobe and others.

The Copilot Three-Layer Cake

During the keynote session, Nadella introduced a three-layered diagram to explain Microsoft’s approach to AI: Copilot is one layer, Copilot-powered devices are another, and Copilot in the AI stack is the third. This organizing rubric is another reminder of Microsoft’s reach among both consumers and businesses, because ongoing improvements to Copilot ripple out across the largest installed base of productivity software in the world (as well as enterprise apps including Power BI and Dynamics ERP), plus Microsoft can introduce whole new classes of computing devices (more on that below) — and then there is all the heavy-duty enterprise IT, especially via Azure.

First, Copilot. What started with essentially a chatbot is now far more than that, crucially incorporating AI agents at every step. Nadella emphasized that expanding Copilots with agents is fundamental to Microsoft’s approach; one Copilot can employ thousands of agents for all kinds of tasks — think anything that involves automation, monitoring, scheduling, planning or collaboration. More than that, agents can be allotted specific roles and permissions tied to job duties, for example by restricting access to certain systems or data to IT or HR staff.

Tools within the Copilot ecosystem can ingest many different multi-modal inputs: numerical data from spreadsheets or your ERP, text or diagrams in documents, social media feeds, e-mails and on and on. The outputs can likewise be almost anything you want: graphs, draft reports, project reminders, webpages, e-mails etc. One presenter even demoed Copilot creating a customized article to help him plan a camping trip, complete with recommended products to buy — all executed onstage in the moment in response to his verbal commands.

The potential productivity gains are pretty compelling. Nadella and other presenters gave examples of synthesizing thousands of reports for risk analysis (saving weeks of laborious reading and collation); helping a salesperson monitor their pipeline and find upsell opportunities; answering questions about a presentation in Teams; extracting information from SharePoint; or, as Nadella himself does, organizing e-mail and prepping for meetings.

Copilot Studio can help you make your own agents, and there was a big point made that creating a basic AI agent should be as easy as creating a Word document. For more advanced functions and more technical uses, there are also Copilot dev tools and coding-friendly agent creation services.

In a separate meeting for analysts, Nadella pointed out that office work hasn’t meaningfully changed since the widespread adoption of the PC; things have gotten incrementally better, of course, but haven’t changed fundamentally. He sees Copilot upending this. From the Ignite stage, he said, “Every employee will have a Copilot that knows their work, helping them unlock productivity, enhancing creativity and saving time.” Importantly, Copilot now includes analytics that will show you exactly what Copilot is doing for your top and bottom line.

In the analyst meeting, Nadella also said that a big first step is agents that can connect organizational silos. During the keynote I was particularly impressed with how Copilot can now connect not only with pre-built or custom agents created using Microsoft, but also with agents from Adobe, SAP, ServiceNow, Workday and even Cohere. This would allow you to keep Copilot as your main AI assistant, but take advantage of all the other agents that may be available through your enterprise systems. Besides connecting silos, this could be a potential solution to the AI agent version of app sprawl that we’re starting to see.

This is highly relevant because the enterprises and global systems integrators I have talked with are having a very tough time integrating these silos of data to be able to do all this magic, and Microsoft is offering solutions to help.

Spreading AI With Copilot Devices

Microsoft is in a position to provide compute everywhere from Azure datacenters to the edge to the device in your hand, and in many cases the company is either making the hardware itself or intimately involved with creating the hardware’s architecture. This of course includes Copilot+ PCs, for which the company has collaborated with all the big PC OEMs, Qualcomm and increasingly AMD and Intel. At Ignite, Microsoft used this opportunity to talk up the benefits of moving your enterprise to Windows 11 — and let me say that if your company is not yet making that move, that’s something you need to fix immediately. From my perspective, Microsoft has removed every single objection to making this transition.

At the event, Microsoft also announced a new piece of hardware called Windows 365 Link. It’s essentially a compute puck that delivers a virtual Windows experience from the cloud. Microsoft has already offered Windows 365 for a few years; the kicker here is that there’s absolutely zero maintenance and zero management of this client, which is adminless, passwordless, cannot be turned off, stores no data and uses the cloud for literally everything. Thinking strategically, I believe that Windows 365 and hardware like the Windows 365 Link are the future of Windows over the next decade. I need to get more information about the Link product, and maybe even get hands-on with it to test the experience, because what broke virtual desktop infrastructure from scaling was the user experience. Windows 365 isn’t VDI, as it’s a managed service — but latency cannot be variable.

Microsoft’s Version Of The Enterprise AI Stack

A lot of the hyperscalers (starting with AWS) and even some of the on-prem vendors (Dell, HPE, Lenovo — each in collaboration with Nvidia) are setting up their own AI factories under one name or another. For Microsoft, it’s the Azure AI Foundry, which I believe makes a big statement about the company’s strategic approach in this area and its dedication to multi-model AI. We’re talking about more than 1,800 models, including all of the latest ones from OpenAI plus open-source models from Meta, Mistral and others.

In this context, I was fascinated to hear about the 20-plus specialized vertical models Microsoft has developed with partner companies outside the tech sphere, including heavy hitters like Bayer, Rockwell and Siemens. This is a very cool phenomenon of companies we don’t consider as high-tech getting into an emergent aspect of the high-tech market to create differentiation.

Bigger picture, Azure AI Foundry unifies everything enterprises need in one place, complete with all the apps and tools to put AI to work. Within it, you can evaluate models, customize them, govern them. And of course it’s built to play nicely with Microsoft’s many other services, from GitHub to Copilot Studio. You can also use it to fine-tune your models with RAG or other methods, and fine-tuning is only growing more important for enterprises as they work to get the most out of their AI investments.

But the number one impediment to enterprise AI adoption and effectiveness? Data. Data is everything, but it presents a lot of sticky problems for enterprise AI. The reason is very simple. When business AI was dominated by machine learning, the data you needed came from inside the stack. You created new efficiencies in your supply chain, for example, by analyzing the structured data in your ERP and SCM systems. But in this era of generative AI, there’s way more data, it comes from everywhere, and it’s often a mess. Again, every show I attend where I talk to enterprises and integrators, they say they are struggling with integrating their data silos.

The need to harness all that data to make it usable is what led Microsoft to launch Fabric, its unified data platform, last year. All of the hyperscalers have something like this, plus many customers achieve parts of this functionality from more specialized vendors such as Cloudera, Snowflake and Databricks. Even though it’s only been in GA since last year’s Ignite conference, Fabric already has 16,000 customers, including 70% of the Fortune 500. Microsoft bills it as an enterprise data platform for all use cases, bringing together operational and analytical workloads in one place. At Ignite 2024, Microsoft announced a significant advance for the platform called Fabric Databases. This brings SQL Server natively into Fabric, which I think will give enterprises an “easy button” to simplify and optimize their databases for AI use.

Making Enterprise AI Digestible And Effective

If we take a broader view of all the experiments in enterprise AI across the two years since the AI hype cycle began, the painful truth is that many of them yielded no meaningful results — and in lots of cases maybe shouldn’t have been built in the first place. By contrast, Microsoft was very early with its investments in generative AI (especially through its support of OpenAI), and since early 2023 it has taken an aggressive but very steady approach to making AI a genuine source of value, for itself and for its customers.

It’s no accident that today big names as diverse as Toyota, Blackrock and NASA are using Microsoft’s AI stack. It’s also no accident that Microsoft has done the hard, unglamorous work of building a data management platform like Fabric, which appeals mainly to IT geeks — but which could be a boon for scaling AI beyond experiments and into pervasive operational deployments.

To be sure, the company has to watch out for over-hyping its capabilities, yet still be inspiring enough to motivate enterprises to move more quickly. The biggest thing Microsoft needs to address is something it didn’t at the show related to data. With 70% to 80% of enterprise data on-prem or on the edge, how does Microsoft maximize its play with AI? From my vantage point, the talking point so far is “Move the data to the public cloud.” Yet we’re 17 years into the public cloud, the on-prem stacks from VMware, RedHat and Nutanix are getting better, and I don’t see a mass public-cloud migration anytime soon. Microsoft can more directly address this.

On the plus side, Microsoft seems to grasp how quickly AI is evolving, from models to apps to use cases to user experiences. And the company certainly doesn’t lack ambition. At the analyst session during Ignite, the question arose of how ubiquitous Nadella thinks Copilot could be. His answer: “How many Excel spreadsheets are there in your organization?” Based on what we saw at Ignite, I think Microsoft’s odds of achieving that are pretty good.