Could Martello’s Copilot 365 Monitoring Service Transform AI Management?

As the AI revolution and evolution continue to accelerate apace, the nascent technology’s business users are gradually adjusting to leveraging it in their daily workflows and uncovering new and exciting use cases that enhance their productivity and collaboration.

However, all these novel possibilities inevitably create fresh challenges, including how to maintain, manage, and oversee organisational AI use so that it’s effective, compliant, and maximises the return on investment. As a result, there will be growing market demand for solutions that support admins and managers in monitoring organisational AI to fulfil those aforementioned goals.

That is where Microsoft Teams management and monitoring specialist Martello and its enhanced Vantage DX platform have stepped in, delivering what the company describes as an “industry first” Microsoft Copilot 365 user experience management solution for enterprises.

“One of the things we are seeing happening in the market with the adoption of Copilot and also with our own customers is that they are purchasing and investing in AI,” Randah McKinnie, VP of Product at Martello, told UC Today. “They’re investing in the Microsoft solution if they’re already using that platform.”

“But interestingly, we actually did a little bit of research when we saw that statistic. We looked at what our customers were doing, and we have access to their Microsoft license information. So we can actually see that across our customer base, while they had purchased a large number of licenses, only about a third of those licenses had yet been assigned to users. So, to me, that really reinforced the fact that they’re still trying to figure out how to do this.”

Spotting An Opportunity

McKinnie’s “main purpose” in her role is “to really look at the value that our products are driving out into our customer base and then make sure that we’re meeting the need, that we’re coming up with things both that the customer is looking for, that the market is looking for, and that can also help us, of course, grow a successful business”.

That focuses on Teams but also monitoring across all of Office 365 and many other impacts occurring across that ecosystem.

McKinnie and her team identified the opportunity to create hugely valuable features that inform and guide customers “still trying to figure out how to do this”.

“You generally see that almost all of those Microsoft licenses that have been purchased have been assigned, and particularly, Copilot licenses, which are expensive,” McKinnie elaborated. “So why would an organisation just be sitting on them?”

“Organisations are still trying to figure out how to use it,” McKinnie said. “So, while we’re not the ones that are working with the end users on how they’re using the solution, we know that one of the things, in order to have them engage with it, is that they’re going to need to have a good experience with it. What we can do is help with the IT deployments and employee engagement departments and make sure that we’re getting rid of any roadblocks to having customers engage with them.”

“That would include reliability, speed of performance, and anything we could do to help them ensure that their networks are configured to support whatever additional capacity the communications require,” McKinnie continued. “I think what’s interesting is that when we released this, it was not like we had a lot of customers asking for it. That’s always what happens when you’re kind of new on the cutting edge of something, right?”

Martello aimed to get something to market as quickly as possible to enable the customer to monitor what’s happening when they deploy Copilot and encourage engagement. This meant having something already there for them so they wouldn’t lack that need.

“If their users started to complain about, ‘Well, I tried, but I couldn’t actually get to the site. I tried to open the side panel in my application, or I tried to go to that Copilot page on the web, and it wasn’t loading, or it wasn’t available.’ The IT department wouldn’t be just sort of sitting there wondering, ‘Now what do I do?'”

“This will give them some overtime in our monitoring tool. You can see over time the stability of the application so they’ll know before the problems happen whether this is going to be a problematic application for them or not.”

A Continuation Of Market Strategy

McKinnie highlighted that, while this feature set is groundbreaking in its Copilot management, it “well aligns” with Martello’s “competencies” in “proactive monitoring” across the Microsoft ecosystem, which means synthetic transactions that can essentially emulate what a user is doing on the network before the user does it.

More advanced than real-time monitoring of users across a network, proactive monitoring takes the user’s actions before the user does on a constant, continual basis to ensure the network is ready and capable and also shows the admin or manager if there are any hiccups that need resolving ahead of time.

“Yes, we can also provide that capability in a way that supports AI,” McKinnie affirmed. “Again, people are not talking about how AI is going to tax our networks, but we’re certainly talking about compute power. That compute power has to fill the data that’s happening with it while it’s a separate tax on your systems. There’s still going to be a lot of back-and-forth traffic, nodes, and everything else, so we want to be there to see what happens, and we want to be able to take care of things as they emerge.”

Catering To Customers’ Needs

Among other recent updates to Vantage DX were major enhancements to Microsoft Teams Rooms and Phone monitoring and recommendations. These empower organisations to refine these high-value solutions and maximise ROI on Teams Rooms infrastructure or Teams Phone migrations after moving on from PBX setups.

“The other thing we’re looking at is that any use of Teams and any collaboration organisation is incredibly valuable,” McKinnie said. “We definitely position the value of wasted time, even in ad hoc meetings, as really what that costs an organisation.

Copilot monitoring will inevitably take the headlines, though. Given this is such a pioneering capability, which Martello launched only late last month, naturally, no customers have approached the business with mature use cases yet, albeit they have seen plenty of initial curiosity.

However, to understand customers’ potential responses, Martello deployed Copilot internally and developed a structured assessment programme with early adopters in its community. McKinnie and her team attempted to parse what customers might try to do, what they might find great about the experience, and what they might find frustrating.

“One of the things is that I’m relatively impatient, so I expect pretty good response times,” McKinnie said. “I did find that when you ask it a question because it has to do all that computing, you are waiting. I’m sure people have had that experience with ChatGPT, right? You’re waiting a while. That’s been an interesting one.”

Another compelling aspect is AI’s context-setting when working in an enterprise organisation and the benefits and challenges that emerge from exploring that more deeply. There’s the context that includes everything on the internet and then the internal office or Microsoft 365 deployment context.

“That’s been kind of an interesting thing, too,” McKinnie added. “I don’t know whether that’s going to affect performance or not. It certainly is going to be a traffic differential. But even from a just usability interest, if there’s something we can do to help users be aware of their context, maybe Microsoft can fix this problem. Maybe it’s not a problem, but it is something the user needs to pay attention to.”

For example, McKinnie outlined a case of Martello doing a search before asking Copilot to generate an article.

“We’re like, ‘Wow, look at that. It’s referencing Martello everywhere. It must be so popular out there on the web.’ And then one of us will actually say, ‘Well, did you check? What was the context? Oh, that was only from our internal sources.'”

“So, of course, we’re top of mind. It’s going to be a little bit different when you go out on the web, especially if the term is something like Copilot, right? We shouldn’t be first. Microsoft has definitely captured all the SEO around that. There are going to be problems to solve for the user that you won’t even think about until people start using it.”

More To Come

Martello naturally considers this only the beginning and intends to maintain Copilot monitoring as a core part of their strategy moving forward. As McKinnie asserted, “Everything AI right now is a core part of our strategy”, highlighting that Martello has produced a specific AI strategy, especially around enabling business efficiencies and conquering big data; both ambitions served to an extent by this initial Copilot monitoring solution.

“We’re going to help take all that data that you see when you enter VDX and help create ways to make that more actionable,” McKinnie said. “Anything we can do to both support how people are using AI in their existing tools and then, of course, how we can be using the capabilities to embed better functionality within our own tools.”

“Not only that, but hopefully, we will learn from the actions taken within an organisation, find out which ones help solve the problem, and then make better recommendations over time. So it’s coming from two places. How can we help users with whatever AI they’re using, and then how can we make our tool better with the technology?”

“The other thing we do there is generate alerts. So we’re just giving that as X-ray vision awareness into what’s happening.”

McKinnie also suggested other future ambitions. While Martello has been focused on Teams and Microsoft, the business is also asking where else it can direct this capability of advanced proactive synthetic monitoring.

“It may be that there’s enough for us to bite off in the Microsoft space, but we’re also seeing a lot of our service partners have got people that aren’t just monitoring Teams,” McKinnie outlined. “They’re monitoring other tools as well. So, looking at how we can expand to spread that capability across other things is also a question mark for us.