How Martello Measures the End-User Experience in Microsoft 365 Hybrid Deployment

Originally posted on gsx.com

Microsoft 365 provides the best collaborative SaaS solution on the market. Keep in mind that a Microsoft SLA is only valid within a Microsoft datacenter.

Once outside it, and without the Martello Robot User, you have no data and hence no visibility into the quality of the service delivered to the end-user.

Why is it important to monitor Microsoft 365 services quality & end-user experience?

• Prevent user complain and tickets
• Prepare for Microsoft 365 deployment
• Measure the ROI of any Network & Infrastructure upgrade
• Put facts to emotions and understand the reality of end-user complaints
• Choose the best route to optimize end-user experience & service quality

At Martello we take this problem seriously and we provide the perfect tool face these challenges, helping you decide what you have to do to prevent, detect and fix Microsoft 365 performance issues.

What is a Martello Robot User?

The Martello Robot User is a Windows service, sitting on any Windows machine. We do recommend dedicating these machines (laptop, stick, etc.) in order to have consistent 24/7 monitoring.

You need constant end-user experience statistics to understand the baseline (what is your “normal” performance) and to compare it instantly with any issue detected. This allows you to understand the reality of complaints, compare before and after a change and understand the root cause of any issue with your Microsoft 365 services.

The GSX Robot User is working to produce and provide these statistics around the clock.

They resides as close as possible to the actual users, at every location you want to monitor for the level of service delivered to the end-user.

What is the Martello Robot User doing?

The Martello Robot User connects to Microsoft 365 through port 443 and TCP and perform multiples actions against it.

1)  Performing real end-user actions on Microsoft 365

• For Microsoft Exchange Online, the Martello Robot User will: open mailbox, create email, create tasks, create folders, create meetings, resolve user, download attachment, perform free/busy lookups, search an item in a mailbox and send real email to check the mail routing internal & external.
• For Microsoft SharePoint Online and OneDrive, the Martello Robot User will: login to SharePoint, upload an attachment and download an attachment.
• For Microsoft Teams, the Robots will use instant messaging, create channels, post messages on them, upload and download documents, search for users, check their presence and of course perform calls. While doing that they capture every network statistic (Jitter, Packet Loss, Healed Ratio, MOS, etc.) to determine whether there is an issue related to them.
• For Microsoft Skype our Robot are also constantly performing calls, measuring the Voice quality and how the network is affecting it.
• For every end-user tests done, the GSX Robot User collects the availability of the Microsoft 365 feature and, of course, the time it takes to perform it.

2)  Performing network health & performance checks

For all of these environments, the GSX Robot User performs network checks to understand the health of your network. Round trip time or DNS resolution availability and performance are among the tests they are constantly doing.

These network statistics allow you to correlate end-user experience problems with potential network issues.

3)  Performing real end-user actions on Microsoft hybrid deployment

GSX Robot Users are also able to measure the end-user experience on Microsoft on-premises environments to provide full visibility on any hybrid deployment.

• For Exchange on-premises, the end-user tests are the same for Exchange Online described above. On top of that GSX is also able to monitor the health of the Exchange Edge Server (availability of the Exchange services & protocol, health check, critical performance counters, mail queues, etc. ) and its connectivity to the Exchange Online environment (mail routing hybrid, scenarios hybrid).

4) Performing hybrid identities health checks

Finally, GSX allows a check of hybrid identities that are key in the Microsoft 365 monitoring. Issues with Azure AD Connect or ADFS really impact the end-user experience.

• For Azure AD Connect, GSX continuously checks the synchronization of the identities to make sure your user can authenticate well.
• For ADFS, GSX performs true authentication to Microsoft 365 with real token exchange in order to make sure the entire process is healthy and performs well.
• As you can see our GSX Robot Users are really a key element integrated in GSX Solutions to capture the Microsoft 365 end-user experience as well as any element of any hybrid deployment (Microsoft server) that can impact end-user performance.

How are they managed?

The GSX Robot Users are centrally managed to allow remote deployment & configuration and also alert coordination.

Every threshold can be customized to the need of the locations that are monitored.

Alerts can be sent to multiple profile through multiple alert system and are smart enough to be triggered only if there is potential end-user impact of Microsoft 365 degradation of services. Of course, they can be connected via Webhook to any major ticketing system (like ServiceNow for example).

The statistics collected through the tests & checks performed by the GSX Robot Users are used in two different places:

• The GSX web interface is very useful to troubleshoot any issue in real time.

And the PowerBI dashboards that we use to report the end-to-end service availability that can be targeted to your needs (overall, site by site, workload by workload, action per action, etc.)

As you can see, at Martello, we build tools to monitor the Microsoft 365 end-user experience & service quality that take into account everything that can impact this user performance.

And we build them to enable you to prevent any operational and financial risks during your deployment, run and performance optimization projects.