Recent posts by Mona

This is a guest post by Hyro - a Mona customer. 

What we learned at Hyro about our production GPT usage after using Mona, a free GPT monitoring platform. 

At Hyro, we’re building the world’s best conversational AI platform that enables businesses to handle extremely high call volumes, provide end to end resolution without a human involved deal with staff shortages in the call center, and mine analytical insights from conversational data, all at the push of a button. We’re bringing automation to customer support at a scale that’s never been seen before, and that brings with it a truly unique set of challenges. We recently partnered with Mona, an AI monitoring company, and used their free GPT monitoring platform to better understand our integration of OpenAI’s GPT into our own services. Because Hyro operates in highly-regulated spaces, including the healthcare industry, it is essential for us that we ensure control, explainability, and compliance in all our product deployments. We can’t risk LLM hallucinations, privacy leaks, and other GPT failure modes that could compromise the integrity of our applications. Additionally, we needed a way to monitor token usage and the latency of the OpenAI service in order to keep costs down and deliver the best possible experience to our customers.

In the rapidly evolving landscape of AI, staying ahead of the curve is crucial for data scientists and engineers. With the increasing adoption of large language models (LLMs) such as OpenAI’s GPT, monitoring the performance, quality and efficiency of the applications that leverage these models has become crucial for businesses. As a leader in intelligent monitoring solutions for AI, we have leveraged our industry expertise and existing platform to develop a monitoring solution specifically tailored for GPT-based products, enabling teams to optimize the performance of their applications and improve their usage of LLMs over time.

Large language models (LLMs) are becoming the bread and butter of modern NLP applications and have, in many ways, replaced a variety of more specialized tools such as named entity recognition models, question-answering models, and text classifiers. As such, it’s difficult to imagine an NLP product that doesn’t use an LLM in at least some fashion. While LLMs bring a host of benefits such as increased personalization and creative dialogue generation, it’s important to understand their pitfalls and how to address them when integrating these models into a software product that serves end users. As it turns out, monitoring is well-posed to address many of these challenges and is an essential part of the toolbox for any business working with LLMs.

Monitoring is often billed by SaaS companies as a general solution that can be commoditized and distributed en-masse to any end user. At Mona, our experience has been far different. Working with AI and ML customers across a variety of industries, and with all different types of data, we have come to understand that specificity is at the core of competent monitoring. Business leaders inherently understand this. One of the most common concerns we find voiced by potential customers is that there’s no way a general monitoring platform will work for their specific use-case. This is what often spurs organizations to attempt to build monitoring solutions on their own; an undertaking they usually later regret. Yet, their concerns are valid, as monitoring is quite sensitive to the intricacies of specific use cases. True monitoring goes far beyond generic concepts such as “drift detection,” and the real challenge lies in developing a monitoring plan that fits an organization’s specific use cases, environment, and goals. Here are just a few of our experiences in bringing monitoring down to the level of the highly specific for our customers.