The Reality of Deploying AI: Lessons from ClaimFox’s Leadership Team

Ali Merchant 00:28 Hello everyone, and welcome to the Enterprise AI podcast by Orvera AI, where we speak with business leaders and operators about how AI, technology, and better workflows are shaping the future of enterprise service delivery. Today, we have with us Michelle Arimes, Director of Customer Experience, and Heidi Villarreal.

Ali Merchant 00:55 Director of Technology, both at claim Fox, Michelle and Heidi. Thank you both for joining me today.

Michelle 01:05 Thank you for having us.

Heidy Villarreal 01:06 Thanks for the invite.

Ali Merchant 01:08 All right, so let's get started. you know, could you each briefly share your professional journey, how it has shaped your approach in improving customer operations so that we get us started from there.

Michelle 01:20 Sure. So I've been with Claim Fox as the director of customer experience for for a little over a year now, but prior to that I was in the also with Claim Fox, but I was in a role of

Michelle 01:35 New business development, which included a little bit of sales and marketing. So I've been able to experience kind of both sides of it on the way we approach the marketplace about who we are and what we do. And now more recently how we communicate with the customers directly, which is you know, what brought us to over AI.

Ali Merchant 02:00 Appreciate the introduction.

Heidy Villarreal 02:02 On my side, I've been in technology for the past 15, 20 years across product, engineering, security, and project management. And now I'm leading technology at Clean Fox. I've been here for about four years now. And I love what I do. it shaped how I see the world and how I approach my work. I'm always looking for

Heidy Villarreal 02:24 how things connect people, processing systems, and seeing opportunities to make them work better together. And that's what excited me about AI and about being here today is that today we can imagine building things and opening up new ways of things that we could do that we wouldn't have considered before.

Ali Merchant 02:46 appreciate that introduction. So let's dive into it so that you know the audience has more context. What does claim Fox handle for clients and what happens between receiving a request and completing it?

Michelle 03:02 Sure, so we are a business process outsourcing partner. So what we do is work with insurance carriers. We partner with them, they become our client. We are fulfilling their incoming record request, so they receive requests for copies of claim documents.

Michelle 03:22 That process is passed along to Claim Fox. So we are managing the process of collecting data, reviewing subpoenas, legal demands, and we are communicating with the parties who request those documents.

Michelle 03:37 That's where our customer experience comes into play. So in between us receiving that request and us delivering the complete, the complete and final package of those requested documents, we are constantly communicating communicating with our customers by email, by phone, by chat, and we are

Michelle 04:02 Able to assist them in answering and resolving their inquiries on whatever that may be for that particular request. It could be status checks, it could be inquiry for dispute of the invoice or payment, and more in-depth customer service that includes the data and the privilege information that they are requesting.

Ali Merchant 04:30 Yeah, I think that sounds like quite a complex operation to handle.

Michelle 04:37 Yes.

Ali Merchant 04:38 Let's dive right into the main theme of AI, right? As we all know, it's kind of changing the world and how we do things. And we would love to learn a little bit on your end, how it is changing your operations. And dive into both of your experience from a customer experience standpoint, from a technology standpoint, that how you both view this so that the viewers who are now thinking on going on a journey of AI, also kind

Ali Merchant 05:08 take some guidance from that. So let's start with how do you choose like the first AI use case which actually solves a real problem in so many things happening. How do you choose that first one to start?

Heidy Villarreal 05:24 we usually start by understanding the problem we're trying to solve. I think as humans, we tend to jump into the solution before we have a deep understanding of the problem. So we take the time to go deeper, understand where is the friction, who does that affect, and what's causing it. Then once we understand that and what a better outcome for that problem looks like, we work backward from there to determine what needs to change. Right? So

Heidy Villarreal 05:52 Then we start evaluating if AI is even the right fit for that problem because sometimes, you know, AI is the big hype right now and everybody wants to do an AI use case, but AI is always not the right path. It may be a straightforward integration or even a process change, right? It doesn't need to be a technology development.

Heidy Villarreal 06:13 If AI is the right fit, then we say, do we need an LLM or a large language model for this specific solution? Because there's other types of AI that could be a better fit, like specialized machine learning or anything that could be better fitted for that task than an actual NLM. And then once we have that, for that first use case, we'll look into having a clear scope.

Heidy Villarreal 06:38 Having reliable data and the outcome we want to measure, while always having a human in the loop. One thing we found is that AI is here to assist in many ways, but we need humans to validate it's actually working, validate the cost associated with that AI investment makes sense. So we always maintain a human in the loop here at Claimbox. And that gives us a manageable starting point to test, learn, and decide whether it's worth expanding or not.

Ali Merchant 07:10 I think that's a very good 360 view. And I think it really helps people to kind of, know, how do do this? And you know, the interesting part about it kind of, know, small, subtle things tell you a lot about an organization's culture. Today, that you both have come together on a podcast instead of one, generally when we invite someone, right, tells you, which is our philosophy as well at Orvera, that it's...

Ali Merchant 07:39 While it's a technology revolution, it goes hand in hand with operations. So generally in tech companies, when deployment happens, you have this forward development deployment engineer, right? But at Orvera, we always have one operations person with them all the times. It generally is not the approach. So I think this will be very interesting to see how you kind of

Ali Merchant 08:10 blend it both together and balance it in a way that really helps the customers. All right. I think one very important part today is that with the avenue of AI, it has become very easy to give impressive demos. I think the biggest challenge right now for any leader is how do they choose a partner with whom they should work?

Ali Merchant 08:38 So I would like to ask you both, know, what are your views? What do you look for in a partner in a demo in a POC, you know, and how do you choose a partner like real practical insights?

Michelle 08:54 So we we choose partners not frequently, right? It takes us, we do a lot of due diligence. And when we started considering a partner for bringing in some AI into our customer experience workflows, your company was at the top of my list because of a relationship that was built many years ago. And we hadn't done business with you before.

Michelle 09:22 We had communication over the years to outsource another function to to your company. And we did not end up partnering with anybody on that particular operation as we solved it in-house.

Michelle 09:37 But as the years went on and AI started coming closer to us, we looked at our processes and thought, okay, this is something that AI can really assist us with to improve the efficiency of the overall company, but drilling it down to this one customer experience department. So when we were looking at potential partners for AI.

Michelle 10:02 answering, calling, customer service, and chatbots, we had looked at a few, and what struck me the most personally was the communication and the relationship, the the data security and infrastructure of you know the overall company, the

Michelle 10:23 The security protocols, measures, and policies that you all have in place were aligned with what we have in place for our customers. And that was very, very important to us.

Michelle 10:34 But the communication has to be number one for me because we have to be able to work together well. We have to be able to reach out and say, hey, we want to, you know, update this or solve it, or we think something's wrong, or we want to do better here, or this is working well. How can we, you know, do something more? And the leaders, the leaders at your company have always been in communication with me over the years to check in, see how things were going, see what we could partner with.

Michelle 11:04 And there were a couple different things that we had explored. Ultimately we landed on the thing that was at top of our at that time, which was the AI calling and chatbot. So to make a long story short, communication is key and the demos from many other companies, a lot of them look the same, right? A lot of the technology can be manipulated or you know managed the same across the board. But for us, it's the people behind the technology that that sold us.

Ali Merchant 11:35 appreciate the kind words.

Heidy Villarreal 11:35 Give me one second, I'll jump right away.

Heidy Villarreal 11:50 I'm back. Sorry about that.

Ali Merchant 11:53 is no worries. So I think you know first of all appreciate the kind words I guess our COO the credit goes to Ash he's always obsessed literally for a lack of better word about you know

Heidy Villarreal 11:55 Okay.

Ali Merchant 12:07 communicating. we all have emails on our phones, right? I see at night, 11, 12, when he's messaging our partner success people that it has been an hour, two hours. Why haven't you replied back? At least tell them it's going to take some time. Tell them that I am going to come back to you tomorrow. And I would tell him that, you know.

Michelle 12:21 Yeah.

Ali Merchant 12:31 They might be asleep at that point of time. And he's like, no, but the response has to go. They need to know that we care and we don't take anything for granted. So,

Michelle 12:39 like that. He was he was you know the original contact that I had and he's been yeah he's been great along the way and he's he's I I like getting his emails at at midnight because you know then it's you know at the top of my list in the morning to to look at.

Ali Merchant 12:55 That makes sense. So, Heidi, how about from a technology standpoint? What do you see? Michelle touched upon security, which is obviously the biggest part when it comes to a domain like yours. How do you look at it from a technology standpoint? How do you evaluate your partner that, OK, their technology is sound, and basically they deliver real results for our customers?

Heidy Villarreal 13:19 Yeah. So for us is before we put the word trust on it, we need to understand the governance around the s the solution, right? So we have a very strong security team, like Michelle said, and we vet every A AI system through hands-on testing, proof of concept exercises, we do prompt injection, bias testing, and guardrail testing. Like we put all that together and that from there we want to understand how the system behaves when some

Heidy Villarreal 13:49 Someone tries to manipulate it and push it outside of the intended scope to see how it performs, right? We want to test stress test the solution to see if it meets our agreed-upon metrics. For example, with the chat bot, we were measuring answer accuracy, successful resolution, and how often there's a need for additional support after there's a chat interaction. And then we want to have a feedback loop that

Heidy Villarreal 14:19 goes back to what Michelle said about communication with the partner to share what we're seeing from the technology standpoint, from the cybersecurity standpoint, from the governance standpoint, review the findings with the vendor, and then use that feedback to make improvements.

Ali Merchant 14:37 And Michelle, one thing that I would like to learn from you is what gives you that confidence that we are ready for a wider rollout for taking it all out.

Michelle 14:49 Sure. So for example, we we started off with the AI calling because it was something that we wanted to tackle first because we get a lot of incoming phone calls. But a lot of that, a lot of those phone calls could easily be answered with a bot or with, you know, 30 seconds. Right? Like have you ever heard if you've ever heard the saying,

Michelle 15:17 that that meeting could have been an email.

Michelle 15:20 This is that's kind of how I relate to all of these phone calls coming in from customers, that it could have been answered, you know, with a quick response and it didn't have to leave a voicemail, create a ticket. It it didn't need so many steps to be resolved. So for the AI calling, we thought it was low-hanging fruit as far as solving a problem and quickly. Not so much a problem. It was just to make us more efficient. So now the AI calling was you know, a little

Heidy Villarreal 15:22 Hmm.

Michelle 15:50 bit less on the data security side, whereas you know somebody's using a phone and you know the system is searching information that we allow it to search for. So that was that was the the the door opener for us. After that was you know had deemed to be successful and it only creates tickets when it needs to. Otherwise it's you know resolving the inquiry and hanging up the call.

Michelle 16:19 But we still get all of the logs and the data and even the voice recordings to be able to listen and make those changes as needed. So secondly, we wanted to add something to resolve the email inquiries. So the phone, the phone was first, second was email, which

Michelle 16:38 We opted for a chat bot on our website. You know, I think I can speak for a lot of people when when I say we you go onto a you know an e-commerce site and you have a question. It's so nice and quick and easy to just have a quick chat and get get your answer, get your question answered. So we implemented a chat bot not long ago.

Michelle 16:58 To do the same thing, to resolve quick inquiries, avoid a phone call, avoid an email, and even avoid a support ticket unless it's truly needed. And the chatbot helps us determine whether a support ticket should be created.

Michelle 17:13 So the next step to that is what we how we want to open it up because this is proving to be successful. The phone calls were successful, the chatbots becoming successful. Now I would like to add in a third tier, which would be having a live agent from our team be able to step into those chats. So perhaps to avoid a ticket being created, if the bot cannot answer a question accurately or there's still

Michelle 17:43 you know, more more concern on that particular question, one of our team members can grab it live and chat through that that interface to get it resolved. Because my team is constantly working in our ticketing system and you know email on the side, this is this is a simple way to kind of just jump in, have it resolved quickly and not have to add anything to another email thread or ticket thread.

Ali Merchant 18:14 Makes sense.

Michelle 18:14 So maybe steps, but all kind of leading into the direction of having a really solid support system of artificial intelligence behind our customer service team.

Ali Merchant 18:26 Makes sense, makes sense. And I think, know, I do need to say this, AI is very helpful, but I think it needs great leaders within organizations like you guys to really take that output out. Meaning, you know, I was just reading and obviously with being in that domain, I saw that only 10 to 15 % people have adopted voice AI.

Ali Merchant 18:54 You are among the few who not only adopted voice AI, but kind of created a whole AI agent group to kind of tackle with multiple things and the workflow. I know being in this domain, doing it for so many people, it's not the easiest. Now that brings me to, first of all, it is very difficult to do what you have done. So I am changing a little.

Michelle 19:20 Mm-hmm.

Ali Merchant 19:22 If I'm a little in awe and it shows on my face, you know, that is very natural. But that being said, tougher part.

Ali Merchant 19:31 is there is always a lot of anxiety when new technology comes in, Especially when AI came in, not only was it an efficiency play, but it was also, it is also a customer experience play. I think in the last few years, one of the most significant technologies that have come in and through AI is where the customer experience has been...

Ali Merchant 20:01 There is impact on that in a positive sense. Now, I think any ways any new technology to be adopted, finance, operations, leaderships, colleagues, there is a lot of anxiety. It is very tough to, or at least a little challenging, would say the right word would be to kind of subside that anxiety and have everyone adopt it.

Ali Merchant 20:28 and try to work in the positive direction, especially when it is impacting the end customer, right? And it is even more challenging. How do you suggest other leaders, that how do they build that case with finance, operations, leadership from a customer experience standpoint, from a technology standpoint, how do they kind of work it through within the organization?

Heidy Villarreal 20:58 I can talk about from the technology standpoint the roadmap of Cleanfox, it whether it's AI or development, it's always informed by

Heidy Villarreal 21:08 the business needs and the client needs. So every release it's tied to a return, like to an ROI. Whether that's reducing manual work or improving turnaround or just making things easier on the on the client side, the buy-in comes from that business case and the value we expect to deliver. So but

Heidy Villarreal 21:32 Then getting the buy-in doesn't mean we're gonna go all the way through if things don't go as we plan it. So measuring every single step of the way, it's what helps us to say yay or nay to a new feature or a new integration. So sometimes during during user acceptance testing, we can say, okay, this is not going the way we want to go. So we bring it that back to engineering to change a few steps and optimize even at that main, like at UAT.

Heidy Villarreal 22:02 then if we're in a pilot and we we're not seeing the expected benefit, we go back again away. So that that that if we go with something that we haven't measured correctly, it doesn't save us any time. It actually puts more effort and cost into the wrong side of the of the picture.

Ali Merchant 22:24 Make sense. Over time. Sorry, Michel. Go ahead.

Michelle 22:24 Yeah, and to sorry, I was gonna just add to that. the Heidi mentioned something about pilots and we are we love doing pilots. We like to test things out, see if it's working, and as

Michelle 22:39 As a company that is a partner for our clients, we offer that same option to our clients. They can pilot our services for a certain amount of time before they decide to use us full-time. And from an from a customer experience, we did that with using AI, but from my perspective, it was piloting how the, not how, not how we received it, but how our customer received it.

Michelle 23:05 So it wasn't working. Are they getting, you know, are they getting frustrated every time they pick up the phone and they can tell that it they they might be able to tell it's not a human? And if that was the case, we would have had to backtrack a little bit and you know rethink that approach. But from a customer experience standpoint, just so just as Heidi was saying for technology, it comes from a use case. It comes from here's a need that's going to improve.

Michelle 23:35 how our customers feel about us and how we operate efficiently. And that's how we kind of go down the path to getting the the buy-in and then ultimately testing it out.

Ali Merchant 23:46 So what I'm hearing is any leader can they need to first spot a problem or a challenge then you know map outings

Ali Merchant 23:57 approximately what efficiencies or what you know let's say positive results like seasats it's going to deliver and then basically you know to do like a small pilot get buying you know from everyone and slowly and gradually expand rather than

Ali Merchant 24:18 saying that we're going to just automate this thing, which kind of puts everyone under pressure. So ease everyone into it. Like, may have very clear benchmarks and achieve them, and then you kind of grow them. That's such a sound advice and way to look at it. Earlier, you were mentioning in one of your answers that you keep human in the loop, right?

Ali Merchant 24:47 Principally, how do you decide, you know, that, okay, this is where the human will be or, know, this is what humans will manage. This is what AI agents will manage. How do you, what is that principle that kind of, you know, helps you determine that?

Heidy Villarreal 25:01 That's a really good question. We usually have a lot of workflow meetings with our teams when they bring when they request a feature or an improvement. We say, let's get together, let's meet, and let's understand the workflow end-to-end so that we can map it and understand where AI is a fit, where do we still need a human? Where does the AI assist the human? Where does the human review and where does AI can AI do something on their own? On the case of the chat.

Heidy Villarreal 25:31 But we have AI answering the status question. Like it's something a human could see, like our user could already see in the portal. But it's easier to chat, so there's a lot of benefit on having that AI. If the AI doesn't solve that problem, it opens a ticket automatically. So we have our customer experience team reviewing everything that the AI chatbot couldn't solve.

Heidy Villarreal 25:56 We have other use cases where we need human judgment along the way. So, you know, we're making a release decision. We cannot rely solely on AI expertise to say this is a dumb product. We need our humans to be part of that work, you know, to say this is, you know, I I reviewed this and it's it is my responsibility as a Clean Fox employee to be this, to have this completed product.

Ali Merchant 26:27 a good insight and how do you bring on board your frontline teams there is an anxiety right that you know we multiple sorts how do you kind of get them get thereby that this will ease their life make the experience for the customers better

Michelle 26:47 We I think getting buy-in from anyone is, you know, in our experience, it just comes down to trusting and being able to being able to adapt to change. Right? Here's here's the basics, here's what we know, and here's how we know we could at least attempt to solve it and do better. And giving

Michelle 27:16 anyone the option to trial and test it out is where where we have been able to thrive. It's like little little little bits versus going all in.

Heidy Villarreal 27:23 What make

Heidy Villarreal 27:28 I was gonna say one big thing with technology development in general is getting the confidence from the users. And we've been, you know, as we've we've been modernizing and changing technology at Clainfox, we've always bring subject matter experts to guide our development. Right. We know where we want the efficiencies as a business, and we know we want to prioritize our clients and our requesters, but also we need to understand what the people doing the work.

Heidy Villarreal 27:57 are doing to make sure adoption is there. You know, it's easier for the user to adopt a product when you have the data ready to back it up, but you also have involvement from the users from day one and they feel part of that development you're doing.

Ali Merchant 28:16 sense. All right, so I guess we are towards the end of it. So one question for both of you would be if you had to leave the audience with an advice from a customer experience standpoint and a technology standpoint on how they should pursue AI, like one big learning that you would have had since you've already deployed it, what would be that one learning that both of you would like to share with the audience?

Michelle 28:46 I would say don't be afraid to try it. And if you don't try it now, you're going to be left behind very quickly. So find the thing that you want to solve and see if any sort of artificial intelligence can help you solve that issue.

Ali Merchant 28:57 it.

Ali Merchant 29:07 sense.

Heidy Villarreal 29:09 On my side, be quick to pilot but have a clear scope on what we're trying to accomplish. we've seen, I think in the past year and a half, we've seen a lot of companies that have been pushing for AI but haven't had

Heidy Villarreal 29:23 a clear cost understanding on the use case that they're implementing, a clear data pipeline set up on the back end, or a clear outcome that they want to accomplish. And I think I was reading an article on about forty six percent of AI initiatives have dropped before they reach production. That's why doing small pilots and having a clear scope and having clear measures of what you want to accomplish and know when to call it quits. Don't just go for the sake of going.

Ali Merchant 29:54 That is some amazing advice actually. think the best advice are the simple ones. But to really understand them, it takes the kind of experience you guys have.

Ali Merchant 30:08 I always give this example that I had seen my father with my sister and I knew daughters are very dear to their father. But having a daughter and knowing are two very different things. So I think you kind of need to, as you mentioned, experience it, take it slow and experience it, learn from it, and obviously, so that you don't get left behind. That being said, I'm going to come. This is my last question. But being a girl dad,

Heidy Villarreal 30:18 Mm.

Ali Merchant 30:38 I always want to put out some positive messages, learnings for girls out there. Because as we have seen, in comparison, we do have less females as leaders. So I always feel that if there are more and more female leaders, that will give my girl a world where she has more strength. So one last question will be, as a female leader, what

Ali Merchant 31:08 One, again, advice or learning that you would like to share with other females who are kind of eyeing senior leadership roles like yourself.

Michelle 31:20 I would say something that is really important to me and advice that I would give is to lean on the people, lean on the people that had the same.

Michelle 31:30 interests as you the the same vision for success so Heidi and I working together in this company we have a shared vision of success so doing the research and partnering and backing each other up for for you know support is really important so it's I I think

Michelle 31:51 leaning on somebody, especially if it's a female, another female is a female, leaning on them having a shared vision of success and and going for going for what you want, going for what's important to the company and for yourselves.

Heidy Villarreal 32:07 For me, don't be afraid to ask questions. You know, a lot of times we're afraid to say we don't know and we don't know what we don't know. So be open, don't be afraid to ask questions and

Heidy Villarreal 32:19 Always keep learning. Like it's never there's also a tendency to say we finished college, we started at a workplace, and we don't have to learn anything else. There's always a learning path, there's always opportunities for growth, and they don't always come to you. You have to chase the learning and continue growing.

Ali Merchant 32:40 with some great advice, I think.

Ali Merchant 32:42 The best part is, know, what Michelle, you said, just go for it. Right. And then continuing to that, go for it. Learn. Ask questions. You know, don't feel shy. My daughter is like three and a half years old and, you know, right out of the gate. That is what I am trying to at least inculcate in her with, know, with my wife that, you know, she just goes out. She doesn't hesitate.

Michelle 32:46 Yeah, we gotta go for it.

Heidy Villarreal 32:47 That's great.

Michelle 33:09 yes. Especially at that age. You can't stop

Heidy Villarreal 33:12 Yes.

Ali Merchant 33:15 I, you know, in my entire life, I've never been worried or anxious. You know, people say that we have more than 2000 people and you know, we have multiple businesses. They're like, you know, do you ever feel, you know, worried or scared? I'm like, never. Only when I got, you know, I became a father to a girl is the first time when I was like, my God, what world is she born, you know, getting, coming into. So.

Michelle 33:39 Yep. That's for sure. That's for sure.