AI Governance in Financial Services with Dave Trier

Alex: Welcome back to Converge. It is the end of July, early August. I want to welcome Dave Trier today. He's the vice president at ModelOp.

Uh we'll be discussing AI governance, which is a topic that I think many organizations in financial services and beyond are interested in um in improving and building a mindset around and developing a point of view around. So, I think we're going to learn a lot here with Dave today. Dave, thanks for joining us. Um, I guess to start, give us a little intro on who you are and what ModelOp does and some of your background and then if you don't mind explaining a little bit about what AI governance really means, defining it for us, that would be a great place to start off.

Dave Trier: Awesome. Thanks so much and thanks for having me today. Dave Trier, VP of product here at ModelOp. spent the last 20 plus years with new technology past 10 plus years in the model and AI governance space mainly helping large enterprises to adopt and leverage AI and ML safely and effectively for their organization and that's what we do here at ModelOp that's the company I work for a software company helping these large organizations to take advantage of AI at scale responsibly and efficiently with the appropriate level of oversight now Alex you mentioned so what Does AI governance mean today?

I'm glad you asked that because a lot of people think, especially in the corporate world, that governance is necessarily a dirty word, meaning it's going to slow things down. It's going to grind things from a halt as it comes to new innovation. But really, what AI governance is and can be is an enabler, an enabler of AI. Because what it's trying to do, what it's out set out to do is not only just help to protect the organization, its customers, its employees, and any consumers, but it's also meant to provide a blueprint, a blueprint for how we can use AI effectively within the organization so that there's not any ambiguity, there's no lack of clarity around what we need to do.

And it by having that in place, it actually becomes an enabler. be because before it's often very hazy as to well I'm not sure what I need to do I know not sure who I need to talk to what reviews need to be done what audits need to be in place and that becomes something that is actually very slow helps sorry slows down the process just where you have a proper AI governance in place with a proper blueprint you know exactly what needs to be done so in that way it turns from something that's slow and manual and ad hoc to something that is known. It's we have all the paths and all the people involved at the right time and it becomes an enabler for innovation.

Alex: Yeah, know that's that's a great way of putting it. Um I guess to help our audience focus on their specific area of interest, why would AI governance be so mission critical um and such a challenge for enterprise financial services or cross-border payments companies to adopt and approach in this specific sector? I know you've worked with um financial services companies. What's your experience of the challenges that they face with enabling AI governance and approaching this um strategic mindset?

Dave Trier: Yeah, great question. So, first off, it's mission critical because AI must be used in order to keep competitive in the financial space just period. Right? If you're not using AI, you're going to start to fall behind from your peers and others that are in the market.

So AI itself is mission critical. Now why AI governance is mission critical is because AI is very much a probabilistic. It is not a deterministic-based approach. You're not sure exactly how it's going to provide outcomes if you will rather it's something that is very tied in with the data.

It's tied into the models that you're using behind the scenes and therefore it introduces risk. That's the re the main reason why AI governance is mission critical. It introduces risks. There's financial risks if AI goes wrong.

There's potentially regulatory risks. It's something is audited, if you will. But there's also brand and reputational risks. If it gets out that you had some AI chatbot that started to give bad customer or consumer advice, then your brand starts to take a hit.

So that's why AI governance is mission critical is to account for and address and make sure you have the right plan in place to mitigate those risks.

Alex: And I know you guys uh at ModelOp recently published an AI governance benchmark report this year. Um what are the main reasons that you cover uncovered in the report that enterprises I guess if you can also focus on financial services are struggling to move AI initiatives from the ideation stage or the conceptual phase to at the actual production and execution.

Dave Trier: Yeah thanks thanks for bringing that up. It was it's very illuminating what was in that report but not only in the report just in our experience at model of what we've seen with these large enterprises there's a couple of key challenges that they face that just slow down these AI initiatives moving from idea and into production. Uh the first is as I mentioned before before somebody like a ModelOp it's very manual it's very ad hoc they may have a documented policy a documented AI governance policy but there's no way to enforce that there's no way to have all the appropriate uh pieces in place to go and do that effectively. So it's very slow.

There's lots of back and forth across different teams. Oh well you forgot to do this particular review. Oh you forgot to fill out this document. And so it's just very manual, ad hoc and slow.

And then it gets even worse because now in the in the past there was really only kind of one type of technology. But with AI there are tens 50 hundreds of different technologies. So not only is the process manual and complex and now you've introduced technologies a variety of different technology that itself is complex. So you got this you know compounding problem of complex technical if you will as well as an undocumented manual process.

So it at the end of the day it just slows down moving from I've got this great technology to well I want to use it because I've got all these barriers in place technical barriers process barriers organizational barriers that just you don't have the right approach to make sure that you can overcome those very quickly.

Alex: So that makes total sense and I guess that explains why your report highlighted that 80% of enterprises have at least 51 generative AI use cases that are in that proposal phase but only a few are actually in production is I mean that gap is persistent especially in regulated sectors like payments and financial services. Do you see that as like a particular pain point um in these highly red taped regulated industries?

Dave Trier: Yes 100%. And that's because with these regulated industries, they're bound by law to go and do things like model reviews or validations as they're called. They need to do risk scoring to understand what's the risk of these different models. So yes, these regulated industries again by law must do this even within the US where there's not a federal-based law so to speak at this time.

There are with the OCC, with the SEC, the Fed in general, there are regulatory uh concerns that they that each financial uh institution needs to comply with. And so yes, that not only just traditional models, it's especially with AI models because of their again probabilistic nature.

Alex: And I and I know that um Alana can speak to this because we both are very frustrated with sort of the ad hoc slow manual nature of a lot of adoption of this and so take it away Alana.

Alana: Yeah. Well so okay in the report when I read it says it takes six to 18 months to move a GenAI project in from intake right to production. What are the biggest contributors that you find right being in product to this delay in fintech and payments?

Dave Trier: Yep. So there's a couple big delays. First is as I said at the outset, there's just not a blueprint in place. It's very manual.

It's it's potentially documented, a lot of times not document, but there's not a blueprint to say here's all the steps. Here's all the different players involved, different stakeholders across the different teams, and here's all the systems that need to be involved in the process. You have your data systems, your IT, your security, you have your validation and risk systems. So there isn't a blueprint that helps to define exactly what needs to be done.

The second biggest delay is because there's not known what that blueprint or playbook is that you get these teams that are either buying or building AI and they get to a point where they say, "All right, I'm ready to go. I want to hit the play button. I want to put it in production." And somebody comes out and says, "Whoa, whoa, whoa, whoa.

Slow down. we haven't done this security review and then you have to go through and figure out who to talk to. What do I need to do? What tests do I need to run?

Oh, you didn't do your documentation. And for those that are in the AI world, you know how fun documentation is. That's another pain that just takes a while. So, you get what you think to the finish line and then you have to go back to the start line because you didn't do the proper reviews, the proper documentation, the proper security scans, etc.

So what we find is that in especially financial services they have to do those things and so again you get to the point where you want to use it but then you have to go back to the starting line and go through all the mandatory steps in the process.

Alana: Right. Right. And yeah because it's not like straight compliance right where there's controls and then people have already ticked off all these controls and then they're operating within these controls. Now this is kind of like a net new world.

And so with those existing systems, right, that are really fragmented andor reliance on manual processes, say spreadsheets, emails, um things like that. Um they're traditionally slowing down that AI governance, right, for organizations that are operating globally, um even global payment providers. Can you kind of give us a little bit more detail around that? You know, I know you kind of mentioned some of these fragmented systems, but what does it look like in practice, right, when you come in, right?

How does the product or the process work to solve for those fragments?

Dave Trier: Yes. No, absolutely. So, as you mentioned before, a lot of times this is emails and spreadsheets. And so, if you think about it on average for especially a large financial institution, there's around 50 to 100 steps that they have to go through from idea to actually being used.

It's kind of crazy, right? But it's it's a it's basically around 50 to 100 different steps, right? And so if you think about trying to manage that with emails and spreadsheets, that's a nightmare, right? And it's even worse with AI.

Why this is a really big problem in AI is that AI is constantly changing. It's not just I move from A to B and I'm done. Now you move to A to B and then you're back to A and constantly iterating. You update some models, you update how you're using it, etc.

So think about it's painful enough to go through it one time with spreadsheet and emails, but imagine having to do updates every single week and you're going through that manual process every single week. So that's why AI governance in today's modern world cannot survive on spreadsheets and emails anymore because it's constantly changing. But also you're constantly have new technologies that come out and that's where the again you just don't know. All right, well we got this new technology.

Well, we've never seen that before. Go talk to Johnny. Johnny says, "Oh, we got to get a security view. Do you go talk to talk to Jane over there in compliance because we don't know what to do, right?"

So, you've got that compounding problem of the new technologies with the unknown process as well as the very traditional spreadsheet based approach that makes everything manual with the this is continuously updating and we need to continuously go through the process, right? So, it's just a it's a compounding problem.

Alana: Yeah. It's like scale is just almost impossible. And then you know kind of the old protocols, the old processes, right? The old ways don't open new doors if you will.

Dave Trier: That's exactly right.

Alana: Oh, that's interesting. Do you have any real world examples that you could share with us? You know, maybe a lot say an 80% use case that people are trying to do and some of the obstacles that they are facing today that you help solve.

Dave Trier: Yeah, absolutely. I It's funny. I just got off with a global financial institution and one of the examples they gave where it stalled, right, was they had this AI system and they thought they were doing everything right. They went and said, "Okay, I filled out my documentation.

I got the uh, you know, the risk team involved. They did their risk review. So, I'm on schedule. I can get it out next week.

Fantastic." Well, what they didn't realize was some of the data that they used. Yes, they looked at the data. It was uh confidential data, but they did the proper data security thing.

So they thought they were good, but what they didn't realize is that inherently in that data, it got down to some very specific indicators of personas, meaning that you could deidentify the data. And because you could deidentify the data, they had to do a very thorough data compliance review. So something that went from well I'm going to get it out in production next week went to stalled for three months because they had to do a very again data specific compliance review to make sure that ident that data would be deidentified to the point you could pick out okay this is Alana's data right so that's just an example where again you thought you had everything right you thought your manual process was okay but you got to the point where nope indeed you forgot to do a particular set of steps because of this data sensitivity nature if you will.

Alana: Interesting. So kind of like portions of the data say need to be anonymized before it's a good data set that you could you know whatever fabricate right interesting. Yeah that is that is interesting.

Alex: Um I was just to pivot here for a minute I was really shocked by the fact that only 14% of enterprises actually enforce what's called AI assurance. And can you confirm for me that what we mean by AI assurance is validating factual information is correct?

Dave Trier: Yeah. So AI assurance is essentially making sure you're doing the appropriate testing, the appropriate oversight and appropriate monitoring of how your AI is actually working, right? So this can be range from anything from making sure it's actually performing, making sure it's robust, making sure that it's stable for your users, especially if it's consumers and customers that are out there.

Alex: So only 14% of enterprises are enforcing this at the enterprise level.

Alana: I was going to say, wait, what? Yeah. What is this? I mean, what does that mean for that means everybody like 85% are hallucinating?

Yeah. What's the ramification there?

Dave Trier: Yeah. So let me give you an example of this. So pretty commonly these AI systems that are out there you have you have a system owner right somebody that is developing and owning the AI system right and so financials because they're regulated of course they have to go and look at it but most commonly what happens is they look at it on the required basis like once a month once a quarter so they will do that right of course they'll do that but they're not looking at it in between and as you can imagine with AI it's constantly changing it also can give you the different answer to the same question that you ask. Right?

So in the world of AI, that process of checking in once a month, it doesn't work anymore. Right? So that's what we mean by only 14% are enforcing AI at the enterprise level is that they are just leaving it to the system owners and model owners to do the right thing as opposed to just having that enterprise level consistent way to do assurance consistent way to report on how these are actually performing and within the bounds of thresholds that you would set.

Alex: Got it. Okay. Okay. Improvements to be done for sure.

Yeah. No doubt about that. Yeah. So with other regulations like the EU AI act and GDPR varying requirements in global markets, how are leading financial services organizations adapting their governance frameworks this year right now in new ways that you'd like to share?

Dave Trier: Yeah. So I think most large financials that especially those that are doing business within the EU, there were specific compliance requirements with the EU AI act that they had to identify which of those different AI systems are high risk systems. That's the first requirement by law to do business within the EU is that you had to identify which ones were high-risk systems and also unacceptable use. there were certain specific AI systems that are just uh not acceptable meaning that you cannot use them.

So that was really the first requirement again legally that they had to comply with and they had to identify which ones were high risk that were fit into a specific criteria. They had to catalog them into an inventory if you will and if they were ever asked that of course they have to report back. Yep. Here's the inventory of the specific systems.

Now down the road that part of the EU AI act but just all the regulations in general there are some common themes. There's that risk tiering like I mentioned is it a high risk or say low risk. There is the second thing which is that you do some independent testing or validations as most financials call it just to have somebody go and test and make sure this thing is not going to be you know ethically biased in any way. It's going to perform.

It's going to be stable and robust. there is the what I would consider the most important part a proper risk management approach and capability to make sure that you're identifying any of the risks around using this AI system and making sure that you have the appropriate mitigating actions in place. So those are just a couple of the key components. There's obviously many others but those are very commonly the some of the key ingredients for regulations like the EU AI act.

There's similar ones within the states within Canada etc.

Alex: And when you're thinking about um cross-border payment uh institutions or businesses doing business uh across borders internationally, what are the challenges that are unique to those businesses compared to you know domestic focused corporations? Obviously we live in a global world where more and more companies are going global but what are what are some of the challenges most unique in cross-border?

Dave Trier: Yeah. So I think what was interesting about the EU AI act and several others have followed this is that it was stated that if you do business in the EU you must comply right so for cross-border payments as you just mentioned if you're doing any sort of cross-border into the EU doesn't matter if you're not headquartered there you have to comply right so that's something that uh was really why the EU AI Act act was monumental and that it required you to do that sort of compliance even if you're not headquartered or even do a lot business in a certain you know country within the EU.

Alana: Understood. So kind of when you're innovating right say you are a fintech and you know you're based in the US and you launch kind of a core product but you know it's focused on smaller businesses and you want to go more upstream and maybe there is right uh like a cross-border use case and now you're doing partnerships and there's different ecosystems and integrations right which is kind of the world I live in um on that report I noticed that 36% of enterprises have budgeted more than 1 million a year um for AI governance software but how are fintechs and cross-border payment providers balancing governance investments right with the pressure to innovate quickly right it's kind of like there is a second player advantage but it's almost you know you want to get to market faster right so you know I mean how are fintech and cross-border payment providers is balancing that you know kind of need to hypers scale need to innovate with um the governance investments that you Yeah.

Dave Trier: Yeah. And those are really are as traditionally opposing forces, right? You have governance which is trying to not it's not trying to slow things down but in the past it did slow things down with innovation where you're trying to move quick quickly. Right.

Alana: So those are typically in the past opposing forces but why you see them disrupt right if you know there's a disruptor use case here. Right. Right.

Dave Trier: And but that's where you see that in 2025 especially that these enterprises have budgeted substantial amount for AI governance software because they can no longer have those traditional methods that slow things down. They can't do that anymore because they need to innovate quickly. So this goes back to what we talked about earlier where that manual approach which slows things down to a halt taking 9 12 18 months to get AI out the door that's not possible anymore because they need the competitive differentiation. So they are spending money on this AI governance software to put a proper approach in place to get out of spreadsheets and email into something that is automated, consistent and scalable.

By having that again automated, consistent and scalable approach, it allows them to open up the floodgates for AI to the entire enterprise to say go innovate quickly, stay within these bounds, stay within these guard rails, if you will, and you know, again, the software will help you to do that, but go innovate. As long as you're moving through that process efficiently and quickly with the software, then you can go and again use AI for what you want to do to really drive that competitive advantage within your business area.

Alana: So that being said, what do you think the minimum viable governance looks like? Right. Can you describe that?

Dave Trier: Right. Yeah. So this is a term that we've coined and ModelOp overall. It's minimum viable governance.

So think about uh in the world of product and minimum viable product just having enough product out there, right? But in minimal viable governance, you think about it as kind of the goldilocks of governance. Just enough governance, not too much, but just enough in order to protect the organization. customers and consumers.

So to get started with minimum viable governance, it's about establishing first and foremost an understanding visibility, if you will, into the different AI systems. You can't govern what you don't see, right? So it's about establishing here are all the different AI systems that are out there across the different business units. Here's an understanding of the risks here.

As I mentioned before, is it high risk versus low risk? And then here is what the intended usage is. So even just starting there is a great way to start to establish this AI governance capability at scale. And then you move into introducing some of the automation to apply different controls to do your risk mitigation.

And then finally move into the third area which is around proper ongoing enterprise AI assurance. Make sure you're constantly continuously testing monitoring tying those back to risks etc.

Alana: Well, I'll give you a phrase. I'll coin you a new phrase then. That in between middle of the road that you described, that one can be, you know, because there's MVP and MLP, right? So, you can do minimal lovable governance, too.

Dave Trier: I love it. I like it. That's great.

Alex: So, as we look to the future, obviously huge projections for AI spending, $631 billion by 2028. Where do you see the greatest opportunities in those years ahead for financial services firms in differentiating themselves through governance? What are the real wins that they could they could leash in the future?

Dave Trier: Yeah, I think you'll start to see this publicly. And what I mean by that is those fintechs and large financial enterprises that are starting to report on where they're using and getting value for their company, value for their customers with AI, those are the ones that are differentiating themselves with governance. And the reason I say that is that there's no way that they could get those out to market and publicly talk about them without the proper AI governance capability in place. So you're going to see those the winners in the space are the ones that are constantly evolving.

They're constantly providing competitive differentiation through AI products or AI based services and those are the ones that have taken this message that we're talking about here to heart and put in the proper AI governance capability that's scalable that actually helps to be that enabler and automate some of those processes so that they're rapidly getting new AI products and services out the door. So I think you're going to see that more publicly, Alex, and you can tell you take it from Dave that they put something behind the scenes to make sure that happens from a governance perspective.

Alex: Yeah, that makes total sense. Uh what about like you know the automation and the life cycle management of AI models which are constantly uh being reborn. How will automation and life cycle management of these models change that risk profile for financial services and payments firms over the next three years in your view?

Dave Trier: Yeah. So, this is the only way to get to scale. You have to use automation, right? There's just too many new technologies.

The technologies are evolving all the time. You can't just keep throwing humans at reviews and the like overall. So, you have to use automation. Automation of the process, automation of the life cycle or life cycle management as we call it to make sure that anytime anybody within the firm or the organization has an idea that you're automating the process.

You're telling them what you need to do. You're taking care of tasks when that you can take care of it on their behalf, pulling in a human when you need a human. But automation is the key to go from again idea to actual usage. If you have that automation, if you have that life cycle management, it does lower the risk of the use of AI because you have that consistent process.

It's auditable. You can retrace its steps and you can as part of it identify the risks related to it and make sure that you have mitigating actions there uh thereof. And by doing so, again, you're able to open up AI, the floodgates of AI to a large organization, knowing that you have this life cycle management that is protecting the organization, cataloging the risk, making sure that you know how you can actually address those risks should they come to fruition.

Alex: Yeah. Just to close out, Dave, this has been great, but I want to just sort of get the takeaways that uh businesses can action upon and leaders can think about as they um as they contemplate their future with AI governance. So, what is your I guess top piece of advice or several pieces of advice for financial services leaders who are looking to set up their very first AI governance framework and what's the single most important step they could take today to get started?

Dave Trier: Yeah. So the first thing I would say is as we talked about AI governance, you should be thought of as an enabler, enabler of innovation to get your products out the door faster. Second is as we talked about with minimum viable governance. This is not a five-year effort.

You get started with just what you need at the right time from a governance perspective and you iterate from there. So I would the advice I'd give to these financial services leaders is you don't need to think about this and continue to delay and delay and delay to the point where you are just using again the old methods of manual spreadsheets and emails and people right put in the base foundation in place that minimum viable governance foundation in place to start and then you can iterate from there over time and that way you get all the benefits you start to build the benefits of having that AI governance in place but then you start to get the benefits of the automation, the scalability, the process improvements by just putting the foundation in place.

Alex: Now, well said. Let's give ModelOp the center stage for a moment. tell us uh what's coming up next for ModelOp that you're excited about that we should all be watching for and then close out if you don't mind just tell us where we can find that AI governance benchmark report that we cited throughout the conversation and how we can um get in touch with you or how organizations can get in touch with ModelOp for further guidance

Dave Trier: thank you so much Alex so what's coming up next for ModelOp everything we've been focusing on is agentic AI right so LLMs were all the rage with generative AI as part of that in the various chat bots etc. But Agentic AI is going to be transformational for many organizations where it starts to actually act. So all our focus over the past six months has been being prepared to help organizations leverage Agentic AI safely and also in a financially and fiscally responsible way. Meaning that you can go overboard with spending on LLMs and Agentic AI.

So not only are we helping to protect the organization from regulatory, financial and other risks, but also helping them to optimize the cost around it. So again, our focus is around Agentic AI with a with a heavy focus as well on the cost management and optimization, if you will.

Alex: Awesome. Well, we're excited to see where this goes and excited to follow your progress as a company. Thanks for joining us. I'm sure our listeners will have a lot to take away today and thanks Alana for being my cohort here today.

Dave, great to meet you again and hope to run into you at one of the industry show floor conferences or something like that.

Dave Trier: Great. Thank you so much, Alex and Alana. It's a pleasure talking.

Alex: Likewise. All the best.