AI: From Pilot to Production | AI for the C Suite

Chad Harvey: Greetings, innovators and leaders. This is AI for the C-Suite, your compass for navigating the exponential age. I'm your host, Chad Harvey, and we're here to bring you cutting edge insights on AI tailored specifically for middle market organizations. Buckle up as we embark on a journey through the transformative world of artificial intelligence.

Today we're joined by Dave Trier, VP of product at ModelOp, who brings over two decades of transformative experience in data analytics, AI strategy, and model operations. Dave's journey is particularly fascinating. He helped pioneer the ModelOp's space during his time as a manager and consultant at Accenture, where he led large-scale enterprise transformation projects for Fortune 500 companies. Before joining ModelOp, Dave held leadership roles at Teradata as VP of advanced analytic services, managing over 350 delivery consultants and served as chief technology officer at Powered by Action.

With multiple patents to his name and current membership in the Forbes Technology Council, Dave has been at the forefront of helping enterprises navigate the complex world of AI implementation and governance. What makes today's conversation especially timely is ModelOp's brand new 2025 AI governance benchmark report which surveyed over 100 senior AI and data leaders across industries including financial services, healthcare and manufacturing. The findings reveal a stark disconnect between enterprise AI ambitions and production results with global AI spending expected to reach $631 billion by 2028. Yet most organizations are still struggling to move beyond pilot projects.

Dave has worked directly with industry giants like Fidelity Investments, Google, AWS, Bristol Myers Squibb, and P&G on AI governance initiatives. This gives him unique insights into what separates successful AI implementations from those that completely stall out. So, let's fly this plane without a stall and dive right in. Dave, welcome to AI for the C-Suite.

Dave Trier: Thank you so much, Chad. A pleasure to be here.

Chad Harvey: I am really excited uh to have you here today given the recent survey that your organization did and I want to dive right into this. So, little context for our listeners. This benchmarking report has shown that 56% of generative AI projects are taking I think it was six to 18 months to move from intake to production and that 80% of enterprises have at least 51 AI use cases in proposal phase but only a small handful in production. So, that was a lot of stats, but basically, we've got a disconnect.

And for leaders and executives that are out there listening today, what is causing this massive disconnect between AI ambition and execution?

Dave Trier: Yeah, Chad, it's a great question. And there's actually a number of different reasons that you can appreciate, but at the highest level, at the executive level, it's actually about just having the proper blueprint of how do I go from idea that seems like a great problem uh problem that we can solve with our business with AI to actually having that in into production usage. And the reason that's such a challenge is that in today's world, especially in generative AI, there are so many different technologies out there. There are different technologies for development for data for how you actually go and execute these in the cloud on prem combination etc.

So you have this variety of different technologies out there and you have a variety of different business scenarios and a variety of different teams that have to get involved. It's not just one team. You have a combination of IT and data and data science and you have some governance legal risk compliance etc. So you have variety of technologies, variety of different teams and all of them are going after hundreds of different use cases.

Without a blueprint, it's like flying a plane without having the proper navigation system. Right? So this is really what it comes down to, Chad, is just having that blueprint, just having that consistent and clear plan of how you're going to go from inception of an idea through the usage and hopefully get really good value out of it.

Chad Harvey: So it's interesting that you start there. um with your answer. One of the things that I find and this is irrespective of AI when I walk into organizations there's a typically a fairly large disconnect between what the strategy for the organization is and what people think it is. And what came through for me in your answer is we're really talking about strategy at the highest level and understanding that this is not an IT project that this is a cross- departmental collaborative initiative that requires strategy and project management on a whole variety of levels.

Do you think that's a fair characterization?

Dave Trier: I think it is. I think that there are there are some companies a lot of companies that have an AI strategy and so they have started at that top level but then it's actually how do you materialize that? How do you turn that into a real plan that you can operationalize across the whole organization. So yeah, I agree with you that it does start with strategy, but it's about turning that strategy, think of a PowerPoint presentation that they probably have.

How do you turn that into something that again you can orchestrate, you can operationalize, you can put into practice and most importantly that you can make sure that you're enforcing that the actual steps in the process are happening with the different teams with the right systems with the right uh personnel and stakeholders at the right time. That's really the really where that the rubber hits the road and the challenges is how do I turn that you know highle strategy that's you know potentially done at the board level into something that's enforceable, executable, operationalized.

Chad Harvey: So you touched on um the board level and I think that's a good add-on question here on this topic. How should leaders begin to realistically set expectations uh with their boards about this subject and about AI timelines and about these types of initiatives?

Dave Trier: Yeah, I think the first thing is that they should set expectations that it is first let's start here. It is a potentially transformational technology. Yes, I think everybody would would agree with that. But you have to set expectations of we need to do this in the right way for our business.

We need to be responsible about it. We need to be thoughtful about it. We're not just going to go out and take every generative AI technology that the company wants to do. That doesn't make sense.

you're going to waste money. You're going to open yourself up to undo risk and you're likely going to have a lot of failed projects. So, there's the right methodical way to do it is something that you should set expectations with the board. We're not just going to go and chase every shiny ball that's out there around generative AI or AI or agentic AI.

It's it's this is something that we're going to have an aligned strategy as you talked about, but here's the steps that we're going to use to go and execute those. And that actually starts all the way back at the I'll call it the intake process. There's going to there's hundreds of ideas throughout your entire enterprise that they may have about how I can use AI, right? But then you have to align it to all right well what are the ones that are going to have the highest impact but what are the ones that are actually achievable right for our you know we're not a digital native company right if you think about the enterprise of the world we're not digital native so we can't do the things that meta or Google might be able to do so you have to line it to what is realizable if you will and then you also have to line to all right well what's our risk appetite right because again most especially fortune 500s are fairly risk averse.

So you need to have that cross-section if you will of the what's the opportunity? What's the highest impact areas? What can we actually realize meaning can we put into practice and then how does that map with our risk appetite and those really are three kind of primary points that you need to have even early on in your intake process. So again if you think about that from a board level saying we're going to be methodical about how we do this.

We're going to make make sure we have a proper intake process and then we're going to have a proper way to make sure that we are piloting these, moving them to production where it's providing value and it's something that is achievable to us and then we'll report back on the results. But we're not going to promise the world to you yet, right? We want to prove this out. We want to do this in a systematic way and then we'll report back to you, you know, board members around how we're seeing the early benefits of it, what the cost and uh benefit trade-offs are in those different types of scenarios or use cases that we're working on and then start to scale it from there.

So, very much a phased, as I said, methodical approach to how you roll this out.

Chad Harvey: I appreciate you taking the time to walk us through there and starting at the at the board level. I know uh you probably weren't expecting a board question right out of the gate. Um but uh I think that as we look at this um and we look at the board level and then we kind of drop down to the leadership level and then we drop down even more into the organization. One of the things that your report highlights is that there's a uh a lot of fragmented systems.

I think it was something like 58% of enterprises are citing that as the biggest challenge that they've got to AI governance adoption. And I'm interested in your perspective because I know you're out there in the trenches all the time. uh could you maybe paint a picture for us of what this fragmentation looks like in a typical middle market company if there is such a thing uh and why it's such a roadblock?

Dave Trier: Sure. Absolutely. So as I talked about before in a in a past life you might be able to just work within a data science COE if you will and it was all-encompassing. They did everything right.

They did all the data, handed all the experimentation, the exploratory data analysis, the development of models, usage, etc. But now with generative AI, it's not just this the data science team that are trying to use AI. You have everybody in the company trying to do that. You have you have marketing, supply chain, back office, finance, everybody that is trying to do, you know, use AI.

So naturally what that means is that you have a variety of data sources that are across the enterprise across whether it's sales force or service now or your data warehouse your data lake your uh data database etc. So you have a variety of different data systems but then as I talked about before it's not just your one team it's not just data science this touches uh you know the data engineering it touches governance legal risk compliance um IT security so each one of those have their own system right so they have a security system you have a GRC you have a you know all the different data platforms you have execution so because of this again you have over in and kind of in practice Chad I see for a given model that you or AI system it touches about 12 systems on average. Just think about it from an idea through actual usage you're touching between 10 and 12 systems. So naturally with a large organization this is a big problem.

If you're trying to you know do one use case no big deal. But if you're trying to do 20 50 100 which is where the real value of AI comes in. How do you do that consistently across the entire organization across different business units lines of businesses teams departments etc. So that's where the fragmented systems really are causing bottlenecks.

They're introducing delays. they're introducing headaches of all right well I need to get data from this system then I need to push it to that system oh I need this various security review and approval so I need to integrate with this system so the fragmentation in today's world before something like a model is what causes that manual effort and therefore manual delays and worse off it's actually ad hoc you go and you say all right well I'm developing this use case great okay talk to Johnny over there hey Johnny says oh you've involved this data you got to talk to Jane in this department and then use this system to go and do a security scan as an example. Right? So if you think about it, all the different systems you need, all the different steps in the process, again coming back to that if you don't have a consistent blueprint that's connecting in the different systems at the right time, that's where those manual delays come into place, steps are missed and potential risks are introduced into the process.

Chad Harvey: Do you think that there is a minimum level of systems integration that's necessary before a company should start to pursue any AI initiatives or do you think that's actually part of the initiative uh and you build that as a preliminary phase in the project?

Dave Trier: Yeah, I think it's something that you can take a phased approach. So it's not something that you need to go and set up all the different integrations before you can get a initial project off the ground. So you can take it as you're building out your first couple of use cases. Think of it as uh what we call it going to minimum viable governance just like a MVP in the product world.

You can have the minimum viable governance of building just enough at the right time and then start to scale out from there. So no, you don't have to have all the integrations in place. You can plan those out, but you should probably should have a couple of them along with your first couple of use cases.

Chad Harvey: Okay, I appreciate that. And I and I want to return to that subject of governance here in just a moment. Um, as you talk about scaling use cases and figuring out those use cases as you're working on that integration, I'm wondering are there any quick wins that companies can achieve uh when they're still working on that integration piece in terms of AI's use cases?

Dave Trier: Yeah, in terms of AI use cases, um to demonstrate progress because um we know that this can take a while, right? And sometimes there's a lack of patience. So what kind of quick wins with use cases could be worked toward or on uh while the integration is ongoing? Yeah.

Yeah. Good good question. So there's actually what we've seen in practice is that there's kind of a dual a dual track around the quick wins. The first phase or wave of use cases one that they start with some actual vendor based um AI solutions that are kind of think of them as point solutions for a particular problem.

So they'll start evaluating okay can we make this work with a vendor so they don't have to spin up a whole bunch of development efforts data and integrations as you as you pointed out. So that's kind of a one work stream and then the other workstream is they take some of the lower risk AI use cases. Think about your you know your very typical uh customer support type use cases with a human in the loop. You could look at things like all right well can we write our own you know chatbot to answer common policy questions.

That's a very common one that we see that they get started with. So it get allows them to introduce them um into how we can create some generative AI solutions using rag architectures which are believe it or not it's kind of crazy that they're already mainstream right came out what 18 24 months ago right now everybody knows what it is oh that's old that's old news right so that's what we I've generally seen in practice is pick a couple of point vendor solutions that can show demonstrable value still lower risk and then in parallel work on a couple of internal facing or internal development I should say based genai use cases and more um tried and true which again it's crazy I'm saying that more tried and true type scenarios like the customer support policy uh chat bots things like that

Chad Harvey: got it thank you I appreciate that I did put a pin in the governance issue so we're going to return to that right now and I really what I really want to ask you is about the stats in uh the study that show that 36% of enterprises have budgeted more than I think it's like a million dollars for AI AI governance software. But before I have you answer that, I what I'm finding is that there is a wide gulf of inconsistency in terms of uh AI language and the language that's around AI and what these terms mean. So before I even ask you about that million-dollar figure, I would like to just hit pause for a second and ask you to define what is AI governance software and what is AI governance?

Dave Trier: Sure, that's a great question. So, um, first off, I'll just say that governance sometimes has is has a negative connotation within large enterprises, especially ones that are trying to move quickly. Um, and so that's the first myth that I want to dispel, right, is that uh governance is not a dirty word actually. It's just an enabler when done correctly.

it can be and absolutely be an enabler to helping you reduce that time to market as you mentioned at the beginning of this chat around six to six six to 18 months. oftentimes that delay is because there's not that consistent blueprint with the governance baked in if you will. And so that's something that again if you think about it when done properly governance can be an enabler if you have the proper blueprint using things like automation and making sure that there's a consistent way to orchestrate all the different steps in there. So first I just want to dis you know again dispel that myth if you will but over at the highest level the way that I think about AI governance is essentially thinking we just want to oversee that you were doing the right thing for the company for your employees for your customers and for consumers.

Are we doing the right thing? So it's just making sure that we're checking off, you know, that we're not, you know, we're following the different policies that we have in place from a again a enterprise perspective, that we're not introducing undue risk to the organization and that there isn't any sort of uh partialities for your customers and for consumers at large. And that can be everybody talks about ethical fairness, but it's not just that. It's it's looking at are we partial to uh different demographic segments?

Are we making sure that we are actually designing the system to be inclusive of all the different types of customers that we may have etc. So again at the highest level and at the simplest level it's just making sure that there's the right level of oversight to do the right thing for your company, your customers and consumers at large. So I'm going to pause there and let you ask any questions on that because we can go into a lot more details on it. when I talk at the at the board level, that's what we really are just making sure that we're doing.

Chad Harvey: That's good. And I appreciate you taking the time to walk us through that. I find that um when I do these uh interviews, when I have these conversations with subject matter experts, sometimes I get caught up in the conversation and I drive past terms that we understand uh the people on the call what we're talking about. But sometimes I'll get emails from listeners and they're like, "What was that acronym that you threw out there?"

I'm like, "Oh, geez, we should have defined it." So, I think thank you very much uh for defining this and talking a little bit about governance. I really did want to get to the meat of that question though because that uh million-dollar figure did jump out at me uh in terms of budgeting for um the uh the AI governance software and I think that there's a tension there u in terms of how leaders and CEOs that are hearing that figure should be thinking about the uh the return on investments of government's uh governance uh investment versus say the investment that they're going to make directly in AI development. And I'm I'm just wondering if you could speak to that and how should a CEO think about it?

I think you already started touching on that in terms of the policies and right driving that through. But what else uh would you add to that?

Dave Trier: Yeah, there's there's actually uh two sides of the coin on that. First is as I mentioned before with the proper governance software in place it becomes an enabler of getting AI solutions to market faster where any AI solution and it's not just AI it was ML any data science project that there's real opportunity cost involved for every minute that it's not being used by the business right at the end of the day you develop some AI solution to drive value whether it's topline or bottom line savings right and if it's taking 18 months months to get it out there. You know, that's 18 months of not getting any value out of it altogether. So, first and foremost on the first side of the coin is as an enabler, you can cut that time substantially by having a proper AI governance and operations software in place.

We worked with large uh customers be in the Fortune 500, the Fortune 50 actually, where we've cut that in more than half by just putting in the proper process, the proper governance and operations software been able to cut that in half. So imagine for any AI use case that you're doing, there's at least a seven figure return. Otherwise, it's not worth their time, right? So every day that you don't have out in there being used in production being used by the business, those are real dollars, right?

So if you're talking about cutting that time in half, there's an opportunity cost by not having a governance solution in place to actually bring those AI solutions to market quicker. That's just one example, right? If you got 50, you know, 100 different use cases that just starts to stack up in terms of the business opportunity cost. On the flip side of the coin is the doom and gloom that you hear, right, around, oh, what happens when AI goes wrong, which is very true.

And that's part of AI governance is just understanding the risk and making sure that you have the right mitigation in place. So on the other side of the coin is how much is a brand exposure event worth to you, right? What happens when you get your name in the paper that oh some AI chatbot started to talk negatively to a customer or when an AI chatbot if you will you know sold a Chevy for a dollar, right? So what is that worth to you?

What is that reputational risk, that brand exposure worth to you as a company? I don't like to talk about that side of it because again, you know, you want to you want to talk about the value of AI because everybody's excited about the value of AI, but there it is two sides of the coin about getting to market faster and making sure that you're keeping your names out of the paper, which is the risk side of the coin. So, if you think about it, a million dollars um for especially a Fortune 50 company, that's nothing in order to keep their names out of the paper. And if you're looking at 50 100 different use cases and just the opportunity cost around those, that cost is negligible.

Chad Harvey: You know, I was going to ask you uh about the level that you thought that uh investment in AI governance would become non-negotiable, but you've you've really kind of addressed that. And so instead, I'll I'll ask you this. Do you feel that AI governance is the AI equivalent of having a strong HR department with policies, practices, and procedures for humans? Because it almost seems to me given the examples you were just talking about what's that um that brand integrity worth to you if you had an incident that we cover that or we attempt to cover that uh from the human perspective with HR policies and manuals and really what you're doing is the uh the AI equivalent of that on the governance side.

Is that a fair comparison do you think or is that more nuanced than that?

Dave Trier: I there's a little bit more nuance in it because when we think about governance um the way we think about it at ModelOp is not just a traditional governance that you would think about for say GRC if you will but it we actually think about it combining the governance with the actual operations and process so that is a continuous cycle and so the analogy we typically use is the AI control tower. So if you think about, you know, in aviation, right, you have the control tower that's making sure that all of the different airlines that are coming and going from different parts of the world are seamlessly executing on schedule, on time with the right safety precautions, etc. So we think about AI governance and operations is that AI control tower to make sure that all the different AI solutions regardless of the geography these particular use case and that's kind of your equivalent of different airlines right what destination they're going to whether it's within finance HR legal uh with your customers etc making sure that everything is orchestrated correctly across all the different teams all the different scenarios all the different use cases inclusive of again security, data, risk, legal, compliance, etc. So for us, it's more about that control tower that is shepherding those AI solutions through their process with the right level of oversight and again making sure that they're going on schedule at the right time etc.

So I just wanted to kind of shift that analogy a little bit because you know oftentimes uh HR is fantastic right but they may not get involved in the day-to-day you know business uh uh processes if you will for us you know we think about it as that control tower we are involved in those day-to-day processes to make sure that it is moving swiftly efficiently and with the right level of oversight does that make sense

Chad Harvey: no that makes a lot of sense and I appreciate you uh parsing that u I had a feeling that you would have more nuance to it than uh the example that I gave What I didn't anticipate is that the uh the metaphors that I threw out at the beginning of our conversation about stalling out and taking off and airplane vibes were going to lead to a control tower and I can't wait to see uh what other kind of aviation metaphors you work in here. So, thanks Dave.

Dave Trier: Yeah, well played.

Chad Harvey: Oh my gosh. So, all right. So, uh, you're you're channeling Fortune 50, Fortune 500 companies and a lot of your answers here and, uh, I want to drill a little bit deeper into this evolution from manual to automated in terms of AI governance. What does that really look like, especially initially when you're working with an organization that wants to migrate from that uh, manual to more of an automated control tower, if you will, uh, type of AI governance structure?

Dave Trier: Yeah, that's a great question. And almost all of our customers before we come in, if you think about the setup that I provided, you have 100 different AI use cases across, you know, 200 different countries, eight lines of businesses, 50 departments and teams. They're using a spreadsheet to manage all that. And on average, Chad, when you for any AI or ML type solution, there's about 50 steps involved to go from idea to actually being used by the business.

So imagine that of using a spreadsheet and trying to coordinate that every single person that is working on the 100 plus use cases are doing everything in that spreadsheet that you're collecting all the information. You're managing the risk. You're collecting the evidence and the test results and the ongoing monitoring results. I mean, how do you do that with a spreadsheet?

It's just it blows my mind that anybody is even trying to do it with a spreadsheet. So, that's the kind of the typical situation is that it's extremely manual spreadsheets, emails, lots of ad hoc because it's just you don't have the blueprint as I started to say. You don't have the flight plan if we wanted to, you know, use that analogy. we don't have the appropriate way to ensure that we are doing the right thing with the right piece uh teams at the right time.

So that's when a you start to shift to automation, right? It's it's 2025, right? We don't need to use spreadsheets. We can use automations to be able to make sure that we are shepherding the AI solutions and the teams involved through that whole process.

So that's really where the power comes in is to go from something that's manual, ad hoc, run on spreadsheets to something that is consistent, i.e. the consistent blueprint that it's orchestrated, meaning that we're pulling in the right people at the right time, the right systems at the right time as well. And that ultimately we're automating as much of those steps as possible. Automating things that people don't want to do, like writing documentation, like running, you know, routine tests, like identifying risk and making sure there's follow-ups on risks, like reaching out to get the right approvals.

So that's really where automation starts to shine is that you have this living breathing system of record that is the automation is keeping up to date as these AI solutions are going through their different cycles you know from inception all the way through usage and retirement.

Chad Harvey: So the three main points I heard near the end of your um answer there were consistency and automation and also uh orchestration essentially and I play a lot in the middle market space and I am still you know to your point it's still 2025 I still walk into organizations and there's a our spreadsheets work fine mentality right and that's with finance uh or this that or the other thing and now we're talking about AI and on top of that we're talking about automating the governance. So, how do you get over that mindset of our spreadsheets work fine? Because I'm sure you run into that all the time. Is it those three points that you just outlined or is there a different type of conversation that has to happen?

Dave Trier: Yeah, the real catalyst is the pace of change, Chad. to be quite honest, it's, you know, like I said, in the past, excuse me, spreadsheets worked fine if you had one or two different technologies, but now you've got, again, a variety of different technologies, 20, 50, 100 different ones. And the other thing is the pace of it, right? So, the pace of change.

So, there's new releases, you know, almost every week for even even the big guys like OpenAI, right? They release all the time. And AI is not deterministic, right? It's probabilistic.

So every time that even you're using it, even if they don't make a change, there's a chance that it'll answer differently, right? So with that in mind, a spreadsheet just it can't keep pace with how fast there's changes. It can't keep up to date or can't m manage and maintain the probabilistic nature of AI. And there's also not they're not they don't provide a feedback mechanism.

Let's be honest with generative AI especially in agentic AI you have to look at what's happening on the output and how it's adjusting how it's you know is there a feedback mechanism to ensure that you know we couldn't test everything ahead of time. So we need to have that production feedback loop production monitoring if you will to make sure that it's not going off course in production. Right? So how do you do that with a spreadsheet?

Right? How do how do you do that with Microsoft Excel? No offense.

Chad Harvey: No, I'm glad you touched on that because um the uh last guest we had on the show, we talked a lot about that probabilistic versus deterministic. It's Andrew Aean from uh 9 uh uh 923 Studios. And I really liked your answer here because I think that's something that is still poorly misunderstood uh or poorly understood. How can it be poorly misunderstood?

I don't know. uh but poorly understood in the market is that issue of the fact that these systems are generative in nature and to constrain them to get reliable output um requires a different type of approach than simply opening up the chat window and typing in you know what's your favorite breakfast cereal or whatever it is we're working on this morning. So um thank you for that. Uh let's switch gears here a little bit and let's uh let's talk about use cases and let's talk about uh production and some of the bottlenecks that you're uh encountering out there.

Uh, I pulled a couple stats from the report that I saw and correct me if I uh I scribbled anything down incorrectly here, but it looked like 72% of enterprises have fewer than 20 AI use cases in production and 90% of the enterprises have 21 or more in development or uh in quality assurance. And to me, this suggests that there's a major bottleneck at the production stage. and maybe you've got some thoughts about what's breaking down here when companies are trying to deploy these models into a live business environment. Yeah.

Dave Trier: So, this is um there's a couple points as usual, but the main one I'll just say at the top level is establishing enough trust in that AI use case in order to unleash it to the business. And it's not just your as everybody talks about around the you know bias and fairness. It's actually trust across many different teams. The data teams, the security teams that it's not going to cause any sort of security breach, the legal teams making sure that it doesn't get them into any legal troubles inclusive of using uh protected data that might have been inherent in the solution.

Right? So it's actually a number of different teams but ultimately around how can you have enough information for them to trust that we can move forward to using this in production. That's where it all stems from and it makes it worse without a kind of a consistent AI governance and operations solution because you don't have a consistent way to present the information the evidence to these different teams to trust it. Right?

You have team one if you don't have a consistent blueprint as I talked about. You have team one that presents it in this form factor and then team two presents half the information in a completely different form. And so now as a legal and security team you're looking at this and it's like well I don't know I need to schedule a project now to go and investigate it. Right?

So then that delays it because you got to go wait and get in their schedule and then they need three weeks to or a month to go and review it in detail. Right? So again, this comes back to that consistent blueprint around making sure that legal and risk and compliance and data know exactly what information that they well they'll tell you what information they need and presenting that information in a consistent way at the right time and based on the right risk tiering if it's a high-risk model versus a low risk. By doing so, by establishing that, I keep saying blueprint, but it's so important.

By establishing that consistent language and blueprint, they will know that all right well I see that you know we've got uh 10 AI solutions in the queue. They're currently in dev. I'll probably get five of them in my desk and I can expect all this information as we agreed upon you know by uh you know July 10th. Fantastic.

I'll slot it in. I'll make sure I get it done. I'll make sure that I ask the questions etc. So, it all comes back to making sure that everybody has the right trust in that solution, which means you're going to have a consistent language, a consistent blueprint, and a consistent understanding about how we're going to manage the risk on a production level because there are risks as part of it.

So, let me pause there and see uh if you have questions on that one.

Chad Harvey: No, I think that makes a ton of sense. One of the things I was wondering was whether you would recommend structuring teams differently, but it sounds less about the structure of the individual teams and more in your words about the blueprint and that cross team collaboration.

Dave Trier: That's right. and making sure that there's a consistent language, a consistent again enterprise understanding of how we're going to evaluate these AI solutions, how we're going to identify the risk first off and then manage and mitigate the risk uh around it. So, yeah, it's just that yeah, consistent, you know, viewpoint of it.

Chad Harvey: you know, the consistency in language really speaks to me because as a uh as an executive coach, one of the things I end up counseling folks on an awful lot is issues with uh humans, right? And the predominant issue with humans is we don't talk uh the same language all the time. And now we've got AI on the scene where we're throwing new terms that are poorly defined quite frankly. Absolutely.

And so when you talk about consistency in languages, that's really what's coming up for me is we've already got um a species that is um you know, we think we're great communicators, but quite frankly we're atrocious at it. Uh and now you're layering on this new technology with all these new terms that don't have consistent definitions in many instances and we're asking people to trust it to your point. Um, so I'm wondering how often do you end up level setting expectations and trying to get people on the same page with that consistency of language and do you find that people are using the same word or the same phrase or concept in different ways uh during these projects?

Dave Trier: So on your last question, absolutely. As you rightfully pointed out, we're not great communicators, right? So I would say on the last point absolutely uh and around it that we do have to spend time helping to get just the common taxonomy right the common language and definition around what these different terms mean but here's the beauty of it we've been doing this for six years right well before generative AI came out well

Chad Harvey: wait there was a there was AI work before ChatGPT I know Dave Sorry. Go ahead.

Dave Trier: No, I know it's it's just crazy how fast pace this environment has gone, but um we've been doing this for a while. And so this is where we're able to come with here are some best practices. You guys need to put your spin on it for your company culture, but here are the best practices for how you define things like how you do a risk tiering or classification or assessment. Here's how you go and do a variety of different independent reviews or peer reviews or validations.

Again, you pick the term that you want, but these are the common ingredients that you need. You can slightly tweak the words if you want, but these are the common ingredients that you need as part of your blueprint and make sure that you have is as part of your overall plan to go and again, operationalize is the term I would use around this. So, we're able to come with all right, here's the menu that's probably going to get you like 80% of the way there. You guys put your spin on it in terms of the exact words that you want to use.

And of course legal comes in and wants to, you know, take a look at it and further put it into an official policy document with compliance. Fantastic. But in that way, you do need to have some common ingredients. Get started with those.

Get your legal and compliance to sign off on the terms around those and then use that as the consistent language that as part of the overall plan. You'll just start to leverage those as you put a variety of AI solutions through it.

Chad Harvey: Very good. Now, I appreciate uh the thoughtful answer on that. I know that um this is going to be a topic that continues to come up, especially as these tools advance and our expectations about what they can do uh continue to advance. And as we all know, CEOs love to chase shiny objects and we can't be left behind.

So, whatever the latest and the best is, we want it. Doesn't matter what it is, we want it.

Dave Trier: All right.

Chad Harvey: Yesterday, we won it yesterday.

Dave Trier: That's right. And for half the budget that uh we've allocated. Yes. Of course.

Chad Harvey: Uh, all right. Let's shift gears here again. Um, we're getting a lot of, uh, inquiries here and for our listeners, uh, I hear you loud and clear. We're going to be setting our fourth quarter 25 interview schedule and we're going to do a deeper dive into some individual industries out there like healthcare and finance.

Um, but I would be remiss if I didn't ask you Dave because we have had some interest on this topic. I know your survey spans multiple industries. I mentioned financial services. I think uh you guys go all the way to manufacturing and I'm wondering about governance challenges that companies in specific regulated industries like financial services again or healthcare face that others don't.

Is there anything there that you can speak to that came out of the survey and your work about more heavily regulated industries uh vis-à-vis AI? Yeah.

Dave Trier: So, first off that they have been a catalyst. Those regulated industries have been a catalyst to actually have some oversight. I won't use a governance term, but to have some oversight. So, that's actually been a good thing for the world to be quite honest, right, of not letting AI just run wild because it already has um if you will, but for companies at least to not let it run wild too far.

So I'll just start there that the financial services and healthcare industry it's great that they have put some level of rigor and oversight into it just to be the shiny beacon or this the northstar however whatever term you want to use around what's the proper way to do this and just I'll finish that and I'll come back to my second point what's interesting is that we have customers that specifically that are in non-regulated industries that came to us and said hey we're not regulated but we agree with the approach they're using. Can you use your best practices and what you learned in the financial industry and apply it to retail or CPG, again, a non-regulated type industry. So again, these companies want to do the right thing and so they are again using the types of processes and approaches that regulated industries are doing within their nonregulated industry, if you will. Uh so I just wanted to point that piece of it out.

Now some of the lessons learned around to your original question around the regulated industries it comes back to those ingredients right so with the especially financial services they've had model oversight in the US especially since the 2008 financial crisis right they've had a model risk management team by law by the OCC uh as well as in the wealth management space in FINRA right they've had by law had to do things such as having an inventory an enterprise inventory of all the different AI and ML models out there, just models in general. They've had to do risk tiering. Is this a high risk versus a low risk and the EU AI act picked that up, which is great. And for all those doing business in the EU of having unacceptable risk, high risk, low or minimal risk, etc.

So having that risk tiering, doing testing and upfront independent reviews or validations as some call it guys, that's just good best practice, but they put it into a law. So great, right? Why would you not want to test this thing before you put it out there? Uh, and then moving to things such as ongoing reviews, monitoring, and making sure that you did identify, you know, potential issues that could occur.

So, monitoring that they don't come to fruition. And then the fifth point is risk management. So, models, as we talked about, probabilistic, they're going to go wrong, right? So, do you understand what those risks are and do you have the mitigation plan in place in case those do go wrong?

So those are really the five key ingredients that I we often see um came out of the financial services industry now are being applied into other regulatory type uh bodies regulatory bodies that are putting out AI specific regulations and ultimately as I said before they're starting to trickle into other industries that just want to do the right thing andor need to do business in the EU so they have to adhere to the AI act.

Chad Harvey: Let's talk a little bit more and maybe a little bit more deeply about risk. Um, again, I keep pulling from this report and we're going to talk a little bit about the uh actual report and where people can get that at the end of our interview here. Uh, but the uh the 86% uh statistic really struck me when we were looking at that in terms of enterprise uh risk uh regarding inconsistent recording and duplicate work. uh but it basically goes back to the idea of AI assurance and I'm interested in what your take is on how that looks uh at an enterprise level and maybe does it look a little different uh in a middle market uh company type level.

So I'll just kind of give you a wide-open field there. I know I didn't tee up a specific question but I think this topic is important and again that statistic was striking to me.

Dave Trier: Yeah, isn't that crazy? 86% inconsistent reporting. Think about that. that AI has been identified as this transformational technology and 86% don't know how they can't report on it accurately.

That's crazy. you have I think there was a stat that I looked at this was a stat a year ago Chad around it was I think it was 70% of the of the S&P 500 had reported to the street that they were going to incorporate AI into their corporate strategy but then 86% can't report on it accurately so that problem is massive that alone is enough to for CEOs and executives to say all right enough's enough spreadsheets you know aren't going work anymore. We need to move something that is again providing the visibility and assurance that we have an understanding.

We have that control tower, right? We have an understanding that all the different AI solutions are being u overseen that we're doing the right thing. We're driving value to the business, right?

Chad Harvey: Okay. So, now let's get to the challenges of why that exists. It comes back to that setup I talked about where you have every single department, every single team that are trying to do AI and without that top level control tower, that enterprise level understanding taxonomy, process, consistency, etc. Each team's doing their own thing.

And so what's happening is that each team is creating a PowerPoint that says, "Here's what I'm doing." So team one is got this PowerPoint here. Team two's got a different PowerPoint and then you are actually having, you know, teams of teams that are putting together and compiling them all the way up to the board level. It's your traditional fun uh company, you know, uh bureaucracy, if you will, or almost corporate waste.

I won't go that far of just spending so much time just compiling different reports because they're inconsistent. They have different information and then you have to go and chase down, all right, well, what's the real information around it? But it all stems from the fact that there's not that consistent blueprint. How many times I said blueprint on this call?

But there isn't that consistent way to uh ensure that we understand all the different AI solutions. We know all the pieces within that AI solution. We know where they are in the process, who's approved it, when they approved it, all of that information. So it really stems from that inconsistency across the different teams.

So when I get to the other side of what we're talking about here and I have assurance, what does assurance look like, feel like, taste like, smell like if I'm in a leadership role? Because I'm probably not going to be poking my fingers at the C-Suite level unless I'm CTO or CIO perhaps into a lot of the different metrics out there. So as I'm as I'm sitting back and I'm saying, "Okay, we've got assurance now."

What does that feel like for me? What does that look like?

Dave Trier: Yep. I'm able to answer at my fingertips. How many AI use cases or solutions are running in my business? Seems like a simple thing, but it's not.

Now, how many AI solutions are running in my business? How many high-risk AI solutions are running in my business? How many open issues and risks do I have related to AI? And then probably most important, which of those AI solutions are driving the most value to my business?

Those are really the four things like even at the C-Suite like I'm dumping potentially millions of dollars into AI. Where's it being used? Where are the risks? And what's the value I'm getting out of them?

Chad Harvey: It's that simple.

Dave Trier: I love it.

Chad Harvey: Okay, then we're done. No. Uh so

Dave Trier: but done at your fingertips not you know requesting you know days upon days of organization and PowerPoint generation and all that at your fingertips anytime you want.

Chad Harvey: Your answer for me really brings up the larger question then to beyond technology, beyond process, uh beyond the specific tech at play, it really brings up a question for me about culture uh and that intersection between culture, organizational culture and the successful governance of AI. And I feel like in many ways there's still this mentality out there um because I just don't subscribe to it anymore. there was a brief window of time where I bought into this whole idea of move fast and break things. Um I just don't think that works.

Um especially at scale yet I still see a lot of organizations out there that have that somewhere in the back of their mind that in order to be innovative and AI is innovative, right? We've got to move fast. We got to break things. Yet that is completely at odds with what you're talking about here in terms of discipline governance, right?

There's I think a spot for play and for taking risks, but that's different than betting the entire farm on that. So, have you seen organizational cultures that are able to make this shift to successful AI governance and have you seen some that because culturally it's part of their DNA that they are not? Yeah.

Dave Trier: So, I'm going to take two different sides of that. Right. So there are there are organizations that are so risk averse that they put too much governance in place and they just slow things down overall because it's too much and especially with AI with the pace of change etc. It really doesn't allow it to get off the ground, right?

You never take flight. And then you have the other side.

Chad Harvey: Here we go again. We're flying.

Dave Trier: Exactly. We're flying. Okay. And then you have the other side which is basically leaves every team to their own devices.

Yeah. To do what they want, which means that you're trying everything under the sun. That leads to duplicative efforts. That leads to a lot of unknown risk and it leads to, of course, inconsistent reporting as we talked about.

So those are really the two opposite ends of the spectrum in between what we've seen work best in practice whether it's Fortune 500 or even mid-size company it doesn't matter is that you have this test fast and fail fast methodology meaning that you go and you allow them first you do have that upfront review does this make sense from a business perspective does it make sense for us to deliver on and does it make sense from a risk appetite as I talked about but once it's there and you say yep Sure, sounds good. Right? Then you have this quick pilot and that pilot is it's not just does the technology work is that does this work in our business as I mentioned and does this drive value for our business and does it make sure that we check off all the risks we know what they are and we've got a plan for all of those. So there's this point of making sure you're doing a quick pilot or PoC, whatever you want to do, and then you come back to the rest of the overall process where you get your, you know, your security, your data, your legal, your independent review, risk, compliance, etc.

Then you pull that in to go into for that productionization type lens. So we found that to work fairly well as this, you know, what do you want to call it? accelerator program or a pilot um um program, but something that's try it, fail fast, come back with us. Does it make us money?

Does it can we deliver on it? And does it drive value to the business? And then you go through the rest. So, I don't know if that answers your question, Chad, but that's just what we've seen in practice.

Chad Harvey: It does. Um it as everything does, it leads to another question. Um, I'm I'm wondering uh do you think that would be a uh a good framework some of the things you outlined for a 90-day action plan if u as it were if I was a let's just pretend I'm a CEO and I'm convinced that we need better AI governance but I don't know where to start. Uh is the core of a 90-day action plan kind of embedded in your prior answer?

Dave Trier: It is right. So the way that let's just say it's a new organization to AI, they've got some, you know, they're in that 72% that have fewer than 20 AI use cases. Great. So the way we get started is, like I said, with that minimum viable governance in those 90 days of action started, what we'll do is lay down the base software solution with all of our best practices, get you about 80% of the way there, and then you take a handful of those use cases that have been prioritized and put them through the process.

In that way, what you're doing is that you're accelerating the use case development and you're starting to build the governance capabilities out as you're as you're moving uh through the overall um you know process for those specific AI solutions. So that way you get a little bit of foundation, put some use cases in, build some more foundation, put more in there, and then eventually you get to the point where you get this scaled roll out, scaled deployment and really can operationalize at that large enterprise-wide uh facet.

Chad Harvey: Perfect. I think we've got time for one or two other questions here and topics um because uh time does go quickly uh as you and I know and we're coming up on an hour here very soon. So if I ask you to put on your futurep proofing strategy hat and understanding uh again I think I mentioned this in the introduction here that AI spending is projected to reach about $631 billion something like that by 2028. What's a billion or two between friends?

Um, I'm I'm interested in how middle market companies and maybe the Fortune 500 as well should position their AI governance strategies to remain competitive yet not overextend their resources and are there as a follow on to that are there any trends that execs should be thinking about right now? I know you touched on best practices and governance. So again, um, to really boil that down, two-parters, uh, future casting hat here, how can you position your company to remain competitive without overextending resources? And then number two, what are some trends that maybe you're seeing out there that execs could be thinking about?

Yeah.

Dave Trier: So on the fir first one of how do you not overextend yourself on that, again, it comes back to making sure you have an upfront intake process that identify here's the high priority ones. We don't need to go chase every shiny ball that's out there just because it might, you know, save one employee one hour every 6 months. Doesn't mean you should do that, right? It's cool, but that's not related to your business.

So, it comes back to that prioritization, but then also the visibility, that consistent reporting. So, even for middle market companies, it's important to understand what AI solutions are you even considering, right? Do I have an understanding of as I put it through that pilot process the quick one we talked about do I have an understanding of the projected value if we move forward and then can we consistently report on okay well what was the actual you know value that we're seeing through the business so in that way you're not going to overextend yourself because you're probably going to narrow it down to you know a manageable size of AI use cases that are going to have an impact to the business overall so I would say that's how you make sure you don't overextend it but even from governance software perspective for middle market. You still still need to have visibility.

You need to have a system of record inventory if you will of what you're working on. You still need to have an understanding of the risks that are associated with it. And you still want to know how they're operating, how they're performing, where do I have potential issues that I need to go and surface and ultimately get that consistent reporting. So even middle market, you're not going to overextend yourself by taking that prioritize approach and having the appropriate again governance capabilities just to support those pieces.

Chad Harvey: Good. I appreciate that. Um, and then any other trends or anything else on the horizon that you've seen that uh you think is worth calling out?

Dave Trier: Oh, yeah. You probably know this, but Agentic, right? Everybody's talking about Agentic AI. It's going to be, you know, the next internet, right?

Um so that's just one where you know again I would make sure that to advise not caution but advise to executives especially that again with any technology agentic has a lot of potential power but you have to use it in a way that makes sense for your business to allow especially core parts of your business to just whole hog use agentic AI to do everything probably doesn't make business sense but when you can combine it with a couple of different more uh controlled type approaches. For example, you know, you may want to have Agentic start the conversation. It may go and allow you to, you know, read different records with agent tools so you can read it, but you may not want it to go and update specific, you know, systems of records. Like you may not want it to update your ticketing system within, uh, you know, your flight booking, if you will.

So, start sensibly, start smart, right? But ultimately, agentic AI. Yes, Chad. That's that's the next big trend.

Yeah.

Chad Harvey: And how um how should folks uh let me ask a different question. I was going to go down one path, but again, time is short. The um the governance frameworks that you're looking at, how uh much additional uh content or structure, if you will, do you think need to be layered onto them to handle that type of agentic AI? Understanding we don't fully know what this is going to all look like quite yet.

Yeah.

Dave Trier: So actually funny enough we've already incorporated agentic um governance into our product. Yeah. And that's that's just the beauty of our product is that we set out from the very beginning to be able to support all different types of a IML and whatever related technology comes out now and in the future. So when GenAI came out took us about a month to add it in.

When agentic started to come out took us you know about a month to add MCP and A2A right? So it's just for us it's really straightforward overall. Um but you know it's it comes down to as you talked about the taxonomy the language how you talk about it and then how it fits in organizationally and into their ecosystem that those are where the challenges come in for a given customer. But for us to be able to do things like inventory and risk classify and moving into doing testing and monitoring those are things that are just it's very easy to incorporate new new solutions as those come out.

Uh, overall

Chad Harvey: perfect. All right, one-stop solution. I love it. All right, Dave, this was fantastic.

I really enjoyed our conversation today, and I did promise that we would uh at least give a uh a little bit of information about where folks can get this report that we keep uh talking about. So, where can our listeners access the full 2025 AI governance benchmark report? Absolutely. So, please go to modelop.com modelop.com and on those in the homepage is a link to go and to download the 2025 AI governance report free of charge.

Um, but please go out there. Some really great insights as we've been talking about on this podcast overall, but uh over a 100 different executives provided insight into what they're doing in their company in real life every single day. So, very uh well worth your time. Very good.

And uh if I am a busy executive, what is the one part of the report that I can't miss? What is the one part I have got to read in your opinion, Dave?

Dave Trier: It's really just the homepage, right? Some of those key findings that we've talked about, it's on page five, believe it or not. But those key findings that we just talked about around, you know, how many use cases are out there, the 86% inconsistent reporting, if you only have five minutes or even two minutes, that key findings page will be really eye opening for you.

Chad Harvey: Perfect. I appreciate it and I appreciate uh your organization, you taking the time to come on our show today and talk about this, but more importantly actually putting that research together because things are moving so fast it's difficult to get very reliable validated information about the current state of affairs uh as it pertains to AI. So I found tremendous amount of value in this and I really appreciate uh the fact that you guys took the time to put this report together. So thank you again.

uh you did very thoughtfully tell us where we can get the report. If someone wants to get in touch with you, how are they going to do that?

Dave Trier: Oh yeah, you know, plea please feel free to reach out. I drop my email or find me on LinkedIn. Um either way, I'd love to talk about this all the days long. So u please any questions you have, reach out to me.

Chad Harvey: All right, very good. Dave Trier, everybody. Uh I really appreciate uh the opportunity to have this space. I appreciate Dave.

I appreciate all of our listeners. And speaking of our listeners, it's that time again. We've got to say goodbye. Thank you for tuning in once more to AI for the C-Suite where we're committed to helping middle market leaders thrive during the exponential age.

If you found value in today's discussion, please be sure to subscribe to our podcast. Follow us on all the socials and visit our site. Join us next time as we continue to unlock the potential of AI for your organization. And until then, keep your algorithms running, your leadership evolving, and your AI in check.

Take care, everybody.