AI Acceleration & Scale Through Governance

Nick Oliveri: Welcome folks to the Entry Point podcast by Cornerstone Technologies. I am your host, Nick Oliveri, the marketing leader over at Cornerstone Technologies. Today, I am joined by two big time thought leaders, professionals in the industry, both with awesome backgrounds, and have made plenty of contributions in the AI space among many others. Skip McCormick, the CTO of Cornerstone Technologies, how are we?

We're doing good. Thanks, Nick. And, I'm also joined by Dave Trier over at ModelOp. He's the VP of product.

Dave, how are we? Doing great. Thanks, Nick. Appreciate you having me.

Looking forward to this conversation. This is going to be great. It's going to be great. Well, let's just dive in right away.

Dave Trier: You know, so many organizations, whether Dave it's through, you know, the for example, ModelOp's 2025, AI governance, benchmark report, and then also anecdotal evidence of all sorts. A lot of organizations seem to be viewing AI governance and AI compliance as obstacles to scalability. What say you both on that topic? Are they correct?

Or is there more nuance? Cause I feel like the latter may be true, but, I'll kind of open it up there. Yeah. I'd I'd love to.

And I'm I'm sure Skip's got a lot of stories on this too. Unfortunately the reality is that it's actually the opposite. Right? And many think and have this, this false conception that AI governance and compliance is actually going to slow things down.

But in reality, they are absolutely can be an enabler to help you get solutions to market faster. And I like to give this analogy. I'll give the first one as a sports analogy. Apologies for this.

But if you think about a football team, American football team, that you have a number of different players that are on that team. And when you think about development of AI solutions, everybody thinks, oh, it's all about the data scientists and the developer, which you kind of equate to the quarterback. Right? But the reality is there's ten other people on your team that you need to work together to get this get your move the ball down the field, if you will.

It's It's the same thing with AI. Your data scientist or AI developer may be the quarterback, but you have people on the line. Right? You have the your data engineering team.

You have your, your your, database management team. You have legal, risk, compliance. You have your IT team, production support team. So you got other players around you, and you have to work together in order to move the football down the field.

So for me, AI governance, I equate to the playbook. The playbook of how you work together effectively, efficiently in order to move that ball down the field. I get your AI products to market overall. If you don't have that playbook, it's like backyard football.

It's every every man for themselves or every woman for themselves, and it ends up being chaos. You don't move the ball down the field. You might have one good run, but then you get into trouble. You get sacked.

Right? So, again, apologies for that. Going to the hospital with a broken collarbone. You got it.

Exactly. So for me, if you have the right playbook in place, you allow all the different players to know exactly what they should be doing, when they should be doing it, and you're really enabled to go and, again, effectively move the ball down the field and score a touchdown. In the AI world, that means you're working together. You know when I'm supposed to be involved to do my data review, my legal review, when I do a security scan.

Everybody's working together because they know exactly their role, when they need to get involved, and ultimately, that just, at the end of the day, actually helps get AI solutions to market faster. I don't know, Skip, if you got a further commentary on that, but that's how I like to think about it with my I love sports analogies, and that works.

Skip McCormick: And as I was thinking about it while you were talking, I thought, okay, who's the defense? And then I realized, even your second and third line of defense and the auditors, they're actually part of your team. Right? It's there's not there's not really an adversary here.

The adversary is this, indifferent planet that we're all trying to survive on. It's it's, the, the various, government agencies that do the auditing, the fed, and those, they're not actually opposed to you. They just have they have their own advocacy that they're looking out for. And if your playbook, is clear, everything goes smoothly.

Right? And so is governance an obstacle? It's absolutely an obstacle if you ignore it until, until, you know, you got a model. And then you say, hey.

We've got a great model that's put in production. Suddenly, it feels like it's, you versus, you know, you know, NFL football team defense. Right? And that's not the case at all.

If you if you, structure how you go about everything, steps from the very beginning are part of the process. Like, what are you trying to build? What techniques are you going to use? What data are you going to use?

There's a whole bunch of things. And if you just have you have a good playbook and you're covering all those bases as you go, it just goes. And you can even schedule when it goes into production. Am I wrong, David?

No. You're you're exactly right. Skip, you said there's not really, forces on the defense, if you will. But actually, if I can take your analogy further, there are there are security threats.

Right? There are inherent risks. Right. There are adversaries.

Right. Those are adversaries that they get in the way. But that's why if you have the right playbook that's inclusive of how do you manage and mitigate those risks, that's part of your playbook, then you can overcome those. When they send a blitz at you, right, that's, you know, some, risk that to you, it was an inherent in the type of methodology that you're using.

Yep. You know how to deal with those. You know how to address them. You know how to handle that blitz and you can work around it, if you will.

I used to love watching what if we're really on this sports analogy, I used to love watching Peyton Manning just undo the defense. Right? He knew what they were doing better than they did. And he would go out there, and he knew what they would do, and he would just pick them apart.

It was it was amazing. And that's if you have a really good, sort of, governance program, you basically have Peyton Manning running your offense. And, and the defense doesn't stand a chance. Yeah.

How about that for taking the analogy to the to the sheer point?

Dave Trier: Exactly right. And one of the things that's, interesting just I'm sure you've got examples of this too, Skip, that sort of analogy, it plays out in even the corporate world. Right? So I just had worked with I worked with a lot of financials and other regulated industries.

And when what happened was that when before we came in, that they were just trying to be the wild west of AI. Right? Everybody is just trying to do cowboy coding. They would do their best to push it, you know, solutions through, and they would just get stopped.

They get stopped.

Skip McCormick: That was the thing I hated the most, would be some line of business calls up and they say, hey. You need to help us get our model into production. I'm like, wait. I'm just hearing about that you're modeling now and you have a model you want in production.

You we need to back this truck up because there's a whole bunch of things that needed to happen before you even started modeling. And then they view me as the defense. Right? Me as the obstacle.

I'm like, dude, I'm just trying to keep you from getting crushed by the real defense out there. Yeah. That's right.

Dave Trier: But what's interesting about it is, again, if we take that playbook to the corporate world, what it was, what we did was we helped to get together all the different players that's on your team and just work together to actually develop the playbook. It was like it was it was like an aha moment for them. It was like, oh, okay. Well, if we work out together what that what we need to be doing, when we need to have the appropriate, you know, data and security reviews, when we need to involve second line.

If we just work that out together, honestly, just getting them in a room was just it was a light bulb moment to them because they said, okay. Well, now I know what we need to do. When do we need to be involved? And the real benefit of that, though, that I saw, Skip, was that it helped to develop trust.

Trust amongst all those different groups that You know, might have put the brakes on in the past. It helped to develop the trust. It was like, okay.

I trust that we all agreed on this process, and this process will work. It will help to keep us out of trouble, right, for many different factors. So I trust that process now. And because of that, that actually, back to your original question, helped to speed things up.

It helped to make sure things get could go faster because those different

Skip McCormick: And not just and, Dave, not just speed up the things that speed up the right things.

Dave Trier: Exactly. Yeah. Yeah.

Skip McCormick: Working on four or five different models, and if you have a decent governance process, you could probably say, no. Don't do that one. Right? That one's not ever going to be approved, or that one's never going to be one the company wants on the top fold of the Wall Street Journal.

These are these are important questions early on. Or the classic, oh, you have this great idea, but you're using imaginary data. The data you're talking about doesn't exist, or you can't get it, or it's illegal. You know?

And there's controls to make sure all those things are addressed early on. Yeah. Absolutely.

Dave Trier: And so it's for me, and I don't know if you, you know, you got other examples, Skip, I know you got a ton of them, but we started there around just, let's just develop the playbook, but then they turned and said, okay, well, you know, we're, we're a fortune one hundred company. We gotta make this scalable. Right? We gotta turn this into something that's scalable.

And so I kind of equate that to you not use a sports analogy about how do we then turn this into more of a factory floor production. And if you think about it, that's about how do you turn these things out in a regular basis. So instead of the playbook, you have more of, like, the bill of materials that goes into the shop floor manufacturing, you know, equipment to make sure that as you're getting this AI solution, it's going to the right, you know, step, that the right mechan the right robots are involved, the right, you know, parts are being placed at the right time. So it's actually helping to form that overall procedure with the bill of materials, like how you get these AI solutions through the factory, you know, more quickly?

And, obviously, we know factory is about scale. So it's I guess the key thing for me is that if you develop that playbook, if you develop that bill of materials plus that process and have the right way to enforce that, it actually can be scalable. It can be robust. It can help get solutions to market faster, which is something that I think a lot of people just don't understand or just don't have the, you know, experience around doing it properly.

I don't know if you have thoughts on that. No. No.

Skip McCormick: And you are highlighting a bunch of the, AI work that isn't even the AI work. Right? It's it's, your data pipelines, your data quality, your, legal approvals. And then and then you can model.

Right? And then once you have a model, these are not simple math. Right? The data changes.

The context changes. The world changes. So the models have to be continuously, retuned and improved. That has to be systematic also.

How many times have you seen where they have a model, it's been in production for a year, and, the people who who created aren't there anymore, and, and now there's a problem. Right? And they're like, what? You know?

And then you say, we have to turn this ModelOp because, we can't trust it anymore. And the end users are like, this model is essential to our business. You're like, you needed to plan for this when you put the model in production. There's it's it requires continuous care and maintenance, and that's that misses lot of places miss that as well.

This the one thing that's guaranteed is your your context or your data, it's going to change. Yep. Absolutely. I that whole, lineage as it's called in the AI world and that process lineage, again, is paramount.

Right? It's it's great to have a playbook. It's great if you turn that into a production factory floor type thing. But if you can't trace it back to exactly what went into that recipe, what went into that bill of materials, who did it, when it was done, what machine, did it, what team was part of it.

You're What are you measuring and why? And who cares? Yeah. You know, I'm I'm thinking back to my CIA days when there was these, Russian factories that were like the best boot factory in the universe.

They made more boots than any other factory. But when you dug in, you found out they only made size nine left. Right? They could be super efficient, and it was like, everybody's wearing a left boot on their right foot and cutting off the toe if their feet are too big.

You know, you end up with stupid things like that if you don't have some process, from the beginning. They missed the upfront design and intended usage. Yep.

Nick Oliveri: Skip, real quick. So, you know, and to Dave's point too, you know, you reference these Fortune one hundred companies. you could say the company if you like. I won't.

But you were a, a very important AI practice lead, at a financial institution. I think I know the number, but could you give an anecdote as to a model that had to sit on the shelf due to and then why it had to and for how long? Because it's crazy.

Skip McCormick: So since I may be talking about, something that's not a good story, I'm not going to say who it was, if it's the other way around. But there was a an excellent model, very, very exciting, and very, very, revolutionary. And it was doing something that everybody thought was kind of impossible in essentially real time. Then, the governance process, engaged the existing model risk management, organization, and they weren't geared up to deal with AI models.

And so even the basic construct of this model retunes every couple weeks or faster, you know, the process that they had in place, they're used to having six months to review the math. Right? And so it just the process, needed to be updated, and that meant that the model sat on the shelf essentially. It went into a beta mode, but couldn't go to production and make a profit.

And I think it was, like, nine months of waiting around. That was that was expensive. Right? So it was, it was, a process that led to, us, meeting with Dave and implementing, a more timely process.

All the right things happen. I mean, there's no villains in the story, just victims. But there was there was a reality, and there is a pain. And that's kind of kind of the point we're trying to make.

Right, Dave? It's like you can you can, wrestle these bears while they're cubs if you choose to. Yeah.

Dave Trier: Just getting ahead of it. There's no there's no reason not to. And people, I think, are afraid of the term governance. They think it's a Yeah.

Bad term. But, again, it's a it's just having the playbook. Just having the playbook that's make sure you can get ahead of these things as you said.

Skip McCormick: My dad always said if you're not speeding, you're not afraid of a cop. Right? You're doing the right thing. And so I think people who are worried about the governance are, probably, no.

They're not doing what they should be doing. Yeah. That's true. Yeah.

Yeah. In one case, if they just don't know, that's different. But, in a in a Fortune 500 or one hundred, situation, you know, we've we've figured out what needs to be done. Now now now just get the, you know, get the right expertise in place and start doing it.

Nick Oliveri: So just to clarify here, for the audience, and also myself, frankly, So Fortune one hundreds, Fortune 500, they're going to have in house model risk management. They're going to have in house risk officers and risk and compliance folks, but

Skip McCormick: They have to, because they have the same requirements. Even if you're a Fortune one thousand or a Fortune five thousand Sure. The regulation laws are the same. And the sort of smaller your company is, maybe the smaller your IT organization is.

And now you have to figure out how to do a lot of things that are kind of, not part of the production, you know, pipeline. I think that's where, that's where ModelOp and, Cornerstone really come in. Because it's like, yeah. We can we can step in and take that burden for you and do it properly.

You don't have to have a giant IT organization. Right? With a really big Fortune one hundred or smaller, a lot of times there's a very large IT organization there, and you end up having a discussion about build versus buy. And if you don't really understand how hard this is, I think it's an illusion that you think you can do it cheaper.

Nick Oliveri: So for lack of a better term, Skip, outsourcing is oftentimes the case, especially the smaller you are.

Skip McCormick: But It's certainly an opportunity. Right? If, if this is if this is taking your data scientists off of data science, maybe you need to look hard at that. I lost great data scientists because they hated doing the governance work.

It wasn't because they didn't think governance was important. Because they just it wasn't it wasn't their kung fu. And it's sort of like you're making hamburger out of your out of your prime rib when you do that. And so agree?

Absolutely.

Dave Trier: And the reason that I would say oftentimes data scientists hate doing that is because they've made it difficult. They made it challenging. There are a lot of unknowns, a lot of manual work. It's just unnecessary.

Right? What they did, and now they have to document it? That's right. Yeah.

So it's just it's just unnecessary. If you get the right, again, playbook upfront, it's not it's not that onerous actually. And at the end of the day for data scientists, the proper way to do this, you can get your AI solution out to market faster. As I said before.

Get more solutions done quicker. And see the fruits of what you've been developing. Right?

Skip McCormick: So that's, that's Dave, you see a lot of the actual modeling. My suspicion is that every model leads to five or six more because suddenly you go, oh, we could do this, we could do that. So it's not it's not one off and all good, we're done. It's like one off and now you have five times as much to do.

That's right. Yeah.

Dave Trier: They develop one and like, oh, man. I we could just tweak it this way, and now we're going to go and target a different segment that wasn't originally intended. Oh, okay. Great.

Well, this is awesome. We just used it. Oh, I've got this. Actually, we can use it for this different organization as well.

So, yeah, it just, it kind of snowballs. And so you want to make sure you're freeing up your, especially data scientists, to go and do that, right? As opposed to thinking about the plumbing, thinking about, you know, how we help to ensure that you're adhering as appropriate. There's just capabilities in place like with ModelOp, with Cornerstone that just make it easy for you.

Yeah.

Skip McCormick: It surprised me to find data scientists who actually enjoy this part of the work. There are some. Right? So put them on that task.

Right? The ones who don't enjoy it or as good as somebody else, You know, don't make your quarterback play center. Exactly.

Nick Oliveri: So, there is a lot of talk and, Cornerstone and an offering per Cornerstone, and as well as ModelOp. There's there's this phrase, and it means a lot, but I want to dive into the nuts and bolts of what that means. It seems to be more of a process that is a loop. It reminds me it's reminiscent of, say like the water cycle I learned in elementary school.

Right. And I can visualize it, AI life cycle governance. So could we hone in on a little bit of the nuts and bolts of what it, what AI life cycle really is, such as humans in the loop, bias, fairness, awareness, and, and go into why that is so important to understand, whether you're outsourcing or for your internal team?

Skip McCormick: This is the bread and butter of ModelOp. Why don't you take it, Dave? Sure. Yeah.

Dave Trier: Life cycle and AI life cycle is really thinking about the full lifetime of a model from idea or inception, use case inception as we call it, through understanding of how it's applied to the business, through developments, testing, independent review or validation as it's called in the financial world, through usage, monitoring, iterations, and refreshes into retirement. So think of it as the full lifetime of a model from soup to nuts, cradle to grave, however you want to call it. It's every step that takes. And fortunately, in a large enterprise, especially, there's a lot of steps involved.

I've worked with companies where there's a hundred steps involved from inception to actual usage. Right? So that life cycle is actually more complicated than one would think. And it involves, as we talked about, a number of different players on the field, involves different processes, a number of different systems, especially for the Fortune 500.

So that life cycle can be an can be an impediment, but doesn't need to be. And so when you think about managing the life cycle, it's about knowing what the full life cycle is, what the different stakeholders are part of that, what are the different processes, different systems, and then helping to streamline that life cycle is how you get solutions to market faster. This is what we specialize in ModelOp. And working with Cornerstone, this is how we help to ensure that you know, large organizations don't have to think about it.

They don't have to think about all the different steps and processes and everything. They just are getting their AI solutions through that factory, through that life cycle as quick as possible to get the actual benefits that you would anticipate with AI.

Skip McCormick: And working as a enterprise instead of a bunch of business units that are all navigating this themselves. There's there's these steps where I need legal's approval, I need public relations approval, I may need certain customer's approval because they're using their data. If you do that every time, uniquely for every model, you wear out those people. Right?

And they stop they don't want to see you darken their door anymore. But if you set it up systematically where there's a understood life cycle and you can routinize it and, like, you know, at my last at my last organization, that, just the, the data review boards, right, was were happening every day because there's so many different models coming. Every department, every organization was working on some kind of model. And, if that was required if they did that piecemeal, it was it was it would just choke everything.

Right? But with a systematic process, a lot of the times, those reviews were really just documenting that it was properly checked and that all the right authorities agreed. And, okay. You're using datasets that have already been approved for this purpose.

You know, it's it as Dave said, it truly accelerated the process. But when I started, it wasn't that at all. You're you're you're, you know, walking through several different offices trying to find somebody, and then you have to explain everything to them. Yeah.

And, it was a lot easier for them to say no because because they didn't know what they were proving, and it was scary as heck. And then they would get so far. They get their first review, and then they realize or they say, oh, well, this involves an LLM. Oh, wait.

You've gotta do these five things. You gotta go back to the start Right. And go in and provide all these other pieces of information. Because, again, that process wasn't efficient.

It wasn't properly, documented. It wasn't there wasn't a systematic way to make sure it was helping you navigate all the different pathways that you have to go in order to unfortunately, the different pathways and multiple pathways you have to go to make it, put it into market with the enterprise. People inventing steps because they're afraid to say yes without getting somebody else on the hook. But you said that another thing, Dave, when you talked about the life cycle, I was really glad to hear you say it, was you said from inception to retirement.

I've seen the life cycle described as from inception to production. It's like, no. No. That's not the end of the line.

And, when you have models that are retired, you need to be able to prove to the auditor that it's been retired, it's no longer used, and here's the last day it was used, and here's what's happened to the data. There's a whole bunch of housekeeping that has to be part of that life cycle. Absolutely.

Dave Trier: I always think about the life cycle of forward and backwards. There's the forward, which everybody's really thrilled about and trying to focus on getting to production. But there's the backwards view as well, which is exactly what you said is I just need to know first from a reporting and management perspective, I just want to know what are all the different AI solutions out there, and did they go through all the steps? When did were those steps approved?

Who approved them? When they were approved? But then to your latter point around decommissioning and auditors, right, they want to know all that information even further down the line, even further in-depth. They want to know, okay.

Well, tell me the exact datasets you used to train this model. Tell them the exact ones you used to test it. Tell me when you did your, your challenge model with your validation or independent review. Right?

And then What happened to all the data? Yeah. Where's the data? And what was this data representative of the challenge you're trying to solve?

That part is, you know, you can sit look back and say, yeah. But you get that documented. Somebody's got it in the document somewhere. But what happens when an auditor calls?

What we call is we call it the governance or audit search parties. Seriously, you've gotta get this team of people together because, you know, Johnny left the company and Johnny was the one who developed the model. So now I gotta go back and find and try to piece this together. But again, that audit search party and trying to fit this puzzle together about, okay.

Well, here was the data. We think it was here. Oh, I think, you know, Jane did the validation, but, you know, Jane's in a different company now. Right?

So that part is what I found, I don't know about you, Skip, to be, really an aha moment. It was, like, oh, wow. These audits are, they're painful. How do we make it better?

Skip McCormick: It's the big win. The first time I had to support a Fed audit for the models, we had no findings. And everybody was like, wait. Wait.

That can't be right. There's always findings. It's just like, no. We covered all the bases.

The auditors, believed that we were actually advocates for the same things they were advocates for. We all cared about, you know, the quality of the model, the, ethics of the model, the privacy, the, you know, the effectiveness. You know, once we sort of earned their trust, the audits actually got pretty easy, and it changed from gotcha to, hey, here's the thing we think you might be able to do a little better if you did this. Right?

It turned proactive, and that was amazing. Yeah.

Dave Trier: If you don't have findings, the auditors will find findings for you, which you don't want.

Skip McCormick: What we earned was the auditors started asking us questions about how to do certain things. We if you could demonstrate that you're that you sort of know what you're doing and you're clear about the things you don't know, there's always a lot of things you don't know. That's why you're modeling. Right?

And if that's all super clear, they'll all audit changes from a confrontation against a defense to, you know, to really somebody on your side just trying to make sure that everybody's, you know, safe and healthy. Hard to imagine, but that you can do.

Nick Oliveri: And to close this out, this has all been about scalability, all about ROI. Skip, you have this, awesome saying, or and it's it's been in some literature and collateral we've been developing over at Cornerstone. Happy auditors equals faster ROI. Absolutely.

Than any ROI at all. So real quick though, since this has all been positive, it's bode towards scalability and ROI. It has to make money if to and it has to make sense to make money and et cetera. Otherwise, companies like ModelOp wouldn't be there.

Otherwise, Skip, you wouldn't have done the things that you have done throughout your career and such. So could we go into more so the cost of not doing so a little bit more specifically? And I want to reference TD Bank, a very large financial financial institution who got fined quite a pretty penny of just a few months back, as of as we're recording this in May 2025. Could, Skip, you go into that on, a few of the specifics where they went wrong and just how many billions of dollars they were they lost.

They had to give up to the government. Yeah.

Skip McCormick: I'll touch on it, but I think Dave Dave will have some some real solid data there. Please. But the basic idea is Please. They had models, that took shortcuts on governance or they skipped governors completely, and so the models didn't work right and advised people to make decisions which were comfortable to blame the model on because the decision, even if it was unethical, was profitable.

And so my sense is when the when the when the investigators hit that, they said, okay. Not only did you not do the right thing in terms of your AI model governance, but then you knew you were doing the wrong thing, but you went with it anyway because it was very profitable. And that's why the fines were so huge. It's like what was it, Dave? Four

Four billion dollars? Yeah. Three point something billion dollars. A lot more money than than I'll make in my lifetime.

What am I leaving out, Dave? Yeah.

Dave Trier: I mean, it was and the and it's not funny. There's nothing funny about this, but this was actually just one type of model. Right? It was just, anti money laundering.

Right? So if you think about a bank, AML or anti money laundering is only a small subset of models. One category. And so, yes, they got hit for this specifically for that.

But you think about if I multiply that by, you know, all the different categories of models that I have and I'm not doing it wrong, it's a lot more than three billion dollars Right? It's, it's Oh, yeah. AML models are only a fraction of the full inventory that banks have. So that was that was one thing for me.

It's like, oh my gosh. Is three billion just for AML? I can't I can't imagine if it really went wrong.

Skip McCormick: There's a side effect here to doing something like that well. Also, I had a friend who built a, cybersecurity company, and, they focused on, cyber insurance audits. So you could so you could, you know, get a cheaper policy for cyber insurance. And what happened was the insurance companies learned that companies who did a good job on cyber defense, they could impute a that they did a good job on a lot of other things too, and so they sold them insurance cheaper in other categories just because it was reflective of the company's, sort of ethos.

And I think I think that will be the case here as well. If you're rushing to, AI models and you're shortcutting your governance, it's going to be reflected in other things that are going wrong just like you said. And so not to pick on TD Bank. I don't know about the rest of it, but it does have that risk.

Or on the other hand, if the auditor shows up and your AI governance is just squared away and spiffy clean, and they're looking for, are there problems, and did your process catch the problems and correct them. Right? They're not looking for problem free. There's always problems like you're you're monitoring the models, the models drifted out of spec or something unexpected happened and you detected it and the process worked.

That's what they're looking for. And if they find that, I think they're you're going to get benefit of doubt in other kinds of audits as well. And you may even get a cut a cut rate on insurance, I would imagine. As the as the as the, insurance, companies, you know, start to realize this, they'll see that.

You know, speculation entirely, but that's that's I think would be the case. Yeah.

Dave Trier: And that just loops us back to the original point we make, Skip, is that the auditors are really looking for, do you have a process? Is the process working? Is the process being followed? Right?

So that, again, loops us back to you got the playbook. You have something that is enforcing the playbook, which is why for us that model life cycle is so imperative because that's what's enforcing the process. That's what's making sure it's happening across all the different departments, lines of business teams, and across all the different types of models, whether you're building it, you're buying it, you got software with AI embedded, that's what's enforcing that process. So for us, there's nothing that's more tangible to help to prove to auditors than a actual life cycle that's Yeah.

Orchestrating and has an auditable trail that you follow that process.

Skip McCormick: Is there a is there an inventory? And am I going to find models that aren't in it? Right? And that's kind of where they start.

And I'll tell you what, the first time they find one model that's not in the governance inventory, the rest of the audit's going badly. Yep. Because you've you've lost you've lost, you've you've sort of violated the trust.

Nick Oliveri: So quick quick, and then, we can wrap up. But for both of you, in a couple sentences or, really, for each of you, I should say. In just a couple sentences, where do you see the future of the market for AI scalability as it relates to governance going? I know that's a broad question, but I ask it broadly for a reason, and it's a tall task to put into less words.

Skip knows that. I have a challenge with that as well. But, yeah, I'm just going to leave that open. There is a there is a self help group for people like you and I, Nick.

It's called On and On and On. Yeah.

Dave Trier: That's I guess I'll I'll I'll start on that one. So I think the future is one that you can kind of see it being paved right now with SLMs, with agents. This is something that because of now that just the variety, the cornucopia of different types of AI that's out there. You have to do something, like, now with governance, especially if you think about agents.

And there was just a recent, you know, publication that came out from the JPMorgan security team that just said around, hey, agents. They're going to cause real issues around opening up different backdoors that weren't there in the past. Right? So that's just an example.

But the point is that because, you know, there's this variety of different types of AI, they're doing different things, they're touching all of your different systems now, they're opening you up to other external partners and other systems via these agents and tools, you gotta do something. You have to have a governance capability in place that's able to encompass all those different types of AI. There's no longer the one ring, one, ring to rule them all, if you will, with a regression model and one platform that'll Yeah. Remain nameless.

Now there's a thousand of them, both in your firewalls, external in your firewalls, pushing with your partners, your vendors, everything of that nature. Just the problem set is just so much more complex and such a wider set now that from an AI governance perspective, it's a necessity. It's no longer nice to have.

Skip McCormick: And the going back to, when I did predictive analytics at the agency, the most powerful models were amalgams. There were models of models. And so now you inherit all of those dependencies for all of the other models. And now your amalgam model, which may be awesome, has dependencies on how reliable are the models that feed it.

So it's it's it compounds, and I think it's I think it's like exponential. The problem is growing faster. My sense is that I barely have an a sense of how fast it's growing, and I think it's growing exponentially, explosively. And I feel like I'm underestimating it.

Right? That's that is that it could be daunting in the sense that how could we ever keep up with it. But I think Dave nailed it. It's like, yeah.

Start, get some systems in place, and then, keep improving them. But, I do see, situations where there's this, assumption that, oh, we can we can figure out this on our own. It's just it's just this is what the auditors want. By the time you solve that problem, ten more problems have come on.

You need you need partners working on this. And, you know, I think, my sense when I was at that bank, which we haven't named, if there had been a product like what Cornerstone and ModelOp are building, if that had been a product, it would have been very welcome. Because, we had we had to figure all this out on our own. and it's not what you it's not what you, think you know.

It's what you don't know, but you think you do. That's the that's the part. You know, it's like, holy crap. I didn't know I had to do that, or this is a problem or whatever.

And adding in all these new metrics, how do you incorporate, explainability and, unfair bias and transparency and all these things? It's it's not simple math anymore. It's models that are measuring models, and those models bring all their own metrics. It's, you know, it's turtles all the way down.

Right, Dave? That's right. Yeah.

Dave Trier: The more that is done, the more that is left undone.

Nick Oliveri: Lao Tzu quoted by Nick Oliveri. So, thank you, gentlemen. Skip for providing the pretty face, the hood ornament, and Dave for all of the substance of this, of this talk. You got it.

So I, I appreciate it immensely. This was the entry point by Cornerstone Technologies. Dave, Skip, awesome. Thank you so much, and, we'll see you on the next one.

Dave Trier: My pleasure.