Exec's Guide to Streamlining AI Integration: Unlocking Speed and Innovation in Business

Dave Trier: I would equate us to the ServiceNow for AI. I call it AI Service Management. It's about making sure that for all the different AI demands that you have, do we have a repeatable, consistent, auditable oversight process to get those AI service requests into market as fast as possible, tracking the SLAs because there's real dollars behind them. We've just hit this wall, and that wall most of the time came into governance and operational challenges around it.

So it takes eighteen months to get it there. By the time it's already being used, it's been degraded to the point where you have to start the process all over. There's not enough trust that these AI systems aren't going to cause undue risk, financial risk, brand or reputational risk, potentially legal or compliance. So there's just a endless wheel of delays and endless wheel of not being able to take advantage of this potentially transformational technology.

By putting in that visibility, the workflows, the monitoring, we were able to reduce the time to market in half. Right? So it didn't take twelve to eighteen months. Oftentimes, we could get it down into a few weeks.

Thanks for having me today. Dave Trier. I'm the VP of product with ModelOp. We're a company that's focused on AI life cycle management and governance, essentially helping those large organizations that are trying to use AI to do so safely, effectively, but most importantly, rapidly to take advantage of the powerful nature of AI.

Caden: What made you get into ModelOp? You've been there for a relatively long period of time. Like, what made you believe in a company to stay there that long? Yeah.

No.

Dave Trier: Great question. So I've actually been with ModelOp for over six years now, and, really, it's based on my background that before ModelOp, I was working with some very large organizations for the better part of oh, jeez, over a decade to really help to leverage machine learning and AI, taking ideas that they had and helping to develop the various AI machine learning models. But routinely, what was happening was that we would just hit this wall in trying to get these solutions out into market, into the business hands, into users' hands so they can take advantage of it. We just hit this wall, and that wall most of the time came into governance and operational challenges around it.

So because of that, we started developing with our current CTO just a way to overcome some of those challenges to shorten the time to market for these solutions, to overcome those impediments. And so I had a passion for doing that because I just hit the wall so many times with these large organizations that there's gotta be a better way. As and it oh, about ten years ago, we started to developing this overall approach, and then about six and a half years ago, decided to come to ModelOp and turn this into proper software that helps to overcome those challenges. So I've always had a passion because I had to live it, breathe it every single day.

So I've had that passion to, again, essentially help to use, in this case, software to address some of those challenges. And at the end of the day, help these organizations leverage AI and ML to the fullest of the extent of the ability. Yeah.

Caden: I was looking at your LinkedIn. It was a VP of solutions, and then so it did seem like more of a service based approach. And then I bet you saw, like, enough repeatable patterns over time that you're like, oh, we could productize this. And so, like, the next logical step was to go, like, VP of product.

Like, I know all the patterns. I've like see I see all the grooves from like the wear and tear of doing this over and over and over again. Like, what are some of the patterns that you've seen in go to market with like machine learning, AI models? Because, I mean, I've never I've always heard ideas of machine learning at companies I've worked at, but we've never fully executed because of I have I who knows what?

Like, I really don't know. Right. What have you seen that's led to monetization? Sure.

So I'm going to speak specifically to the large enterprise because the focus that's been my focus for many years. There's a couple of patterns that we have been seeing with the large enterprise over the past decade or so as it relates to machine learning and AI. The first is actually that, especially in the past ten years, that prior to ten years ago, there was really just one way of doing things as it relates to machine learning and statistics. There was one data platform, one machine learning development tool, one way to actually execute those.

What happened though is that with open source, with, again, big data technologies was really precursor to this, but with cloud, the availability of compute, but also just the innovations from different vendors, they moved from just having one tool to rule them all, as they would say, to leveraging a variety of different technologies that were more purpose built for the scenario, the use case that they were working on. So you go from this place for the large enterprise where I have one way to do things to now I have, oh, ten, twenty different way to go and build a model to execute a particular machine learning based scenario or use case. So that's the first challenge is just to change involved with it, the technology, the heterogeneity, all of that is what started that change, if you will, and started some of the initial problems. So these organizations are trying to catch up, say, oh my gosh.

Okay. Now I've got different teams that are starting to use open source tools and cloud based tools. How do I have a consistent way to operationalize those, to oversee them with the right level of governance, if you will? So that was really one of the first challenges that started to see that wall that we're starting to hit.

The second pattern that started was just the expansion of where you can use machine learning and AI. In the past, it was always focused around of the various highly skilled mathematicians, maybe some physicists, if you will. But more recently in the past couple years of patterns is that with generative AI, it has leveled the playing field where you don't need a PhD in order to develop AI based solutions. And rather and you also don't need to have it focus in just areas like if you think about underwriting or you think about giving a loan, if you will.

Those are were very much the areas of focus for these large AI and ML projects. But now with generative AI, it's every team. Right? It's marketing.

It's finance. It's supply chain. It's back office. Every team is actually able to use it.

You don't need the PhDs, and it doesn't and it not necessarily needs to be isolated to a very small portion of your business. So that's, I think the second pattern that has generated the demand for these AI solutions overall because, again, it's been the especially generative AI has now expanded to all reaches of the business, if you will. So that's a demand portion. And then what becomes in more focus is the first pattern that I mentioned.

You have this demand now, but now we're hitting this wall of, okay, now it was ten different technologies even five years ago. Generative AI now have a hundred different technologies. So that problem is exacerbated by generative AI. And these large organizations that move are trying to accommodate all of the demands, all the requests for these different generative AI technologies, but do so in a way that adheres to their corporate policies, their IT policies, their data, their security, their governance, legal risk compliance.

That's a real challenge that we see day in and day out, Caden. It makes me think about why did the Google cofounders like Larry Page, Sergey Brin, as soon as this AI stuff started getting traction, they left retirement. They're going to the office every day now because they have to go in the founder mode and figure out like, okay, this is an existential threat to the business. It's so threatening that they have to come out of retirement and really take the helm again at Google.

Because how scary it is, I can't imagine a lot of non founder executives, like, they know what to do. Like, with like this new because a lot of like SaaS business models are getting disrupted by like generative AI because you like simple SaaS use cases. You could just upload a CSV, like for expense management. I just upload my bank statements into like a GPT.

And then that allows me to just ask questions like I'm talking to my accountant or something like that, which is very disruptive like personal finance apps. I never, like, looked at AI through the angle of, like, governance. Right? Because you can be, like, a forward thinking, like, leader at a company and just say, like, hey, guys, start using cursor.

Start using AI tools. But I've never considered all like the compliance aspects of it, like how to do it where it's not risky. This isn't really something you want as like a grassroots approach. Like where like the engineers are like buying their own licenses to different models and, like, incorporating them into the tech stack.

What's, like can you walk me through like, before this episode, we talked a little bit about, like, ModelOp's, mark a report on AS kind of market quagmire, and it shows, like, why a few models are going into production. From, like, the angle of, like, compliance, like, what's, like, stopping, like, big enterprises from getting into applying these, like, models into their tech stack? Is it does this have to do with the fact that compliance is, like, lawyers, and they don't understand it? And so it's very they're very risk averse.

What's going on there?

Dave Trier: So there's a couple of things overall. But at the end of the day, what's stopping it is that there is not full trust across all the different teams. It's not just compliance. It's not just legal, but all the different teams.

There's not enough trust that these AI systems aren't going to cause undue risk, financial risk, brand or reputational risk, potentially legal or compliance, even performance or just internal risk of change management. So what's happening is, again, it's not just the legal and compliance, but there's different teams. The IT, the data teams, the security teams, even the business teams aren't fully trusting that these AI solutions aren't going to cause issues. So that's what slows it down.

It isn't anything related to technology, to be honest. It's typically not related to anything around, is this a purpose fit? It's just that all the teams aren't on board and actually starting to use this. So because of that, they put a halt on it, and they want to do very in-depth reviews.

And every time is a one off process. They don't have a consistent way to go in, have all those different oversight committees, oversight meetings, insight into what's happening. So every time they're starting anew, and it's a one off. And just with one offs, there's a lot of delays, a lot of back and forth.

So with that uncertainty, it just introduces that time to market quagmire as we talked about in the report overall where, yeah, there's a large organizations. Most of them are looking at fifty, a hundred plus different generative AI use cases, but few of them actually have moved it into production less than seventy two percent and less than twenty AI generative AI use cases overall. And then a large majority of them take anywhere between six to twelve to eighteen months just to get one use case into business hands. And the challenge with AI is that a lot of times, they start to degrade over time.

Right? So if it takes eighteen months to get it there, by the time it's already being used, it's been degraded to the point where you have to start the process all over. So there's just a endless wheel of delays and endless wheel of not being able to take advantage of this potentially transformational technology.

Caden: Is there, like, a case study that you could walk me through? I'm trying to imagine, like, okay. They you engage the services of ModelOp. You hire you know, the job to be done has been pretty well defined by you right now up to this point in the interview.

They're like, okay. ModelOp, count me out. Can you walk me through, like, one of I guess, your engagements with one of these enterprises and, like, what that looks like, how you address those problems? Yeah.

Absolutely.

Dave Trier: I'll take a financial organization that we've with and pretty much had the similar challenges that I laid out at the start of this is that even within one line of business, they had over ten different teams, each of which were using different technologies, a combination of open source and cloud and proprietary. So they had a variety of different technologies that they were using. They had a variety of different demand, I. Different types of scenarios or use cases that they wanted to use AI and ML for.

But then they also were heavily regulated, so they needed to make sure that they were following not only the regulatory framework, but also their IT policy, their data policy, their privacy, their security. And that's something that was taken very seriously, very robust, very sensitive data as part of these overall solutions. So the challenge or what resulted is that they were trying to do this manually. And as a result, it was taking them about eighteen months to get any new ML solution or AI solution into the hands.

So we came in and said, okay. Let's first start with laying out your playbook. If I take the sports analogy, it's laying out let's take the American football analogy. It's laying out what is your playbook.

Right? In your playbook, you have eleven players on the field. And they all need to be working together. They need to make sure that they're all moving in the same direction, that they have consistent way to go and operationalize these AI initiatives.

And so what we did was we helped to get all eleven players that are on the field, so to speak, get them into a room, have helped to define what that consistent playbook is across, again, IT, data science, legal, risk, compliance, security, et cetera. And then we implemented our software. The software, the first thing it did was help to establish visibility first off into what every player on the field is doing so we know who needs to be involved and when. The second thing we did was we put in into place some automated workflows, and think of that as the actual execution to enforce that or making sure that all the right players are involved with the proper reviews, the proper tests, the proper oversight, proper testing and validation, if you will.

So we help to put these workflows in place so that we can quickly move these AI solutions through their full life cycle, ensuring that all the right checkpoints, all the right stakeholders are involved at the right time. We're getting all the various evidence and documentation that makes everybody feel comfortable and trust that this thing is not going to cause any problems. So we implemented our workflows to help to streamline that process. And then we also helped to put into place some monitoring to ensure that, alright, once we have this thing being used by the business that we're going and getting feedback.

We know what is happening for this AI solution. We know if it's performing well, if it's starting to deviate from what we expected, and so we get that feedback loop, if you will. By putting in those three pieces, that visibility, that workflows, the monitoring, we were able to reduce the time to market in half. Right?

So it didn't take twelve to eighteen months. We cut that in more than half. Oftentimes, we could get it down into a few weeks. So for them, that's a material opportunity cost that they were able to capture by leveraging our consistent and scalable approach with our software.

As you can imagine, each one of these AI or ML models are extremely valuable. So we're talking hundreds of thousands, potentially millions of benefit for even an individual AI or ML solution. And so if it's sitting on the shelf for eighteen months, that's real dollars of opportunity cost that is just sitting on the shelf. So it was a material impact that we helped to make by cutting that in half, being able to capture the value more quickly for any of those AI and ML solutions, and we're sitting back in waiting mode while it went through that process.

Does that make sense? Yeah.

Caden: I'm thinking about a lot of the I'm going to make it an assumption about some of, like, your road map decisions to solve that problem. Eighteen months when I see eighteen months, tons of reviews, and I've experienced that multiple times in my career for a lot of different things, not just technology, but it's usually like a wait and see mode where there's not really sure, like, what to do or, like, what to look for. Like, they don't know there might be, like, a lack of expertise on how to measure health of an initiative or understand, like, what KPIs to understand. Just going through the ModelOp website, a lot of, like, the marketing UIs have, like, these dashboards that show, like, the health of an initiative by use case.

And so one of the repeatable patterns that you must have seen is, okay. First, understand what they're trying to do, what use cases they're trying to use AI for, and then use software to provide visibility into those KPIs of how you're addressing those use cases. And I'd imagine once you implemented that solution, like most things, was probably PDF reports from Excel, from complex back end databases, like manual reports to automated reporting to show the health of an initiative, implementing a use case. It probably gave your customers a lot of certainty that they could be a little bit more aggressive to implement these models.

Dave Trier: The first problem when we set out developing our product was exactly that of just establishing visibility. Before ModelOp, and even today, for those that aren't our customers, it's the Wild West of AIs that are just going and building things or buying things, and there's not an understanding of who's doing what. Right? So it's the Wild West of AI.

And so the first problem that we helped to solve and tackle with our product was around visibility. Give all stakeholders, whether they're data scientists or business owners, management executives, just give them visibility into where and how AI is being used across the organization. Understand, okay. Here is what we're trying to do.

Here's the business scenario we're trying to solve. Here's the output that which comes out of it. So just get visibility into that. Now we combine that with visibility into how do we actually operationalize this.

Right? So is it going to be on cloud? Is it on prem? When's it going to run?

What's the data to be used? So you get this full 360-degree visibility of just the top line here, what it is and the value, but also the bottom up, if you will, up here is the operational characteristics. Here's the operational details to be able to bring this to life. You need both, right, so that you can say, yep.

I know what we're doing, and I know that it's running smoothly. It's running smoothly, and it's driving value. So that's the first problem that we wanted to tackle from a product perspective is just giving visibility to all aspects of AI inclusive of, again, the business drivers, the technical drivers, and also the operational characteristics so that we can ensure that, yes, we do have this out there, and it is actually driving value back to the business overall. That's the first problem we set out.

I'm happy to go to the second one, but let's first pause and see if you have questions on that one. Yeah. Well, no.

Caden: I love that. Like, especially in enterprise, reporting is always the low hanging fruit. One of the things I like to do on this podcast when talking about people's roadmaps, you know, historically, maybe we don't like, know, I won't respect privacy on any, like, future roadmap decisions unless you're comfortable, like, talking about, like, your vision for ModelOp as a VP product. I always like to ask, like, okay.

Like, you obviously prioritize this first thing, but there's always trade offs to, like, allocating resources to solve one problem. What was the trade off? Like, what was the thing that you had to choose not to, like, optimize right away when you chose to, like, optimize the visibility problem? And what did you have to do as a result to make up for that gap in the product?

And maybe that was the second problem. Sure.

Dave Trier: It wasn't so much a trade off, but there were other adjacent product markets that were going after a different problem first. And that was focused more on, I would say, data science or analytical monitoring, if you will. So you think about things like looking at drift and characteristic profiles, looking at performance and stability. So, again, adjacent markets went after that problem first, but, specifically, they went after teams.

Right? So an individual department at a time to try to help them with that piece. Whereas we saw this as a more transformational program. AI would be a transformational program at the enterprise level.

So in order to establish this at an enterprise level, not a department level, but at an enterprise level, you first had to establish where it's being used. So for us, we wanted to make sure that we were starting the groundworks of an enterprise grade solution, and that starts with just getting that visibility at the Because as I mentioned, you heard me a couple times now even with the case study, the biggest challenge was that you had pockets of innovation all over the place, the wild west of AI. And so from a department level, they could probably just keep on going as is. But because this is something that was demanding enterprise level investments in terms of dollars, you needed to elevate AI to an enterprise level asset or first class citizen, if you will.

So that was one thing that we decided. It's like, no. We could go after these individual departments that are trying to get some more statistical monitoring capabilities, but rather we decided, nope. We want to establish this as that enterprise level capability to elevate AI to the same sort that you would in ERP world.

You elevate the customer or the transaction, right, to that enterprise grade asset in the ERP. AI is so transformational that we believe it is a first class citizen just like your customer objects or your transaction objects or your supply chain type stuff. That's

Caden: So you target the C suite as, like, the selling persona, not director of a department you target when you're, like, selling. Yeah.

Dave Trier: Because you need it to come from the top, like, when it comes to budget. The CEO is the only one that could make sure that the right resources are allocated across the enterprise.

Caden: And, you know, I never really thought of it that way when I was thinking about enterprise SaaS or like normal SaaS. I never thought of it as what's the difference between enterprise and normal SaaS? It's all b to b, but the way that you described it really made sense to me just from a political buy in perspective of getting something adopted into a company is that enterprises, you're targeting the c suite.

Dave Trier: It's really actually comes down to the budgets. You've seen all the headlines where gen enterprises are making hundred million dollar investments, some billion dollar investments in AI. Because of that, that is a C suite decision. Right?

It's not a department decision. It's a C suite decision. And because of that, you need to manage that at the enterprise because, you know, that's a hundred million dollars a billion dollars. That's material.

Right? So that's that's what it's come down to, and that's all driven with every single product or new wave of technology. It's all driven by demand, and that it comes back to my original comment around that there's demand no longer in just a small data science COE, but it's all teams, all departments. That demand is what then elevates it to, okay, if we have all this demand, we need a much larger investment.

If we need that larger investment, we need better oversight, and that oversight starts from a top down, if you will.

Caden: Hence, the whole, like, governance positioning that ModelOp has is governance is literally the leadership team, yeah, that's operating the business, and you're giving them a chance to incorporate AI, be able to innovate with all the other companies that are effectively implementing these models and helping fill those gaps. Obviously, the first use case is going to be reporting. As you started implementing, like, the product side of ModelOp, you know, it's not as much about solutions architecture where, like, you're doing the mental gymnastics to make the tech stack, like, fit fit the use case. Like, you're trying to create a repeatable product company, which is I imagine it's still very difficult, especially with enterprise.

Like, I mean, you mentioned ERPs. What I'm hearing when I hear, like, how ModelOp is being implemented from an AI perspective, it just kind of sounds like ServiceNow for AI. Would I say that's too simplistic of a view?

Dave Trier: Or It's funny you said that, Caden. I would equate us to the ServiceNow for AI. If you think about what ServiceNow set out to do, it's IT service management, right, which is around you've got request to IT. Let's make sure that we have a repeatable process that's consistent, that's tracked, that gets your SLAs around IT service requests, you know, tracked and got through that process as fast as possible.

We are equivalent to that for AI. I call it AI service management, where it's about making sure that for all the different AI demands that you have, do we have a repeatable, consistent, auditable oversight process to get those AI service requests into market as fast as possible, tracking the SLAs because there's real dollars behind them. So for us, AI was so transformational that you need a tool like a ServiceNow to specifically handle the AI throughput, the AI times of market. So, yes, I think that's an analogy that I've used in the past of if you think enterprise level program, we are the AI service management capability similar to how ServiceNow is for IT service management.

Does that make sense? Yeah.

Caden: Like, it's managing provisioning. It's showing approved use cases and programs for the company. So you start at a high level for, like, reporting and just governance use cases, but then you start working your way down to, like, self-service, service desk use cases.

Dave Trier: So how do you And you think about IT service requests, you generally are getting involved in application team or product support team. So there's one or two involved in the process. AI is actually more complex because you have data. You have code.

You may have a vendor that's involved, and then you have IT and, like I said, security and legal and risk and compliance. So it's not one or two teams. It's ten to twelve teams. And then also you have typically ten to twelve systems involved.

Right? So you have your security system, like I mentioned. You have a variety of data platforms. You have your infrastructure.

Right? So your and then you have your traditional for large enterprise, you have your homegrown systems that are having your application registry, if you will. So this it's ten different teams, There's also ten different systems that are part of that overall process. So as you can imagine, if you're trying to build this scalable, efficient, self-service, auditable approach, that's when you need a purpose built kind of AI service management type capability, which is why we exist.

That's why IT now is a great example, but AI has a much higher demand. It has a higher closer affiliation to the business because you're driving actual business results, but also there's a higher risk as part of it. Right? So there's more inherent risk involved with AI, and that's why you need something that's purpose built in helping to manage that end-to-end process be consistent and scalable and, again, auditable fashion.

Caden: It's because AI is nondeterministic. Like, every iteration of output of an LLM is not going to be the same as the last because it's not pulling from an index. It's pulling from human knowledge that's been trained on. There's this quote I keep repeating in I mean, I'm having a lot of AI interviews lately on the podcast.

Reminds me back in, like, 2019 when, like, VR was big. That had a lot of VR related guests. And so there's definitely hype cycle, but the one thing that resonated with me the most, I forgot who wrote on Twitter, it was that AI can automate tasks, but it can't automate responsibility. And the That's right.

Obvious use case for me when people get fearful about AI of, like, oh, it could it's going to take my job is, hey, just because a Tesla could drive itself doesn't mean I'm going to let my seven year old in the driver's seat to, like, supervise the car while it's driving. There's a responsibility you have even if, like, your car is, like, doing most of the work. Like, you have to, you know, understand, like, if the car is making good judgment calls, and so they've invested a lot in visibility interfaces. Like, I'm always auditing, like, how they're indicating, like, where the car intends to turn or, like, what does it see around itself.

When it's at an intersection, the UI goes for an overhead view so that it could show you everything it sees to give you the reassurance that it's going to make a good judgment call when Right. The light turns green. And it like, just because I'm not doing anything, I'm not, like, going to just let it act like a normal car dashboard with just, a radio, and there's no screen, like, showing me how it's driving. So the responsibility element to AI that I think is going to make it so that it's going to be really hard to take someone's job unless you're like, if your job's not a responsibility, it's like you're literally just being paid to do a task, like generate basic artifacts.

Yeah. There should be some concern for like what that job looks like in the future, but I think most of the knowledge work jobs are you're paid to make judgment calls, and that's the risk. Right? It's like, hey, we're not going to automate our legal and compliance to AI because, like, it decides to make a mistake or say something racist or off putting.

It's like that legal team's still on the hook even though they're not the ones that made that call AI did.

Dave Trier: But AI absolutely is very valuable. It helps what I found is to be very helpful with is content generation. It's pretty good about getting some ideas flowing. It doesn't get it exactly right, but it's helps with getting ideas flowing.

Right? But I'll still have to put it in the perspective of what am I trying to do? What's my intent around this? I have to give it direction because but that first requires thinking through, okay.

What is my outcome? Or what's my goal here? So a human has to think about that first, then they can use AI to assist, right, in the content generation, which is very helpful. But then you still end up tweaking a little bit to the exact to this exact original design intent and putting it into the overall flow of what you're intending for whether it's a customer interaction or whatever it may be.

So, yeah, the workers are always going to need to be involved first in that upfront design thinking around how it could be used, how it fits into the flow, and then there's the oversight after the fact of just, okay. This isn't exactly what I wanted. Yes. It is good.

Right? Or I need to tweak it in this way, shape, or form. Now there are certain cases around just transactional based items, but, again, that you still need the oversight to make sure that it's not making a wrong decision as part of it. So, yes, I definitely agree with your opinion that overall AI is going to be helpful in certain places, but it's not going to replace humans full out because you still need to make sure it fits into the overall design intent to how you operationalize it, the workflows, the business scenarios, and then, again, that oversight, as I mentioned.

Caden: So as VP product to ModelOp, like, where do you see the use cases evolving to with AI governance systems? Or I guess a do you say AI management system? Similar to like an I information technology management system. Like, how do you, like, govern the model?

Do you do you have workflows that allow them to review the results and help, like, train the model? What does it look like for a department head now that you've implemented it at the enterprise level? How's that look? Because you said it's more complex.

It's not just ServiceNow. It's like you implement it. The department's driving the instance of ServiceNow. But there's so many more departments involved, like you said.

Yeah. So there's a couple things to pick out of that just to make a few highlights there. So first off, that, yes, this is at an enterprise level, but you do set up some a part of the organization, whether it's part of the AI COE or if it's part of a digital transformation group. You have somebody that is really serving as the oversight or governance leader.

Think of them as the conductor of the orchestra. Right? So you have somebody that is saying, okay. Here's the process.

Here's, you know, what we need to do across the different teams, and then they use ModelOp to enforce that process. And so they have a team that is using ModelOp to make sure that, one, they have visibility into to all the different solutions as I mentioned. But second, that they're actually overseeing those processes. Again, the conductor of the orchestra.

There are not any issues or any bottlenecks for any risks that are raised that we have the appropriate action team that's going in and looking at the risk, helping to put in mitigation plans, et cetera. Just from a organizational perspective, that's what we're starting to see. Put in practice is that there is a governance or a operational oversight team that is handling that piece of it overall. What department have you noticed?

What traditional part department usually ends up being the system owner Yep. For ModelOp?

Dave Trier: So the software itself, the system owner is in IT or technology. That's under the CIO, if you will. They're the ones that are running the software, making sure everything is integrated correctly into the existing ecosystem, making sure that the common SLAs around the software itself are in place. So that's generally under the technology or CIO department itself.

Now, again, that operational or governance team, unfortunately, that varies from organization to organization. Some have wants to take COE, you know, that has a governance team or an operational team. Some have this team under the CIO because, again, they've recognized that AI is such a transformational capability. It needs to be had proper SLAs, and that's generally the responsibility of a CIO just like the CIO might own the ITSM or ServiceNow.

But it varies on terms of that operational team that's overseeing the process, the inventory, that information.

Caden: And so let's say that, like, I'm on a product team. I come in as, like, a product manager with a company that's integrated model. Is this me, like, getting my cloud account provisioned or as an engineer, like, getting my API keys to certain, like, LLM models? Like, what are the actual use cases now that you've implemented this?

Yeah. No.

Dave Trier: Great questions. If you're a product owner on a business team and as it first starts with a great idea, I'll just take the most common example. I'm going to create a customer service chatbot, right, using an LLM, the most common one out there, if you will. Awesome.

Yeah. First off, I don't even know how to get started. I know that I'm going to run into some policy concerns, so I'm not sure what those are. But I can come to ModelOp and start to say, hey.

I've got this new idea, this new use case, put in the information. Want a customer service chatbot. I'm going to use a foundation model. Oh, ModelOp has here's all the approved foundation models because not all of them are approved.

Right? Whether it's a Claude or GPT-4o or what have you, but here are all the approved ones. Awesome. Here are all the approved environments, if you will, whether Azure, AWS, GCP, Oracle, IBM.

It doesn't matter to us. Right? And then when I go and put in my information about here's what I'm intending to do from a business perspective, it aligns to our company goals of driving productivity and cost savings. Great.

Hit the submit button. And then what ModelOp does, it says, okay. I recognize this is a customer service chatbot. It's in the service organization.

It's going to be using a foundation model. Because of that, we're going to do a risk tiering and just say, okay. Based on these inputs, it's probably like a low risk. It's not a high risk.

We're not using sensitive information or sensitive data. It's a lower risk. And, oh, by the way, here are the various policies you need to adhere to. So you will need to go and get a legal approval because you're going to be pulling in potentially a large language model that has pretrained on other data.

Right? So you will need a legal approval, but don't worry. We'll facilitate that. You'll need to go and get a data review just in case you're using any internal datasets.

So we help to then prompt the user to use that terminology. Prompt the users, like, okay. Here's what you need to go and get a couple of reviews done, but go ahead and start building or buying your model. You can go use, like I said, these approved foundation models.

Go build them. Go buy them. We can help to facilitate that. We can kick off a pipeline, if you will, that goes in provisions, an AWS account or Azure account to go and start to build your model if that's what you want to do.

From there, you go and build it. You come back and say, yep. I built it. Here's all my assets.

Right? Here's the data that I used. What happens to sit in our Snowflake or Databricks, it doesn't really matter where it is. We have integrations with all of them.

Here's the code that I developed around the foundational model. We integrate with all the Git, repositories so we can pull in all of the source code materials, configuration files. And, you know, here's where I want to operationalize. I want to put it in this production environment.

So they associate that all back. They hit the update button, and then, again, we help to prompt the user what's next. We go and kick off the various reviews. We help to open up any risks that have been identified as part of a proper risk management framework And so that you're mitigating not only just the you know, have an understanding of what's going on, but then you have all the risks that are cataloged and you have mitigating actions or basically how you're going to remediate those should they come to fruition.

So that's kind of the day in the life. It's it's before us, it's very much, I got this great idea. I talked to my manager. Our manager says, I think you gotta talk to Jim in legal and Joe and in compliance and then talk to Jane down in IT, and they'll try to figure it out for you to something that's now, again, like the ITSM process.

It's consistent. It has a orchestrated flow. It makes sure it's pulling in the right people, the right systems at the right time, and all the while we're collecting the various pieces of evidence, if you will, around this AI solution as we go systematically. I'm not filling out a hundred page spreadsheet and then a fifty page report.

It's just something that we collect all along the way. So it becomes a nice seamless self-service type experience, which is ultimately what you want.

Caden: I've gone through that just trying to use a new p like a new web browser internally, and it's okay. I think you need to talk with this person. Ends up being the wrong person. So I actually know you talk to this person.

And yeah. Like, even like using AI, it's like they'll do they give you a training of like, basically everything you can't do with AI, but they only give you an idea of what you can do with it. And it's like frustrating because, oh, there's so many simple use cases Like, an LLM could handle. Like, you've prototyped it in your own instance of Claude.

Maybe against policy. Like, you don't know. Because, like, the governance isn't very well defined. And that yeah.

That sounds very valuable. Quick lightning round question because I don't get a lot of leaders with, like, strong, like, solutions architecture backgrounds in their career. There's a lot of talk about, you know, product led growth. Like, that's the way to go for, like, you build a product for all customers and from the get go.

How has starting out with more of a solutions architecture, like, implementations, like, business fulfillment model and ModelOp benefited ModelOp in the long run? And, like, when should you decide if you're starting a company, like, if the problem merits, like, having some more hands on, like, solution architect, like, type roles, or you try to hire, like, do a more traditional, like, product model? Yeah.

Dave Trier: I think there's there's not one answer to fit that question exactly. Yeah.

Caden: I guess give me the heuristic. Go ahead. Maybe, like, a heuristic or the thing that informs the it depends answer.

Dave Trier: So I think if you take a look at it, and especially if in our situation that it was we're pioneer in the industry. Myself and CTO had run into this a couple times, but we're a pioneer in the industry. The other piece of it is that we knew it was an enterprise problem. It wasn't something that was a small medium business problem.

I would say PLG makes a lot of sense anytime that you're trying to go up for small medium business. That totally makes sense. But we knew this was strictly an enterprise problem. So because of that nature, in general, if you're thinking about, one, it's an enterprise problem, and two, it needs to be at the enterprise level, not at the department level, that's an area where you have to do a little bit of a, okay.

Go build and then go and implement with some Lighthouse customers, which is the approach that we took. Figure it out.

Caden: I was saying, was your hypothesis correct?

Dave Trier: We got it about sixty percent correct. We knew that there was this problem. Right? But it wasn't until we started working with enterprise customers, those Lighthouse customers or development partners almost, to go and implement this in action?

Did we help to refine our original thesis, if you will, to go iterate based on that, and they can do and develop the product itself? So I would say, again, there's not a one size fits all answer for this, but if you are looking at something that's strictly enterprise, you're looking something at the enterprise level, not a department level, and, of course, it's brand new, leading edge. Not we're not even bleeding edge. Right?

So if something that's completely out there in the being ahead of the market, if you will, That's generally a good case that you might want to take a bit of a solution, build product, solution it, implement it with customers, build more. So that's the approach that we took. Yeah.

Caden: Like, if especially if you're directly interfacing with an executive who they're not getting their hands dirty, like, on the execution work of the company. They're more of a decision maker. And, I mean, where I worked at, like, a buy now, pay later company, and it led to interfacing with Best Buy and stuff. And it's just, hey, we don't really know exactly like Best Buy's all other use cases.

Like we have to do like a more of a solutions approach first. And then over time, they figured out how to standardize the APIs and be more of a PayPal, like, where you could just implement it into your ecommerce website. Is there anything that you feel like hasn't been said that should be said before we sign off? No.

Dave Trier: I think you had alluded to it a couple questions ago around what's the future. And for us, it's it's all about agentic AI. Right? This is all the buzz from large enterprise to small, medium business is around agentic AI.

How do you take advantage of it? How do you do it safely? How do you not open your company up to undue risk? So our focus will continue to be support all types of AI, ML, and agentic is just another type of AI.

In our road map, we've already developed capabilities around agentic, but we'll continue to build them, to refine them, to make it easy for customers to go and put those into their overall ecosystem, but in a way that you have the proper oversight, the proper risk mitigation, et cetera. So that's the only thing I would leave with is that agentic AI is certainly a buzz worthy topic, if you will. There's a lot of opportunity there, but it's one where it fits very nicely into our overall ecosystem, our overall product, and just our thesis in general as a company.

Caden: And requires a ton of governance because it's multi turn. It could go off for seven hours and come back with a result. Like, how do you visualize that? How do you know what their judgment's going to be?

Very interesting challenge and excited to see what you guys do. Oh, and one last thing. How do you want people to reach out to you directly?

Dave Trier: Oh, sure. Thank you so much. So, yeah, please visit our website, ModelOp dot com. Reach out to us or on LinkedIn if you'd like.

If you have any questions that you'd like to answer, I live and breathe this every day. I love talking about it. Feel free to reach out on our website. Find me on LinkedIn.

I'm always happy to have a conversation on this subject.

Caden: Perfect. I'll make sure all that stuff's linked into the show notes. Dave, thank you so much for coming on the show. You have a good one.

Yeah.

Dave Trier: Thank you so much for having me. I really appreciate it.

Caden: Hey, listener. Thanks again for listening to The Way of Product. As we power down today's session on the fusion of creativity and technology, let's solidify the energy you've gotten from our dialogue today. If you've gleaned insights or strategies that could turbocharge your projects, I have a small but powerful ask.

Swing by the platform where you're hearing this and leave a review. Think of it as your digital high five to us. It's a fuel that propels this content to more creative technologists like you, amplifying the impact of the shared knowledge. So until our next interview, keep pioneering and pushing the envelope.

Thanks again for lending your ears and your imagination to our community. Catch you on the next download.