As enterprises progress in their AI journey, model operations (ModelOps) represents a foundational capability, a way for teams to establish end-to-end governance. ModelOps is an integral requirement for any organization that is making significant investments in data science and AI, regardless of industry. Through ModelOps, teams can be positioned to fully unleash the value of their AI investments, while addressing requirements to boost trust and minimize risk. A lot of exciting developments have happened in this area recently, so I wanted to provide an update.
Gartner Market Guide: An Important Milestone for the ModelOps Market
Last year, Gartner reported that 50% of data science initiatives don’t make it into enterprise production. In addition, a Corinium Intelligence survey found 80% of respondents indicated that difficulty managing risk and ensuring regulatory compliance was a barrier to AI adoption.
This reflects a critical “AI gap” that exists in many enterprises. Recently, Gartner released an exciting piece of research for teams looking to address this AI gap. I’d encourage anyone looking to govern and scale their AI initiatives to review this important report, which is entitled “Market Guide for AI Trust, Risk and Security Management.”
This market guide provides enterprise teams with insights for addressing the AI gap, and it discusses how ModelOps is a key pillar in AI trust, risk and security management (TRiSM). With the release of the market guide, they’re providing some much-needed clarity in terms of the various vendors and often confusing array of capabilities that are available in the market currently.
Key Insights and Takeaways
This market guide is very comprehensive. The report features a breakdown of how the market is expected to evolve over time, breaking the market’s evolution into five phases represented by a pyramid. Phase one represents the current capabilities available today, and phase five represents the vision of how the market, technologies and vendors will converge over time.
The diagram depicts how the AI TRiSM market consists of capabilities in five categories: explainability, data anomaly detection, ModelOps, adversarial attack resistance and data protection. The report shows that, as the market enters into the next phase of its evolution, the five categories of capabilities are expected to be condensed into two: ModelOps and data protection.
The report features the following definition of ModelOps:
“Model operationalization (ModelOps) is primarily focused on the end-to-end governance and life cycle management of all analytics, AI and decision models (including analytical models and models based on machine learning, knowledge graphs, rules, optimization, linguistics, agents and others).”
In reviewing the report, I came away with a couple key takeaways:
- First, there are critical capabilities that need to be in place if large enterprises are to make the most of AI investments, and these capabilities are fundamentally distinct and separate from what’s happening in the data science tools arena. Further, the more investments are made in AI and data science, the greater the need for these capabilities.
- Second, the report points to ModelOps as a key pillar moving forward, and its strategic and growing importance in the years ahead, which is key for leaders to consider as they plot their investment strategies.
ModelOps Definition and Requirements
It is essential to make enterprise AI assets fully governed and visible, from the board level to the operational level, establishing and tracking concrete metrics in terms of how data science investments are contributing to AI and the business. Fundamentally, this is the role of ModelOps.
The Gartner definition of ModelOps included a key phrase: “end-to-end governance.” Over the course of their life cycle, there are so many other components that models need to interact with, which can make establishing this end-to-end coverage a challenge. The diagram below provides an idea of all the capabilities a ModelOps solution needs to provide an enterprise.
Following are a few of the capabilities we see as vital as enterprises embark on ModelOps:
- Flexible, complete integration with AI ecosystem. Integrations with data science platforms on the left of the diagram are required. Assets from these environments need to be leveraged and interacted with in the lifecycle. Teams also need to integrate ModelOps with business processes and technical systems.
- Governance and monitoring. It is vital to establish evergreen, persistent visibility of where all models are, and leverage capabilities for governance and risk that can be fully automated. Everything needs to be managed in order to ensure models go through the right regulatory, compliance and risk management workflows. If something happens with the model that requires action, remediation responses need to be triggered.
- Comprehensive reporting and auditability. Throughout the model lifecycle, and for all models, comprehensive reporting is required. Teams need to be able to track the provenance and lineage of each model. All model activity needs to be orchestrated and documented so it is fully auditable. ModelOps tools need to deliver board level reports, as well as operational reports, including for both business and technology teams. All this intelligence needs to be accessible at the push of a button.
- Flexible, adaptable workflows. Complicated software delivery processes need to be managed, and governance needs to be aligned with different model types. For example, there may be very different tolerances for an AI model that is run weekly to optimize supply chain decisions as opposed to a model that is run during the fulfillment of customer transactions.
Conclusion
Complete ModelOps capabilities are essential in enabling enterprises to ensure they gain maximum business value from their AI investments—while addressing their risk and governance objectives. If you’re interested in learning more, I’d encourage you to review information about ModelOp Center, a powerful ModelOps solution.
