Company

Designing Your Analytic Development Life Cycle

 

designing your ADLC blog-1It has become clear that analytics are critical intellectual property for the enterprise with the potential to have huge business impacts.  However, the value companies derive from their models may be hindered by lack of a strong analytic development life cycle(ADLC) process.  Without a well conceived analytic development life cycle, models in production may not work to entitlement, and the rate of iteration may be hindered. It is essential to design your analytic development life cycle to create a scalable and efficient downstream pipeline that moves models efficiently from creation to production.

According to “How Data Science Teams Leverage Machine Learning and Other Advanced Analytics” by Gartner research, many companies still struggle to get data science initiatives up and running and drive them forward, often felled by common obstacles such as poor data quality or cultural resistance.  Today as data science pipelines continue to expand – companies are investing time, money and human resources to turn their analytic development life cycle dreams into a reality.

One useful framework for building an analytic development life cycle comes from considering how software projects are generally run today, and applying those same principles to data science assets. As industry leaders, Open Data Group has come to understand what it means to design an ideal analytic development life cycle for our customers by applying those same principles.  In addition, we have identified some of the common pitfalls to avoid that will impede your analytic journey.  Most issues come from the different focus and ownership of each stage, which may reside with different people and a lack of well-defined processes. Some of the more overlooked considerations include:

  • Data compliance – Data will come from different sources across the analytic development life cycle journey. For example, Dev may leverage smaller, curated data sets while production is exposed to the more natural and wild data. It’s possible each stage may even find different name mappings of the same data.  It’s important to require the model creator (e.g. the data science team) to define the schema for their model.
  • Model code is execution ready – another large issue is that the Dev environment may leverage different tooling and software from the downstream process steps. It’s important to give the model creation team a requirement to PROVE the model is execution ready, in the production execution framework chosen by the organization.
  • Model is safe – the organization should provide processes and tooling to ensure to every extent possible the model is safe for execution.  Teams should be able to enforce library restrictions, have registered images of containers, and provide clear documentation of the underlying assets to prevent unwanted access or operations from impacting mission critical systems.
  • Model is governed – as analytic assets become more like IP, and as they solve for business cases that may be regulated, organizations must develop the ability to track and maintain model lineage and metadata.  Questions like “what data was used to train this model?” and “which libraries were used in producing these scores?” should be easily and readily answered.
  • Model is monitored – business teams will expect to know if and when an issue occurs with their mission critical analytic.  Systems of data capture, monitoring systems and data logging should be in place to allow for intelligent action, alerts and automated responses when an issue is found.

 

In the end, it starts to become apparent that a standard, agnostic way to execute, monitor and manage each analytic asset is required. New approaches, such as combining FastScore, by Open Data Group with a well thought out analytic development life cycle, allows for the journey of analytic models to move and organically scale with the production needs of the organization.  The right deployment approach will be agnostic to other choices in the enterprise and will be capable of enabling the analytic development life cycle that is required for long-term value creation from the analytics investment.

 

All ModelOp Blog Posts 

ModelOp Golden Ale Takes a Holiday – Part 2

ModelOp Golden Ale Takes a Holiday – Part 2

2 Minute Read By Greg Lorence Before we go much further, I feel obligated to state what is likely already obvious: I’m not all about that #InstaLife. All accompanying photography was snapped with little regard for composition, typically while stretching out from 4-6...

Q&A with Ben Mackenzie, AI Architect

Q&A with Ben Mackenzie, AI Architect

2 Minute Read By Ben Mackenzie & Linda Maggi How AI Architects are the Key to Operationalize and Scale Your AI Initiatives Each week we meet more and more clients who are realizing the importance of operationalizing the AI model lifecycle and who are dismissing...

Behind the scene of ModelOp by our Brewmasters- Part1

Behind the scene of ModelOp by our Brewmasters- Part1

2 Minute Read By Greg Lorence As a long-time homebrewer, when our President, Scott asked me, “wouldn’t it be cool if you and Jim brewed a beer to commemorate our rebrand later this year?” my reaction, after the immediate “heck yeah! Beer is awesome”, was honestly...

Open Data Group Officially Becomes ModelOp

Open Data Group Officially Becomes ModelOp

2 Minute Read By ModelOp Today, Open Data Group rebrands as ModelOp. Read more on Globe Newswire It is an exciting day for us, if only because people will stop asking “Why are you called Open Data Group?” after they understand what we do. More importantly the name...

Gartner & WIA Conferences Exit Poll

Gartner & WIA Conferences Exit Poll

2 Minute Read By Garrett Long As we continue into our “Year of Model Operations”, I thought it would be useful to highlight some of the key things I observed, learned and shared over the last few weeks at both the Gartner Data and Analytics Summit March 18-21, 2019 in...

Machine Learning Model Interpretation

To either a model-driven company or a company catching up with the rapid adoption of AI in the industry, machine learning model interpretation has become a key factor that helps to make decisions towards promoting models into business. This is not an easy task --...

Matching for Non Random Studies

Experimental designs such as A/B testing are a cornerstone of statistical practice. By randomly assigning treatments to subjects, we can test the effect of a test versus a control (as in a clinical trial for a proposed new drug) or can determine which of several web...

Distances And Data Science

We're all aware of what 'distance' means in real-life scenarios, and how our notion of what 'distance' means can change with context. If we're talking about the distance from the ODG office to one of our favorite lunch spots, we probably mean the distance we walk when...