News

For Large Orgs, Agile AI Is the Key to a Successful Transition

• By John K. Waters
• 06/25/2018

It's hard to overstate the impact of Artificial Intelligence (AI) and Machine Learning (ML) on ... well ... everything. The industry watchers at IDC expect spending on AI and ML initiatives to increase from $12B to $57.6B by 2021. The AI writing is certainly on the wall -- and yet, according to Nathaniel Gates, a surprising number of data scientists are still struggling to implement AI initiatives with the outdated waterfall development model.

"This is something we've been seeing for a while," Gates said. "We'd be exhibiting at trade shows and get swarmed by frustrated data science teams that had these very high-profile AI projects within their organizations. These are projects that everyone is banking on to disrupt their industry or keep them from being disrupted from behind. They're also very high risk and very slow, and it became clear to us that the way these data scientists were approaching AI was similar to the way application developers approached their work back in the '90s and early 2000s. They were using the waterfall method, which meant they had to get the entire project up and off the ground with up to 80 percent confidence before they could put it into production."

Gates is the CEO and co-founder of Alegion, which provides a training platform for AI and machine learning initiatives. Alegion (pronounced a legion, as in "a legion of workers") started out in 2012 focused on providing automation for organizations using Amazon's Mechanical Turk (MTurk) crowdsourcing marketplace for on-demand human labor. Organizations used the MTurk service to cover such tasks as identifying objects in a photo or video, performing data de-duplication, transcribing audio recordings and researching data details. Toward the end of 2016, Gates and company began getting requests from data science teams, who wanted them to use the crowds to develop training data for their AI projects.

"They needed data to be assembled, or they needed classification, labeling, image annotation, video annotation -- and they needed humans to do it at very large scale," Gates explained. "These organizations were clearly looking for new insights into the data they had, which seemed like an opportunity to us. So, we made the decision to re-launch the company with a new software platform on which organizations could create training data, score their models and validate their AI and ML models, and provide a human-based exception-handling loop in their process throughout development."

The data scientists who approached Alegion were frustrated by the waterfall approach they were using and its tendency to introduce risk and its inability to discover issues early and correct them along the way. Such linear development processes are time consuming, set unrealistic expectations and are an inefficient use of data scientists' time and skillsets, Gates said. He believes that, as AI and ML initiatives take priority on CIOs' agendas, the waterfall approach must be retired and replaced with a kind of "Agile AI" development methodology.

"Because we have humans involved throughout the process [on our platform]," Gates said, "and there's a built-in exception-handling capability, as soon as one pillar of the AI project is stood up, we can put it into production."

Gates illustrated the problem with a hypothetical: "Let's say you want to develop a chatbot. You have 3,000 hours of customer service phone calls and interactions from which you want to create a natural language processing model that includes things like understanding the urgency of the speaker, understanding the products they're talking about, and recommending a solution or an ancillary product or service they can buy -- all things that had been handled by humans. We typically see organizations tackle all four of those things at once, which means, until they can replace the human completely with an AI, they have keep the human there. They would have spent 80 percent of their money and still only be at 50 percent confidence in the model, so they would get no value."

Despite its drawbacks, the waterfall methodology is the status quo in 8 out of 10 of the large enterprises Alegion works with that are trying to transition existing processes to AI and ML, Gates added,

"They want the whole thing to be up and running before they make the transition," he said. "They think it's either/or, that they can either have the AI or their existing, manual, human-based processes. What they don't understand is that you can employ what we call a 'human in the loop' model. You can put a human into the AI processes and accelerate the value of your AI pillars as you complete them."

Agile AI enables AI initiatives to be broken down into manageable, bite-sized pieces, Gates said, which helps teams learn continuously and prioritize where necessary to deliver value quicker to build projects more efficiently. It also allows organizations to get the value from the projects much earlier.

"Let's say the only thing your chatbot can do right now is determine the product the customer is talking about," he said. "You can implement that piece and provide that data to your existing customer service people, which allows you to realize value sooner, and effectively accelerates the whole process. With Agile AI, you divide and conquer."