Practical AI

'Practical' is in the AI of the Beholder

• By Howard M. Cohen
• 10/31/2024

Impractical, unproductive, frivolous, and trivial applications are all pretty easy for most of us to spot. Especially when it comes to applications using artificial intelligence. Consider the following:

• Having an AI write your emails, your reports, and other documents may be somewhat productive in that they save you time, but not when you consider the cost of running that AI engine.
• Chatbots have the potential to free human operators to do more productive work, but its likely that most people’s response to a business chatbot will be to request a human operator.
• Having DALL-E or other similar graphic apps augment a photo of you so you look more rugged. Yeah... no.

On October 14, 2024, Inc. Magazine published an article entitled "Inside AI’s $1Trillion Cash Bonfire," with the subtitle "Generative AI companies have spent a fortune still searching for a killer app."

For those who have been around IT long enough, this observation and its implications should sound drearily familiar. Three or four decades ago it was videoconferencing. Looked great. Lots of fun. But what was it good for? Today we know the answers to that. Complex platform products, such as Microsoft SharePoint, for which many products were woven together into a larger fabric, took a few years to demonstrate some really practical applications. At first, "the Cloud" was panned as being impractical, pointless, insecure, and even dangerous. Today, we all feel very differently. In fact, for many if not most enterprises, they're indispensable.

It has been pointed out that the mobile phone took more than 16 years to reach the point where there were 100 million users. ChatGPT had a million users five days after its release, and it reached 100 million two months after its launch. When something hits this big so suddenly, it's tempting to assume it's a passing fancy. A fad. Here today, gone tomorrow. But if we follow the likely consequences of Moore’s Law, and the fact that Intel founder Gordon Moore's prescient assertion that the number of transistors on a microchip will double every two years at minimal cost continues to accelerate, we must also re-examine our ideas about how new technologies will develop in our environment. AI is clearly not a fad, and it continues evolve and proliferate at a meteoric rate.

But headlines like the one in the Inc. article do surface the reality that truly practical, pragmatic, productive, and profitable applications of this new computer science have been scarce at best.
Finding those applications is the raison d'etre of this column.

One of the foremost AI developers, CrushBank CTO David Tan, explained it to me simply. "Most people don’t understand how all this works," he said. "People think this stuff is magic."

CrushBank was formed in 2017 by Tan and Evan Leonard, execs of an IT service provider company that had been serving its market for more than 40 years. They realized that much of the operational side of what they did for a living could be handled by AI, making those operations far more efficient, and far more profitable. Partnering with IBM WatsonX, they built their solution which is today enjoying broad adoption within their industry.

"It's things like ticket classification, ticket budgeting, ticket prioritization, and summarization," Tan explained. "A ticket comes into the help desk, and all these things a person needs to do are really, really good use cases to train AI to do because it's sort of finite. But you can't do it by just sending a ticket to ChatGPT and saying, 'What's the priority on this ticket?' You need to do it the right way, by sending a bunch of training data, examples, building these custom models and this custom tuning, and then you can deliver really quality results."

What kind of results? "We get about a 90%—and this is from our clients—about 90% to 93% accuracy on those automations. Things like how long the ticket should take. What's the priority on it? What's the category? You know, summarize the problem for me, because we've trained it very specifically for that use case. We're not using OpenAI. It's too huge, a 17 trillion parameter model. We're using a model that was custom tuned just for IT support."

This is an excellent example that takes an operating process they were very familiar with and training a large language model (LLM) specifically for that application.

Tan also talked about how to extend the value of the data by building a data lake, one central location in which all your data resides.

"You can then leverage it," he said, "and that becomes the foundation for everything you do. If you need to find something, you need to get an answer that lives across various documents. All of that is part of knowledge management. We're ingesting documents, runbooks, user manuals, guides and ticketing history. Now, we can go in and ask, how do I do this? What is this address? Where do I find this? You can go in and ask, how many tickets did each technician close last week." What's the average response time on tickets related to a VPN setup?"

Beyond their own IT service industry, CrushBank is working on practical applications for a variety of customers:

• A legal vertical application. They're ingesting all of their previous cases so they can ask things like, who was the expert witness on this trial, or who was the other special counsel on this case? All things that used to take hours to look up. That's a very good use for AI, because it takes a simple natural language question, turns it into a query, finds the right data, and generates a simple natural language result.

• A construction company that does many large construction projects. They have 15 people that generate quotes and estimates, and they're accurate only 20% of the time. CrushBank is taking all of their accurate and inaccurate quotes and all of their intellectual property and they're training a custom quoting model for them. All they'll have to do going forward is to put in a bunch of parameters and generate a proposal, or a quote. They're seeing that 20% accuracy improve to somewhere north of 80%, which means they'll soon be able to cut down from 15 people quoting to 1.

As with so many things in IT, it’s all about the data. If you have good data collected, summarized, and available, you can use it to train an LLM to do what you need it to do, and so much more.
This means that practical, pragmatical, purpose-built, profitable applications will arrive as more experts gain the ability to properly train the required data models.

There's a learning curve here that will take considerably more time than learning how to program, but not that much in IT time.

In this column, I'll continue exploring the valuable applications leaders like Tan and CrushBank are creating, how they’ve created them, and what it can mean for you.