AI Castaway

The AI Agents Are Coming (For Better or Worse)

Micro-implementations of generative AI bots are hitting retailers, restaurants and healthcare. So far, the results have been mixed. Here's a snapshot.

• By Ginger Grant
• 06/26/2024

For AI to be useful, it needs to move on from writing things for us and start doing tasks for us. The way to do that is through AI agents.

Agents are AI implementations (whether in a device or a chatbot) that take action to complete tasks like sending email, diagnosing conditions or detecting fraud.

This is not the same as building a machine that automates mindless tasks performed by humans, such as cooking burgers the same way every time (although some people want to equate it to that). Instead, agents take action based upon varied input, such as network traffic analysis. In this example, it can determine that a denial-of-service attack is in progress and take moves to block traffic to stop it. These actions are autonomous based on the learned experience of the models.

Companies are starting to adopt AI agents, with varying results. This development is occurring across different industries and focusing on a variety of ML and AI technologies. Here are a couple of recent examples.

AI Advances in Healthcare
By now, it's no secret that AIs based on large language models (LLMs) may result in hallucinations, which is AI-speak for incorrect results. Discerning AI users understand, expect and work around this. However, there are situations where hallucinations are simply unacceptable.

For example, within the healthcare industry, if an AI hallucinates 10 percent of the time, the results could be catastrophic.

As of May 2024, the FDA has approved over 191 AI and ML devices used for things such as detecting anomalies in lung X-rays, developing tooth models for crowns or implants from images, and determining treatment plans based on heart images. As I am sure the FDA is being inundated with applications, it issued guidelines this month for what healthcare devices with ML or AI need to get approval. Reading through the guidelines, they appear to favor more traditional ML models with test and training datasets.

The list of recently approved research and development is mostly focused on improving analysis by studying anomalies and speeding up the identifications of known issues using curated datasets. The FDA guidelines reflect this; they focus on evaluating the data used for testing instead of the more recent developments in LLMs. Reading the guidelines, LLMs would probably not be approved.

Rogue Agents
There have been several high-profile instances of AI agents gone bad.

Exhibit A: A Chevy dealership in Watsonville, Calif. using a ChatGPT chatbot built by Fullpath offered to sell someone a new SUV for a dollar.

Exhibit B: Air Canada's chatbot made up a bereavement rate and later claimed that the chatbot was wrong to offer it. The court found that since it was Air Canada's chatbot that provided the information, the airline was required to honor it. Air Canada has since discontinued using the chatbot.

Exhibit C: McDonald's worked with IBM to test AI voice-automated ordering. Users who had severely screwed-up orders were quick to go on TikTok to show 21 orders of chicken nuggets added to their order, or nine sweet teas when only one was ordered. McDonald's since announced it is discontinuing the AI-ordering test.

Handling Hallucinations
But AI agents are not all bad news. Fast-food companies Rally's and Checkers also decided to implement AI for their ordering systems, for the same reason that McDonald's did. In 2021, their parent company, Oak Hill, invested in "restaurant of the future" concepts, which included AI, with Hi Auto, a company that focuses on developing conversational AI to quick-service restaurants.

Hi Auto's Web site advertises a 94 percent accuracy rate on fulfilling orders, and Checkers is featured front and center endorsing them. When Rally's and Checkers originally tested the AI ordering, they found it had a 98 percent accuracy rating on orders.

While the restaurants still had issues with some orders, they have staff evaluating the accuracy of the AI in-store to resolve issues that it might have. Oak Hill has determined that AI has improved restaurant efficiency and it continues to roll out the technology to all of its stores.

Conclusion
It is just a matter of time before we see more AI agents coming to different industries in the future. It is not realistic to think it won't come to where you work, too. And just like when working with people, there will be times where the AI is wrong.

In healthcare, there are a lot of rules, restrictions and testing requirements. Other industries rely more on human verification to ensure that, for example, the code you asked GitHub Copilot to create does what you want or your company's customer service bot will not create new pricing structures and will complete orders correctly. Currently, 25 percent of all new chatbots do not meet the needs of the companies that implement them, but as the safeguards improve, this will change.