News
How AI Is Accelerating Machine Learning Development
- By Pure AI Editors
- 12/03/2025
Generative AI is dramatically accelerating how quickly data scientists can create machine learning prediction models. Briefly, AI can 1.) accurately determine which machine learning techniques are most likely to produce the best prediction results, 2.) generate machine learning program code, and 3.) tune machine learning model hyperparameters. As a crude rule of thumb, using generative AI typically reduces the time needed to create a machine learning prediction model by at least half, and often much more.
What Is Machine Learning?
The three most common forms of machine learning are regression, classification, and clustering. AI accelerates the creation of all three types of ML systems. The goal of a regression problem is to predict a single numeric value. An example is predicting the bank account balance of a person based on age, sex, annual income, and so on.
The goal of a classification problem is to predict a discrete non-numeric value, for example, predicting wine quality (poor, average, good) based on pH acidity, alcohol content, and so on. When there are only two possible values to predict, the problem is called binary classification.
The goal of a clustering problem is to group similar data items together. The clustered data can then be analyzed to identify anomalous items, or to find unusual patterns in the data. One example is clustering credit card transaction data to look for fraudulent activity.
Picking the Best Machine Learning Technique
One of the significant challenges of using machine learning is that there are dozens of techniques, and each technique has dozens of variations. For example, for regression problems, basic techniques include linear regression, nearest neighbors regression, polynomial regression, kernel ridge regression, Gaussian process regression, support vector regression, neural network regression, random forest regression, adaptive boosting regression, gradient boosting regression, and neural network regression. Each technique has pros and cons related to the size of the problem dataset, the hidden underlying structure of the data, and other factors.
A traditional exploration of different regression techniques for a dataset with moderate size and moderate complexity might take roughly five days of experimentation. Generative AI can reduce this time to approximately half a day.
After uploading the source dataset to an AI application, suppose you submit a prompt like:
"In the uploaded file, the first five values on each line are the predictors, and the last value on the line is the target value to predict. What are the best machine learning regression techniques for the data?"
The AI application might give response advice similar to:
"Because the dataset predictors are continuous, bounded, and appear to have nonlinear interactions, the best models will tend to be tree-based ensemble regressors or kernel-based models, such as kernel ridge regression or gradient boosting regression."
At the time this article was written, current AI technique selection advice for machine learning techniques operates at an expert level, roughly equivalent to a recent PhD graduate.
Generating Machine Learning Program Code
Generative AI systems can produce code for computer programs. As recently as about 10 months ago, AI was limited to relatively simple programs, such as SQL queries, common C++ functions like binary search, and HTML/JavaScript web sites. But AI has advanced to the point where relatively sophisticated machine learning programs can be generated with good (but not always perfect) accuracy.
Key training data sources for AI computer program generation are source code repositories, notably GitHub, web blog sites and question-answer sites such as Stack Exchange, software documentation, and digitized computer science textbooks. After initial pre-training, the code generation models are refined using supervised learning and reinforcement learning from human feedback.
[Click on image for larger view.] Figure 1: Example of an AI-Generated Machine Learning Program
Suppose you submit a prompt such as:
"Please give me a simple program, using the scikit-learn library, that creates a kernel ridge regression model for the dataset."
The AI application might generate a computer program like:
import numpy as np
import pandas as pd
from sklearn.kernel_ridge import KernelRidge
np.set_printoptions(precision=4, suppress=True,
floatmode='fixed')
# Load the dataset
data = \
pd.read_csv("synthetic_train_200.txt",
comment="#")
X = data.iloc[:, :-1].values # predictors
y = data.iloc[:, -1].values # target
# Create and train the Kernel Ridge Regression model
model = KernelRidge(kernel="rbf", alpha=1.0, gamma=0.5)
model.fit(X, y)
# Predict on the data (no split)
y_pred = model.predict(X)
print("\nFirst 3 actual values:")
print(y[:3])
print("\nFirst 3 predictions:")
print(y_pred[:3])
The generated program is minimal, but correct. When the program is executed, the output is:
First 3 actual values:
[0.1568, 0.8054, 0.1345]
First 3 predictions:
[0.1798, 0.7659, 0.1279]
Because most machine learning examples use the Python language scikit-learn, AI is especially good at generating programs that use that library.
Machine Learning Hyperparameter Tuning
One of the most time-consuming tasks in machine learning is tuning the prediction model hyperparameters. For example, for a nearest neighbors regression model, you must find the best value for the number of nearest neighbors to use. For a kernel ridge regression model, you must find the best values for the alpha and gamma parameters.
Suppose you submit a prompt like:
"Create and execute a Python language program that will perform a grid search for the alpha and gamma parameters of the KRR model. Use three different possible values for both alpha and gamma."
The AI response might be similar to:
Best hyperparameters:
{'alpha': 0.01, 'gamma': 0.1}
Best cross-validation MSE (negative):
-0.000196
All results:
MSE = -0.000196 Params={'alpha': 0.01, 'gamma': 0.1}
MSE = -0.000692 Params={'alpha': 0.01, 'gamma': 0.5}
MSE = -0.003388 Params={'alpha': 0.01, 'gamma': 1.0}
MSE = -0.000616 Params={'alpha': 0.10, 'gamma': 0.1}
MSE = -0.001192 Params={'alpha': 0.10, 'gamma': 0.5}
MSE = -0.004365 Params={'alpha': 0.10, 'gamma': 1.0}
MSE = -0.002953 Params={'alpha': 1.00, 'gamma': 0.1}
MSE = -0.004636 Params={'alpha': 1.00, 'gamma': 0.5}
MSE = -0.010316 Params={'alpha': 1.00, 'gamma': 1.0}
Using AI-generated hyperparameter tuning can save days of effort.
And So?
The Pure AI editors asked Dr. James McCaffrey to comment. McCaffrey was one of the original members of the Microsoft Research Deep Learning Group. He noted, "AI is definitely going to eliminate many tech jobs, but I don't think AI will replace the data scientist role. I suspect that it's more likely that AI will dramatically increase the speed and efficiency of data science activities.
"However, it's quite possible that AI will negatively impact the job market for entry-level data scientist positions, which is a role that tends to be filled by recent college graduates."