Nvidia's ChatRTX Update Adds New AI Models, Features -- Pure AI

Nvidia's ChatRTX Update Adds New AI Models, Features

By John K. Waters
05/03/2024

Nvidia is enhancing its experimental ChatRTX chatbot by integrating additional AI models for owners of RTX GPUs. The chatbot, which runs on a Windows PC, could already utilize Mistral or Llama 2 for querying personal documents. But the list of supported AI models is expanding to encompass Google’s Gemma, ChatGLM3, and OpenAI’s CLIP model.

ChatRTX is a demo app designed to allow a user to tailor a GPT large language model (LLM) to his or her individual content, including documents, notes, and images. Employing retrieval-augmented generation (RAG), TensorRT-LLM, and RTX acceleration, users can swiftly obtain contextually pertinent responses from a customized chatbot. Because the process occurs locally on the Windows RTX PC or workstation, the company says, the results are both rapid and secure.

Nvidia initially unveiled ChatRTX as "Chat with RTX" in February as a demo app requiring an RTX 30- or 40-series GPU with a minimum of 8GB of VRAM for operation. By essentially creating a local chatbot server accessible from a browser, the app enables users to input local documents and even YouTube videos, yielding a robust search tool complete with summaries and answers derived from personal data.

Google’s Gemma model, tailored for powerful laptops or desktop PCs, finds its place within ChatRTX seamlessly. Nvidia’s app streamlines the complexity associated with local model execution, offering a user-friendly chatbot interface that facilitates model selection, ensuring alignment with the user’s specific data for analysis or search purposes.

Available as a 36GB download from Nvidia’s website, ChatRTX now also supports ChatGLM3, an open bilingual (English and Chinese) large language model based on the general language model framework. Furthermore, OpenAI’s Contrastive Language–Image Pre-training (CLIP) has been incorporated, enabling interaction and search functionality with local photo data, essentially training the model in image recognition.

In addition, Nvidia is enhancing ChatRTX to accommodate voice queries, integrating Whisper, an AI speech recognition system facilitating data searches via voice commands.

About the Author

John K. Waters is the editor in chief of a number of Converge360.com sites, with a focus on high-end development, AI and future tech. He's been writing about cutting-edge technologies and culture of Silicon Valley for more than two decades, and he's written more than a dozen books. He also co-scripted the documentary film Silicon Valley: A 100 Year Renaissance, which aired on PBS. He can be reached at jwaters@converge360.com.