News

Nvidia's Nemotron 3.5 Lightning Targets the Workhorse Role in Agentic AI

• By David Ramel
• 08/11/2026

Nvidia has released Nemotron 3.5 Lightning, a new open AI model designed less as an all-purpose answer engine than as a fast workhorse inside long-running AI agent systems.

Released Aug. 11, Nemotron 3.5 Lightning is a 30 billion-parameter mixture-of-experts model that activates 3 billion parameters per token. Nvidia says it supports context windows of up to 1 million tokens and can deliver up to four times the output speed of similar-sized models. The company is offering the model with open weights, training data and recipes under its OpenMDW-1.1 license.

The more consequential part of the Nemotron 3.5 Lightning release is the job Nvidia expects models like it to perform. Rather than sending every step of an AI agent's work to a large frontier model, Nvidia envisions systems in which different models handle different classes of tasks.

A larger reasoning model might plan a workflow or handle a difficult decision, while a faster specialized model performs the repeated tool calls, validations, formatting tasks and other routine operations generated as an agent carries out that plan. That distinction becomes more significant as AI applications move beyond single prompts and responses toward agents that can generate dozens or potentially many more model calls while completing a task.

From One Model to a System of Models
Nvidia describes this approach as a system of models. Its enterprise announcement says modern agents increasingly operate as model ensembles, with different models specializing in different work.

That changes the equation around choosing an AI model. Raw capability remains one factor, but latency, inference cost, deployment location and task specialization become part of the same decision. A model that is not the strongest choice for difficult reasoning may still make sense for a large volume of narrower tasks if it can complete them more quickly and with sufficient accuracy.

Nvidia is reinforcing that idea by releasing NeMo Switchyard alongside Lightning. The open source routing library is designed to direct individual requests to different models based on requirements such as quality, latency and cost. Nvidia says organizations can route work across their own mix of open, proprietary and Nvidia models without rewriting the applications making the requests.

The combination points toward an AI architecture in which model selection becomes a runtime decision rather than a one-time platform choice. Teams building agents may increasingly evaluate the performance of the entire workflow -- including which model handles which task -- rather than choosing one model to serve every request.

Speed Over Maximum Intelligence
Lightning's design reflects that tradeoff. Its mixture-of-experts architecture gives the model more total capacity than the number of parameters actually used for each token, while Nvidia has also added multi-token prediction, speculative decoding and an NVFP4 quantized checkpoint to increase inference efficiency.

Nvidia reports that Lightning reaches about 86% accuracy on PinchBench while completing 10,000 tasks 30% faster than Qwen3.6 35B at similar accuracy. Those are Nvidia-reported results, but independent testing also illustrates the model's emphasis on speed rather than simply topping intelligence rankings.

Artificial Analysis gave Nemotron 3.5 Lightning a score of 24 on its Intelligence Index, behind small models including Qwen3.6 35B A3B and Muse Glimmer. However, using the final NVFP4 weights on a prerelease endpoint, it measured median output speed at nearly 670 tokens per second. Artificial Analysis characterized the model as occupying a different point on the performance-efficiency curve from models optimized primarily for higher intelligence scores.

For production AI teams, that distinction puts more emphasis on matching the model to the workload. An application handling code review, tool execution, data extraction or another repeated specialized task does not necessarily have the same requirements as an agent performing complex planning or open-ended reasoning.

More Deployment and Customization Choices
Nvidia is also making deployment flexibility part of the pitch. Nemotron 3.5 Lightning can run on local systems and on-premises infrastructure as well as in data centers and cloud environments. Nvidia lists systems ranging from GeForce RTX PCs and DGX Spark to larger data-center deployments.

Organizations can also post-train the model for their own domains, tools and workflows. Nvidia says companies including CrowdStrike, Harvey and CodeRabbit have been working with customized versions for specialized tasks in areas including cybersecurity, legal services and code review.

That combination of smaller active parameter count, open weights and local deployment gives organizations another option between consuming a large proprietary model exclusively through an external API and building or training a model from scratch. It also makes model evaluation more complicated: the relevant comparison can include task accuracy, latency, infrastructure requirements, customization and the number of calls an agent generates, rather than a single benchmark score.

Nemotron 3.5 Lightning is available through Nvidia and model repositories including Hugging Face and ModelScope. NeMo Switchyard is available separately on GitHub.

The release adds another piece to Nvidia's broader push around agentic AI, but the larger theme extends beyond one model. As agents perform longer workflows containing many individual model calls, Nvidia is betting that the winning architecture will not necessarily be one model doing everything. Nemotron 3.5 Lightning is designed around the alternative: reserve the most capable models for the jobs that require them and use smaller, faster models for the rest.