Nvidia Nemotron 3.5 Lightning 与 Nemo Switchyard

Nvidia Nemotron 3.5 Lightning 与 Nemo Switchyard
Nvidia Nemotron 3.5 Lightning and NeMo Switchyard

原始链接: https://blogs.nvidia.com/blog/nemotron-lightning-switchyard-rtx-dgx/

NVIDIA 推出了 **Nemotron 3.5 Lightning** 和 **NeMo Switchyard**，旨在提升自主 AI 代理系统的效率与控制力。 **Nemotron 3.5 Lightning** 是一款拥有 300 亿参数的混合专家（MoE）开源模型，专为高容量、专业化的代理任务而优化。与同类模型相比，它在保持前沿准确性的同时，输出速度提升了 4 倍，任务完成速度提高了 30%。该模型专为定制化设计，允许企业针对网络安全、编程或法律服务等特定领域的工作流程进行微调，并通过在工作站、数据中心或云端本地运行，确保了对隐私的完全掌控。作为补充，NVIDIA 发布了 **NeMo Switchyard**，这是一个旨在实现“模型路由”自动化的开源库。它能智能地将每个请求导向代理集合中最合适的模型——无论是开源模型、专有模型还是 NVIDIA 旗下的模型。通过确保每个特定任务由最合适的模型处理，Switchyard 能够优化性能、显著降低延迟并减少运营成本，且无需开发人员重写应用程序。这些工具共同助力企业构建更智能、更具成本效益的 AI 代理，使其能够在利用多元化模型生态系统的同时，保持现代企业部署所需的自主性与可扩展性。

这篇 Hacker News 帖子讨论了英伟达近期发布的 **Nemotron 3.5 Lightning 和 NeMo Switchyard**。用户们分享了对这些可本地部署的小型（约 30B 参数）模型的使用体验，看法不一。一个普遍的共识是，在处理编程等复杂的智能体任务时，**稠密模型**（如 Muse Glimmer 或 Qwen 3.6-27B）的表现往往优于 **混合专家模型（MoE）**（如 Nemotron 3.5），尽管 MoE 架构在速度上更具优势。主要讨论点包括： * **模型混淆**：参与者指出，当前的标签体系（如在 Ollama 等工具中）未能清晰区分稠密模型与 MoE 变体，导致用户在性能上感到困扰。 * **硬件限制**：内存不足的问题正推动行业关注模型效率。一些人寄希望于未来出现高内存的消费级硬件，而另一些人则认为，软件优化和智能模型路由（如 NeMo Switchyard）是当前必经之路。 * **对路由技术的质疑**：社区对“智能路由”工具持怀疑态度，质疑其相比直接使用子智能体是否具有实际价值，并指出缓存粘性依然是多模型系统难以逾越的障碍。总体而言，对于“模型小型化”究竟是通往通用人工智能（AGI）的可行路径，还是仅仅为了应对资源受限的临时权宜之计，社区内部仍存在分歧。

原文

As AI shifts from chatbots to autonomous agents, open models are serving market demands for full control over where AI runs and how it’s deployed and evolves.

Today, NVIDIA is expanding its Nemotron 3 model family with Nemotron 3.5 Lightning, the highest-efficiency model in its class for long-running agentic AI workloads. This release follows Nemotron 3 Nano and reflects NVIDIA’s commitment to continually improving open models for greater accuracy and speed.

Built for specialized tasks within larger multi-agent systems, Nemotron 3.5 Lightning, a 30-billion-parameter mixture-of-experts model, helps create smarter and more efficient agentic applications.

Also, NVIDIA is releasing NeMo Switchyard, an open source library for smart routing inside popular agent tools. Enterprises can use it to build a router based on their specific needs. When deployed, NeMo Switchyard can intelligently direct each request to the most capable and suitable model for the job, across developers’ own mix of open, proprietary and NVIDIA models, without requiring developers to rewrite their applications.

Together, Nemotron 3.5 Lightning and NeMo Switchyard deliver greater control over how AI is deployed, where it runs and how efficiently it operates — across PCs, workstations, data centers and the cloud.

Nemotron 3.5 Lightning delivers frontier-level intelligence in a small, customizable open model built for high-volume agentic workflows.

Always-On Agents Need a System of Models

Modern agentic systems — always-on agents — increasingly operate as systems of models, or model ensembles, with different models specialized for different tasks.

NVIDIA Nemotron open models are designed for this architecture. A frontier reasoning model such as Nemotron 3 Ultra or GPT-5.6 may plan and orchestrate a workflow, while smaller specialized models like Nemotron 3.5 Lightning can perform targeted tasks such as code review, tool use, security alert monitoring and answering billing questions.

Powering High-Volume Specialized Tasks With Nemotron 3.5 Lightning

NVIDIA Nemotron 3.5 Lightning is a fully customizable open model built for high-volume tasks powering always-on agents. It was developed with contributions from the Nemotron Coalition, whose members provided evaluation methodologies, inference software and datasets to help advance the model.

The model delivers up to 4x faster output speed, leading to 30% faster agentic task completion compared with other models in its class. And because it’s open and customizable, Nemotron 3.5 Lightning can be easily post-trained with NVIDIA NeMo on an organization’s own domain data, tools and workflows to improve accuracy for specialized tasks.

PinchBench benchmarks demonstrate that Nemotron 3.5 Lightning delivers faster agentic task completion with frontier-level accuracy compared to other models in its class.

AI leaders across industries are customizing Nemotron 3.5 Lightning for their workloads, including CrowdStrike for cybersecurity, Harvey with Trajectory for legal services and CodeRabbit with Baseten for code review, helping improve accuracy for domain-specific agentic tasks. Additionally, Lila Sciences is helping to improve reasoning capabilities for agentic tasks across physical and life sciences, and Fastino Labs customized the model and is seeing leading accuracies for software development, finance and healthcare workloads.

Enterprises have customized Nemotron 3.5 Lightning to achieve leading accuracy for their specialized task in their agentic workflows.

Nemotron 3.5 Lightning also gives organizations control over privacy and deployment. It can run on local AI systems — including NVIDIA RTX PCs, NVIDIA DGX Spark, NVIDIA DGX Station and NVIDIA Jetson — to help users maximize existing infrastructure investments, or scale across edge AI devices, NVIDIA RTX PRO workstations, data centers and cloud environments for enterprise use cases. And Nemotron 3.5 Lightning can run locally or on premises for high-volume, specialized tasks that require fast responses.

Also, as with every Nemotron launch, NVIDIA publishes as much of the training data and techniques as licensing permits, which allows for traceability, auditing and training of other models. Alongside Lightning, NVIDIA is releasing Nemotron-RL-Agentic-Terminal-Pivot, an agentic reinforcement learning dataset used to post-train it for coding agent capabilities.

More Efficient AI Apps With Model Routing

Some models are better for coding, some for reasoning, some for lightweight tasks and some are optimized to run locally for greater privacy and efficiency. If customers rely on one default model, they might either overspend or lose quality; if they manage routing manually, it becomes integration work that can slow down a deployment.

NVIDIA NeMo Switchyard is an open source model routing library for AI agents. The technology routes prompts to the most capable and efficient model for each step of an agent workflow automatically, based on specific needs. Agent application developers can tune or modify the router with different routing algorithms to match their priorities, such as quality, latency and cost requirements. In a system of models, enterprises can create powerful AI agents with improved tokenomics.

Internal benchmarks show that NeMo Switchyard maintains frontier-level accuracy while reducing task completion cost to nearly one-third of Opus 4.8 alone.

NVIDIA internal benchmarks show that NeMo Switchyard maintains frontier-level accuracy while reducing task completion cost to nearly one-third of Opus 4.8 alone.

NVIDIA is working with partners across the AI ecosystem to bring intelligent model routing into the tools and platforms developers already use.

Boomi: Evaluated Switchyard across five routing capabilities, achieving 100% domain-routing accuracy, sending 59% of traffic to a 5x faster fine-tuned model and reducing later-turn latency by 21%.
Cadence: Improved efficiency by 9.9% by using the ChipStack AI Super Agent for a formal verification use case.
Classmethod: Is running opencode and Fireworks workloads using NeMo Switchyard internally, with initial testing showing a 27% cost reduction while maintaining quality.
Cognition: Integrated the NVIDIA NeMo Switchyard staged router into Devin Desktop for NVIDIA internal use, achieving near-frontier performance on FrontierCode Main while reducing mean cost by 28% relative to routing all requests to a single underlying frontier model.
Kong: Delivers routing with NeMo Switchyard natively through Kong AI Gateway.
LangChain: With NeMo Switchyard, achieved 74% lower cost in 145 multi-turn Deep Agents tasks by routing only 7% of calls to a frontier model, at a 6% accuracy tradeoff.
LiteLLM: Is adding NeMo Switchyard as a plug-in into its proxy layer so developers can access these benefits without changing their existing stack.
Nous Research: Integrated NeMo Switchyard into Hermes to provide developers with an easy-to-configure routing system to improve agent efficiency.
Ramp: Used NeMo Switchyard to match a frontier model’s performance while cutting costs by 58% and runtime by 33% in Ramp SWE-Bench.
Siemens: Is benchmarking to improve efficiency in its Fuse EDA AI Agent.

Nemotron 3.5 Lightning is available on Hugging Face, ModelScope, OpenRouter and build.nvidia.com as an NVIDIA NIM microservice as well as through a broad ecosystem of NVIDIA Cloud Partners, post-training platforms, inference platforms and cloud service providers. NeMo Switchyard is available on GitHub and coming to partner platforms soon.