Fine

Model customization transforms general-purpose AI models into specialized enterprise assets. By fine-tuning foundation models (FMs) on domain-specific data, businesses teach AI their unique workflows, terminology, and deep domain specialization, along with strict adherence to brand voice and fewer hallucinations. For enterprises, this is more than an optimization. It’s the creation of proprietary intellectual property. A fine-tuned model encodes an organization’s unique intelligence and best practices into its architecture. This builds a competitive advantage that is difficult to replicate with off-the-shelf public frontier models. At the same time, fine-tuning smaller, open-weight models on targeted tasks often matches or exceeds the performance of much larger proprietary models. This approach delivers significant cost savings while keeping sensitive data within secure, private infrastructure.

Amazon SageMaker AI offers a wide selection of open source models and fine-tuning techniques to help organizations tailor foundation models to their unique needs. Now, SageMaker AI introduces serverless model customization for NVIDIA Nemotron 3 models, starting with Nemotron 3 Nano (30B total parameters, 3B active) and Nemotron 3 Super (120B total parameters, 12B active). With supervised fine-tuning (SFT), reinforcement learning with verifiable rewards (RLVR), and reinforcement learning with AI feedback (RLAIF), you can adapt these high-performance open-weight models to your specific domains and workflows without provisioning or managing any infrastructure. For a complete list of open models available for serverless model customization, see Customize open weight models in the Amazon SageMaker AI documentation.

In this post, we explore what makes the Nemotron 3 architecture unique, walk through the fine-tuning techniques available, and show you step-by-step how to get started with serverless customization using SageMaker Studio.

Overview of NVIDIA Nemotron 3 models on Amazon SageMaker AI

NVIDIA Nemotron 3 is a family of open-weight large language models (LLMs) built on a hybrid Mamba-Transformer Mixture-of-Experts (MoE) architecture with native support for up to 1M-token context lengths. The architecture interleaves three complementary layer types: Mamba-2 layers for efficient linear-time sequence processing, Transformer attention layers for precise associative recall, and Latent Mixture-of-Experts (LatentMoE) layers that compress tokens before routing to specialized experts. This design activates only a fraction of total parameters per forward pass (for example, 12B of 120B in the Super variant), delivering high throughput and strong accuracy at significantly lower compute cost. The models use multi-environment reinforcement learning through NeMo Gym, which aligns them to real-world, multi-step agentic tasks across domains such as coding, reasoning, and long-context analysis.

Nemotron 3 Nano is a small language model optimized for high compute efficiency while maintaining strong accuracy on specialized tasks. Nemotron 3 Nano performs strongly on coding and reasoning tasks among open language models in its size class. Trained using multi-environment reinforcement learning through NeMo Gym, the model achieves 4x higher throughput than its predecessor Nemotron 2 Nano. Its efficient 3B active parameter footprint makes it ideal for high-volume, multi-agent workloads where cost and latency matter. For a deeper look at the architecture and training techniques, see the NVIDIA developer blog.

Nemotron 3 Super is a larger model designed for high-efficiency multi-agent AI and complex reasoning tasks that require more capacity than Nano while maintaining cost efficiency. Nemotron 3 Super delivers high compute efficiency, throughput, and accuracy for complex multi-agent applications such as software development and cybersecurity triaging. The model performs well at reasoning, coding, and long-context analysis, while remaining efficient enough to run continuously at scale. This makes it a good fit for IT ticket automation, enterprise workflow orchestration, and autonomous agent systems that require sustained multi-step reasoning. For more details, see the NVIDIA developer blog on Nemotron 3 Super.

SageMaker AI serverless model customization

Let’s walk through how to get started with serverless model customization for Nemotron 3 models. While the base Nemotron 3 models deliver strong general-purpose performance, enterprise use cases need domain-specific behavior that base models alone cannot achieve. With model customization, you can adapt these models for industry-specific terminology and decision patterns, train reliable tool calling with your organization’s APIs, align outputs with your brand voice, refine multi-step agentic reasoning for your architectures, and optimize cost by specializing the smaller Nano model to match larger model performance on targeted tasks.

You can get started with serverless model customization through the Amazon SageMaker Studio console or programmatically using the SageMaker Python SDK. On the console, navigate to the Models page, select your Nemotron 3 model, and follow the guided workflow to configure your training data and launch a customization job. Alternatively, if you’re already working within SageMaker AI, you can use the agentic functionality with agent skills to accelerate your model customization workflow. The following sections walk you through the prerequisites, data preparation, and step-by-step instructions using the SageMaker Studio console. For a detailed programmatic example with the SageMaker Python SDK for customizing an open-source model, see the AWS samples GitHub repository.

Before you begin, verify that you have:

Follow these steps to customize a Nemotron 3 model using the SageMaker AI Studio console.

SageMaker AI automatically provisions the required compute, executes the training job, and captures continuous logs. The training metrics are automatically logged to the SageMaker MLflow App by default for training tracking.

You can monitor the status on the model home page, which displays training performance, as shown in the following screenshot. Several high-level metrics are worth monitoring. Train Reward (for RLVR) should increase steadily. Training Loss and Validation Loss should decrease and track generalization, respectively. Policy Entropy (for RLVR) decreases as the model gains confidence. Gradient Norm should stabilize to indicate convergence.

The detailed training and validation metrics are also logged to the associated SageMaker AI MLflow App, as shown in the following screenshot. This captures a comprehensive set of metrics and parameters that track training progress, and model behavior. In the MLflow tracking UI, these metrics are organized by the component they measure (actor, critic, rollout, performance), so you can diagnose training health at a glance.

You can also turn on Compare with base model in evaluation to directly measure how your post-trained model performs relative to the base model. Together with the previous training metrics, MLflow tracks the training dynamics (rewards, KL divergence, loss). The evaluation measures output quality from an end-user perspective, giving you a complete picture of the model fine-tuning effectiveness.

Deploy your customized model directly from the model details page on the console. You can also deploy to SageMaker Inference endpoints, or you can download model weights from an Amazon Simple Storage Service (Amazon S3) bucket for self-managed deployment. The deployment options auto-populate defaults, giving you full flexibility over compute and scaling based on your traffic and throughput requirements. The following screenshot shows the deployment of the fine-tuned NVIDIA Nemotron Nano 30B using an ml.g6e instance powered by NVIDIA L40S Tensor Core GPUs. The deployment uses SageMaker inference components and, by default, serves the merged model weights, where the base model and LoRA adapter are combined into a single set of weights for optimized inference. Because this is a LoRA fine-tune, you can also self-host and serve the unmerged LoRA adapter separately, because you have access to both the base weights and the adapter weights in your S3 bucket. After deployment, you invoke the endpoint using the invoke method with the AWS Command Line Interface (AWS CLI) or SDK.

With serverless model customization for NVIDIA Nemotron 3 models on Amazon SageMaker AI, you can now adapt these high-performance open-weight models to your specific domains and workflows. Whether you’re fine-tuning Nemotron 3 Nano for cost-efficient agentic task execution or customizing Nemotron 3 Super for complex multi-agent orchestration, SageMaker AI handles compute provisioning, training orchestration, and metric tracking so you can focus on your data, evaluation, and deployment.

Get started today with serverless Model Customization on Amazon SageMaker AI. For detailed examples of customizing open-source models, see the AWS samples GitHub repository. To learn more, see the Amazon SageMaker AI model customization documentation.