AWS Savings Plans for SageMaker AI Workloads

Machine learning workloads on AWS have a distinctive cost shape: bursty, expensive training jobs on accelerated instances, plus steadier inference and notebook usage. SageMaker has a dedicated Savings Plans instrument that covers this usage — and it is separate from the Compute Savings Plans most teams already know. Confusing the two leaves discount on the table or, worse, double-buys coverage.

Across 500+ engagements and $2.4B+ in reviewed AWS spend, ML spend is the fastest-growing line in many portfolios and the least disciplined. This guide covers how SageMaker Savings Plans actually work and how to commit to a workload that changes month to month.

SageMaker Savings Plans are a separate instrument

A SageMaker Savings Plan commits to a dollar-per-hour rate on eligible SageMaker usage in exchange for a discount versus SageMaker On-Demand. Critically, Compute Savings Plans do not cover SageMaker, and SageMaker Savings Plans do not cover general EC2. They are distinct commitment pools.

This matters because teams running both general compute and SageMaker need two commitment strategies. A Compute Savings Plan sized to "all our compute" will silently fail to cover the SageMaker line, leaving expensive accelerated-instance usage at full On-Demand.

What SageMaker Savings Plans cover

Eligible usage spans the major SageMaker components — training, real-time and batch inference, processing, and notebook/Studio compute — across instance families including the accelerated families used for deep learning. The commitment floats across these components: usage that shifts from training to inference is still covered, as long as total SageMaker usage clears the committed rate.

The training versus inference shape

The defining challenge of ML commitments is the difference between the two dominant usage modes.

Training is bursty and project-driven. A model trains for hours or days on expensive accelerated instances, then stops. Committing to peak training capacity guarantees over-commitment during the long gaps between training runs.

Inference is steadier — production endpoints serving predictions around the clock. This is the part of the ML workload that resembles traditional compute: a floor of consistent usage that is a good candidate for commitment.

The disciplined approach is to commit against the inference floor, where utilization is high and predictable, and leave bursty training largely On-Demand unless you have a sustained training pipeline that produces a genuine floor of its own.

Sizing a moving target

ML workloads change faster than almost anything else in a typical AWS estate — new models, new accelerated instance generations, shifting from self-managed inference to managed endpoints, or adopting newer AI services entirely. That argues strongly for:

• Shorter terms. One-year SageMaker Savings Plans over three-year, unless a specific inference workload is genuinely stable. The flexibility premium is worth the modest discount give-up when the underlying technology turns over this fast.
• Conservative coverage. Cover the demonstrable inference floor, not the average that bursty training inflates.
• Laddering. Stagger commitments so the portfolio re-prices as new accelerated instances and lower rates arrive — the same logic as our laddering strategy.

SageMaker Savings Plans versus the alternatives

Before committing, confirm the workload belongs in SageMaker at all. Some teams run training on raw EC2 accelerated instances, which are covered by EC2 Instance or Compute Savings Plans, not SageMaker plans. Others use managed SageMaker for the operational tooling. The commitment instrument follows the service:

• SageMaker managed training/inference → SageMaker Savings Plans.
• Raw EC2 accelerated instances → EC2 Instance or Compute Savings Plans.
• Bedrock and other managed AI services → their own pricing and commitment mechanics, separate again.

Getting the mapping right is the first step; sizing comes second. Our broader SageMaker Savings Plans guide and Graviton instance coverage go deeper on the instance-level decisions.

Notebooks and the idle-cost problem

One under-discussed driver of SageMaker spend is notebook and Studio compute that runs when nobody is using it. Data scientists spin up notebook instances for interactive work and frequently leave them running overnight, over weekends, and across vacations. Unlike training jobs, which terminate when complete, notebook instances bill continuously until explicitly stopped. On large data-science teams this idle notebook cost can rival the inference bill.

This matters for commitment strategy because idle notebook usage is not a floor you should commit to — it is waste you should eliminate. Committing a SageMaker Savings Plan against a baseline inflated by idle notebooks locks in the very inefficiency you want to remove. The correct sequence is to fix the operational hygiene first — auto-stop idle notebooks, set lifecycle policies, right-size instance types for interactive work — and only then measure the genuine floor and size a commitment against it. Cleaning up idle notebooks before committing routinely lowers the SageMaker baseline enough that the right commitment is materially smaller than a naive reading of the bill would suggest. The same discipline applies to leftover inference endpoints serving deprecated models: decommission them before you commit, not after, so your Savings Plan covers live production rather than forgotten infrastructure. This is the ML-specific version of the floor discipline that runs through all of our Savings Plans optimization work.

The negotiation dimension for AI spend

Fast-growing ML spend is leverage. Accelerated-instance capacity, SageMaker commitments, and AI service usage are exactly the categories AWS is most motivated to grow, which makes them negotiable inside an EDP or custom pricing arrangement. Buyers scaling AI workloads should bring that growth to the table rather than absorbing list rates — and should plan SageMaker commitments alongside the EDP, as covered in EDP and Savings Plans stacking. Committing to SageMaker in isolation from the broader AI negotiation is a missed opportunity.

Double-buying coverage by accident

The most common SageMaker commitment error after under-coverage is the opposite: paying for the same usage twice. A team buys a SageMaker Savings Plan and, separately, a Compute Savings Plan sized to "all compute" that the buyer assumed included the ML estate. Because the two pools do not overlap, the Compute plan never applies to SageMaker, so it sits partly unused against general compute while the SageMaker plan covers ML. The fix is to size each pool independently against its own floor, confirm which service actually emits the usage, and never assume one Savings Plan type bleeds into another. This separation discipline is the ML corollary of the floor method that runs through every commitment we size.

What to do this quarter

Separate your SageMaker line from general compute in Cost Explorer and chart its hourly usage over 90 days. Identify the inference floor distinct from training bursts. Size a one-year SageMaker Savings Plan to that floor only, and keep bursty training On-Demand until it demonstrates a floor of its own. Re-check quarterly — ML usage shape changes faster than your commitment term.

For an independent review of ML and SageMaker commitments as part of a broader AWS negotiation, Contact Us. See also the Savings Plans optimization guide and our hourly commitment sizing method.