AWS Batch Adds Quota Management for SageMaker Training Jobs
AWS Batch has launched a new quota management capability designed to help organizations divide, share, and prioritize GPU compute resources across multiple machine learning teams and projects.
Radwan AbdulhadiEditor and technology writer
Overview of AWS Batch Quota Management
As artificial intelligence and machine learning operations scale across organizations, coordinating shared graphics processing unit (GPU) capacity can introduce substantial administrative overhead. To address this, <a href="https://aws.amazon.com/batch/">AWS Batch</a> has introduced a new scheduling capability that allows administrators to divide compute capacity into quota shares organized by team, project, or workload.
According to the <a href="https://aws.amazon.com/blogs/machine-learning/lend-and-borrow-resources-across-teams-with-aws-batchs-quota-management-for-sagemaker-training-jobs/">AWS Machine Learning Blog</a>, this new feature works backward from multi-team training environments by providing an automated way to define allocations per team, share idle capacity, and run high-priority workloads first without requiring manual coordination.
Resource-Sharing Strategies and Quota Shares
Each quota share defined within <a href="https://docs.aws.amazon.com/batch/latest/userguide/quota-management.html">quota management</a> includes specific capacity limits and resource-sharing configurations. These strategies dictate how idle compute is handled: it can be reserved until jobs arrive using the RESERVE strategy, lent to other shares using LEND, or configured to both lend when idle and borrow when needed via LEND_AND_BORROW.
When scheduling jobs, AWS Batch prioritizes quota shares that remain under their respective capacity limits. If incoming jobs arrive in a share that has previously lent out its capacity, the system performs cross-share preemption to automatically reclaim those resources without requiring manual intervention from administrators.
Preemption and Priority Handling
Administrators can also turn on priority-based in-share preemption when configuring individual quota shares. While cross-share preemption reclaims lent capacity, in-share preemption targets lower-priority jobs that are currently in the SCHEDULED, STARTING, or RUNNING state. This mechanism ensures that higher-priority jobs waiting at the front of the queue can begin execution much sooner.
For example, background projects might utilize a LEND_AND_BORROW strategy with in-share preemption enabled, allowing higher-priority background tasks to preempt lower-priority tasks within the same share. Conversely, critical workloads can be configured to reserve capacity strictly for themselves without lending or borrowing.
Configuration and SDK Integration
To get started with these capabilities, administrators must configure a <a href="https://docs.aws.amazon.com/batch/latest/APIReference/API_CreateSchedulingPolicy.html">Scheduling Policy</a> containing a quota share policy alongside a <a href="https://docs.aws.amazon.com/batch/latest/APIReference/API_CreateServiceEnvironment.html">Service Environment</a> specifying instance capacity limits.
These components are then connected to <a href="https://docs.aws.amazon.com/batch/latest/APIReference/API_CreateJobQueue.html">a SageMaker Training job queue</a>. Detailed instructions for <a href="https://docs.aws.amazon.com/batch/latest/userguide/create-quota-management-resources.html">configuring quota management resources</a> and establishing <a href="https://docs.aws.amazon.com/batch/latest/userguide/create-quota-shares.html">quota shares</a> are available in the official AWS user guide.
Practical Implementation and Getting Started
The integration supports programmatic job submission via the updated <a href="https://github.com/aws/sagemaker-python-sdk/tree/master/sagemaker-train/src/sagemaker/train/aws_batch">aws_batch</a> module within the SageMaker Python SDK v3. Developers can reference <a href="https://github.com/aws/amazon-sagemaker-examples/blob/default/%20%20%20%20%20%20training/sm-training-queues_getting_started_with_model_trainer/utils/aws_batch_resource_management.py#L571-L707">sample IAM permissions</a> to ensure correct API access when invoking AWS Batch and SageMaker resources.
By leveraging these updated SDK tools and scheduling configurations, machine learning engineers and administrators can efficiently share constrained GPU resources across diverse enterprise projects.
Sources
- AWS Machine Learning BlogLend and borrow resources across teams with AWS Batch’s quota management for SageMaker Training jobs
Continue chronologically
Related entity coverage
Related stories
Manage SageMaker HyperPod Spaces Directly from Studio
Data scientists and machine learning engineers can now create, configure, start, stop, and open Amazon SageMaker Spaces directly from SageMaker Studio.
AWS Launches Amazon SageMaker HyperPod Inference Gateway
Amazon introduces the SageMaker HyperPod Inference Gateway, a GPU-aware routing add-on for EKS designed to slash inference latency for LLM deployments.
AWS Introduces Support for Reflection AI's Beam
Amazon Web Services announces support for Reflection AI's new open-weight agentic coding model, Beam, on Amazon SageMaker.
AWS Introduces aws-ai-ml Skill for Coding Agents
Amazon SageMaker AI introduces the aws-ai-ml skill via Agent Toolkit for AWS, equipping coding agents like Kiro and Claude Code with inference optimization.
