Apps & Software

Orchestrating SageMaker AI Auto Scaling with AWS CDK

A new architectural approach uses AWS CDK and CloudFormation custom resources to seamlessly manage Amazon SageMaker AI real-time endpoint updates and auto scaling configurations in a single deployment.

Radwan AbdulhadiEditor and technology writer

Published: October 23, 2026 · Updated: October 23, 2026 · 3 min read

Orchestrating SageMaker AI Auto Scaling with AWS CDK — Image: AWS Machine Learning Blog

The Challenge of Updating SageMaker AI Endpoints

Updating real-time endpoint configurations in Amazon SageMaker AI—such as changing instance types, instance counts, or environment variables—frequently causes deployments managed through AWS CloudFormation or the AWS Cloud Development Kit (AWS CDK) to fail. Because SageMaker AI requires users to deregister scalable targets before updating endpoints, native CloudFormation templates lose track of the underlying AWS Application Auto Scaling resources.

When attempting standard updates with auto scaling active, SageMaker AI blocks changes before CloudFormation reaches the scaling resources. Conversely, manually deregistering targets via the command line can leave stacks in broken states or running without auto scaling entirely, often triggering a rollback failure that requires manual console intervention.

Automating Orchestration with Reusable Constructs

To solve these deployment blockers, developers can implement a specialized architecture detailed on the [AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/orchestrating-sagemaker-ai-endpoint-auto-scaling-with-aws-cdk/). This approach replaces native CloudFormation resources with three dedicated custom resources managed inside a reusable construct.

The custom resources coordinate the entire update workflow using explicit dependency ordering enforced by programmatic calls within the CDK. On initial deployment, the workflow handles missing targets gracefully, allowing seamless transitions when updating production environments.

Step-by-Step Execution Sequence

The complete deployment sequence begins with a deregistration custom resource that removes the scalable target before endpoint modifications start. Next, the standard CDK-managed update modifies the endpoint configuration itself.

Following the endpoint update, a register scalable target resource waits for the endpoint to reach InService status before re-registering the target. Finally, an attach scaling policy resource applies target tracking parameters once the scalable target becomes fully queryable.

Asynchronous Completion and Provider Frameworks

To manage timing constraints across infrastructure updates, the registration and scaling policy components utilize the provider's asynchronous completion pattern. Each custom resource features an on_event handler that returns immediately alongside an is_complete handler.

A framework waiter re-invokes readiness checks at regular thirty-second intervals, bound by total timeouts of one hour for target registration and fifteen minutes for scaling policies. A unified configuration hash ensures that modifications to parameters automatically force CloudFormation to re-invoke the underlying handler functions.

Getting Started with the Walkthrough

Developers looking to test this deployment pattern can explore resources and implementation scripts available in the [linked GitHub repository](https://github.com/aws-samples/sample-sagemaker-endpoint-autoscaling). The walkthrough requires an active AWS account with appropriate permissions for SageMaker AI, Lambda, IAM, Application Auto Scaling, and CloudWatch.

Prerequisites also include model artifacts stored in Amazon S3 alongside a container image in Amazon ECR. After cloning the repository, developers can bootstrap their environment and execute deployments cleanly using standard command-line tools.