AWS coordinates SageMaker updates and automatic scaling

The AWS example addresses a deployment trap: manually removing automatic scaling before changing an endpoint can leave it unrestored without an error when scaling settings themselves remain unchanged. Custom resources coordinate this. They remove the scaling target before the endpoint changes, register it again only after the endpoint reaches its running state, InService, and attach the scaling policy once the target can be queried. The implementation targets teams deploying endpoints through CDK.

The update documentation narrows when deregistration is required: changing an automatically scaled production variant's instance type or removing that variant. Replacing only its model is allowed, but operators must check that performance and scaling behavior are not significantly affected. During an update, automatic scaling is blocked. AWS describes adding instances for that period as an optional precaution, rather than a requirement for every update.

Automatic scaling changes the number of instances serving a model as demand changes: it adds capacity when workload grows and removes unnecessary instances when workload falls.

For policies using SageMakerVariantInvocationsPerInstance, the load-testing guide starts with one instance and increasing parallel requests, observing throughput and response latency. Capacity depends on instance type, typical inputs and external dependencies. It also calls for testing scaling with both rising and falling traffic.