SageMaker idle inference endpoint cost is the hosting instance-hour that continues while a real-time endpoint has no invocations. Amazon SageMaker charges real-time inference for the instance type you leave in service. Invocations are not the meter. A model that nobody calls, on an endpoint that is still InService, bills the same hours as a model that is busy, plus the storage attached to instances that use it. The data-processed line is the part that goes quiet.
Rutagon finds this on demo endpoints, on shadow variants left after a launch, and on "production" endpoints whose callers were deleted. AWS cloud infrastructure work includes reading the hosting line before the training line. The SageMaker pricing page is the source for the examples below. Serverless cost is a different bill, Lambda duration and memory. This article is the Host: usage type.
Where SageMaker idle inference endpoint cost is billed
Real-time hosting usage types look like USE1-Host:ml.c5.xlarge for the instance, USE1-Host:VolumeUsage.gp2 for the volume SageMaker attaches when the instance does not already have local SSD, and Hst:Data-Bytes-In and Hst:Data-Bytes-Out for bytes through the endpoint. Filter Cost Explorer on Host: and you see the instance hours. Filter the data-byte types and you see traffic. An idle endpoint keeps the first and drops the second. Instance families with local NVMe do not get that extra gp2 volume. The instance hour is still there.
The pricing page's real-time example deploys a model to two ml.c5.xlarge instances for multi-AZ hosting. The hour count is 24 * 31 * 2 = 1488. At $0.204 per hour that hosting subtotal is $303.522. The same example moves 100 MB in per day and treats inferences as a tenth of that, priced at $0.016 per GB, about five cents of data against three hundred dollars of instances. Zero invocations remove the five cents. They do not remove the $303.522 if those two instances stay up for the month. The page states the rule in one line: real-time inference charges you for the instance type you choose.
bill: REGION-Host:instanceType for each instance hour the endpoint keeps
bill: REGION-Host:VolumeUsage.gp2 when an EBS volume is attached
bill: Hst:Data-Bytes-In and Hst:Data-Bytes-Out, which follow invocations
example: 2 x ml.c5.xlarge x 24 x 31 hours = 1488 x $0.204 = $303.522
idle: instance hours remain, data-processed dollars do not A Savings Plan can discount eligible real-time instance usage. The hour is still consumed. An idle endpoint draws down the commitment you bought for traffic you thought you had. The discount does not detect that Invocations is zero. We look at the usage type first and the discount second.
Serverless inference is a different product on the same pricing page. On-demand serverless charges for the compute used to process requests, by the millisecond, and for data processed. There is no Host:ml.c5.xlarge line, because you did not leave an instance in service. Provisioned concurrency on a serverless endpoint is the warm capacity you asked to keep, priced from the memory size, the time you provisioned, and the concurrency you set. That charge is real, and it is not the idle real-time instance hour. Mixing the two names is how a team "moves to serverless" and keeps paying for warmth they configured on top.
Asynchronous inference charges for the instances the endpoint uses, and the pricing page's example scales the instance count to zero when it is not processing. The worked case is one ml.c5.xlarge up 2 hours a day plus a 30-minute cooldown, so 2.5 hours a day, 2.5 * 31 = 77.5 hours. That example prices the instance at $0.20 per hour, $15.81 for the month, plus the storage on the volume. The cooldown is the part people miss. Scale-to-zero does not stop the clock on the last request. It stops after the cooldown. We quote that example as published. We do not blend its $0.20 with the real-time example's $0.204.
Batch transform charges for the instance for the duration of the job. There is no endpoint left behind when the job ends. A nightly batch that was implemented as a real-time endpoint with zero daytime invocations is the idle pattern. The job's hours replace the month's hours.
The engineering fix on a zero-invocation endpoint
We join Host: hours to the Invocations metric on AWS/SageMaker for that endpoint name. A full month of instance hours and a sum of invocations at zero is an endpoint that can go away or scale to zero instances. Deleting the endpoint stops the hosting line and the volume line. Leaving the endpoint and scaling the in-service count to zero also stops the instance charge. New requests then fail until SageMaker provisions an instance again, and that provisioning takes minutes. Callers in that window get errors. That trade is acceptable for a demo, a nightly path, or an internal tool. It is the wrong trade for a user-facing model that must answer immediately. Those endpoints keep their instances, and the idle question does not apply, because invocations are not zero.
Shadow variants and old production variants are the usual leftovers. A variant that receives no traffic still has a weight or an instance count. We set the unwanted variant's instance count to zero or we delete it. The endpoint that remains is the one the application invokes. Two ml.c5.xlarge instances in the pricing example are a multi-AZ choice for a model that is called. Copying that shape onto an endpoint with no caller copies the $303.522 shape without the five cents of data.
Model Monitor is a separate instance line. The pricing page includes up to 30 hours of built-in monitoring rules each month at no charge, aggregated across endpoints, and charges the instance type after that. A monitor schedule on an endpoint with no invocations still runs the schedule. We disable the schedule when we delete or zero the endpoint. Otherwise the Host: line goes away and a monitor instance remains on a timer.
The async path is the fix when the caller can wait and the payload is large. The endpoint scales out for work and back to zero after the cooldown, and the bill looks like the 77.5-hour example instead of 1,488 hours. We keep the cooldown in the estimate. Serverless on-demand is the fix when the traffic is spiky, the latency can move, and we do not want a hosting instance at all. We do not add provisioned concurrency "just in case" and then describe the endpoint as idle-free. Provisioned concurrency is the case.
Real-time stays when the latency has to be steady and the invocations justify the instances. The fix there is the instance count and the size, checked against invocations that are not zero. An idle endpoint is not a rightsizing problem. Rightsizing a server that receives no calls still leaves a server. Removal, or scale-to-zero with the error window understood, is the change.
What we stop confusing with this line
Training jobs, notebook hours, and processing jobs use instance hours too, and they stop when the job stops. They do not show up as Host:. A review that sorts all SageMaker spend together will "optimize" a training run that already ended and miss the endpoint that has been InService since the proof of concept. The usage type is the filter.
Data processed at $0.016 per GB in the real-time example will not explain a three-hundred-dollar line. We do not chase payload compression on an endpoint whose invocation sum is zero. Compression matters when the data line is the one that moved. Here the instance hour moved, and the data line did not.
Rutagon runs this as a short bill review: Host: by instance type, volume usage beside it, Invocations per endpoint, monitor schedules on the zeros, and a decision per endpoint to delete, scale to zero, move to async or serverless, or keep the instances because the callers are real. The pricing page is the source for the 1,488 hours at $0.204 and the async 77.5 hours at $0.20. The metric is the source for whether anyone called the model.
Frequently Asked Questions
Which usage type is the idle endpoint?
Host: plus the instance type, for example USE1-Host:ml.c5.xlarge, is the real-time instance hour. Host:VolumeUsage.gp2 is the attached volume on instance types that get one. Hst:Data-Bytes-In and Hst:Data-Bytes-Out track traffic and fall when invocations do. Zero invocations do not fall the Host: line.
Does a Savings Plan make an idle endpoint free?
A Savings Plan can discount eligible real-time instance hours. The hours still count against the commitment. An endpoint with no invocations keeps drawing that commitment for as long as the instances stay in service. The fix is to remove the instances, not to buy a larger plan.
What does scale-to-zero change for callers?
Instance hours stop once the in-service count is zero. The next invocation fails until an instance is provisioned, which takes minutes. Async endpoints in the pricing example also bill a cooldown, 30 minutes in that writeup, before the count drops. We use scale-to-zero when callers can take the error or the wait.
How is serverless inference billed when nobody calls?
On-demand serverless inference bills compute for the time spent processing requests, and bills data processed. No request means no compute charge of that kind. Provisioned concurrency is separate: you pay for the concurrency you asked to keep warm. A real-time Host: hour and a serverless provisioned-concurrency charge are different lines, and an idle review has to say which one is on the endpoint.
Ask for an AWS bill review that joins SageMaker Host: hours to endpoints with no invocations. Contact Rutagon at rutagon.com/contact, 907-841-8407, or contact@rutagon.com.