Amazon MSK IAM authentication lets a producer or consumer assume an IAM role and reach the cluster without a SASL/SCRAM password. IAM authenticates the client and authorizes the topic. The password that used to sit in the client configuration, or in a secret the client fetched at startup, comes out after that client is observed on the IAM path. Rotating the SCRAM password is the control we are leaving. Rotation still leaves a shared secret. The role does not.
Rutagon builds the client cutover one family at a time and files the before-and-after in the GRC you already run. The client owns the cluster. Rutagon builds the identity change. Keeping a SCRAM secret "for emergencies" after every client has moved is how the password returns. The emergency path is the IAM role you can grant for a bounded session, not a second copy of the password.
What IAM access control changes
Amazon MSK can use IAM for both authentication and authorization, so you are not pairing one mechanism to prove identity with another to prove topic rights. AWS describes that in IAM access control. Apache Kafka ACLs can still be called, and they do not authorize IAM identities. The IAM policy does. A deny that you try to fix by editing an ACL will not fix an IAM client. We teach the on-call that split before the first cutover, because the first denied produce will otherwise generate a night of ACL edits.
The setting allow.everyone.if.no.acl.found has no effect on a cluster that uses IAM access control. Teams that relied on that default for a quiet topic have to name the IAM permissions explicitly. That is a behavior change, and it belongs in the test plan for the first client. We would rather see a deny in a lower environment than a produce that succeeded because an ACL default was still in someone's head.
IAM access control does not apply to Apache ZooKeeper nodes on clusters that still have them. A password used only for ZooKeeper is a separate secret. Closing the client SCRAM password does not close that one. We list it so it is not marked done by accident.
Amazon MSK IAM authentication on the client
Java clients can use the AWS_MSK_IAM mechanism. AWS documents the properties in Configure clients for IAM access control. The cutover on a Java client is the security protocol, the mechanism, and the login module:
security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
The trust store line is optional. When it is omitted, the Java process uses its default trust. We do not add a named profile on a production client. A named profile is a laptop habit and it tends to drag a long-lived key back onto the host. Production uses the role the compute platform already delivers to the process. The same role pattern is what we want for other workloads that no longer carry keys.
Non-Java clients use SASL_SSL and the OAUTHBEARER mechanism, with the signer library for that language. AWS documents signer libraries for Python, JavaScript, Go, .NET, and Java. IAM access control for non-Java clients is available on clusters running Kafka 2.7.1 or above. We check the cluster version before we schedule a Python consumer. A client pinned below that floor stays on its current authentication until the cluster can accept the signer, and the password stays with an owner until then.
The SCRAM password leaves that client's configuration only after the client has produced or consumed with the role. Amazon MSK logs access events. We want the IAM identity visible in that audit before the secret is deleted. Deleting the password first, while a shared library still sends SCRAM, is an outage. The listener that still accepts the password stays until the last client has moved. Then the password is removed from the remaining configs and from the secret store those configs read.
Policy, not a shared secret
Authorization is an IAM policy on the role: which cluster, which topic, produce or consume. We start from the topics that client already uses, not from a star on every topic. A role shared by unrelated services is the same smell as the shared password. Split the role when the services do not share a reason to read the same topic. Secrets sprawl remediation is where other copies of the SCRAM password get hunted. Rotation versus elimination is why we do not schedule a rotation Lambda as the finish line.
The evidence pack is the role name, the clients moved, the mechanism each client uses, and the date the SCRAM material was deleted from those clients. "Date" here is the event time on the change, stored as a timestamp, not a slogan. The pack is filed in the GRC you already run. A console checkbox that IAM auth is enabled on the cluster is not evidence that any producer stopped using a password.
Rollback for one client is returning that client to SCRAM while the password listener still exists. Once the password is deleted, rollback is an IAM change, not a paste from a password manager. We keep the password until the last client is observed, and we do not keep it in a ticket comment.
What we leave alone
Cluster sizing, partition count, and the bill are not this cutover. A client that assumes a role and a client that presents a password can cost the same and still be different controls. We do not bury the credential change inside a resizing project. The people who must review the IAM policy are not the people who approve a broker type.
Applications that embed a SCRAM username in a Helm value or a task definition get that value removed in the same change as the mechanism switch. A later deploy from an old chart will put the password back. The chart is in the inventory with the running process. This is AWS cloud infrastructure work on the cluster you already operate. Rutagon's own event pipelines use roles for AWS APIs rather than long-lived keys. The MSK client should be able to say the same, with the SCRAM password gone.
Frequently Asked Questions
Do Kafka ACLs authorize Amazon MSK IAM authentication?
They do not control authorization for IAM identities. IAM policies do. You can still call ACL APIs, and those ACLs will not grant or deny the IAM client. A produce failure after cutover is a policy problem to fix on the role.
When is the SASL/SCRAM password removed?
After that client has produced or consumed with the IAM role, and after no remaining process in that family still loads the password. The password listener stays until the last client moves. Then the secret store entry those clients read is deleted. A cluster flag flipped first is an outage.
What do non-Java clients configure?
SASL_SSL with the OAUTHBEARER mechanism and the signer library for the language. The cluster needs Kafka 2.7.1 or above for non-Java IAM clients. Java clients can use the AWS_MSK_IAM mechanism and the IAM login module AWS documents.
Does this cover ZooKeeper?
IAM access control does not apply to ZooKeeper nodes. A password used there remains its own secret with its own owner. We do not mark it retired because the Kafka clients moved to IAM.
What is the audit artifact?
The role, the clients and their SASL mechanism, and the deletion of the SCRAM material from those clients, filed in the GRC you already run. A cluster setting that shows IAM is enabled does not prove a given producer dropped its password.
Talk to us about your credential elimination pilot. Contact Rutagon at rutagon.com/contact, 907-841-8407, or contact@rutagon.com.