HCP Terraform workspaces that still store AWS_ACCESS_KEY_ID are standing cloud admin in a variable set. Mid-market SaaS teams rotate those keys and still leak them into run logs, state backends, and leftover variable versions. Rutagon builds Terraform Cloud dynamic credentials AWS as an OIDC cutover: plan and apply assume separate IAM roles, then the workspace static keys are deleted. This is not a GitHub Actions rewrite. The identity provider is HCP Terraform. The pipelines stay where they are.
The buyer pain is a workspace that can create production resources with a user key that three former contractors still remember. Dynamic credentials replace that user with a role that only HCP Terraform can assume, with a subject that names the organization, project, and workspace.
We already ship GitLab OIDC to AWS for GitLab runners. The pattern is related and the trust policy is not the same document. Copy-pasting a GitHub Actions role onto Terraform Cloud is how apply starts failing with a subject mismatch.
Terraform Cloud Dynamic Credentials AWS for Plan and Apply
Terraform Cloud dynamic credentials AWS means HCP Terraform presents an OIDC token at plan and at apply, and AWS STS issues short-lived credentials for a role you name. We create an OIDC provider for the HCP Terraform issuer, then two roles when the blast radius should differ: a read-mostly plan role and a tighter apply role. Variable sets lose the access keys. The AWS provider uses the web identity that HCP Terraform injects.
Plan and apply are not the same permission set. Plan should not be able to delete the production database. Apply should not be assumed by a speculative plan from a fork workspace you did not intend. We encode workspace and phase in the role trust, then we prove it with a failed assume from the wrong workspace.
Static keys stay until that proof exists. We do not delete the variable on Monday and hope Tuesday’s apply works. The cutover run uses dynamic credentials on a clone workspace or a tagged run, then production, then key deactivate.
What We Delete and What We Keep
We delete IAM user access keys that existed only to serve that workspace. We keep the OIDC provider and the roles. We keep CloudTrail. We do not keep a “backup key” in a different variable set. That backup is the old control.
If a workspace also talked to a second cloud or a SaaS API with a static token, that token is a separate elimination. This engagement is AWS credentials for Terraform Cloud. Mixing a GitHub Actions migration into the same change set is out of scope here; use the access-key to OIDC path for those runners.
State remains in your existing backend. Dynamic credentials do not require a state migration. If state was reachable only because the old user key was over-broad, we notice that during the call-set inventory and we fix the backend policy as part of the role, not as a surprise.
workspace: prod-network
was: variable_set.aws_access_key
now:
oidc_issuer: hcp-terraform
plan_role: terraform-plan-prod-network
apply_role: terraform-apply-prod-network
prove:
wrong_workspace_assume: denied
key_status: deleted
AWS documents OIDC identity providers for this trust. We use that mechanism. We do not invent a long-lived session token stored as a “dynamic” variable.
Evidence and Rollback
Evidence is the provider ARN, the trust policies, a successful plan and apply principal in CloudTrail, a denied assume from a throwaway workspace, and the IAM user key deletion timestamp. Variable set versions that still show the key names are archived as before-state, not left live.
Rollback is re-enabling a key you hoped you would never need. We treat that as an incident, not a documented operating mode. If apply fails after cutover, we fix the role or the provider configuration. We do not paste the old key back into the variable set to “unblock Friday.”
AWS cloud infrastructure covers the account layout when plan and apply roles must live in a shared services account and assume into workload accounts. Cross-account assume is part of the design when you already use that layout. It is not a reason to keep a user key in the workspace.
Variable sets, run agents, and the workspace that was never in the inventory
HCP Terraform variable sets are how one key infects twenty workspaces. We inventory every set that contains AWS key names, including sets attached at the project or organization layer. A workspace-level delete that leaves the org set in place is not a cutover. The apply will still see the static key and may prefer it. We remove the key from every set that the target workspaces inherit, then we prove a run without those variables present.
Agent-backed runs need the same identity story. If you run Terraform on a self-hosted agent that already has an instance role, we do not add a second standing user “for Terraform Cloud.” Either HCP Terraform’s OIDC assume is the principal, or the agent role is the principal, not both with a user key as a tie-breaker. Two principals plus a leftover key is how nobody can explain CloudTrail.
Workspaces that were never tagged as production still matter. A “playground” workspace with the production key is the leak. The inventory is all workspaces that can reach the account, not the ones in the official module repo. We include no-code and discarded workspaces that still have variable values. Discarded is not deleted.
Module registries and VCS connections can hold their own tokens. Those tokens are not AWS access keys and they are out of this pilot unless they were used to mint AWS keys. We note them so nobody claims the org is credential-clean. The AWS close is the OIDC roles plus deleted user keys.
State locking and the backend still need permission on the apply role. We put those actions on the apply role, not on a leftover user. If plan must read state, the plan role gets read. If plan can write state, you have already given plan apply-shaped power. We call that out in the packet.
A failed first apply after cutover is usually a missing IAM action or a wrong subject claim, not a reason to restore the variable set. We keep the old key offline in a break-glass store you already use for incidents, with a ticket, not in HCP Terraform. Putting it back in the variable set is the rollback we refuse to document as normal.
Frequently Asked Questions
Is this the same setup as GitHub Actions OIDC?
No. The issuer, the audience, and the subject claims differ. We write a trust policy for HCP Terraform, not for token.actions.githubusercontent.com. Reusing a GitHub Actions role is a common failed apply. This engagement does not rewrite your GitHub workflows.
Do we need one role per workspace?
We split when workspaces have different blast radius or different account targets. A single role for every workspace is how a sandbox apply becomes production. The diagnostic decides the split. The deliverable names the roles you got.
What happens to existing AWS provider environment variables?
They are removed from the workspace and the variable set after the OIDC run is proven. Leftover AWS_ACCESS_KEY_ID in the workspace wins over the role in some provider versions. Leftover is a failed close.
Can plan use the apply role to save IAM objects?
It can, and we advise against it. Plan with apply permissions turns every speculative run into a privileged principal. Separate roles are the default we ship unless you record a reason.
Talk to us about your credential elimination pilot. Call 907-841-8407, email contact@rutagon.com, or use rutagon.com/contact.