Skip to main content
INS // Insights

Eliminate Databricks Personal Access Tokens

Updated November 2026 · 7 min read

Teams ask for databricks personal access token elimination when a long-lived token, usually minted for a person, still authenticates a workload that should be a service principal using a short-lived federated token. The personal access token sits in a notebook, a CI variable, a partner connection, or a laptop script. It outlives the job that needed it. It authenticates to workspace APIs as the owner, so the grants are that person's grants, not a scope someone designed for the job. Rotation replaces the string and leaves the pattern in place.

Rutagon builds the replacement and the revoke, in that order. Databricks documents the direction in monitor and revoke personal access tokens: use OAuth instead of personal access tokens, then audit what remains. OAuth with a client secret is still a secret. Elimination means the caller presents a token from a workload identity provider, Databricks exchanges it, and no Databricks secret is stored in the pipeline. Workload identity federation migration is the same shape we use when an AWS key is the thing being removed. On a production SaaS platform we operate, CI assumes cloud roles with OpenID Connect and does not keep a personal cloud token. The Databricks cutover is that pattern against a different API.

How databricks personal access token elimination is sequenced

The inventory comes from an administrator, not from a request that each user check a settings page. Account admins can see tokens across the account. Workspace admins can list and revoke tokens in the workspace. Databricks states that managing tokens in the workspace requires the Premium plan or above. A workspace that cannot list tokens is not "clean." It is an inventory gap, and the campaign stays open until an account admin can see the tokens or the plan allows the workspace list. We store token id, owner, comment, created time, and expiry. We do not treat the comment as evidence of which job uses the token. A comment that says "prod etl" is a hint for the owner, not a last-used field.

The replacement identity is a service principal per workload, not one principal for the whole company. A single principal reused by every job recreates the shared login. The principal receives only the workspace entitlements and Unity Catalog grants that job needs. Human PATs are in the same inventory. A person who minted a token for a one-off script is not "solved" by a service principal. That script either becomes a real workload with federation, or the token is revoked and the person uses their normal login. We split the list: workload tokens, human tokens, partner tokens.

Federation is a service principal federation policy. Databricks describes it in configure a federation policy. The policy names the issuer, the audience, and the subject. The subject must match the workload token's subject claim exactly. A policy can omit the audience, and Databricks then defaults the audience to the account id. We set the audience explicitly so a token minted for a different recipient does not match. There is a cap of 20 federation policies on one service principal. That cap is a reason to keep one principal per workload rather than pile every repository onto one identity.

Account-wide token federation is the other mode. It lets users and service principals exchange tokens from the corporate identity provider, often beside SCIM. It is the right replacement for interactive users. It is the wrong first step for a job that should not run as a person. Workload identity federation is the path for CI and scheduled jobs. We do not flip the account-wide switch and call the PAT list done.

service_principal: etl-prod
oidc_policy:
  issuer: https://token.actions.githubusercontent.com
  audiences:
    - https://github.com/acme
  subject: repo:acme/etl:environment:prod
pat_still_in_job: false
successful_federated_run: required before revoke

The subject in that example is the name-only form. Some repositories emit a subject that includes immutable organization and repository ids instead. Databricks matches the policy subject to the claim exactly, so the policy has to use the form the runner actually sends. A copied example that does not match the claim fails closed, which is what we want, and it is not a reason to widen the subject to a wildcard. We do not put a wildcard subject on a federation policy. A subject we cannot name is a policy we do not create.

Disable is not revoke

Workspace personal access token authentication is enabled by default. An admin can turn it off under workspace settings, Advanced, Personal Access Tokens. Databricks documents the effect: tokens cannot be created or used while the setting is off, and turning it off does not delete tokens. If an admin turns the setting back on, non-expired tokens work again. The toggle is a gate. The revoke is a delete of each token id. We record both. A workspace that is "tokens disabled" and still holds a list of non-expired tokens will light those tokens up the moment the gate opens.

We do not close the gate as the first move. Partner connections and older integrations may still present a personal access token. Databricks warns that some of those connections expect token authentication to stay available. The sequence we run is: federated authentication succeeds for the workload, the job configuration no longer contains the token, one scheduled run completes on the service principal, then we revoke that token, and only then we consider the workspace gate. A job that still has "run as" set to the user who owned the token is not migrated. The run-as identity has to be the service principal, or the job is still a person.

OAuth machine-to-machine client secrets are a waypoint, not the end state. They are shorter-lived to manage than a personal access token only if someone rotates them. They are still a Databricks secret in a store. We allow a secret only when the caller cannot present an issuer token, we date that exception, and we list it beside the PAT inventory so it does not become the new unreviewed credential. GitHub deploy key and token elimination is the same discipline on the source-control side: remove the long-lived credential after the federated path is the one that actually runs.

The GRC platform the team already runs can hold the decision and the export. It will not create the federation policy or prove the next job run lacked the token. Security automation is the build that does both and writes the result back.

What we keep as evidence

For each token, the evidence is the owner, the replacement principal, the federation policy subject, the id of a successful run that used federation, the revoke, and a later list that does not contain the token id. A policy without a successful run is not evidence of elimination. It is a plan. A revoke without a policy is an outage followed by someone minting a new token the same day.

Shared tokens are called out. One token used by eight jobs is eight owners and one string. We fan the work out by caller. The token is revoked when the last caller has a federated run, not when the first one does. Leaving the token in place for the straggler is a dated exception with a named owner, not a silent keep.

Human tokens that remain after the workload cutover are a separate campaign with a shorter due date. They are standing credentials on a person. The reviewer is the person's manager. Maintain is rare and expires. The reason code has to name the API that cannot accept OAuth yet. "Convenience" is not a reason code.

Talk to us about a credential elimination pilot for the tokens still sitting in jobs and CI variables. Contact Rutagon at rutagon.com/contact, 907-841-8407, or contact@rutagon.com.

Frequently Asked Questions

Does turning off personal access tokens delete them?

No. Databricks documents that disabling token authentication blocks creation and use, and does not delete existing tokens. If the setting is enabled again, non-expired tokens authenticate again. Revoke is a separate delete of each token id after the replacement path has already run.

Is an OAuth client secret the same as elimination?

No. A client secret is still a long-lived Databricks credential. Federation exchanges a workload token from an issuer you already trust, and the pipeline does not store a Databricks secret. A client secret is a dated exception when the caller cannot present an issuer token.

What has to match on the federation policy?

The issuer, the audience, and the subject. The subject must equal the subject claim the runner sends, including whichever subject form that repository emits. A default audience is the Databricks account id only when the policy omits the audience. We set the audience and refuse wildcard subjects.

Who owns a token whose comment names a team?

The comment is not an owner. The admin list has a created-by identity. That person, or the service owner of the job we can prove still sends the token, decides. A token with no proven caller and no owner is a revoke, not a maintain.