Skip to main content
Version: 17.x

Certificate Authority Rotation

Components of a Teleport cluster authenticate to one another using either X.509 or SSH certificates. To issue certificates, Teleport maintains several certificate authorities. You can rotate Teleport CAs to prevent malicious actors from impersonating part of your Teleport cluster. This guide explains the CAs that Teleport maintains and how to rotate them.

We recommend becoming familiar with the entire guide before following the steps, as you should be ready to roll back the CA rotation if it does not proceed as expected.

How it works​

Teleport maintains its CAs independently of one another, and rotating one CA does not affect the rotation status of the others. The rotation process is designed to take place in phases, which give operators time to update their infrastructure and roll back a CA rotation if necessary.

Teleport CA rotation takes place in five phases for each CA. The phases have the following order:

  1. standby: No rotation in progress. No operations have begun.
  2. init: A new certificate authority is issued, but not used.
  3. update_clients: The Teleport Auth Service uses the new CA to sign new certificates but continues to trust certificates signed by the original CA.
  4. update_servers: Any server components in the cluster that accept incoming connections from clients reload their identity and start serving certificates issued by the new CA. During this phase clients accept server certificates issued by the original CA or the new CA. When rotating the host CA Teleport Agents, Auth Service and Proxy Service instances will automatically reload their identity. OpenSSH hosts must be issued new SSH host certificates during this phase.
  5. standby: No rotation in progress. All operations have completed. All components will stop trusting the old CA and only trust the new CA.

Before the final standby phase, you can also put the rotation in the rollback phase to abort the rotation return to the original certificate authority. After the rollback phase you will then proceed to the standby phase.

CA rotations can be manual or semi-automatic. In manual mode, admins must instruct the Teleport Auth Service to advance from one phase to the next. Between phases, admins can prepare their infrastructure to adjust to each change. In semi-automatic mode, the Teleport Auth Service cycles through each phase automatically, with a grace period between each phase.

In 17.0.5+ tctl auth rotate (with no arguments) starts an interactive terminal UI for CA rotations. The interactive UI displays a live cluster status, allows you to choose a CA to rotate and guides you through each phase, automatically performs certain checks to make sure the cluster is ready for the next phase, and lists any manual steps that need to be completed.

Prerequisites​

  • A running Teleport cluster version 17.4.6 or above. If you want to get started with Teleport, sign up for a free trial or set up a demo environment.

  • The tctl admin tool and tsh client tool.

    Visit Installation for instructions on downloading tctl and tsh.

  • To check that you can connect to your Teleport cluster, sign in with tsh login, then verify that you can run tctl commands using your current credentials. For example, run the following command, assigning teleport.example.com to the domain name of the Teleport Proxy Service in your cluster and email@example.com to your Teleport username: 
    tsh login --proxy=teleport.example.com --user=email@example.com
    tctl status
    Cluster  teleport.example.com
    Version  17.4.6
    CA pin   sha256:abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678abdc1245efgh5678
    If you can connect to the cluster and run the tctl status command, you can use your current credentials to run subsequent tctl commands from your workstation. If you host your own Teleport cluster, you can also run tctl commands on the computer that hosts the Teleport Auth Service for full permissions.

Step 1/4. Choose a CA to rotate​

When rotating a CA, during each phase you should check that any infrastructure that relies on the CA has not lost connectivity. You may also need to export the new CA to your infrastructure, or issue new certificates to any self-hosted services. Choose one of the CAs below to determine how to keep it up to date during the migration.

CA typeCertificate subjects
hostTeleport Agents. Auth Service and Proxy Service instances.
userTeleport users.
dbSelf-hosted databases protected by Teleport (users must distribute certificates to databases).
db_clientThe Teleport Database Service.
opensshOpenSSH servers enrolled in your Teleport cluster.
jwtTeleport users accessing web applications.
saml_idpThe Teleport SAML IdP.
oidc_idpThe Teleport OIDC IdP integration.

host​

The host CA issues certificates to Teleport Agents as well as Auth Service and Proxy Service instances so Teleport clients and the Teleport Auth Service can verify them. The host CA also issues SSH host certificates to any enrolled agentless OpenSSH servers.

Teleport Agents and Proxy Service instances use heartbeats to periodically report their status to the Teleport Auth Service and update their internal data to reflect data held by the Auth Service. This internal data includes the status of the host CA rotation if one is in progress.

To check the rotation status of an agent or Proxy Service instance, run a variation of the following command, assigning resource to the name of an agent or Proxy Service instance:

tctl get resource --format=json | jq '.[] | {hostname: .spec.hostname, rotation: .spec.rotation.state, phase: .spec.rotation.phase}'
{  "hostname": "terminal",  "rotation": "in_progress",  "phase": "init"}

In this example, the Teleport instance named terminal has updated its status to phase init. This means it has downloaded a new CA public key and is ready for state transitions.

You can use the tctl get command with the following resources to determine the rotation state of the host CA on each agent kind:

Roletctl get value
Application Serviceapp_server
Auth Serviceauth_server
Database Servicedb_server
Kubernetes Servicekube_server
Proxy Serviceproxies
SSH Servicenodes
Windows Desktop Servicewindows_desktop_service

During each phase of the host CA rotation, make sure all Agents and Proxy Service instances have completed the transition to target phase before proceeding to the next phase. We will explain the phases in Step 2.

Any OpenSSH hosts must be issued new host certificates during the update_servers phase of the host CA rotation.

Any Teleport processes that connect directly to the Auth Service need a CA pin in order to trust the TLS certificate presented by the Auth Service. This includes all Proxy Service instances and any agents in a self-hosted cluster that connect directly to the Auth Service. During the CA rotation, tctl status will report that there are 2 CA pins. If you are adding any new Teleport Agents to the cluster during the CA rotation you must configure them to trust both reported CA pins. After the rotation is complete, only the new CA pin will be reported.

Note: Teleport Agents that connect to the Proxy Service never need a CA pin because the Proxy's TLS certificate should issued by a trusted CA.

user​

The user CA issues a certificate when a user authenticates to Teleport. It also signs client certificates for users connecting to Windows desktops and Teleport SSH servers. Teleport-protected servers and Windows desktops use these certificates.

Before you complete the rotation and reach the final standby phase, users who have signed into Teleport must reauthenticate to receive a user certificate from the new CA, otherwise web sessions and Teleport client commands will fail and users may have to log out and log back in. To avoid this, you can wait longer than the max user session TTL between the update_clients and standby phases so that all user certificates signed by the old CA will have already expired and required a re-login.

If you have registered Windows desktops with Teleport, follow the guide to export the Teleport user CA so the Windows Desktop Service can authenticate to RDP hosts. Verify that you can connect to registered desktops throughout the rotation.

db and db_client​

The db and db_client CAs issue certificates that the Teleport Database Service uses to communicate with self-hosted databases.

The Teleport Database Service presents a certificate signed by the db_client CA when communicating with a self-hosted database, which an admin configures to trust certificates issued by this CA.

Admins can configure self-hosted databases to present a certificate signed by the db CA, which the Database Service uses to verify that a database server is a genuine Teleport-protected resource. Alternatively, self-hosted databases can present a certificate signed by a custom CA, and admins can configure the Teleport Database Service to trust the CA.

Rotating the database CAs​

These steps provide instructions to rotate both the db and db_client CAs together, but it is also possible to rotate just one or the other and follow the same steps.

Start by rotating both the db and db_client CAs to the init phase. During the init phase, tctl auth sign will issue database server certificates signed by the new db CA keys, and will output a CAs file including both the old and new db_client CA certificates. To avoid losing access to your self-hosted databases at any point, you should reconfigure your databases during the init phase with new certificates and trusted CAs.

note

tctl auth sign --format db is an exception to the usual behavior of the init rotation phase. When the db CA is in the init phase, tctl auth sign --format db will issue database server certificates signed by the new CA keys. This is so that self-hosted databases only need to be reconfigured twice during a CA rotation: first during the init phase to get a certificate signed by the new db CA and start trusting the new db_client CA, and second during the final standby phase to stop trusting the old db_client CA.

Consult the appropriate documentation for configuring your databases before proceeding to the update_clients rotation phase.

As soon as you proceed to the update_clients phase, the Teleport Database Service will start using client certificates issued by the new db_client CA to connect to databases. Verify that you can still access your databases before and after transitioning both CAs to the update_clients phase.

If all is well, proceed rotating both CAs to the update_servers and standby phases. After reaching the standby phase, you may once again reconfigure your databases to stop trusting the old CA certificate that has now been rotated out.

Rolling back the rotation​

The most common reason you would want to roll back is if you cannot reconfigure your databases. If you have connectivity issues after reconfiguring a database, it's likely that you misconfigured the database.

If you reconfigured any of your databases during the rotation, you will need to reconfigure them again during the rollback phase before proceeding to the standby phase.

openssh​

The openssh CA issues ephemeral SSH user certificates that the Proxy Service uses to authenticate to OpenSSH servers registered with Teleport. The OpenSSH agent verifies these certificates when it receives incoming connections from the Proxy Service.

During the init phase of openssh CA rotation all OpenSSH servers must be updated to trust the new CA public key in addition to the existing public key. This is necessary to avoid any loss of connectivity when the Proxy Service starts using certificates signed by the new CA keys during the update_clients phase.

If you used the manual method to enroll any OpenSSH servers, you must follow the instructions to export the new openssh CA public key and provide it to your OpenSSH servers before you transition the rotation to the update_clients phase.

If you used the automated method you should reconfigure sshd by following the same steps before proceeding to the update_clients phase.

note

OpenSSH servers use SSH host certificates issued by the host CA and trust incoming certificates issued by the openssh CA. Make sure you also reconfigure OpenSSH servers with a new host certificates when rotating the host CA during the update_servers phase.

jwt​

The Teleport Auth Service uses the jwt CA to sign JSON web tokens. The Teleport Application Service includes JSON web tokens in HTTP messages that it forwards to Teleport-protected applications, which use the jwt CA to verify the tokens.

If you have enrolled web applications with Teleport, and those applications authenticate traffic from the Teleport Application Service by verifying JSON web tokens against the Teleport certificate authority, you need to ensure that these applications continue to trust the rotated CA.

Teleport-protected JWT applications use one of two methods to retrieve the public key of the Teleport jwt CA. Depending on the method, you may need to take action after the init phase and before the rotation reaches the final standby phase:

  • The application queries the /.well-known/jwks.json endpoint of the Teleport Proxy Service. In this case, no action is required as long as the application can continue to access the endpoint. If the application caches jwks.json, invalidate the cache.
  • The application accesses the jwks.json file on the local filesystem. Obtain a new jwks.json file by querying the /.well-known/jwks.json endpoint and re-uploading the file.

For an example of exporting the jwt CA so a web application can trust Teleport-issued JWTs, see the guide to using JWT authentication with Elasticsearch.

saml_idp​

The saml_idp CA signs SAML messages sent by the Teleport IdP so services that rely on the Teleport IdP can verify them.

If you are rotating this CA, then before entering the update_clients phase, you must configure any service providers that rely on the Teleport SAML IdP to trust the Teleport saml_idp CA. Follow the instructions in the SAML IdP documentation to export an XML metadata file and make it available to your service provider.

oidc_idp​

The oidc_idp CA signs messages sent by the Teleport OIDC IdP integration. Relying parties (e.g., AWS) verify these messages to authenticate your Teleport account for features like External Audit Storage, Auto-Discovery, and AWS Sync for Access Graph.

The Teleport Proxy Service serves the JSON Web Key Sets for the OIDC IdP integration from the /.well-known/jwks-oidc path of the Web API.

The /.well-known/jwks-oidc path of the Teleport Proxy Service Web API is always enabled. The Teleport Proxy Service updates the endpoint automatically.

You can retrieve the full URL of the integration's JSON Web Key Sets by querying the /.well-known/openid-configuration path of the Web UI and reading the jwks_uri field:

curl https://example.teleport.sh/.well-known/openid-configuration | jq '.jwks_uri'
"https://example.teleport.sh/.well-known-jwks-oidc"

Step 2/4. Start a manual rotation​

Once you have chosen a CA to rotate and have planned to check or update the infrastructure that relies on that CA, you are ready to begin a manual rotation.

tip

In 17.0.5+ tctl auth rotate (with no arguments) starts an interactive terminal UI for CA rotations. The interactive UI displays a live cluster status, allows you to choose a CA to rotate and guides you through each phase, automatically performs certain checks to make sure the cluster is ready for the next phase, and lists manual steps that need to be completed. We recommend using the interactive rotation whenever possible, but you can read on to learn how to manually initiate each rotation phase.

init phase​

In the init phase, the Teleport Auth Service issues a new certificate authority of the chosen type, but does not use it to sign certificates.

  1. Initiate the manual rotation of host certificate authorities:

    tctl auth rotate --manual --type=type --phase=init
    Updated rotation phase to "init". To check status use 'tctl status'

  2. Use tctl to confirm that there is an active rotation in progress. This command prints the rotation status of all CAs that the Teleport Auth Service maintains in your cluster:

    tctl status
    Cluster       teleport.example.comVersion       17.4.6CA pins: sha256:0000000000000000000000000000000000000000000000000000000000000000         sha256:1000000000000000000000000000000000000000000000000000000000000000
    authority rotation                                protocol status  algorithm   storage--------- --------------------------------------- -------- ------- ----------- --------host      in progress (mode: manual, phase: init) SSH      active  Ed25519     software                                                  SSH      trusted Ed25519     software                                                  TLS      active  ECDSA P-256 software                                                  TLS      trusted ECDSA P-256 softwareuser      standby (never rotated)                 SSH      active  Ed25519     software                                                  TLS      active  ECDSA P-256 softwaredb        standby (never rotated)                 TLS      active  RSA 2048    softwaredb_client standby (never rotated)                 TLS      active  RSA 2048    softwareopenssh   standby (never rotated)                 SSH      active  Ed25519     softwarejwt       standby (never rotated)                 JWT      active  ECDSA P-256 softwaresaml_idp  standby (never rotated)                 TLS      active  RSA 2048    softwareoidc_idp  standby (never rotated)                 JWT      active  RSA 2048    softwarespiffe    standby (never rotated)                 JWT      active  RSA 2048    software                                                  TLS      active  ECDSA P-256 softwareokta      standby (never rotated)                 JWT      active  ECDSA P-256 software

  3. Perform checks and updates on your infrastructure, depending on the CA type.

update_clients​

Execute the transition from init to update_clients. In this phase, The Teleport Auth Service uses the new CA to sign certificates but continues to trust certificates signed by the original CA.

  1. Transition to the update_clients phase:

    tctl auth rotate --manual --type=type --phase=update_clients
    Updated rotation phase to "update_clients". To check status use 'tctl status'
    tctl status
    Cluster       teleport.example.comVersion       17.4.6CA pins: sha256:0000000000000000000000000000000000000000000000000000000000000000         sha256:1000000000000000000000000000000000000000000000000000000000000000
    authority rotation                                          protocol status  algorithm   storage--------- ------------------------------------------------- -------- ------- ----------- --------host      in progress (mode: manual, phase: update_clients) SSH      active  Ed25519     software                                                            SSH      trusted Ed25519     software                                                            TLS      active  ECDSA P-256 software                                                            TLS      trusted ECDSA P-256 softwareuser      standby (never rotated)                           SSH      active  Ed25519     software                                                            TLS      active  ECDSA P-256 softwaredb        standby (never rotated)                           TLS      active  RSA 2048    softwaredb_client standby (never rotated)                           TLS      active  RSA 2048    softwareopenssh   standby (never rotated)                           SSH      active  Ed25519     softwarejwt       standby (never rotated)                           JWT      active  ECDSA P-256 softwaresaml_idp  standby (never rotated)                           TLS      active  RSA 2048    softwareoidc_idp  standby (never rotated)                           JWT      active  RSA 2048    softwarespiffe    standby (never rotated)                           JWT      active  RSA 2048    software                                                            TLS      active  ECDSA P-256 softwareokta      standby (never rotated)                           JWT      active  ECDSA P-256 software

  2. Check or update infrastructure that depends on your CA before proceeding to the next step.

  3. If you lose connectivity to your resources, see if you need to reconfigure them to accept the new CA. If that does not restore access or you are unable to reconfigure a database, then roll back to the original certificate authority.

update_servers​

Initiate the update_servers phase. In this phase, Teleport cluster components (Agents, Auth Service, and Proxy Service instances) reload and start serving TLS and SSH certificates signed by the new certificate authority, but still accept certificates issued by the original certificate authority. This phase only affects the host CA.

  1. Execute the transition:

    tctl auth rotate --manual --type=type --phase=update_servers
    Updated rotation phase to "update_servers". To check status use 'tctl status'

    tctl status
    Cluster       teleport.example.comVersion       17.4.6CA pins: sha256:0000000000000000000000000000000000000000000000000000000000000000         sha256:1000000000000000000000000000000000000000000000000000000000000000
    authority rotation                                          protocol status  algorithm   storage--------- ------------------------------------------------- -------- ------- ----------- --------host      in progress (mode: manual, phase: update_servers) SSH      active  Ed25519     software                                                            SSH      trusted Ed25519     software                                                            TLS      active  ECDSA P-256 software                                                            TLS      trusted ECDSA P-256 softwareuser      standby (never rotated)                           SSH      active  Ed25519     software                                                            TLS      active  ECDSA P-256 softwaredb        standby (never rotated)                           TLS      active  RSA 2048    softwaredb_client standby (never rotated)                           TLS      active  RSA 2048    softwareopenssh   standby (never rotated)                           SSH      active  Ed25519     softwarejwt       standby (never rotated)                           JWT      active  ECDSA P-256 softwaresaml_idp  standby (never rotated)                           TLS      active  RSA 2048    softwareoidc_idp  standby (never rotated)                           JWT      active  RSA 2048    softwarespiffe    standby (never rotated)                           JWT      active  RSA 2048    software                                                            TLS      active  ECDSA P-256 softwareokta      standby (never rotated)                           JWT      active  ECDSA P-256 software

  2. Configure and check resources depending on the CA you are rotating. This is your final chance to update Teleport-protected resources before transitioning to the standby phase.

  3. If you have lost connectivity to Teleport-protected resources, roll back to the original certificate authority before entering the final standby phase, when rolling back is no longer possible.

Final standby​

Before wrapping up, verify that you have not lost access to Teleport-protected resources that rely on the CA that you rotated.

  1. Execute the transition:

    tctl auth rotate --manual --type=type --phase=standby

  2. Verify that the rotation has completed with tctl:

    tctl status
    Cluster       teleport.example.comVersion       17.4.6CA pins: sha256:0000000000000000000000000000000000000000000000000000000000000000
    authority rotation                                        protocol status algorithm   storage--------- ----------------------------------------------- -------- ------ ----------- --------host      standby (last rotated: Apr 4 2025 10:14:51 UTC) SSH      active Ed25519     software                                                          TLS      active ECDSA P-256 softwareuser      standby (never rotated)                         SSH      active Ed25519     software                                                          TLS      active ECDSA P-256 softwaredb        standby (never rotated)                         TLS      active RSA 2048    softwaredb_client standby (never rotated)                         TLS      active RSA 2048    softwareopenssh   standby (never rotated)                         SSH      active Ed25519     softwarejwt       standby (never rotated)                         JWT      active ECDSA P-256 softwaresaml_idp  standby (never rotated)                         TLS      active RSA 2048    softwareoidc_idp  standby (never rotated)                         JWT      active RSA 2048    softwarespiffe    standby (never rotated)                         JWT      active RSA 2048    software                                                          TLS      active ECDSA P-256 softwareokta      standby (never rotated)                         JWT      active ECDSA P-256 software

  3. Follow the instructions for your CA to ensure that you can connect to Teleport-protected resources that rely on the CA that you rotated. This is the last stage where you have the opportunity to roll back. If you have lost connectivity to Teleport-protected resources, roll back to the original certificate authority.

Step 3/4. [Optional] Run a semi-automatic rotation​

You can instruct Teleport to manage the CA rotation semi-automatically. Semi-automatic rotation transitions between the phases of a rotation for you, and there is no need to run a tctl auth rotate command for each phase. After a grace period elapses, the Teleport Auth Service updates the phase of the CA rotation to the next step.

Determine whether to use a semi-automatic rotation​

Aside from automatic phase updates, a semi-automatic rotation is identical to a manual one. It is up to the operator to update any infrastructure to accommodate the current phase before the grace period elapses.

Teleport does not check the status of any infrastructure that relies on the CA, meaning that you can lose connectivity if things go wrong. As a result, you should not carry out a semi-automatic rotation if you need to export a CA to your infrastructure.

Complete a rotation in manual mode first to understand all the edge-cases and hazards before attempting a semi-automatic rotation.

Initiate a semi-automatic rotation​

If you want to run a semi-automatic rotation, initiate it with tctl and monitor the status of the rotation.

You can trigger semi-automatic rotation with the following command:

tctl auth rotate --type=type

The command triggers a rotation process for hosts with a default grace period of 48 hours.

Configuring the grace period

You can customize grace period and CA type with additional flags:

Rotate only user certificates with a grace period of 200 hours:
tctl auth rotate --type=user --grace-period=200h

Rotate only host certificates with a grace period of 8 hours:
tctl auth rotate --type=host --grace-period=8h

Be careful when choosing a grace period when rotating the host CA.

The grace period needs to be long enough for all Agents and Proxy Service instances in a cluster to request a new certificate. If some hosts go offline during the rotation and come back only after the grace period has ended, they will be forced to leave the cluster.

During semi-automatic rotations, Teleport attempts to divide the grace period so that it spends an equal amount of time in each phase before transitioning to the next phase. This means that using a shorter grace period will result in faster state transitions.

Step 4/4. [Optional] Roll back the rotation​

You must perform a rollback before the rotation enters standby state.

  1. Enter the rollback phase with a manual phase transition:

    tctl auth rotate --phase=rollback --type=type --manual
    Updated rotation phase to "rollback". To check status use 'tctl status'

  2. Ensure that you can connect to Teleport resources that depend on the CA you were rotating.

  3. Finish rolling back the CA rotation:

    tctl auth rotate --phase=standby --type=type --manual
    Updated rotation phase to "standby". To check status use 'tctl status'

Further reading​

How Teleport certificate authorities work.