Bots occasionally fail to join with `missing identity groups or scope pin` due to invalid `user_login_state`

[timothyb89]

Expected behavior:

Bots should join the cluster and receive certificates successfully.

Current behavior:

Occasionally for reasons still not known, an invalid /user_login_state/bot-foo entry is created in the backend for bots. For example:

{
  "kind": "user_login_state",
  "version": "v1",
  "metadata": {
    "name": "bot-ansible",
    "expires": "0001-01-01T00:00:00Z"
  },
  "spec": {
    "original_roles": null,
    "original_traits": null,
    "roles": null,
    "traits": null,
    "user_type": "local"
  }
}

Note the "roles": null. When present, the authentication process preferentially returns this cached (and invalid) user login state over user roles defined on the user, which results in certificates with no defined groups, such as in this x509 subject:

Subject: L=-teleport-internal-join + L=-teleport-nologin-f1b5021e-5195-4ba7-a7c9-b0500e0d1e70, street=example.teleport.sh, postalCode=null, CN=bot-ansible, 1.3.9999.1.7=example.teleport.sh, 1.3.9999.1.9=10.0.47.68, 1.3.9999.1.14=1, 1.3.9999.2.18=ansible, 1.3.9999.2.20=80a06b50-1674-43c4-fa9d-a545e7581ef4, 1.3.9999.2.23=example-token, 1.3.9999.1.20=local, 1.3.9999.1.15=none, 1.3.9999.2.21={"meta":{"join_token_name":"example-token","join_method":"bound_keypair"}}

Note the lack of an O = ..., indicating no attached roles. When tbot attempts to parse the returned subject into an identity, it fails:

$ tbot start identity --storage=./storage --destination=./destination --allow-reissue --proxy-server example.teleport.sh:443 --join-method=token --token=foo
2026-02-17T17:24:30.044-07:00 INFO [TBOT]      Initializing tbot version.teleport:18.6.8 version.teleport_git:v18.6.8-0-g9a94677 version.go:go1.25.7 tbot/tbot.go:126
2026-02-17T17:24:30.044-07:00 INFO [TBOT]      Anonymous telemetry is not enabled. Find out more about anonymous telemetry at https://goteleport.com/docs/reference/machine-workload-identity/machine-id/telemetry/ tbot/anonymous_telemetry.go:87
2026-02-17T17:24:30.045-07:00 INFO [TBOT]      Created directory path:./storage-token destination/directory.go:144
2026-02-17T17:24:30.045-07:00 INFO [TBOT]      Created directory path:./destination-token destination/directory.go:144
2026-02-17T17:24:30.045-07:00 WARN [TBOT]      Secure symlinks not supported on this platform, set `symlinks: insecure` to disable this message path:storage-token/.write-test botfs/fs_unix.go:55
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Initializing bot identity identity/service.go:292
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Loading existing bot identity from store store:directory: ./storage-token identity/service.go:207
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] No existing bot identity found in store identity/service.go:212
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Bot was unable to load a valid existing identity from the store, will attempt to join using configured token identity/service.go:302
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Fetching bot identity using token identity/service.go:700
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Trying to join with the new join service joinclient/join.go:74
2026-02-17T17:24:30.045-07:00 INFO [TBOT:IDEN] Attempting to join cluster via Proxy joinclient/join.go:114
ERROR: building identity service
	initializing identity service
		joining with token
			parsing tls identity
				parse tls identity
					missing identity groups or scope pin

Workarounds:

• Create a bot with a new name to bypass the invalid user login state
• Delete the offending /user_login_state/bot-foo entry from the Teleport backend

Possible triggers:

• Frequent deletion and creation of bots
• Okta sync enabled (?)

Bug details:

• Teleport version: v18.6.4, v18.6.8, presumably earlier versions