NATS continues to run with an expired operator #7570
Description
Observed behavior
A NATS server can be started with an expired operator. It functions normally and can authenticate users.
Log:
$ nats-server -c nats.conf -DV
[493998] 2025/11/21 12:45:48.856648 [INF] Starting nats-server
[493998] 2025/11/21 12:45:48.856711 [INF] Version: 2.12.2
[493998] 2025/11/21 12:45:48.856717 [INF] Git: [not set]
[493998] 2025/11/21 12:45:48.856735 [DBG] Go build: go1.24.6
[493998] 2025/11/21 12:45:48.856744 [INF] Name: NAPI2OPUWETQJJAEOYF7LRFUCN7TNDG6VQNW4KXUPTM4WDZQVTCP4PSZ
[493998] 2025/11/21 12:45:48.856751 [INF] ID: NAPI2OPUWETQJJAEOYF7LRFUCN7TNDG6VQNW4KXUPTM4WDZQVTCP4PSZ
[493998] 2025/11/21 12:45:48.856761 [INF] Using configuration file: nats.conf (sha256:6a276fa4e31800f6202a2c8704194c1139c265b9095d485f4e097980580b03ad)
[493998] 2025/11/21 12:45:48.856768 [INF] Trusted Operators
[493998] 2025/11/21 12:45:48.856773 [INF] System : ""
[493998] 2025/11/21 12:45:48.856779 [INF] Operator: "xpired"
[493998] 2025/11/21 12:45:48.856785 [INF] Issued : 2025-11-19 19:19:33 -0600 CST
[493998] 2025/11/21 12:45:48.856813 [INF] Expires : 2025-11-19 19:20:33 -0600 CST
[493998] 2025/11/21 12:45:48.857636 [INF] Listening for client connections on 0.0.0.0:4222
[493998] 2025/11/21 12:45:48.857647 [DBG] Get non local IPs for "0.0.0.0"
[493998] 2025/11/21 12:45:48.857836 [DBG] ip=[[redacted]]
[493998] 2025/11/21 12:45:48.857855 [INF] Server is ready
[493998] 2025/11/21 12:45:48.858207 [DBG] maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined
[493998] 2025/11/21 12:45:53.069457 [DBG] 127.0.0.1:56342 - cid:5 - Client connection created
[493998] 2025/11/21 12:45:53.071946 [TRC] 127.0.0.1:56342 - cid:5 - <<- [CONNECT {"verbose":false,"pedantic":false,"jwt":"eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJqdGkiOiIzNk40SlZOMkpOSkxKUjNYWEhJUUZHM05LS0tNTDZJSVpJNEIyV1VLWEIzUlRST1lNTlJRIiwiaWF0IjoxNzYzNTkwNjAxLCJpc3MiOiJBQzdFSzRFNEJSRU1aWEFSSlZSNkM1VlMzQ0tIM1lRSDNNMkQ0NVZPVkFJNUxGRk5JQkM0VVdPSiIsIm5hbWUiOiJzeXMiLCJzdWIiOiJVQ05NNFlaUkxHWjZFNzM2WkZTTExKREMyVlFPVVVINE9FRkNGQVVVQUsyNDMzNFRMSFFBQVJVNiIsIm5hdHMiOnsicHViIjp7fSwic3ViIjp7fSwic3VicyI6LTEsImRhdGEiOi0xLCJwYXlsb2FkIjotMSwidHlwZSI6InVzZXIiLCJ2ZXJzaW9uIjoyfX0.exeCX_j4TvKWjg12huH0PLVeR0as5QhT88IkNZEmfotz_ByaFEcj6x_NNHa5J4TO5rMGGb5utQKO41XYurRNDg","sig":"fEBHOrt5xr0J8kJ1xUwpTdtI01gZEcfapCeqoUkwt1AyfdFzXiJA4UTgsNKUlj09VqGzFThwI_NTcim4r6oNAQ","tls_required":false,"name":"NATS CLI Version v0.3.0","lang":"go","version":"1.45.0","protocol":1,"echo":true,"headers":true,"no_responders":true}]
[493998] 2025/11/21 12:45:53.072539 [DBG] 127.0.0.1:56342 - cid:5 - Authenticated JWT: Client "UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" (claim-name: "sys", claim-tags: []) signed with "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ" by Account "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ" (claim-name: "SYS", claim-tags: []) signed with "OAZVKZJMWGA7DMFI67R5XIASKPUREAXWNA5CV2RNLZIPGA463NTXX6BT" has mappings false accused 0xc0001c6a88
[493998] 2025/11/21 12:45:53.072590 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - <<- [PING]
[493998] 2025/11/21 12:45:53.072600 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - ->> [PONG]
[493998] 2025/11/21 12:45:53.072944 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - <<- [PUB foo 2]
[493998] 2025/11/21 12:45:53.072968 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - <<- MSG_PAYLOAD: ["hi"]
[493998] 2025/11/21 12:45:53.072979 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - <<- [PING]
[493998] 2025/11/21 12:45:53.072989 [TRC] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - ->> [PONG]
[493998] 2025/11/21 12:45:53.073161 [DBG] 127.0.0.1:56342 - cid:5 - "v1.45.0:go:NATS CLI Version v0.3.0" - "AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ/SYS/jwt:UCNM4YZRLGZ6E736ZFSLLJDC2VQOUUH4OEFCFAUUAK24334TLHQAARU6" - Client connection closed: Client Closed
^C[493998] 2025/11/21 12:46:01.024169 [INF] Trapped "interrupt" signal
[493998] 2025/11/21 12:46:01.024309 [INF] Initiating Shutdown...
[493998] 2025/11/21 12:46:01.024409 [DBG] Client accept loop exiting..
[493998] 2025/11/21 12:46:01.024425 [DBG] SYSTEM - System connection closed: Client Closed
[493998] 2025/11/21 12:46:01.025127 [INF] Server Exiting..
Expected behavior
To fully respect the exp claim of its operator JWT, NATS would have to consider all accounts signed by an operator to be invalid as soon as that operator expires.
That may be impractical, but a partial solution that confers most of the benefits of expiry would be for the NATS server to refuse to start when the operator is expired. This would have the advantage of causing a CrashLoopBackoff and escalating the situation through normal Kubernetes alerting mechanisms.
This would be a breaking change for anyone who created an expiring operator (not the default when using nsc) and then relied on the fact that NATS would do nothing when it expired. Although that's a silly thing to do, it technically means this behavior should be gated behind a config option.
Server and client version
Server: 2.12.2
CLI client: 0.3.0
Host environment
Ubuntu 22.04 in a VMware virtual machine on a Xeon Platinum 8168.
Steps to reproduce
Run in two separate terminals:
nats-server -c nats.conf -DV
nats --creds user.creds pub foo hi
Use the following files in the current directory:
operator.jwt
eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJleHAiOjE3NjM2MDE2MzMsImp0aSI6IkFKRFBKR0ZGWVRRUFRGSUFLRTZZSjdHN0NNUjZHRUFXWU1RRVNVTUxLTUFPQ0tEQjZFQkEiLCJpYXQiOjE3NjM2MDE1NzMsImlzcyI6Ik9BWlZLWkpNV0dBN0RNRkk2N1I1WElBU0tQVVJFQVhXTkE1Q1YyUk5MWklQR0E0NjNOVFhYNkJUIiwibmFtZSI6InhwaXJlZCIsInN1YiI6Ik9BWlZLWkpNV0dBN0RNRkk2N1I1WElBU0tQVVJFQVhXTkE1Q1YyUk5MWklQR0E0NjNOVFhYNkJUIiwibmF0cyI6eyJ0eXBlIjoib3BlcmF0b3IiLCJ2ZXJzaW9uIjoyfX0.pfAjYwvSd33dzpLs4XOOp0SzBdaoF-V72vTM444hcXpIYdWNBMYq8Y0bQV7yutTyLpT55qwTl6ttwaOA3hZ-AQ
user.creds (these are not creds to any of our actual deployments)
----BEGIN NATS USER JWT-----
eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJqdGkiOiIzNk40SlZOMkpOSkxKUjNYWEhJUUZHM05LS0tNTDZJSVpJNEIyV1VLWEIzUlRST1lNTlJRIiwiaWF0IjoxNzYzNTkwNjAxLCJpc3MiOiJBQzdFSzRFNEJSRU1aWEFSSlZSNkM1VlMzQ0tIM1lRSDNNMkQ0NVZPVkFJNUxGRk5JQkM0VVdPSiIsIm5hbWUiOiJzeXMiLCJzdWIiOiJVQ05NNFlaUkxHWjZFNzM2WkZTTExKREMyVlFPVVVINE9FRkNGQVVVQUsyNDMzNFRMSFFBQVJVNiIsIm5hdHMiOnsicHViIjp7fSwic3ViIjp7fSwic3VicyI6LTEsImRhdGEiOi0xLCJwYXlsb2FkIjotMSwidHlwZSI6InVzZXIiLCJ2ZXJzaW9uIjoyfX0.exeCX_j4TvKWjg12huH0PLVeR0as5QhT88IkNZEmfotz_ByaFEcj6x_NNHa5J4TO5rMGGb5utQKO41XYurRNDg
----END NATS USER JWT-----
----BEGIN USER NKEY SEED-----
SUAL4S6WPOUBXOOCFSTFOIIMOZCY7GW54PQ7I6U46UNB3U5D2M6YZ5YSFU
----END USER NKEY SEED-----
nats.conf
operator: operator.jwt
system_account: AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ
resolver: memory
resolver_preload: {
AC7EK4E4BREMZXARJVR6C5VS3CKH3YQH3M2D45VOVAI5LFFNIBC4UWOJ: eyJ0eXAiOiJKV1QiLCJhbGciOiJlZDI1NTE5LW5rZXkifQ.eyJqdGkiOiIzV1lQVFdMSDJEVVlXVEc0VENSTE9OQkVGR0xTNE4yWEJLRTVBUFhEU1BFV1JHNU0zMlFRIiwiaWF0IjoxNzYzNTkwNDgzLCJpc3MiOiJPQVpWS1pKTVdHQTdETUZJNjdSNVhJQVNLUFVSRUFYV05BNUNWMlJOTFpJUEdBNDYzTlRYWDZCVCIsIm5hbWUiOiJTWVMiLCJzdWIiOiJBQzdFSzRFNEJSRU1aWEFSSlZSNkM1VlMzQ0tIM1lRSDNNMkQ0NVZPVkFJNUxGRk5JQkM0VVdPSiIsIm5hdHMiOnsibGltaXRzIjp7InN1YnMiOi0xLCJkYXRhIjotMSwicGF5bG9hZCI6LTEsImltcG9ydHMiOi0xLCJleHBvcnRzIjotMSwid2lsZGNhcmRzIjp0cnVlLCJjb25uIjotMSwibGVhZiI6LTF9LCJkZWZhdWx0X3Blcm1pc3Npb25zIjp7InB1YiI6e30sInN1YiI6e319LCJhdXRob3JpemF0aW9uIjp7ImF1dGhfdXNlcnMiOm51bGx9LCJ0eXBlIjoiYWNjb3VudCIsInZlcnNpb24iOjJ9fQ.mFnwt3nnYIoqcset-bfaHnjacKiL8bP6MGru4ito-32dMqs8xaOIAn8KzHzbvTiuth7IpMaW-xglfxQo8J5bCw
}