OustandingToken only created when refresh token is used to get a new key pair and not at creation ? Bug ?

[StitiFatah]

from the doc :

If the blacklist app is detected in INSTALLED_APPS, Simple JWT will add any generated refresh or sliding tokens to a list of outstanding tokens. It will also check that any refresh or sliding token does not appear in a blacklist of tokens before it considers it as valid.

I'm working with 'ROTATE_REFRESH_TOKENS': True and 'BLACKLIST_AFTER_ROTATION': True

When asking for a token pair through TokenObtainPairView it indeed directly create an OutstandingToken referencing the given refresh_token but when asking for a new pair via TokenRefreshView no OutstandingToken referencing the new given refresh token is created but instead it creates an OutstandingToken for the refresh_token which was used to ask a new pair and thus a BlacklistedToken.
Don't know if it's clear enough. In summary it's :

1-Login through /api/token --> 1 pair given ( let's call the refresh one refresh_token_0 ) --> outstanding_token_0 created from refresh_token_0
2-Claiming a new pair through /api/token/refresh ---> 1 new pair given, outstanding_token_0 is blacklisted but no outstanding_token_1 created
3-Claiming a new pair through /api/token/refresh with token_refresh_1, 1 new pair given and then oustanding_token_1 created + blacklisted

Expected behavior :

1-Login through /api/token --> 1 pair given ( let's call the refresh one refresh_token_0 ) --> outstanding_token_0 created from refresh_token_0
2-Claiming a new pair through /api/token/refresh ---> 1 new pair given, outstanding_token_0 is blacklisted and outstanding_token_1 created
3-Claiming a new pair through /api/token/refresh with token_refresh_1, new pair given, outsatnding_token_1 blacklisted then oustanding_token_2 created

So apart from the first claim when the user ask for credentials himself the new generated refresh tokens aren't creating outstanding token (when generated from /api/token/refresh), It waits for a new /api/token/refresh claim with this token to create outsanding token referecing it + blacklist in the same time.

Is this the expected behavior and I am missing something ? What's the point of Outstanding Token if it's directly blacklisted ( apart from the first time at user's login ) and not created one cycle before ? Is it possible to create them when generated ?

Thanks in advance

[Andrew-Chen-Wang]

Really late sorry. Will come back to this on the weekend. Sit tight.

[write2anitab]

I am also observing similar issue with the api/login/refresh not inserting the new generated refresh token to the outstanding list

The api/login correctly adds the generated token to the outstanding list

 select count(*) from token_blacklist_outstandingtoken; 
 count = 1

 select count(*) from token_blacklist_blacklistedtoken; 
  count = 0

The api/login/refresh correctly blacklists the refresh token , but does not add the generated refresh token to the outstanding list

 select count(*) from token_blacklist_outstandingtoken; (**expecting count 2 here **)
  count = 1 
  
 select count(*) from token_blacklist_blacklistedtoken;
  count = 1

django-rest-framework-simplejwt/rest_framework_simplejwt/serializers.py

For the api/login the TokenObtainPairSerializer implements the TokenObtainSerializer get_token method which adds to the outstanding list

class TokenObtainPairSerializer(TokenObtainSerializer):
    @classmethod
    def get_token(cls, user):
        return RefreshToken.for_user(user)
    ..........

However for the api/login/refresh the TokenRefreshSerializer is not implementing the TokenObtainSerializer . It blacklists the old access token, but does not add the new generated token to the outstanding list

class TokenRefreshSerializer(serializers.Serializer):
    refresh = serializers.CharField()
 ............
 if api_settings.ROTATE_REFRESH_TOKENS:
       if api_settings.BLACKLIST_AFTER_ROTATION:
            try:
                  # Attempt to blacklist the given refresh token
                  refresh.blacklist()
              except AttributeError:
                  # If blacklist app not installed, `blacklist` method will
                   # not be present
                   pass

[adeng322]

Im having the same issue as well

[Andrew-Chen-Wang]

Sorry, not sure why I was having a hard time parsing through this. @write2anitab the code doing refresh.blacklist() creates an outstanding token:

djangorestframework-simplejwt/rest_framework_simplejwt/tokens.py

Lines 196 to 213 in 1c7f005

	def blacklist(self):
	"""
	Ensures this token is included in the outstanding token list and
	adds it to the blacklist.
	"""
	jti = self.payload[api_settings.JTI_CLAIM]
	exp = self.payload['exp']
	# Ensure outstanding token exists with given jti
	token, _ = OutstandingToken.objects.get_or_create(
	jti=jti,
	defaults={
	'token': str(self),
	'expires_at': datetime_from_epoch(exp),
	},
	)
	return BlacklistedToken.objects.get_or_create(token=token)

We also have tests to verify this:

djangorestframework-simplejwt/tests/test_serializers.py

Lines 285 to 322 in 1c7f005

	def test_it_should_blacklist_refresh_token_if_tokens_should_be_rotated_and_blacklisted(self):
	self.assertEqual(OutstandingToken.objects.count(), 0)
	self.assertEqual(BlacklistedToken.objects.count(), 0)
	refresh = RefreshToken()
	refresh['test_claim'] = 'arst'
	old_jti = refresh['jti']
	old_exp = refresh['exp']
	# Serializer validates
	ser = TokenRefreshSerializer(data={'refresh': str(refresh)})
	now = aware_utcnow() - api_settings.ACCESS_TOKEN_LIFETIME / 2
	with override_api_settings(ROTATE_REFRESH_TOKENS=True, BLACKLIST_AFTER_ROTATION=True):
	with patch('rest_framework_simplejwt.tokens.aware_utcnow') as fake_aware_utcnow:
	fake_aware_utcnow.return_value = now
	self.assertTrue(ser.is_valid())
	access = AccessToken(ser.validated_data['access'])
	new_refresh = RefreshToken(ser.validated_data['refresh'])
	self.assertEqual(refresh['test_claim'], access['test_claim'])
	self.assertEqual(refresh['test_claim'], new_refresh['test_claim'])
	self.assertNotEqual(old_jti, new_refresh['jti'])
	self.assertNotEqual(old_exp, new_refresh['exp'])
	self.assertEqual(access['exp'], datetime_to_epoch(now + api_settings.ACCESS_TOKEN_LIFETIME))
	self.assertEqual(new_refresh['exp'], datetime_to_epoch(now + api_settings.REFRESH_TOKEN_LIFETIME))
	self.assertEqual(OutstandingToken.objects.count(), 1)
	self.assertEqual(BlacklistedToken.objects.count(), 1)
	# Assert old refresh token is blacklisted
	self.assertEqual(BlacklistedToken.objects.first().token.jti, old_jti)

Will need more debugging information and a reproducible git repository if possible.

[adeng322]

Sorry, not sure why you having a hard time parsing through this. The issue is "with the api/login/refresh not inserting the newly generated refresh token to the outstanding list". I believe the code snippet you provided has nothing to do with this issue but just to show how a refresh token is blacklisted. Correct me if I'm wrong, please. @StitiFatah @write2anitab provided such a detailed explanation. 3 people reflected the same issue. how can this just being simply labeled as invalid?

[Andrew-Chen-Wang]

@adeng322 They're great logs and fantastic explanations, just really long run-on sentences that personally made it hard for me to decipher...

For reproducibility, the settings ROTATE_REFRESH_TOKENS and BLACKLIST_AFTER_ROTATION are True. You need to send username/pw credentials to get your initial tokens. Then you refresh your access token and expect a new refresh and access token.

The "bug" is that the OutstandingToken is not created. The first permalink is where we generate the OutstandingToken and a BlacklistedToken. If that OutstandingToken is not created, then deduce: why is the BlacklistToken still created? It's because the get_or_create QuerySet method got an existing OutstandingToken that matched the token value.

The second snippet is our test code that I added for y'all to verify if we implemented our test case correctly. Please note the lines:

    now = aware_utcnow() - api_settings.ACCESS_TOKEN_LIFETIME / 2 
  
     with override_api_settings(ROTATE_REFRESH_TOKENS=True, BLACKLIST_AFTER_ROTATION=True): 
         with patch('rest_framework_simplejwt.tokens.aware_utcnow') as fake_aware_utcnow: 
             fake_aware_utcnow.return_value = now 
             self.assertTrue(ser.is_valid()) 

In other words, give it some time. Hope that explains it @adeng322