Opauth occasionally fails with unauthorized response

When I originally log into my website through Opauth, everything works perfectly. I have some pages that will import contacts or calendar events, and these pages work well too. The only issue is that after a while, I'll get an error when I go to these page, as if my request token has expired or I'm not authorized anymore.

I checked my refresh token, but it says it's not supposed to expire for another day. Is there a way to automatically try and refresh my session on the backend without having to make the user log back into Google?

Here is the error:

'code' => 'userinfo_error',
'message' => 'Failed when attempting to query for user information',
'raw' => [
    'response' => false,
    'headers' => 'HTTP/1.0 401 Unauthorized
Vary: Origin
Vary: X-Origin
WWW-Authenticate: Bearer realm="https://accounts.google.com/AuthSubRequest", error=invalid_token
Content-Type: application/json; charset=UTF-8
Date: Mon, 24 Nov 2014 02:55:13 GMT
Expires: Mon, 24 Nov 2014 02:55:13 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic,p=0.02'