Opening a workspace for a GitLab project occasionally runs into an error page

[gtsiolis]

Bug description

Following up from #10090, this is still happening if you haven't tried opening a workspace for a GitLab repository. I've run into this page two times of over the last couple of weeks.

Reloading, not even granting access, resolves the issue. ⚠️

See also relevant discussion (internal). Cc @AlexTugarev

Steps to reproduce

From what I've seen so far, opening a workspace for any GitLab repository after a while (maybe a week a several hours) triggers this issue.

[AlexTugarev]

Quick investigation

The recent change made to the relevant parts does seem to have a bug. Maybe the assumption that the @singleton TokenService is instantiated only once is wrong, or the local caching itself is not working as intended, either way I do see 2 tries to refresh the token because it was recognized as outdated. One of the tries fails, the other succeeds, which is the reason why it's failing once, but working again on reload.

[svenefftinge]

Not sure if relevant but there are two calls to this.tokenProvider.getTokenForHost(user, host); in WorkspaceStarter#createGitInitializer. [1][2].

[AlexTugarev]

'Having hard time to reproduce it on a preview environment. The odd scenario on prod is best to observe in logs after filtering out the noise. One can see, there are two competing refreshes, which should not happen on the same server pod.

I instrumented the code with detailed logs and could not spot the issue on two tries. The following example shows, that in L.4 there is a cache hit, and it works as expected.

{"time":"12:13:24.460Z","message":".getTokenForHost gitlab.com-67 (fresh)"}
{"time":"12:13:24.470Z","message":".doGetTokenForHost (e: 12:11:25.672Z < a: 12:18:24.470Z)"}
{"time":"12:13:24.470Z","message":"(Auth-With-gitlab.com) Token to be refreshed.","payload":{"userId":"67"}}
{"time":"12:13:24.476Z","message":".getTokenForHost gitlab.com-67 (from cache)"}
{"time":"12:13:24.748Z","message":"(Auth-With-gitlab.com) Token refreshed and updated.","payload":{"userId":"67","updateDate":"12:13:24.725Z","expiryDate":"14:13:24.725Z"}}
{"time":"12:13:25.105Z","message":".getTokenForHost gitlab.com-67 (fresh)"}
{"time":"12:13:25.107Z","message":".doGetTokenForHost (e: 14:13:24.725Z > a: 12:18:25.107Z)"}
{"time":"12:13:25.406Z","message":".getTokenForHost gitlab.com-67 (fresh)"}
{"time":"12:13:25.406Z","message":".getTokenForHost gitlab.com-67 (from cache)"}

Also, while trying to reproduce again on prod, it did not fail, so there might be another factor to consider.

[AlexTugarev]

Dropping assignment, as not working on it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.