pandas-gbq tests timing out

[noahdietz]

LGTM, with the exception of the pandas-gbq prererelease nox session.
This has been an ongoing problem for a number of PRs.
pandas-gbq has been stalling in all the PRs I have worked related to pandas-gbq.
When it stalls, the logs just end without a clue to indicate why the pandas-gbq prerelease session has stalled, so troubleshooting is incredibly problematic.

Originally posted by @chalmerlowe in #16739 (review)

[parthea]

Assigning to @ohmayr to check if the fix from #16746 would help