Confirming this issue on 3.0.0-rc5 (and Drupal 10, drupal/entity_browser:2.9.0 with patches from #3181631 and #3301376)
Applying patch from #8 on `drupal/entity_browser` does not solve the issue for me.
Replicating it is possible following #5 except, sometimes it also happens when the viewport is small and the button is in the middle of the screen.
I also switched to claro theme to see if it is happening there but as mentioned in #10 it doesn't.
After some investigation this doesn't seem to be a problem with the calculation of the position of the dialog to me. It's looks more like the canvas (the grey aria) on which the dialog is shown, is being misplaced: Sometimes it overflows and the top is placed a bit higher outside the viewport. Which causes the dialog to seem misplaced compared to the viewport. After clicking on the canvas it snaps back into place and it looks like the dialog is centered again. I came to this conclusion after trying to drag the dialog off viewport. Sometimes I can and sometimes I cannot drag it off viewport. I also observed that the position (the calculated top&left px) of the dialog did not change compared to the canvas.
Not sure what causes this issue. I could not find a way to confirm that what I concluded is correct.
What fixes the placement for now is adding position: fixed, so the dialog position is calculated on the viewport and not the canvas (disclaimer, this does not actually solve the underlying problem but just the symptom and might cause accessibility issues)
.ui-dialog {
position: fixed !important;
}
I am getting error with drupal 10
Uncaught TypeError: $(...).once is not a function
So I changed patch 3 a little.