The Reload class does not provide all necessary route parameters.

It looks like the issue is related to my project.
The following code sets view_mode,
https://git.drupalcode.org/project/flag/-/blob/8.x-4.x/src/ActionLink/Ac...

It works as expected; the issue was related to customization, which had to be rewritten after the module upgrade.

@alorenc - what was you customization?

We have a custom ActionLinkType, and I had to provide four parameters to lazy_builder (flag.link_builder:build) - view mode.
At the same time, I had to update getAsFlagLink method, as ActionLinkTypePluginInterface requires $view_mode as well.