Impossible to assign different permissions on translations

Problem/Motivation

When a translatable field is used to set permissions, then it is not possible to set different permissions depending on the translation.

Steps to reproduce

- Assign permissions on a term for example on a role.
- Set a translatable field to be a reference to that vocabulary
- Create a node having for example the default language without any term
- Translate the node setting the field to the term

Proposed resolution

No idea tbh. This seems to be quite hard to achieve if i got it right, because of the way the AccessStorage class is getting permissions out of the database. As it uses taxonomy_index tables and there's not a language set there, then i don't know.
The node_access table could be fixed, but i don't think that would help that much.