Problem/Motivation
Today I worked with a member of my team to add a calculated entity reference base field to the user entity. We expected to see token pick that new field up and offer the tokens for it out of the box, given token's entity, fields, and entity reference support. We had done the same thing in Drupal 7 with success, so we expected at least as much from Drupal 8. Instead we hit up against an inconsistency in Token's entity support, and an inconsistency in token's field support.
(1.) The entity issue: If an entity type is already described to token by another module, token does not add tokens to it.
So in our case, because the User module defines some custom tokens for user entities, token would not expose our new base field and its referenced entity's tokens. We were forced to code this by hand, duplicating code that is already in the token module. It looks like this would be the case for many core entity types. Ughh.
(2.) The field issue: If an entity type defines a calculated base field, token does not provide a token for it.
So we realized that even if we fixed (1.) we were still out of luck because our field wasn't stored in the database. I can't imagine this restriction is by design.
Proposed resolution
(1.) The entity issue: If an entity type is already described to token by another module, token does not add tokens to it.
Berdir has made it clear that this is by design "...so we don't end up with duplicated custom tokens and generic tokens".
I looked at the code and there's already a check to ensure we don't overwrite any tokens already provided by other modules, so custom tokens are always safe. That then leaves us with the circumstance where a custom token provides the same data as a generic (token-provided) token.
I'd argue that the potential for duplication here is far outweighed by the effort and time you'll save the community by always providing these entity tokens. I'd imagine most developers and site-builders would much rather choose amongst a few extra (even potentially duplicating) tokens than have to implement token hooks and potentially complex token replacements themselves in order to get basic entity field tokens in place. If they want to do that for some reason we should totally let them, but let's not force that on people just because their entity-providing module decided to include a tokens.inc file.
I propose that we always add the entity tokens, regardless of whether or not custom tokens are also provided for the entity type. Lets empower our users with all the tokens.
(2.) The field issue: If an entity type defines a calculated base field, token does not provide a token for it.
I propose we expose calculated base fields along with stored base fields.
Remaining tasks
Discussion, patch, review.
User interface changes
Consistent tokens support across all entities and fields.
API changes
Consistent tokens support across all entities and fields.
Data model changes
None.
