Update Token system documentation for new drupal.org docs structure

I had a bit of free time and ended up starting work on the Token API guide. Here's what I have so far.

• Homepage →
• Overview →
• Usage info for devs →
• Hooks →
• Tokens provided by Drupal core →
• The Token contrib module →
• Upgrade guides →

The bits that I'm hoping will be most useful are the lists of available tokens and the hook implementation guides. It's pretty hard to generate lists of tokens in pretty-printed documented form so hopefully the former shouldn't change too much.
I thought that we couldn't really explain the Token system without having some info about Token module, since it provides entity field tokens which are pretty critical for extending the functionality of Token API itself.

I spent some time going over the Token API codebase and reviewing the ins and outs of each section, so hopefully it's reasonably accurate. I'd say there are more passes to do before it fully fits the "voice" of the Drupal community, but hopefully it's useful.

Thinking about how https://www.drupal.org/node/390482 → provides a comprehensive list of D7 tokens with a lot of examples of chained tokens, but trying to replicate this would get very long and would be tedious to write. I wonder if there's some way to write a JS tool that does the same thing as Token module's Token Tree that lets people explore the list of default/Token module tokens and generate ones with the appropriate nesting... probably not something that would be hostable on drupal.org though.

Noting also that the old Token module handbook at https://www.drupal.org/documentation/modules/token → has some extra functionality that our guide doesn't yet cover such as using token_element_validate in #element_validate. I think I'd probably want buy-in from the module maintainers themselves before I consider adding that -- don't want to duplicate their entire handbook without their say so beyond what's necessary to say what about Token module makes Token API really click.

Probably the better project

Everything else on the APIs page → is a core api - it works without any additional modules. I'd prefer a module API not be on a core documentation page, but if there are strong feelings to the contrary it should ad least be incredibly clear, probably on all pages, that installing the contrib Token module is necessary for any of this to work.

The the APIs page → states that "Each version of Drupal introduces new APIs and keeps only some of the previous major versions’ APIs." For me, that is clear that the guide is for Drupal core API only. The benefits are that is that reader know the scope of the material and the maintenance is simplified. All contributed modules are documented at https://www.drupal.org/docs/extending-drupal/contributed-modules → . I do not think exceptions should be made.

What needs to be reviewed here? What are the links?

Thanks @quietone (and apologies to @geoffreyr ) I may have spoke too soon in #7 . I'm glad you responded as it motivated me to double check my findings and it turns out my IDE was delivering incomplete info.

There is in fact a Core API and it's fine to have this as part of the drupal-apis → guide.

Something to consider adding to the intro page → is an explanation of what this API does vs the Token module. More often than not, I wouldn't want this kind of pre-emptive clarification in a core API guide, but in this case it's an incredibly popular module that makes use of this identically-named API so it might be beneficial.

@bnjmnm All good, no worries. I agree that I could do more to explain the difference between Token API and Token module, and why the latter is really useful for extending the capabilities of the former, while at the same time not being a requirement for Drupal core. I'll see if I can do this and get back to you and @quietone when I've made some progress.