Contrib.social

Update Token system documentation for new drupal.org docs structure

Open on Drupal.org →
Created on 15 April 2025, about 1 month ago

Problem/Motivation

The token system needs some better documentation. Working with tokens has been a complicated and obscure matter as long as I've worked with Drupal, and I can't seem to find any dedicated section for it under the new drupal.org documentation structure → .

I'm reaching out to figure out how much demand there is for reworking the existing docs and writing up new docs to fit the new structure. Also, please point me in the right direction if anyone's already working on this or if I'm jumping the gun. All feedback is very much welcome.

Proposed resolution

Since the token system is mainly used by site builders and developers instead of content editors or end users, it probably makes the most sense to add a section to the Develop wiki → , at least as far as I can tell. As the Drupal project itself describes the "token system" as a component of Drupal on its own, I reckon it makes sense to add a guide for it under the Drupal APIs → section -- even if it's a bit smaller than most APIs.

A dedicated section for tokens for site builders could be added to the main Drupal wiki or the Drupal User Guide, with examples on how to use it. This in turn could link back to the Token API section.

The Token API section might contain information about the following:

  • rationale for why the token system exists
  • how to replace tokens via code
  • how it works, in terms of scanning for tokens, linking replacements through hooks, and replacing into strings
  • how the hooks are structured, in both functional and OO style
  • what tokens are available by default
  • how tokens for entities work
  • ecosystem modules that work with the token system
  • an upgrade guide for older Drupal sites, similar to the Tokens for D6 -> D7 guide →

Remaining tasks

  • Aggregate existing Token system docs for current and previous Drupal versions
  • Determine a structure for new docs that will be navigable and useful
  • Work out what docs to write
  • Determine best location for new documentation, obtain necessary stakeholder approvals
  • Write docs, get out for others to review
📌 Task
Status

Active

Version

11.0 🔥

Component

token system

Created by

🇦🇺Australia geoffreyr Sydney, AU / Gadigal country

Live updates comments and jobs are added and updated live.
  • Needs documentation

    A documentation change is requested elsewhere. For Drupal core (and possibly other projects), once the change has been committed, this status should be recorded in a change record node.

Sign in to follow issues

Comments & Activities

  • Issue created by @geoffreyr
  • Comment 27 days ago →
    🇦🇺Australia geoffreyr Sydney, AU / Gadigal country

    I had a bit of free time and ended up starting work on the Token API guide. Here's what I have so far.

    The bits that I'm hoping will be most useful are the lists of available tokens and the hook implementation guides. It's pretty hard to generate lists of tokens in pretty-printed documented form so hopefully the former shouldn't change too much.
    I thought that we couldn't really explain the Token system without having some info about Token module, since it provides entity field tokens which are pretty critical for extending the functionality of Token API itself.

    I spent some time going over the Token API codebase and reviewing the ins and outs of each section, so hopefully it's reasonably accurate. I'd say there are more passes to do before it fully fits the "voice" of the Drupal community, but hopefully it's useful.

  • Comment 27 days ago →
    🇦🇺Australia geoffreyr Sydney, AU / Gadigal country

    Thinking about how https://www.drupal.org/node/390482 → provides a comprehensive list of D7 tokens with a lot of examples of chained tokens, but trying to replicate this would get very long and would be tedious to write. I wonder if there's some way to write a JS tool that does the same thing as Token module's Token Tree that lets people explore the list of default/Token module tokens and generate ones with the appropriate nesting... probably not something that would be hostable on drupal.org though.

  • Comment 27 days ago →
    🇦🇺Australia geoffreyr Sydney, AU / Gadigal country

    Noting also that the old Token module handbook at https://www.drupal.org/documentation/modules/token → has some extra functionality that our guide doesn't yet cover such as using token_element_validate in #element_validate. I think I'd probably want buy-in from the module maintainers themselves before I consider adding that -- don't want to duplicate their entire handbook without their say so beyond what's necessary to say what about Token module makes Token API really click.

  • Comment 26 days ago →
    🇺🇸United States smustgrave

    Probably the better project

  • Comment 26 days ago →
    🇮🇹Italy apaderno Brescia, 🇮🇹
contrib.social Blog FAQ Discussions
Production build 0.71.5 2024