Twig debug output does not display all suggestions when an array of theme hooks is passed to #theme

Created on 23 October 2013, over 11 years ago
Updated 9 August 2023, over 1 year ago

Updated: Comment #206

Problem/Motivation

Twig debugging output is supposed to show all template suggestions. Currently, the debug output does not show all the template suggestions that the Drupal render system knows about.

Here are a few examples where this feature fails:

  • Views (a core feature used in many places) templates do not show any template suggestions. For example, go to /admin/content and look at the Filter form at the top of the page; Twig debug shows: 'views_exposed_form', but Drupal's theme system is given this list of suggestions: ['views_exposed_form__content__page_1', 'views_exposed_form__page_1', 'views_exposed_form__default', 'views_exposed_form__content__page', 'views_exposed_form__page', 'views_exposed_form__content', 'views_exposed_form'] (defined in Drupal\views\Form\ViewsExposedForm::buildForm()).
  • Using the default Bartik theme, use Drupal's search and look at the list of search results. Twig debug shows: x item-list--search-results.html.twig, x item-list--search-results.html.twig, * item-list.html.twig, but Drupal's theme system is given this list of suggestions: ['item_list__search_results__node_search', 'item_list__search_results'] (defined in Drupal\search\Controller\SearchController::view()).

The underlying cause of this issue is when #theme is set as an array of suggestions. The twig_debug output only works* when #theme is a string or when suggestions are added via suggestions_alter hooks.

* where "works" is defined as super buggy output, showing duplicates and out-of-order suggestions. See 🐛 Incorrect order and duplicate theme hook suggestions Needs work

When #theme is set as an array of suggestions, Drupal\Core\Theme\ThemeManager::render() will call the theme suggestions alter hooks with the base hook and will conditionally add an entry from the #theme array if that entry is actually implemented in the system. If none of #theme array suggestion entries are implemented, only the base hook is sent to the theme suggestions alter hooks and the code handling the twig debug comments code.

Here's a specific example of what Drupal core does now:

  1. #theme is set to ['item_list__search_results__node_search', 'item_list__search_results'] in Drupal\search\Controller\SearchController::view()
  2. Drupal\Core\Theme\ThemeManager::render() will look for implementations of the theme suggestions in the #theme entry.
    • If the only implemented Twig template is item-list.html.twig (like in the Stable themes), the theme suggestion alter hooks will be passed this list of suggestions: item_list, [list of suggestions from hook_theme_suggestions_item_list() (if this function existed)]
    • If the item-list--search-results.html.twig Twig template is implemented (like most core themes), the theme suggestion alter hooks will be passed this list of suggestions: item_list, [list of suggestions from hook_theme_suggestions_item_list()], item_list__search_results
  3. The twig_render_template() generates the Twig debugging comments by looking at the list of theme suggestions returned by the theme suggestion alter hooks (passed in $variables['theme_hook_suggestions']). But it doesn't know about the item_list__search_results__node_search suggestion in Step 1, so the list is wrong.

Proposed resolution

Theoretically, we could fix this by #2923506: Deprecate and discourage use of the "array of theme hooks" feature , but that is a loooong road of deprecations that is completely stalled. And since the "array of theme hooks" feature has been in core since 6.x and has been used frequently in contrib and in core 9.1.x and earlier, we need to fix the bug now.

OMG. No. This is technically possible, but a "proper list of theme suggestions" requires super ugly code if we're not going to modify existing lines of code. See comment #204 for proof in the form of a patch that implements this option.

Instead of conditionally appending suggestions from #theme on to the end of the suggestions from hook_theme_suggestions_HOOK(), we should always append those suggestions so that hook_theme_suggestions_alter() and hook_theme_suggestions_HOOK_alter() can see them.

Extending the example above, no matter what item_list Twig templates are implemented, the theme suggestion alter hooks will be passed this list of suggestions: item_list, [list of suggestions from hook_theme_suggestions_item_list()], item_list__search_results, item_list__search_results__node_search

This will make it easy to output an accurate suggestion list in twig debugging comments.

We will need to add extensive tests to make sure we aren't breaking anything.

This solution also fixes 🐛 Incorrect order and duplicate theme hook suggestions Needs work .

Remaining tasks

  1. Final polishing and reviews
  2. Commit

User interface changes

Twig debug output will display all templates suggestions, no matter the source of those suggestions.

API changes

The theme suggestions from #theme will always be added to the end of the hook_theme_suggestions_HOOK() list of suggestions before sending them to hook_theme_suggestions_alter() and hook_theme_suggestions_HOOK_alter().

Release notes snippet

Previously, theme suggestions from #theme (like views_exposed_form__content__page) were only sometimes added to the $suggestions variable passed to hook_theme_suggestions_alter() and hook_theme_suggestions_HOOK_alter(). Now those suggestions are always passed to alter hooks.

🐛 Bug report
Status

Needs work

Version

11.0 🔥

Component
Theme 

Last updated about 12 hours ago

Created by

🇨🇦Canada star-szr

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
  • Open on Drupal.org →
    Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    Not currently mergeable.
  • Open on Drupal.org →
    Environment: PHP 8.2 & MySQL 8
    7:02
    7:02
    Queueing
  • First commit to issue fork.
  • last update over 1 year ago
    Custom Commands Failed
  • last update over 1 year ago
    Custom Commands Failed
  • Assigned to johnalbin
  • 🇹🇼Taiwan johnalbin Taipei, Taiwan

    It looks like all the new code handling for 🐛 Improve documentation of hook_theme_suggestions_HOOK_alter() Fixed has been reverted during a rebase and force push to the GitLab branch.

    I'll work on properly porting this to the main (11.x) branch.

  • Open on Drupal.org →
    Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    Not currently mergeable.
  • @johnalbin opened merge request.
  • last update over 1 year ago
    Custom Commands Failed
  • last update over 1 year ago
    Custom Commands Failed
  • last update over 1 year ago
    Custom Commands Failed
  • last update over 1 year ago
    30,082 pass, 1 fail
  • 🇹🇼Taiwan johnalbin Taipei, Taiwan

    Still working on this. The solutions in 🐛 Improve documentation of hook_theme_suggestions_HOOK_alter() Fixed and 📌 Refactor out theme hook suggestion building from ThemeManager::render() into a separate function. Fixed need to be refactored so they can work with this issue.

  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 year ago
    #81972
  • 🇪🇸Spain alexismmd

    alexismmd changed the visibility of the branch 2118743-twig-debug-output-updated-11-x to hidden.

  • 🇮🇹Italy Giuseppe87

    @JohnAlbin about #225: I tried to rebase the MR according the documentation , I'm sorry if I messed something up.

    I needed to reroll the 10.x patch again, this time against 10.2.x, as it doesn't work anymore for that version.

    In order to avoid to mess again, I'm attaching as .patch file, if it is good, should I rebase\or push on the MR request?

  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 year ago
    Total: 192s
    #95360
  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 year ago
    Total: 573s
    #98454
  • Pipeline finished with Failed
    about 1 year ago
    Total: 529s
    #98485
  • Pipeline finished with Failed
    about 1 year ago
    Total: 597s
    #99259
  • Pipeline finished with Failed
    about 1 year ago
    Total: 650s
    #99298
  • Pipeline finished with Failed
    about 1 year ago
    Total: 583s
    #99353
  • Pipeline finished with Failed
    about 1 year ago
    Total: 239s
    #99570
  • Pipeline finished with Failed
    about 1 year ago
    Total: 483s
    #99580
  • Pipeline finished with Failed
    about 1 year ago
    Total: 163s
    #99628
  • Pipeline finished with Failed
    about 1 year ago
    Total: 562s
    #99635
  • Pipeline finished with Success
    about 1 year ago
    Total: 735s
    #99760
  • Issue was unassigned.
  • Status changed to Needs review about 1 year ago
  • 🇷🇴Romania vasike Ramnicu Valcea

    Some updates on MR to make it "green"

    - "Refactor" for related refactor https://www.drupal.org/project/drupal/issues/2953921 📌 Refactor out theme hook suggestion building from ThemeManager::render() into a separate function. Fixed (i think also tried on #234 patch)
    - Updates for the related refactor "regression" https://www.drupal.org/project/drupal/issues/3409982 🐛 [D10.2 regression] Theme suggestions cannot alter variables anymore Active
    - Some updates for "invalid suggestions"
    - Updates (forced sometimes) for tests to pass

    In conclusion ...the MR got dirty (in its "moments") ... commits and merges
    And, i think, it needs more than review ... maybe a "resume" ... and make it happen!!!

    For now "Needs review" ... but it's "Needs work" (imho) ,,, still

  • Status changed to RTBC about 1 year ago
  • 🇨🇦Canada joseph.olstad

    This is a 10 year old issue that will improve DX / TX by a large margin. I'm favoring a community RTBC on this as we'll need theming system manager review on this and from there lets try to get this to the finish line and into the end zone ( a release ) while we have some momentum here.

  • 🇮🇳India bhanu951

    Bhanu951 changed the visibility of the branch 2118743-twig-debug-output-d10 to hidden.

  • 🇮🇳India bhanu951

    Bhanu951 changed the visibility of the branch 11.x to hidden.

  • Status changed to Needs work about 1 year ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Needs to merge 11.x and fix conflicts due to Make it more obvious that a Twig template is overridden Fixed

  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 year ago
    Total: 8855s
    #108130
  • Pipeline finished with Failed
    about 1 year ago
    Total: 174s
    #108463
  • Pipeline finished with Failed
    about 1 year ago
    Total: 649s
    #108468
  • Pipeline finished with Failed
    about 1 year ago
    Total: 710s
    #108829
  • Pipeline finished with Failed
    about 1 year ago
    Total: 718s
    #108863
  • Pipeline finished with Success
    about 1 year ago
    Total: 644s
    #108885
  • Status changed to Needs review about 1 year ago
  • 🇷🇴Romania vasike Ramnicu Valcea

    MR green again
    Most of errors were from https://www.drupal.org/project/drupal/issues/3420709 Make it more obvious that a Twig template is overridden Fixed update .. mentioned in #241.
    So updated the MR - put back things and fix tests

    There also others changes missed in latest merges and rebases. I hope I covered them all.

    Let's review again

  • Status changed to RTBC about 1 year ago
  • 🇨🇦Canada joseph.olstad
    • Has new and adjusted test coverage
    • Passes all tests
    • Conflicts are resolved.
    • Code Quality hasn't changed.
    • 11 years and counting, it's time
  • Status changed to Needs work about 1 year ago
  • The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • Pipeline finished with Failed
    about 1 year ago
    Total: 186s
    #118492
  • Pipeline finished with Success
    about 1 year ago
    Total: 634s
    #118494
  • Status changed to RTBC about 1 year ago
  • 🇷🇴Romania vasike Ramnicu Valcea

    MR updated - solved conflicts
    so RTBC ... once more

  • Pipeline finished with Canceled
    about 1 year ago
    #129494
  • Pipeline finished with Success
    about 1 year ago
    #129495
  • Status changed to Needs work about 1 year ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    I think we need a separate CR detailing the new variables that are going to be available ie.

    +    // Add two read-only variables that help the template engine understand
    +    // how the template was chosen from among all suggestions.
    +    $variables['template_suggestions'] = $template_suggestions;
    +    $variables['template_suggestion'] = $hook;
    

    And specifically how they related to any existing variables, ie. theme_hook_original.

    Also this is adding an info variable everywhere - this feels very generic and likely to be used elsewhere - is this necessary?

  • 🇺🇸United States evilehk

    evilehk changed the visibility of the branch 2118743-twig-debug-output to active.

  • Pipeline finished with Failed
    12 months ago
    Total: 183s
    #149899
  • 🇷🇴Romania vasike Ramnicu Valcea

    vasike changed the visibility of the branch 2118743-twig-debug-output to hidden.

  • Pipeline finished with Failed
    12 months ago
    Total: 1075s
    #149912
  • Pipeline finished with Failed
    12 months ago
    Total: 963s
    #149987
  • Pipeline finished with Success
    12 months ago
    Total: 1111s
    #150055
  • 🇩🇪Germany metalbote Aachen

    metalbote changed the visibility of the branch 2118743-twig-debug-output-d10 to active.

  • Pipeline finished with Success
    12 months ago
    Total: 504s
    #166191
  • 🇺🇸United States mrweiner

    Re-rolled #234 for latest 10.2.6.

    There was one conflict:

    CONFLICT (modify/delete): core/modules/system/tests/src/Functional/Theme/TwigDebugMarkupTest.php deleted in 51703edde0 (Applying patch 231 from issue 2118743) and modified in HEAD.  Version HEAD of core/modules/system/tests/src/Functional/Theme/TwigDebugMarkupTest.php left in tree
    

    But, just left the file in per HEAD.

  • 🇪🇨Ecuador jwilson3

    We had to remove the latest patch from a project because it had some strange knock-on effect with Paragraphs module and the Paragraphs EE module, which is supposed to load a list or grid of Paragraphs inside a modal. After several hours of exploration and git bisect, it turns out that this patch was causing an issue where that modal never loads and instead the page reloads when clicking the "Add Paragraph" button. It looks like it may have something to do with how paragraphs_ee_field_widget_complete_form_alter() is implemented. Due to how elusive this bug became, we ran out of time and budget to keep pushing the investigation. If this patch lands as is, there may be more issues like this showing up in contrib land.

  • 🇵🇱Poland mscieszka

    When applying 2118743-251.patch using Drupal 10.3.1 there was an error in the core/themes/engines/twig/twig.engine file. I have taken changes from 2118743-251.patch into the 10.3.x branch taking into consideration 2 latest commits from https://git.drupalcode.org/project/drupal/-/commits/10.3.x/core/themes/engines/twig/twig.engine - patch now applies successfully.

  • 🇵🇱Poland mscieszka

    Please disregard the previous comment, the patch provided breaks the suggestions functionality and should not be used.

  • 🇺🇸United States omerida

    Are there any details on how the patch in #254 and earlier break the suggestions functionality? I applied it cleanly and did a diff between the HTML source pre- and post- patch. The only difference highlighted was that the patch removes the spaces between the dot and filenames in one case. I re-rolled the patch to fix that and then the only difference with the patch is that views suggestions show up in the HTML comments.

    Line 1737 of the patch is missng a space:

    $suggestion = '▪️ ' . strtr($suggestion, '_', '-') . $extension;

  • 🇬🇷Greece mkoul

    Fixed the patch to work for Drupal 11.1.1

  • Pipeline finished with Failed
    2 months ago
    Total: 140s
    #420137
  • Pipeline finished with Failed
    2 months ago
    Total: 203s
    #420146
  • Pipeline finished with Failed
    2 months ago
    Total: 164s
    #420164
  • Pipeline finished with Failed
    2 months ago
    Total: 137s
    #420177
  • Pipeline finished with Failed
    2 months ago
    Total: 344s
    #420185
  • Pipeline finished with Failed
    2 months ago
    Total: 106s
    #420898
  • Pipeline finished with Failed
    2 months ago
    Total: 391s
    #420915
  • Pipeline finished with Failed
    2 months ago
    Total: 1021s
    #420976
Production build 0.71.5 2024