Provide some way to rebuild the persistent bundle field map

Created on 20 April 2020, over 4 years ago
Updated 30 January 2023, almost 2 years ago

Problem/Motivation

In #2482295: Rebuilding field map with many bundles/fields is very slow β†’ , the bundle field map was moved from being generated on demand and cached to being saved in the KV store. I've encountered a number of projects where this has gone out of sync.

When this happens it can be really hard to track down, since there aren't any immediate adverse effects you can notice the issue long after it was caused. However it can still cause errors β†’ and create noisy log entries that emanate from views.

Proposed resolution

The map used to be generated on demand, so at least provide some method of programatically rebuilding the map, so when troubleshooting issues that are caused by an out of sync map, there is an easier way to diagnose and fix the problem.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

✨ Feature request
Status

Needs work

Version

10.1 ✨

Component
EntityΒ  β†’

Last updated about 3 hours ago

Created by

πŸ‡¦πŸ‡ΊAustralia Sam152

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • The Needs Review Queue Bot β†’ tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • First commit to issue fork.
  • last update over 1 year ago
    30,342 pass
  • πŸ‡ΊπŸ‡ΈUnited States pmagunia Philadelphia πŸ‡ΊπŸ‡Έ

    Adding a file patch in case anyone want to link. Identical to #38

  • Status changed to Needs review over 1 year ago
  • last update over 1 year ago
    30,099 pass
  • πŸ‡ΊπŸ‡ΈUnited States pmagunia Philadelphia πŸ‡ΊπŸ‡Έ
  • πŸ‡ΊπŸ‡ΈUnited States pmagunia Philadelphia πŸ‡ΊπŸ‡Έ

    If someone wanted to use the code in the patch:

    function MYMODULE_update_10000(&$sandbox) {
      \Drupal::service('entity_field.manager')->rebuildBundleFieldMap();
    }
    
  • Status changed to Needs work over 1 year ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Seems to be an open question in #34 could that be answered?

  • πŸ‡¬πŸ‡§United Kingdom joachim

    > Should I assert against an explicit expected bundle field map, or just test that it's the same before and after a rebuild. Normally I like to be really explicit in tests, but in this case I thought that would introduce additional brittleness, so I just test it's the same before and after.

    I think testing it's the same before and after is potentially good, in that as you say it keeps the test simple.

    And I see that the test is deleting the map -- so the intention is that we're checking that the rebuild actually can rebuild from scratch. However, we're not checking that after the deletion the map is empty!

    > How complex a bundle field map should I be making for the test? I've kept it simple for the time being.

    What are the different cases we should consider?

  • πŸ‡¬πŸ‡§United Kingdom joachim

    Also,

    > public function rebuildBundleFieldMap() {

    The other methods that work with the field map call it just 'field map', so the use of 'bundle' in this method name is confusing.

  • πŸ‡¬πŸ‡§United Kingdom joachim

    rebuildBundleFieldMap() should finish off by clearing the 'entity_field_map cache item, as otherwise it won't have any apparent effect.

  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    Separate thought but: perhaps this can be leveraged to help with #3045509: EntityFieldManager::getFieldMap() doesn't show bundle fields β†’ ?

  • First commit to issue fork.
  • Pipeline finished with Failed
    5 months ago
    Total: 168s
    #227396
  • Pipeline finished with Failed
    5 months ago
    Total: 530s
    #227400
  • Pipeline finished with Failed
    5 months ago
    Total: 491s
    #227412
  • Pipeline finished with Failed
    5 months ago
    Total: 582s
    #227423
  • Pipeline finished with Success
    5 months ago
    Total: 3339s
    #227428
  • πŸ‡ΊπŸ‡ΈUnited States paul121 Spokane, WA

    I've opened a new MR against 11.x. Chose to cherry-pick the existing commits into a new MR because there were unrelated merge conflicts in the core branches between 9.3.x and 11.x. and I didn't want to force-push this old branch. Attaching a patch file for composer builds.

    rebuildBundleFieldMap() should finish off by clearing the 'entity_field_map cache item, as otherwise it won't have any apparent effect.

    I've added a final step to clear this cache item and updated tests to account for this. One thing I noticed is that these tests are implemented in a new Kernel test file - all of the existing EntityFieldManager tests seem to be unit tests. I'm not sure if that is a blocker for merging.

  • Status changed to Needs review 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States paul121 Spokane, WA
  • Status changed to RTBC 5 months ago
  • πŸ‡¬πŸ‡§United Kingdom scott_euser

    Thanks for the work on this! Gave it a whirl and seems to be working correctly for me:

    • Changes in #45 seem to be addressed + match the way UseCacheBackendTrait does it
    • Running drush php:eval "\Drupal::service('entity_field.manager')->rebuildBundleFieldMap()" to check works fine
  • πŸ‡«πŸ‡·France nod_ Lille

    nod_ β†’ changed the visibility of the branch 9.3.x to hidden.

  • πŸ‡³πŸ‡ΏNew Zealand quietone

    I left a question in the MR and some suggestions for the comments.

  • Status changed to Needs work 8 days ago
  • πŸ‡¦πŸ‡ΊAustralia nterbogt

    I ran this on a site with 31 entities and 1039 fields and it took 1.9 seconds on my local machine. I'm not sure this qualifies it as slow; it's certainly not slow compared to rebuilding node access as an example.

Production build 0.71.5 2024