Many calls to ContextRepository::getAvailableContexts() due to entity upcasting

Created on 9 July 2020, over 4 years ago
Updated 7 May 2024, 8 months ago

Problem/Motivation

The documentation of \Drupal\Core\Plugin\Context\ContextRepositoryInterface::getAvailableContexts() says:

Gets all available contexts *for the purposes of configuration.*

The expected flow is that you have something like a block edit form with context mapping. You call the available contexts, you configure it, and then at runtime, you only load the contexts that you need.

There is no caching, static or otherwise on this method. There's nothing very expensive in core, but we have have context providers expose terms of certain bundles as context, and we run an entity query in that method. And @EclipseGc always had similar plans for core as well. I didn't notice until recently, but the entity upcasting can easily be called dozens of times on a page when upcasting entities of menu link access checks. That means dozens of entity queries for our project.

I honestly don't understand why we use the plugin context system on that level. The implementation is hardcoded, we only care about two specific language contexts, and we have the language manager injected, there is no reason why the current language contexts would return anything else than getCurrentLanguage($type).

The only actual context that we pass along we have to create as a fake context is the the translation operation context, which IMHO shows quite nicely that this is the wrong approach. Plugin context is meant to be provided as a global thing to you, similar to cache contexts, not that you hardcode it and pass it to something else

Maybe I should have pushed back more on that in the original issue :)

Proposed resolution

Remove the getAvailableContexts() calls in EntityRepository and EntityConverter.
Remove the user context workaround that's apparently still causing some problems as we don't need the user context at all.
Switch EntityRepository::getContentLanguageFromContexts() to use LanguageManager::getCurrentLanguage().

Remaining tasks

Decide if we need BC here.

(For BC, we could keep using those language context for the unlikely case that someone did use it. And maybe switch "back" to simple context keys like operation and language if you do want to get the the canonical entity for a specific language.)

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Fixed

Version

10.3

Component
Entity 

Last updated about 17 hours ago

Created by

🇨🇭Switzerland berdir Switzerland

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • last update about 1 year ago
    30,392 pass
  • 🇨🇭Switzerland berdir Switzerland

    Reroll for 11.x.

  • Status changed to Needs work about 1 year ago
  • 🇺🇸United States smustgrave

    For the API change mentioned in #28 and CR in #27

  • 🇺🇸United States jasonawant New Orleans, USA

    Here's a patch for 10.1.x branch.

    This resolves an issue I was experiencing when attempting to apply runtime contexts to condition plugins. The condition plugins, specifically entity_bundle:node, was not able to evaluate b/c of missing context value.

            $contexts = $this->contextRepository->getRuntimeContexts($condition_plugin->getContextMapping());
            $this->contextHandler->applyContextMapping($condition_plugin, $contexts);
    

    Sorry, I'm not able to help with CR or the issue summary; I'm not sure what exactly is needed.

  • First commit to issue fork.
  • Pipeline finished with Failed
    10 months ago
    Total: 502s
    #118666
  • 🇺🇦Ukraine voleger Ukraine, Rivne

    Created MR based on #30.
    Attaching the patch for 10.2.x branch

  • 🇬🇧United Kingdom longwave UK

    Ran into this today when doing some profiling. Tracing the code before finding this issue I also discovered that we load all the contexts to only use the language ones; I don't fully understand this subsystem but it seems like we could optimise here. LazyContextRepository::getAvailableContexts() takes 15% of the request time in some cases according to xhprof so if we can remove this entirely it looks like it would be a good performance gain.

  • Status changed to Needs review 9 months ago
  • 🇬🇧United Kingdom longwave UK

    Fixed the performance test assertions (decreasing some values!) and rebased.

  • 🇬🇧United Kingdom longwave UK

    Updated IS. Does this need a change record? Not sure who this would affect or who to write the change record for.

  • Pipeline finished with Success
    9 months ago
    Total: 494s
    #130743
  • Merge request !7222Performance fix with BC for EntityConverter. → (Open) created by longwave
  • 🇬🇧United Kingdom longwave UK

    @Berdir thanks for the explanation - this is quite the mess and I'm not sure either how to immediately start untangling it.

    To fix the immediate performance issue (all contexts are loaded but only two are ever used) I attempted a stopgap approach in MR!7222. This only loads the contexts that EntityRepository needs - if the site isn't multilingual then it doesn't load them at all. This also removes one of the @todos but replaces it with another :)

    I think longer term that just passing around an optional $langcode would be a better idea and perhaps we should try to untangle it all in one go by switching to that, but without diving in and trying it I'm not sure of the full implications.

  • Pipeline finished with Canceled
    9 months ago
    Total: 542s
    #131303
  • Pipeline finished with Failed
    9 months ago
    Total: 523s
    #131320
  • Pipeline finished with Canceled
    9 months ago
    Total: 106s
    #131347
  • Pipeline finished with Canceled
    9 months ago
    Total: 157s
    #131352
  • Pipeline finished with Failed
    9 months ago
    Total: 168s
    #131358
  • Status changed to Needs work 9 months ago
  • The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • Pipeline finished with Failed
    9 months ago
    #131365
  • Status changed to Needs review 9 months ago
  • 🇬🇧United Kingdom longwave UK
  • Pipeline finished with Success
    9 months ago
    Total: 520s
    #131465
  • 🇬🇧United Kingdom longwave UK

    On my test site MR!7222 shaves at least 2-3ms off every page load that invokes EntityConverter::convert().

  • Pipeline finished with Failed
    9 months ago
    Total: 183s
    #131782
  • 🇨🇭Switzerland berdir Switzerland

    I updated the main MR with an attempt to implement what I suggested in my comment, I also added the test from 📌 Unpublished translations should fallback to the original language Needs work because this should allow that one to pass as well. Didn't really test yet beyond that.

  • Pipeline finished with Canceled
    9 months ago
    Total: 22s
    #131801
  • Pipeline finished with Failed
    9 months ago
    Total: 533s
    #131802
  • Pipeline finished with Failed
    9 months ago
    Total: 171s
    #131826
  • Pipeline finished with Success
    9 months ago
    Total: 492s
    #131919
  • 🇨🇭Switzerland berdir Switzerland

    So my proposal basically works, but after working through 📌 Deprecate the "entity_upcast" operation specified in EntityConverter Active and #2942907: Entity system does not provide an API for retrieving an entity variant that is safe for editing once again, the operation/entity_upcast thing is not as straight forward as the langcode thing is, so I think that needs to stay for now to avoid too much complexity and change in a single issue. I think I can revert it to the point where the operation is just passed in as a regular array key, in sync with existing language fallback context parameters, without those pseudo-plugin-context-ids.

  • Status changed to Needs work 9 months ago
  • 🇺🇸United States smustgrave

    Not sure which MR is to be reviewed. Could one be hidden please?

  • 🇬🇧United Kingdom longwave UK

    longwave changed the visibility of the branch 3158130-performance-fix to hidden.

  • 🇬🇧United Kingdom longwave UK

    Hidden mine, while it looks like it works I think Berdir's covers more ground in the direction we want to go in.

    Still NW for now as it needs rebasing.

  • First commit to issue fork.
  • Pipeline finished with Canceled
    9 months ago
    Total: 325s
    #134597
  • Pipeline finished with Canceled
    9 months ago
    Total: 97s
    #134601
  • Pipeline finished with Canceled
    9 months ago
    Total: 330s
    #134604
  • 🇺🇸United States weekbeforenext Asheville, NC

    Resolved merge conflicts in the MR, then realized that the patch from #35 applies to 10.2.4. :shrug:

  • Pipeline finished with Failed
    9 months ago
    Total: 630s
    #134614
  • Status changed to Needs review 9 months ago
  • 🇬🇧United Kingdom longwave UK

    Rebased again.

  • Pipeline finished with Success
    9 months ago
    Total: 665s
    #135182
  • Status changed to Needs work 9 months ago
  • 🇺🇸United States smustgrave

    Left some comments on the MR.

  • Status changed to Needs review 9 months ago
  • 🇬🇧United Kingdom longwave UK

    Had a go at a change record, @Berdir can hopefully correct my mistakes and flesh this out a bit more! https://www.drupal.org/node/3437685

    Changing the argument defaults from NULL to [] means changing the interface, not worth doing here with all the other changes, let's just leave this as-is for now.

  • Status changed to Needs work 9 months ago
  • 🇨🇭Switzerland berdir Switzerland

    Did a review, will try to do that myself when I find time, but feel free to pick it up if you have time earlier, not exactly sure when I will.

  • Pipeline finished with Failed
    9 months ago
    #146350
  • Status changed to Needs review 9 months ago
  • 🇨🇭Switzerland berdir Switzerland

    Addressed my review. Also deprecated the constant, FWIW, there's not a single usage of this in all of contrib.

  • Pipeline finished with Success
    9 months ago
    Total: 986s
    #146357
  • Status changed to Needs work 8 months ago
  • 🇬🇧United Kingdom longwave UK

    Added a question to the MR as it appears we're also implementing 📌 Unpublished translations should fallback to the original language Needs work here. If this is unavoidable I think we need a change notice for that issue explaining the change and also how to revert it back to the previous behaviour.

  • Pipeline finished with Failed
    8 months ago
    Total: 176s
    #147370
  • Status changed to Needs review 8 months ago
  • 🇨🇭Switzerland berdir Switzerland

    Found the problem, I needed to drop the $legacy_contexts and instead just pass $contexts through. Thanks for catching that!

    And yes, that single 200/403 line is literally the whole test coverage that exists in core for the current entity_upcast behaviour, and we only added that in a separate issue as part of the translate-editable-content permission. There's that unit test mock, but that does not test what's going on inside EntityRepository.

  • Status changed to Needs work 8 months ago
  • The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • Pipeline finished with Success
    8 months ago
    Total: 1109s
    #147389
  • Status changed to RTBC 8 months ago
  • 🇬🇧United Kingdom longwave UK

    Thanks - the explanation and fix makes perfect sense, and it seems in that other issue people are relying on that test assertion so let's keep it as is and defer fixing that until later (or never).

    Everything else here looks great so marking RTBC, we have some other good performance improvements in 10.3 so would be nice to land this one as well.

    • catch committed 5535002d on 10.3.x
      Issue #3158130 by longwave, Berdir, voleger, mayurjadhav,...
    • catch committed adedbb17 on 11.x
      Issue #3158130 by longwave, Berdir, voleger, mayurjadhav,...
  • Status changed to Fixed 8 months ago
  • 🇬🇧United Kingdom catch

    Nice work untangling the various scope issues here, hopefully we can keep going in the related issues and make all this a bit more understandable.

    Also pleased to see the performance tests pick up that there is some improvement (even if the improvement is not really that we're removing the cache get, at least we can see that something is happening less than it previously was).

    Committed/pushed to 11.x and cherry-picked to 10.3.x, thanks!

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • Here's a patch for Drupal 10.2.6.

Production build 0.71.5 2024