Batch mode does not allow other modules to alter query

Created on 26 September 2020, about 4 years ago
Updated 23 August 2024, 3 months ago

Problem/Motivation

The website encountered an unexpected error. Please try again later.</br></br><em class="placeholder">Drupal\Component\Plugin\Exception\PluginNotFoundException</em>: The &quot;&quot; entity type does not exist. in <em class="placeholder">Drupal\Core\Entity\EntityTypeManager-&gt;getDefinition()</em> (line <em class="placeholder">150</em> of <em class="placeholder">core/lib/Drupal/Core/Entity/EntityTypeManager.php</em>). <pre class="backtrace">group_query_entity_query_alter(Object) (Line: 259)
group_query_views_entity_query_alter(Object, NULL, NULL) (Line: 539)
Drupal\Core\Extension\ModuleHandler-&gt;alter(&#039;query&#039;, Object) (Line: 480)
Drupal\Core\Database\Query\Select-&gt;preExecute() (Line: 488)
Drupal\Core\Database\Query\Select-&gt;preExecute() (Line: 505)
Drupal\Core\Database\Query\Select-&gt;execute() (Line: 80)
Drupal\views_data_export\Plugin\views\display\DataExport::buildBatch(Object, Array) (Line: 52)
Drupal\views_data_export\Plugin\views\display\DataExport::buildResponse(&#039;user_admin_people&#039;, &#039;data_export_1&#039;, Array) (Line: 53)

Google teaches that for other people who have hit the same error, the root cause is this VBO issue: #3163912: VBO does not preserve query metadata added by other modules . I don't have VBO installed, and I don't hit the same error when using the 'standard' rathe than 'batch' export mode. Therefore we can conclude that this module's bath export mode is making the same mistake that VBO was, and failing to allow other modules to alter the query.

This may or may not be the same issue as #3155609: query_alter addWhere clause not applied to csv .

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

RTBC

Version

1.0

Component

Code

Created by

🇬🇧United Kingdom jonathanshaw Stroud, UK

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇳🇿New Zealand RoSk0 Wellington

    Spent several hours debugging 🐛 _access tag can leading to fatal error Closed: won't fix that lead to creating 🐛 Query metadata get lost when $view->query->query() is called multiple times during page build Active .

    I was going to propose a similar change to #12, but surprised people reporting it as problematic as my understanding was quite firm on that being the correct approach. It would be awesome if people can share details about what that approach breaks.

    I completely share the concerns of #23 around performance , but mostly afraid of the failing tests . Quick patch test however succeeded, but that is only a quick test full test suite is running and I will report back here later.

    I don't quite understand the purpose of modifying cache configuration of the view, so proposing a variation of #10 without changing cache settings , but with restoration of the original configuration as I believe this is crucial part that was missing in the original implementation - without it modified view is passed further for batch processing.

  • 🇳🇿New Zealand RoSk0 Wellington

    Remove unused use statement.

  • 🇳🇿New Zealand RoSk0 Wellington

    Test suite we have on a site has passed many times with patch from #25.

  • 🇮🇷Iran sgarsivaz

    I use views_data_export:^1.2 and have more than 10K row and 5 column to export. three of this 5 column is views_field_view which fetch data from different views.
    without any patch I can not export any thing. by using #25 patch problem solved.
    but I have to use batch size = 100. any batch size more than that result in an error. by this batch size it takes me 30 to 40 minutes.

  • 🇬🇧United Kingdom jonathanshaw Stroud, UK

    #25 looks reasonable.

    This seems tricky to debug and reason about, so a test would be really good. This would also establish that it was fine to follow #25 in not unsetting the cache.

  • 🇨🇦Canada Ron Collins

    #25 fixed the issue for me.

  • heddn Nicaragua

    #25 fixed the issue for me too. It wasn't clear if that causes a re-execution of the query or not. Can someone confirm?

  • 🇨🇦Canada nruest

    #25 fixes the issue for me as well.

  • 🇮🇳India vishal.kadam Mumbai

    Those who are experiencing data inconsistency issue in batch as compared to standard mode.

    I've identified the source of the batch export limit problem. It has to do with the sequence of query results.

    In order to prevent changes in the query result sequence, add the unique fields to the sort.

  • 🇸🇮Slovenia alesbencina

    I've tried patch #25 and worked well. The issue I had was that the view exposed filters weren't taken in countQuery() and therefore it always resulted into max number of rows (I have more than 100k in a table).

    Acctually $view->build(); helped a lot which added the filters from the request.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.2.x + Environment: php8.3_mysql8_ignore
    last update 5 months ago
    3 pass
  • Status changed to RTBC 3 months ago
  • 🇺🇦Ukraine rollins

    I can confirm that patch #25 resolved the bug
    Thank you @rosk0

  • 🇺🇸United States maskedjellybean Portland, OR

    #25 worked for me as well. Thanks!

Production build 0.71.5 2024