Data Export in Batch fails to acknowledge Facet selections

Created on 18 October 2023, almost 2 years ago

Problem/Motivation

I have a view with >50,000 records. I am trying to export several thousand records as CSV. Standard mode for Views Data Export fails to complete, so I must use Batch mode. But batch mode does not return the correct records if using facets, nor does it return the correct number of records.

I'm running Drupal 9.5.11, PHP 8.1.23.

A similar issue is discussed here (see comment #193), but for 8.x-1.x-dev:
https://www.drupal.org/project/views_data_export/issues/2789531#comment-... ✨ Support for batch operations Fixed

Steps to reproduce

Create a view and add search facets. Add a download CSV button and set the mode to "batch" rather than "standard". Set the batch size to 100 records. If my search facet selections return 50 records, when I click the download button I get 100 records (instead of 50), and none of these downloaded records recognize my facet selections. If my faceted search returns 140 records, when I click the download button I get 200 records (none of which match my facet selections).

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

πŸ’¬ Support request
Status

Active

Version

1.3

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States jhinds

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @jhinds
  • πŸ‡ΊπŸ‡ΈUnited States jhinds
  • Status changed to Needs work over 1 year ago
  • πŸ‡­πŸ‡ΊHungary nevergone NyΓ­regyhΓ‘za, Hungary, Europe

    Please test and review.

  • πŸ‡ͺπŸ‡ΈSpain markus_petrux

    For us it is working properly. The data export display provides an option to choose the Facet source. Using the correct option worked for us... we're using facets_form module too.

    Kind regards

  • πŸ‡©πŸ‡ͺGermany DiDebru

    Patch did the opposite in my case.

  • πŸ‡ΊπŸ‡ΈUnited States jhinds

    The patch in #3 works for me. Results are reflecting facet selections. Thanks!

  • πŸ‡¬πŸ‡§United Kingdom steven jones

    Can we change to a Merge Request please, also:

    +++ b/src/Plugin/views/display/DataExport.php
    @@ -668,7 +668,12 @@ class DataExport extends RestExport {
    +        \Drupal::request()->request->add($query_parameters);
    

    It should be fine to simply change to using this version in all cases? I don't think we need to detect anything here.

  • First commit to issue fork.
  • πŸ‡ΊπŸ‡ΈUnited States jonraedeke

    jonraedeke β†’ changed the visibility of the branch 8.x-1.x to hidden.

  • Pipeline finished with Success
    3 months ago
    Total: 415s
    #499029
  • πŸ‡ΊπŸ‡ΈUnited States jonraedeke

    I created an MR for review based on the comment by steve Jones in #7.

    This working really well for me.

  • πŸ‡¬πŸ‡§United Kingdom mike-kelly

    I am still having problems with this in Drupal 10.4.6, with views_data_export v1.6.

    I applied the patch in #3 (also tried just using #7), but the facet-related query params are ignored when returning results in batch export mode. Facet filtering works fine in standard export mode.

    Setting breakpoints I can see that in display/DataExport.php file, in the buildBatch function, the number of $total_rows is correct and takes accounts of facets:

     if ($view->getQuery() instanceof SearchApiQuery) {
          // this line returns the correct value
          $total_rows = $view->query->getSearchApiQuery()->range(NULL, 1)->execute()->getResultCount(); 
        }
        else {
          $count_query_results = $view->query->query()->countQuery()->execute();
          $total_rows = (int) $count_query_results->fetchField();
        }
    

    However, in batchProcess, the results returned do not take account of facets.

    I have selected the correct facet source in the view configuration, as evidenced by the success of standard export.
    I am using a searchAPI datasource as a contextual filter if that is relevant.

    Any suggestions how to troubleshoot this? Thanks.

Production build 0.71.5 2024