VBO 4 doesn't process every selected node

Created on 8 December 2021, about 3 years ago
Updated 16 August 2023, over 1 year ago

Problem/Motivation

The site I'm working on has a large number of nodes (~15,500). We fairly consistently run into an issue where views_bulk_operations doesn't process all of the selected items when I "Select all on this page", nor when I "Select all results in the view" I've included some steps to reproduce with the "Select all results in the view"

When I did some debugging for the "Select all on this page", I find that view_bulk_operations seems to successfully POST all 50 checkbox values to the backend (i.e.: /views-bulk-operations/ajax/content/page_1, and the confirmation step seems to show 50 nodes; but when the actions actually run, they don't act on all the items. The number of items processed also seems to vary... sometimes its 42, sometimes it's 46, sometimes it's 48, sometimes its all 50. When I select all items in the view, it's often low, like 2 items processed, or 0 items processed.

Doing a bit more debugging of the data stored in the private tempstore, I see...

{
  "owner": "25",
  "data": [
    "view_id" => "content",
    "display_id" => "page_1",
    "list" => [
      // Sometimes this is empty and, when exclude_mode = false, nothing gets processed.
      // Sometimes there are less than 50 nodes here, and less than 50 nodes get processed.
      // Sometimes there are 50 nodes here, and all 50 nodes get processed.
    ],
    "exclude_mode" => false,
    "batch" => true,
    "batch_size" => 10,
    "total_results" => "15472",
    "relationship_id" => "none",
    "arguments" => [],
    "exposed_input" => [
      // (redacted)
    ],
    "bulk_form_keys" => [
      // This seems to always have the correct number of entries, i.e.: if I select all on a page, I see 50 values here, even if there are less than 50 values in object.data['list'] above.
    ],
    "redirect_url" => Drupal\Core\Url {#5336},
  ],
  "updated": 1637280422,
}

My guess is that whatever is supposed to copy/move data from object.data['bulk_form_keys'] to object.data['list'] doesn't always move everything.

It's also hard to reproduce this on sites without many nodes, fields, or content types. Maybe it runs out of time or memory? But I'd expect to see an error or message in the log if that happened.

Steps to reproduce

  1. git clone --branch '9.3.x' https://git.drupalcode.org/project/drupal.git
    • The latest commit when I ran this test case was 72d15850a271e3718bf74d71a8207614c79b6918, committed Tue Dec 7 16:55:28 2021 +0100 (authored Thu Nov 25 15:52:00 2021 +0100 - it was a cherry-pick).
  2. composer install
  3. composer require 'drush/drush' 'drupal/devel:^4.1'
  4. cd modules/
  5. git clone --branch '4.0.x' https://git.drupalcode.org/project/views_bulk_operations.git
    • The latest commit when I ran this test case was d83b999e87dac745057a1a746a0ba038d63258fd committed Mon Nov 15 10:26:13 2021 +0100.
  6. cd ..
  7. drush -y si minimal
  8. Log in as the super-administrator
  9. Go to /admin/modules. Install the modules with machine names views_bulk_operations, views_ui, devel_generate and all their dependencies.
  10. Go to /admin/structure/views/view/content:
    1. Add field "Global: Views bulk operations". Select Action "Unpublish content item" and check "Add confirmation step". Select Action "Publish content item" and check "Add confirmation step". Apply.
    2. Remove field "Content: Node operations bulk form". Apply.
    3. Save the view.
  11. Go to /admin/structure/types/add. Set name = Page. Leave all other settings at default. Click Save content type
  12. Go to /admin/config/development/generate/content. Set: Content type = (checked) (this will check "Page" automatically), and How many nodes would you like to generate? = 12000. Leave all other settings at their default values. Click Generate
  13. Go to /admin/content. You see a table of 50 nodes, ordered by "Updated", descending.
  14. Select all items on the page using the checkbox. Select Action "Unpublish content item".
    • Expected behavior: The #edit-multipage details element says "Selected 50 items in this view". Expanding the details element shows a list of all 50 items selected.
    • Actual behavior: The #edit-multipage details element says "Selected 50 items in this view". Expanding the details element shows "No items".
  15. Click Apply to selected items.
  16. You see a page saying "Are you sure you wish to perform "Unpublish content item" action on 50 entities", and a list of 50 "Items selected". Click Execute action.
  17. A batch process runs.
  18. You are returned to /admin/content. You see the status message "Action processing results: Unpublish content item (50).". You see a table of 50 nodes, ordered by "Updated", descending. The first 50 rows in the table are unpublished.
  19. Check "Select / deselect all results in this view (all pages, 12000 total)" Select Action "Unpublish content item". Click Apply to selected items.
  20. You see a page saying "Are you sure you wish to perform "Publish content item" action on 12000 entities? / Action will be executed on all items in the view." Click Execute action.
  21. A batch process runs.
  22. You are returned to /admin/content. You see the status message "Action processing results: Publish content item (12000)."
  23. You see a table of 50 nodes, ordered by "Updated", descending.
    • Expected behavior: The first 50 rows in the table are published.
    • Actual behavior: The first 28 rows in the table are published. The remaining 22 rows are unpublished. (or at least, that's what happened in my run)

Note that there are no notice, warning, or error messages:

$ drush -y wd-list
---- -------------- ---------------- ---------- -----------------------------------------
ID   Date           Type             Severity   Message
---- -------------- ---------------- ---------- -----------------------------------------
21   08/Dec 09:13   node             Notice     Added content type Page.
20   08/Dec 09:12   system           Info       devel_generate module installed.
19   08/Dec 09:03   page not found   Warning    /admin/stucture
18   08/Dec 09:02   system           Info       views_ui module installed.
17   08/Dec 09:02   page not found   Warning    /admin/structure/views
16   08/Dec 09:02   page not found   Warning    /admin/structure/veiws
15   08/Dec 09:02   system           Info       views_bulk_operations module installed.
14   08/Dec 09:02   system           Info       views module installed.
13   08/Dec 09:01   user             Notice     Session opened for admin.
12   08/Dec 09:01   cron             Info       Cron run completed.
---- -------------- ---------------- ---------- -----------------------------------------

Proposed resolution

(To be determined)

Remaining tasks

(To be determined)

User interface changes

(To be determined)

API changes

(To be determined)

Data model changes

(To be determined)

🐛 Bug report
Status

Fixed

Version

4.2

Component

Core

Created by

🇨🇦Canada mparker17 UTC-4

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇪🇸Spain grota

    I think this is the way to reproduce it:
    - create a view using a table style plugin.
    - add some fields: at least a vbo field, configured to use batch, and a date field, used for the next point.
    - configure the table style plugin to use a date field as default sorting option.

    This should be enough.

    The problem is in ViewsBulkOperationsActionProcessor::getPageList.
    The workaround from 3020922 is nullified when it calls $this->view->build(); since it calls $this->style_plugin->buildSortPost() which in our case is in Drupal\views\Plugin\views\style\Table::buildSortPost() which in turn ends up calling Drupal\views\Plugin\views\field\EntityField->clickSort() which sets the order by statement based on the table's default sorting field.

    I don't know what a proper fix should be, but I hope the previous description can be useful to find one.

    I have put the following workaround in place based on hook_views_query_alter:
    As a note, I also tried an approach based on hook_views_pre_execute without success (probably because the query was already built).

    use Drupal\views\Plugin\views\query\Sql;
    
    /**
     * Whether the view is configured in batch mode.
     */
    function xxx_vbo_view_is_vbo_configured_in_batch_mode(ViewExecutable $view) : bool {
      $fieldHandlers = $view->getHandlers('field');
      foreach ($fieldHandlers as $fieldHandler) {
        if (($fieldHandler['plugin_id'] ?? NULL) === 'views_bulk_operations_bulk_form') {
          return $fieldHandler['batch'] ?? FALSE;
        }
      }
      return FALSE;
    }
    
    /**
     * Get base field alias for a view.
     *
     * Algorithm copied from
     * \Drupal\views_bulk_operations\Service\ViewsBulkOperationsActionProcessor::populateQueue.
     */
    function xxx_get_view_base_field_alias(ViewExecutable $view) : string {
      $base_field = $view->storage->get('base_field');
      if (isset($view->query->fields[$base_field])) {
        if (!empty($view->query->fields[$base_field]['table'])) {
          $base_field_alias = $view->query->fields[$base_field]['table'] . '.' . $view->query->fields[$base_field]['alias'];
        }
        else {
          $base_field_alias = $view->query->fields[$base_field]['alias'];
        }
      }
      else {
        $base_field_alias = $base_field;
      }
      return $base_field_alias;
    }
    
    /**
     * Implements hook_views_query_alter().
     */
    function xxx_views_query_alter(ViewExecutable $view, QueryPluginBase $query) {
      // Operate only on views currently run by VBO.
      if (!isset($view->views_bulk_operations_processor_built)) {
        return;
      }
      // Operate only on VBOs that are configured in batch mode.
      if (!xxx_vbo_view_is_vbo_configured_in_batch_mode($view)) {
        return;
      }
      if (!$query instanceof Sql) {
        return;
      }
      $base_field_alias = xxx_get_view_base_field_alias($view);
      $query->orderby = [
        [
          'field' => $base_field_alias,
          'direction' => 'ASC',
        ],
      ];
    }
    
  • 🇵🇱Poland Graber

    Hmm, can you try setting style plugin to default just before $this->view->build(); in the action processor?
    $this->view->setHandler($this->bulkFormData['display_id'], 'style', 'default', []); (not sure about the last argument..).

  • 🇵🇱Poland Graber

    ok, more like
    $this->view->style_plugin = Views::pluginManager('style')->createInstance('default');
    The plugin needs to be initialized as well though..

  • 🇪🇸Spain grota

    Hi @Graber, sorry for the delay in answering. My proposed fix above had to be reverted because it actually introduced other regressions (incorrect elements on which ultimately the action is executed on) while selecting items across pages.
    As soon as we'll be able to have another go at fixing the problem I'll let you know.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    15 pass
  • @grota opened merge request.
  • Status changed to Needs review over 1 year ago
  • 🇪🇸Spain grota

    Hi @Graber,
    I saw that you added those 2 lines in 22d6240fd295ec59bb758e8279046507fbbc528b. I tested them and they fixed the issue for me.
    I created this MR to add a test.

    As far as I know there are no other scenarios right now where the set of selected rows is incorrect, so for me this issue can be closed.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.2 & MySQL 8
    last update over 1 year ago
    15 pass
  • Status changed to Fixed over 1 year ago
  • 🇵🇱Poland Graber

    TBH.. I accidentally added those lines in that commit, that should've been tested.. Oh well, I'm glad it worked and thank you for the test coverage!

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024