Make non-progressive batch operations possible

Created on 20 November 2009, about 15 years ago
Updated 4 July 2024, 6 months ago

My module does some tasks via Batch API. To avoid duplicating code, I want to do those same tasks during a cron run by programmatically starting and executing the batch without trying to redirect the user to the standard batch job progress bar thingie. There seems to be the facility for this by setting the 'progressive' value of the $process_info array to FALSE, but by the same token, there doesn't seem to be a way to do that - it's hard-coded to TRUE . From D6's form.inc (D7's version is the same in this regard):

function batch_process($redirect = NULL, $url = NULL) {
  $batch =& batch_get();

  if (isset($batch)) {
    // Add process information
    $url = isset($url) ? $url : 'batch';
    $process_info = array(
      'current_set' => 0,
      'progressive' => TRUE, // <= HERE
      'url' => isset($url) ? $url : 'batch',
      'source_page' => $_GET['q'],
      'redirect' => $redirect,
    );
    $batch += $process_info;

    if ($batch['progressive']) {
      // Clear the way for the drupal_goto redirection to the batch processing
      // page, by saving and unsetting the 'destination' if any, on both places
      // drupal_goto looks for it.
      // […snip…]
      drupal_goto($batch['url'], 'op=start&id='. $batch['id']);
    }
    else {
      // Non-progressive execution: bypass the whole progressbar workflow
      // and execute the batch in one pass.
      require_once './includes/batch.inc';
      _batch_process();
    }
  }
}

Unless I'm missing something, this means that, though the functionality exists to do what I want to do, it's not actually possible to do so without hacking core - the code in that "else" branch will never, ever execute. One dead kitten later, I get the expected result.

However, I will allow that I'm just misunderstanding how to use an esoteric feature of this esoteric API and that doing what I need to do is possible without slaughtering kittens; in which case, a bit of edumacation would be appreciated.

Proposed resolution

Set the progressive status in the batch based on batch definition e.g.

  $operations = ['my operations'];
  $batch = [
    'title' => 'My batch runs in single run',
    'operations' => $operations,
    'progressive' => FALSE,
  ];
  batch_set($batch);
  batch_process();

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Needs work

Version

11.0 🔥

Component
Batch 

Last updated 4 days ago

Created by

🇺🇸United States Garrett Albright

Live updates comments and jobs are added and updated live.
  • Needs framework manager review

    It is used to alert the framework manager core committer(s) that an issue significantly impacts (or has the potential to impact) multiple subsystems or represents a significant change or addition in architecture or public APIs, and their signoff is needed (see the governance policy draft for more information). If an issue significantly impacts only one subsystem, use Needs subsystem maintainer review instead, and make sure the issue component is set to the correct subsystem.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
  • Merge request !7999non-progressive batch fix → (Open) created by sukr_s
  • Pipeline finished with Failed
    8 months ago
    Total: 606s
    #168509
  • Pipeline finished with Failed
    8 months ago
    Total: 668s
    #168554
  • Pipeline finished with Failed
    8 months ago
    Total: 556s
    #168576
  • Pipeline finished with Failed
    8 months ago
    Total: 166s
    #168609
  • Pipeline finished with Failed
    8 months ago
    #168610
  • Pipeline finished with Failed
    8 months ago
    Total: 604s
    #168696
  • Pipeline finished with Success
    8 months ago
    Total: 569s
    #168729
  • 🇮🇳India sukr_s

    sukr_s changed the visibility of the branch 638712-make-non-progressive-batch to hidden.

  • 🇮🇳India sukr_s

    sukr_s changed the visibility of the branch 11.x to hidden.

  • Status changed to Needs review 8 months ago
  • 🇮🇳India sukr_s

    With this fix progressive value can be set in batch definition. if progressive is false then it run in a single run else it uses multiple run. usage e.g

    $operations = ['my operations'];
      $batch = [
        'title' => 'My batch runs in single run',
        'operations' => $operations,
        'progressive' => FALSE,
      ];
      batch_set($batch);
      batch_process();
    
  • 🇮🇳India bhanu951

    Bhanu951 changed the visibility of the branch 11.x to hidden.

  • Status changed to RTBC 7 months ago
  • 🇺🇸United States smustgrave

    Removing tests tag as coverage can be seen here https://git.drupalcode.org/issue/drupal-638712/-/jobs/1645989
    Hiding patches since fix is in the MR.

    Issue summary appears complete and believe matches the solution.

    Only one I'm iffy about is

      $this->drupalGet('admin/reports/dblog');
        $this->assertSession()->pageTextContains('Non progressive operation 100');
    

    If there is just a better way to check the values without having to go to the page but may not be a big deal. Was the only thing I saw.

  • Status changed to Needs work 7 months ago
  • 🇳🇿New Zealand quietone

    I read the issue summary, the comments and the MR.

    @satbir.singh, thanks for getting this working. To help reviewers, the information in #69 should be in the issue summary in the proposed resolution section. That is the best place for people to find and understand the consequences of the proposed change. Thanks.

    In #54 the solution of the patch in #47 was questioned as doing too much work. They then preferred the solution from #41. The solution in the MR is difference again. There should be discussion here on the reason the current solution was chosen over the others.

    I am setting back to NW for a comment in the MR and to get agreement on the solution.

  • Pipeline finished with Canceled
    7 months ago
    Total: 1572s
    #183182
  • Pipeline finished with Failed
    7 months ago
    Total: 877s
    #183206
  • Pipeline finished with Success
    7 months ago
    Total: 4432s
    #183276
  • Status changed to Needs review 7 months ago
  • 🇮🇳India sukr_s

    batch_set function accepts the batch definition with progressive as a parameter. #41 introduces a new parameter to the batch_process. Having a new parameter in batch_process can cause conflict between the value set when creating the batch with batch_set vis-a-vis calling the batch_process.

    The proposed solution uses the value set in the batch definition so that it's consistently available throughout the processing of the batch.

  • Status changed to RTBC 6 months ago
  • 🇺🇸United States smustgrave

    Feedback appears to be addressed here

  • Status changed to Needs work 6 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    I think we can make two improvements to the test.

  • Status changed to RTBC 6 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    After reviewing batch_test_stack() I applied all my suggestions to the test and marking back to rtbc as my changes are only test changes.

  • Pipeline finished with Success
    6 months ago
    Total: 511s
    #213809
  • Status changed to Needs work 6 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    So I think that the key comment by @yched in #12 is largely ignored by the current fix. @yched points out that whether or not the batch is process progressively or not should be an option on the batch processor (using the batch_process() function) and not the batch definition. I think we should change tack here and do one of the options suggested in #12. We should also deprecate \Drupal\Core\Batch\BatchBuilder::setProgressive() as this currently does not work and is incorrect for the reasons suggest in that comment.

  • Status changed to RTBC 6 months ago
  • 🇮🇳India sukr_s

    My suggestion is that the current fix is commited so it's immediately usable. In addition open a new ticket for implementing the approach suggested in #12 🐛 Make non-progressive batch operations possible RTBC . So setting back to RTBC.

  • 🇮🇳India sukr_s

    🐛 Refactor Batch API Active has been opened for comment 12

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    @sukr_s but the point about #12 is that the approach of having progressive-ness set by the batch is incorrect. It is a property of the processor not the batch.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Pinging the framework managers to chime in here because I'm not convinced about the current direction as per #77.

  • 🇮🇳India sukr_s

    @alexpott I understand that. As per the current design the progressiveness is a part of the batch_set / batch definition. I'm suggesting that we make the current design work as it was designed by fixing the current bug.

    The improvement of moving the progressive definition from batch definition to batch_process can be taken up in a new ticket.

  • 🇦🇺Australia larowlan 🇦🇺🏝.au GMT+10

    #77 makes sense to me and I think captures the original intent

  • Status changed to Needs work 6 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    @sukr_s as both @larowlan and I agree going to set this back to needs work. This issue should:
    1. Deprecate setting progressive as part of the batch
    2. Convert batch_process and _batch_process into a service
    3. Add a way to process the batch progressively and non-progressively via the new service.

    The reason the API did not work is because it didn't make sense. Making the current change here will cause problems due to

    - it will affect the whole processing, so what if set A specifies 'progressive' and batch B specifies 'non progressive' ?

  • 🇮🇳India sukr_s

    noted. I've closed the other ticket.

Production build 0.71.5 2024