Add batch support

Created on 10 June 2016, over 8 years ago
Updated 14 November 2023, about 1 year ago

We should add batch support to our action plugins to prevent timeouts on massive exports. It seems like there is an existing effort to improve batch operations in Drupal 8 core. In the meantime, I think that we could use a callback function. The callback would determine if batch processing is necessary, say over 10 entities exported or something (we can test that later), and it would set the batch if necessary. Then we would need to implement callback_batch_operations() and callback_batch_finished().

To be clear, this is a functional approach since I am not aware of an existing object oriented solution available through core. If someone is aware of a better approach, I would be very excited to hear about it.

I have already implemented a very similar solution in a custom export to Word module at work, so I'm confident that this will at least get the job done. We could/should improve the feature if/when Drupal 8 core provides a better solution to batch operations.

Feature request
Status

Active

Version

2.0

Component

Code

Created by

🇺🇸United States jordanpagewhite

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇧🇪Belgium matthieu_collet

    Hello
    We have a problem with a large export, is it still in the roadmap to add batch support ?

    thank you

    Matthieu

  • 🇨🇦Canada teknocat

    Plus 1 on this! I have a client with many rows to export and it canney handle it.

    After reviewing the source code, I'm not even sure how the action could be updated to work with Views Bulk Operations properly. The action just wants to stream it directly out.

  • 🇧🇷Brazil carolpettirossi Campinas - SP

    +1 to this!

  • 🇮🇹Italy trickfun

    Very usefull with view.
    Thank you

  • 🇮🇹Italy trickfun

    This is my controller to generate PDF view with batch

      /**
       * Starts the batch process to flush image styles.
       */
      public function generate() {
    
        $destination = \Drupal::request()->get('destination');
    
        // Create a new batch builder.
        $batch_builder = new BatchBuilder();
    
        // Set batch title and messages.
        $batch_builder->setTitle($this->t('Generating PDF'))
          ->setInitMessage($this->t('Starting generating PDF...'))
          ->setProgressMessage($this->t('Generate PDF.'))
          ->setErrorMessage($this->t('An error occurred during the generating process.'));
    
        // Add a batch operation 
        $batch_builder->addOperation([__CLASS__, 'buildPDF'], [$destination]);
    
        // Set a callback to handle completion.
        $batch_builder->setFinishCallback([__CLASS__, 'finishBatch']);
    
        // Set the batch.
        batch_set($batch_builder->toArray());
    
        // Return a redirect to the batch processing page.
        return batch_process();
      }
    
      /**
       * Flushes a single image style.
       *
       * @param string $style_name
       *   The name of the image style to be flushed.
       * @param array $context
       *   The batch context.
       */
      public static function buildPDF($destination, &$context) {
    
        // Initialize progress counter, which will keep track of how many items
        // we've processed.
        $context['sandbox']['progress'] = 0;
    
        $context['results'] = [
          'automatic_download' => TRUE,
          'redirect_url' => $destination,
        ];
    
        $export_type = 'pdf';
        $view_name = '<view_name>';
        $display_id = 'page_1';
    
        // Create the Print engine plugin.
        $config = \Drupal::config('entity_print.settings');
        $view = \Drupal::entityTypeManager()->getStorage('view')->load($view_name);
        
        $executable = $view->getExecutable();
        $executable->setDisplay($display_id);
    
        $scheme = 'public';
        $filename = 'pdf_catalog/name.pdf'; 
    
        $print_engine = \Drupal::service('plugin.manager.entity_print.print_engine')->createSelectedInstance($export_type);
        // The Print is sent straight to the browser.
        $path = \Drupal::service('entity_print.print_builder')->savePrintable([$view], $print_engine, $scheme, $filename);
    
        $context['sandbox']['progress'] = 1;
        $context['results']['vde_file'] = $path;
        $context['finished'] = 1;
      }
    
      /**
       * Callback to handle batch completion.
       *
       * @param bool $success
       *   Whether the batch process was successful.
       * @param array $results
       *   The batch results.
       * @param array $operations
       *   The remaining batch operations.
       */
      public static function finishBatch($success, $results, $operations) {
    
        // Set Drupal status message to let the user know the results of the export.
        // The 'success' parameter means no fatal PHP errors were detected.
        // All other error management should be handled using 'results'.
        $response = new RedirectResponse($results['redirect_url']);
        if ($success && isset($results['vde_file']) && file_exists($results['vde_file'])) {
          // Check the permissions of the file to grant access and allow
          // modules to hook into permissions via hook_file_download().
          $headers = \Drupal::moduleHandler()->invokeAll('file_download', [$results['vde_file']]);
    
          // Require at least one module granting access and none denying access.
    
          // Create a web server accessible URL for the private file.
          // Permissions for accessing this URL will be inherited from the View
          // display's configuration.
          $url = \Drupal::service('file_url_generator')->generateAbsoluteString($results['vde_file']);
          $message = t('Export complete. Download the file <a target="_blank" download href=":download_url"  data-download-enabled="false" id="vde-automatic-download">here</a>.', [':download_url' => $url]);
          // If the user specified instant download than redirect to the file.
          if ($results['automatic_download']) {
    
            // Prevents browser from displaying JSON data if automatic download
            // is selected.
            if (!preg_match("/^.*\.(json)$/i", $results['vde_file'])) {
              $message = t('Export complete. Download the file <a target="_blank" download href=":download_url" data-download-enabled="true" id="vde-automatic-download">here</a> if file is not automatically downloaded.', [':download_url' => $url]);
            }
          }
    
          \Drupal::messenger()->addMessage($message);
          return $response;
        }
        else {
          $message = t('Export failed. Make sure the private file system is configured and check the error log.');
          \Drupal::messenger()->addError($message);
          return $response;
        }
    
      }
    
    
Production build 0.71.5 2024