Workspaces with thousands of items can't be published

Hello all here,

Here is implementation of Batch usage on publishing (to Live workspace).

Thanks for opening. Feature sounds interesting but fixes should be in an MR.

Also will need test coverage.

We can't use a batch because workspaces can also be published via the CLI or by cron, for example during a scheduled deployment. In order for that work correctly, we would need something like ✨ Add an 'instant' queue runner Needs work and/or 📌 Make batch a wrapper around the queue system Needs work .

We've solved this problem in the Workspace Extras → module by extending the PHP execution time after each entity save, but we can improve that solution and only do it for large workspaces.

Opened a MR for this. Testing this would only increase the testing times artificially so I don't think it's worth it. Manual testing can be done locally using Devel generate or some custom code to create 1000 or more entities in a workspace.

Should this be postponed on ✨ Add an 'instant' queue runner Needs work ?

@smustgrave, I don't think so. The current MR provides an immediate fix for the problem, while using a queue or something similar would need to figure out how to provide data integrity at the database level, which is very important when there's an error during the deployment of a workspace and every change needs to be rolled back.

Agree that testing would probably not be worth it to extend time.

Reviewing the code to execution believe it's right
Tested by creating a workspace and generating a number of content items.
Publishing still works.

@smustgrave, how many content items did you create and test?

Since that is important to this particular change I am changing the status.

Here's a relatively quick way to test this:

- change your php_execution_time to something very low, like 3 seconds
- run this script with Drush

drush ev '                             
  $workspace = \Drupal::entityTypeManager()->getStorage("workspace")
    ->create(["id" => "test_3469792", "label" => "Test 3469792"]);
  $workspace->save();

  \Drupal::service("workspaces.manager")->executeInWorkspace($workspace->id(), function () use ($workspace) {
    for ($i = 1; $i <= 5000; $i++) {
      \Drupal::entityTypeManager()->getStorage("path_alias")
        ->createWithSampleValues("path_alias", [
          "workspace" => $workspace->id(),
        ])
        ->save();
    }
  });
'

Then go to /admin/config/workflow/workspaces, activate that workspace and try to publish it. Without the MR you should get an error about max execution time exceeded, and with it the workspace will be published successfully.

Sorry I should of been more clear but I followed the suggestion of using devel generate to create 1000 dummy nodes.

Environment::setTimeLimit() is on its way out, see 📌 Deprecate Environment::setTimeLimit() with no replacement Active and [#74617], should we just use the raw PHP API instead here?

The Needs Review Queue Bot → tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.

Thanks @catch, I didn't know about those issues. Updated the MR to use set_time_limit() instead, then verified using the steps in #12 that it still works as expected.

Not sure what's up with the CSpell failures from the NR bot, so tentatively putting back to RTBC.

Thanks that looks good.

Committed/pushed to 11.x and cherry-picked back through to 10.3.x, thanks!

catch → closed merge request !9325

Automatically closed - issue fixed for 2 weeks with no activity.