Cron update either exceeding time-out or exhausting server memory

Oddly enough, this OOME happened on dev while testing, but not production. Running drush cron on production has memory holding steady at 10.3 GB with the CPU sitting ~85% of that core for ~44 minutes now.

We are also experiencing this issue on a site, it appears to run out of memory at the line:

$entities = $entity_storage->loadMultiple(array_unique($flat_ids));

when trying to load ~65k node ids, in web/core/modules/views/src/Plugin/views/query/Sql.php:1625

Digging into it a bit further, it seems to happen when processing field values for an entity_reference field that references other nodes, in our case when $table equals node__field_member_of in the following loop, it runs out of memory:

web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php:1229

I don't have a solution to this yet, just letting you know I'm also seeing it and this is where it borks for me. This also started happening after upgrading Drupal from 9.4.14 to 9.5.9.

This happens when:
* REST OAI-PMH Queue worker fires on cron
* Viewing the feed at /oai/request?verb=ListRecords&metadataPrefix=oai_dc
* Rebuilding the feed at /admin/config/services/rest/oai-pmh/queue

It looks to me like there should be a limit set when executing the view inside RestOaiPmhViewsCacheBase::processItem($data), similar to how it is done in rest_oai_pmh_cache_views() - because when the processItem() method is called we already know the offset and limit we're working with, so we shouldn't have to load all results for the view - if I'm understanding this correctly?

• RestOaiPmhViewsCacheBase.php#L35
• rest_oai_pmh.module#L194

I've got a small patch I'll upload shortly that gets things working again for me, would be keen to get some feedback on whether I've understood this correctly or am missing how the queue/processItem system works?

Steps I used to reproduce the issue

1. Create a Drupal 10 Lando development environment
2. Composer install drupal/rest_oai_pmh, drupal/devel_generate
3. Generate 100,000 nodes of type Article
4. Create an entity reference source view of those articles
5. Configure rest_oai_pmh to use that view and set cache strategy to Conservative (also set metadata format to oai_dc)
6. Rebuild feed and observe out of memory error in SqlContentEntityStorage
7. Apply patch, rebuild feed and it rebuilds successfully

I think the reason the entity reference view display tries to load all results by default is that the entity_reference_options has limit set to 0 in https://git.drupalcode.org/project/drupal/-/blob/9.5.x/core/modules/view... and so it has to be limited by a call to:

    $view->getDisplay()->setOption('entity_reference_options', ['limit' => $view->getItemsPerPage()]);

I applied the MR to our site and cron is no longer crashing and I can see the records and set membership increasing in record counts each time. 👍

Can we get this merged?

Thank you Seth for reporting and @code_brown for the patch. The patch has been merged and beta12 released with the fix.

Automatically closed - issue fixed for 2 weeks with no activity.