Subscription emails can send multiple times to same user

Problem/Motivation

If a very large number of subscribers receive immediate email notifications, it is possible that the system will send duplicate emails to the same user. I believe this is because the module's email queuing system can place user/emails in the queue multiple times.

A modification/simplification of the system may be needed.

From what I can tell, after you publish an Announcement, cron claims the cron_announcement_builder item from the queue. The item is given an "expires" time of around 5 seconds or less.

The processing of the item is done in the group_subscription module. It queries the users that should receive the announcement and adds new cron_announcement_immediate items to the queue for each of those emails. It sometimes doesn’t finish this before the next cron run (in our site's case, that is every 5 minutes), and “releases” the item back to the queue.

When an item is “released” with a DatabaseQueue, it sets its “expires” back to 0 so that another worker knows to claim it. I think this is just what happens — and the whole process starts all over again, adding new items to the queue to send out emails.

TL;DR: Processing the first cron_announcement_builder queue item (that creates other queue items) takes too long, gets put back into the queue, and gets run again.

Steps to reproduce

This is very difficult to reproduce without a very large number of subscribers. Our site has around 10K-20K subscribers.

Proposed resolution

The process of having a queue item that generates other queue items may be too complicated — and too prone to error. Basically, the site just needs to know whether to create cron_announcement_immediate items or not. It's either a "yes" or "no."

Would it make sense to use a state variable instead? Something like:

if (\Drupal::state()->get('group_subscription.should_queue_items') {
  // Add cron_announcement_immediate items.
  ...
  \Drupal::state()->set('group_subscription.should_queue_items', FALSE);
}

(I'm sure there is some important logic I'm missing or something I've forgotten about queueing, but I hope that conveys my idea.)