Update fails if there are no newsletters with subscriptions

Created on 1 December 2023, 12 months ago
Updated 19 January 2024, 10 months ago

Problem/Motivation

Probably not much of a case on live sites, but still can happen.

In simplenews_update_840002 the NOT IN condition assumes the result of the result of querying simplenews_subscriber__subscriptions is non empty array.

This can result in

>  [error]  Query condition 'id NOT IN ()' cannot be empty. 
>  [error]  Update failed: simplenews_update_840002 

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Fixed

Version

4.0

Component

Code

Created by

🇸🇮Slovenia primsi

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @primsi
  • Status changed to Needs review 12 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update 12 months ago
    61 pass
  • Status changed to Closed: duplicate 12 months ago
  • 🇸🇮Slovenia primsi

    Oh snap, this was already fixed in {#3396668}. Was working on alpha version so I noticed just now when I created the patch :) closing.

  • Status changed to Needs work 12 months ago
  • 🇨🇭Switzerland berdir Switzerland

    Reopening this.

    @AdamPS: FYI, the fix in the other issue is not correct, the issue is also not correctly explained in the issue summary there.

    You select subscribers *with* a subscription and then update all others. Now the query is skipped, but it's a NOT IN condition, that could mean that you have no subscribers with a subscription but still have some without and now they are not updated.

    The query also has scalability issues, if you have tens of thousands of subscribers, it loads all of their ids and then creates a _huge_ IN condition with tens of thousands of separate placeholders.

    One option is to change the query so it can be an IN condition, with a query on subscriber, left join on subscriptions with a condition on subscription being NULL. But if you have a lot those that's still a problem.

    So we could try to same with a nested condition, I think ->condition() supports passing in another select query object, or we could try with an expression. doing a subquery on an UPDATE condition on the same table afaik has some issues, we'll investigate a bit.

  • Status changed to Needs review 12 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update 12 months ago
    61 pass
  • 🇸🇮Slovenia primsi

    Something like this then?

  • Status changed to Needs work 12 months ago
  • 🇬🇧United Kingdom adamps

    Thanks @Berdir for explaining and @Primsi for the patch.

    I believe #5 is a bit mixed up and $has_subscriptions will always be empty. Are you aiming to do this?

    One option is to change the query so it can be an IN condition, with a query on subscriber, left join on subscriptions with a condition on subscription being NULL. But if you have a lot those that's still a problem.

    In which case you would need to

    • rename $has_subscriptions to $no_subscriptions
    • fix the query
    • change NOT IN to IN

    However as Berdir says this solution anyway still has a problem and he suggests some alternative.

    we'll investigate a bit.

    Thanks! Let's see if Berdir can work some magic. I don't know much myself about optimising SQL queries 😃.

  • 🇬🇧United Kingdom adamps
  • 🇨🇭Switzerland berdir Switzerland

    Yes, the patch is not correct, it is indeed a mix.

    As you said, the variable should be renamed and the query should remain condition with an IN condition inside, just like before we only want to run the query if we have any matches.

  • Status changed to Needs review 11 months ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update 11 months ago
    61 pass
  • 🇨🇭Switzerland berdir Switzerland

    Like this.

    inline conditions might be possible but are differently more complicated to handle and I'm not sure about different database backends. This should be performant enough for our case and most others too.

  • Status changed to Fixed 11 months ago
  • 🇬🇧United Kingdom adamps

    Great thanks

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024