Contrib.social

Explore additional ways to load large number of items

Open on Drupal.org →
Created on 12 April 2023, over 1 year ago
Updated 13 January 2024, 10 months ago

Problem/Motivation

Follow up from 🐛 New Checks/NamePasswords does not scale with a large number of user accounts Fixed

Possible further solutions:
#1 More Queue. Each test is already being processed as a queued item, I'm not sure how we would then create and run another dependent queue as part of this tests execution. If the original test could populate a new queue of users, and then use its time to process that queue, we could go as slow as we want and scale forever.

#2 Skip all the user loading that costs overhead by leaving authService out of the picture. We could then pull only name/password from db and then jump straight to the important part of the code where the PasswordInterface's check() function is called. We could just call that ourselves and hopefully skip a bunch of intermediary function calls that might incur sql queries of their own.

#3 Skip drupal by pushing the logic all into sql. Since the heart of the password check is an md5 hash comparison (ignoring old style D7 passwords), we can check the existing stored hash against and the username as part of the query directly. This might theoretically be faster but would have a fragile assumption that the md5 in use by the database is comparable to the md5 used by the php crypt library.

Would be nice to implement something generic

Steps to reproduce

Create 15-20K users

Proposed resolution

TBD

Remaining tasks

Figure out best appraoch
Implement
Review

User interface changes

TBD

API changes

TBD

Data model changes

TBD

✨ Feature request
Status

Fixed

Version

3.0

Component

Code

Created by

🇺🇸United States smustgrave

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @smustgrave
  • Status changed to Postponed about 1 year ago
  • Comment about 1 year ago →
    🇺🇸United States smustgrave

    So currently floating the idea of redoing security review completely for 3.x and think this would be a good feature.

  • Comment 12 months ago →
    🇺🇸United States rex.barkdoll

    I just want to add my support of this request. We also have a site with 15K+ users and this is one check we can't perform due to system resource limitations. Batching the process or being able to filter by role would be amazing.

  • Status changed to Active 12 months ago
  • Comment 12 months ago →
    🇺🇸United States smustgrave

    3.x isn't taking off quickly, so unpostponing this.

  • Status changed to Needs review 11 months ago
  • Comment 11 months ago →
    🇺🇸United States smustgrave

    Not production ready yet but currently redoing the module to use plugins and more batch jobs. I have the name_password check converted, if someone could maybe test locally?

  • Status changed to Fixed 11 months ago
  • Comment 11 months ago →
    🇺🇸United States smustgrave

    Going to stick with the batches for now. If a different method is needed we can reopen. Hoping to have a stable release next month.

  • Comment 10 months ago →
    System Message

    Automatically closed - issue fixed for 2 weeks with no activity.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024