Performance Issues, Out of memory with large number of users

Created on 8 November 2022, about 2 years ago
Updated 23 November 2023, about 1 year ago

Problem/Motivation

I have a case where this module is being used in a site with over 20k users.
We had to disable the module because cron was running out of memory when being executed.

Proposed resolution

I've searched around and could probably implement queue API.

Another option is creating a separate drush command to run batch operations in a separate cron.

✨ Feature request
Status

RTBC

Version

2.0

Component

Code

Created by

πŸ‡§πŸ‡·Brazil igoragatti

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • First commit to issue fork.
  • πŸ‡¬πŸ‡§United Kingdom 3li U.K. πŸ‡¬πŸ‡§
  • πŸ‡¬πŸ‡§United Kingdom 3li U.K. πŸ‡¬πŸ‡§

    This is just the plain patch of the MR.
    Just to note this is quite a refactor as I created another service to handle the optional batch processing and added in a drush command so you can run the command with a separate cron.

    Have tested this on a site that has 40,000 users that we class as inactive and via drush seemed to run fine.

    To note I did not add batching to the delete command as this was not something I wanted, but someone else might want to update it to do that.

    I also added an option to enable/disable cron so you can run everything when you want, will also need to run update to make sure that cron gets enabled.

  • πŸ‡¬πŸ‡§United Kingdom 3li U.K. πŸ‡¬πŸ‡§

    Oh forgot to mention the drush commands:

    • drush block-inactive-users:block --run

    Without --run no updates will be made just for testing/count.

  • Status changed to Needs review over 1 year ago
  • πŸ‡¬πŸ‡§United Kingdom 3li U.K. πŸ‡¬πŸ‡§
  • πŸ‡ΏπŸ‡¦South Africa rawdreeg

    This is a massive effort. Thanks for your contribution @3li. I will need to review this and probably release it as part of 3.0 beta. Especially, that we'll be dropping support for < php 7.4.

  • πŸ‡¨πŸ‡­Switzerland colorfield Lausanne

    Thanks @3li, I did a manual test on a large set of users and confirming that it works as expected so RTBC+1

  • Status changed to RTBC over 1 year ago
  • πŸ‡ΏπŸ‡¦South Africa rawdreeg
  • πŸ‡ΏπŸ‡¦South Africa rawdreeg

    Hey 3li,

    Decide to move this to 3.x and create a beta build with the changes. I am happy to take over the PR and sort out the conflicts. Let me know if that's okay.

  • πŸ‡¬πŸ‡§United Kingdom 3li U.K. πŸ‡¬πŸ‡§

    Hey rawdreeg,
    Yeh thats fine, let me know if I can do anything though.

Production build 0.71.5 2024