Hi all.
We've got a site that is having approx 7300 users and we've been plagued as well with the xhr/post request on the `user_selection/users` callback. The server hosting this site is having PHP 8.1 with 512Mb of available PHP memory and 3 minutes timeout, with the `opigno_learning_path` module at version 3.1.1
While it is quite nicely written in terms of coding, ie respecting the entities etc, it just is not that performant on a high volume of users (and trainings/classes)
This is mainly due to the fact that each entity is trying to get cached so if you have a database of more than 5kg users on the site, the caches break due to out of memory limitations, if the php timeout doesn't kick in.
One would say, why don't you just add more memory ? Well, this will buy you some time but only for a while: I tried to pump up my test site with 10k contacts and everything collapsed on the `group/xxx/content/add/group_membership` page, I got a WSOD only by visiting that page (`group/xxx/content/add/group_membership`).
Phase 1
As a first countermeasure, I tried to simplify/lighten the arrays by removing the email & avatar rendering process on each php call on the submodule
`opigno_user_selection` of `opigno_learning_path` -> https://git.drupalcode.org/project/opigno_learning_path/-/blob/3.1.1/mod... and https://git.drupalcode.org/project/opigno_learning_path/-/blob/3.1.1/mod...
Additionally, in order to make it more understandable to the end-user, I replaced the 'name' => $user->getDisplayName() with the 'name' => $user->getDisplayName() . ' [ ' . $user->id() . ']', so that they could see the user IDs along with the display names.
Then, I replaced the call opigno_messaging_get_all_recipients with my own call where I do a direct DB query to fetch the list of users based on conditions (user status -> 1 and user is not anonymous)
The above corrections started to make things functional again, on my original site with 7300 users, however, it was not enough with 10k users so I had to dig deeper.
Phase 2
On top of the above changes, I've started doing a more aggressive work, I've decided to ditch anything that relates to the function loadMultiple as this was the source of memory exhaustion and replaced them with direct SQL queries.
I've also replaced the getDisplayName() calls since I was having first + last name inside my SQL query results so there was no point to re-request that function. Additionally, for 7k contacts, the amount of calls to this function was massive.
The last thing I did was to drop most nested php calls of this sort:
'member' => array_filter(array_map(function (GroupMembership $membership) use (&$groups_id, $map) {
$group = $membership->getGroup();
if (array_key_exists($group->bundle(), $map)) {
$groups_id[] = ($gid = (int) $group->id());
return $gid;
}
return NULL;
}, $memberships)),
by preparing the sql statement, running it, creating an array of users and assigning that array to the `members` element instead of the nested functions.
Doing the above (Phase 2) did gave me a huge improvement and the data are now rendering properly and quickly. I have some doubts on some parts of my changes (especially one that does a permission check).
I can provide a patch though it needs to be carefully tested on the output as I might have missed something!
I believe this issue could be related with the following:
*
https://www.drupal.org/project/opigno_learning_path/issues/3244973 →
*
https://www.drupal.org/project/opigno_learning_path/issues/3326247
🐛
Imposible Add member to learningPath
Active
*
https://www.drupal.org/project/opigno_learning_path/issues/3397054
🐛
Opigno user selection when adding a new user to opigno_class returns 500 error
Active