- Issue created by @alexpott
- π¬π§United Kingdom alexpott πͺπΊπ
With this change we see a massive difference
[notice] Synchronized configuration: update views.view.archive. [6.76 sec, 135.48 MB] [notice] Synchronized configuration: update views.view.association_entity_reference. [6.77 sec, 136.38 MB] [notice] Synchronized configuration: update views.view.associations_multiply. [6.79 sec, 137.76 MB] [notice] Synchronized configuration: update views.view.authmap. [6.8 sec, 138.55 MB] [notice] Synchronized configuration: update views.view.changelogs. [6.8 sec, 138.95 MB] [notice] Synchronized configuration: update views.view.channel_errors. [6.82 sec, 140.29 MB] [notice] Synchronized configuration: update views.view.channels. [6.83 sec, 141.26 MB] [notice] Synchronized configuration: update views.view.code_pools_collections. [6.84 sec, 141.95 MB] [notice] Synchronized configuration: update views.view.content. [6.86 sec, 135.06 MB] [notice] Synchronized configuration: update views.view.current_user_organizations. [6.88 sec, 136 MB] [notice] Synchronized configuration: update views.view.custom_forms. [6.9 sec, 138.02 MB] [notice] Synchronized configuration: update views.view.customer. [6.95 sec, 138.05 MB] [notice] Synchronized configuration: update views.view.customer_emails. [7.32 sec, 155.2 MB] [notice] Synchronized configuration: update views.view.easy_email_content. [7.33 sec, 156.01 MB] [notice] Synchronized configuration: update views.view.email_evs_to_customer_report. [7.34 sec, 157.57 MB] [notice] Synchronized configuration: update views.view.event. [7.35 sec, 158.29 MB] [notice] Synchronized configuration: update views.view.event_winners. [7.38 sec, 153.9 MB] [notice] Synchronized configuration: update views.view.events_list. [7.39 sec, 154.64 MB] [notice] Synchronized configuration: update views.view.events_summary. [7.4 sec, 155.43 MB] [notice] Synchronized configuration: update views.view.evs_file_upload_type. [7.4 sec, 155.73 MB] [notice] Synchronized configuration: update views.view.evs_to_customer_submissions. [7.43 sec, 154.71 MB] [notice] Synchronized configuration: update views.view.ffh_approval_queue. [7.47 sec, 154.67 MB] [notice] Synchronized configuration: update views.view.files. [7.49 sec, 156.25 MB] [notice] Synchronized configuration: update views.view.frontpage. [7.51 sec, 153.5 MB] [notice] Synchronized configuration: update views.view.glossary. [7.52 sec, 154.24 MB] [notice] Synchronized configuration: update views.view.manage_affiliates_banners. [7.53 sec, 154.92 MB] [notice] Synchronized configuration: update views.view.manage_channels. [7.54 sec, 155.62 MB] [notice] Synchronized configuration: update views.view.manage_offer_banners. [7.56 sec, 153.71 MB] [notice] Synchronized configuration: update views.view.manage_offer_groups. [7.58 sec, 155.09 MB] [notice] Synchronized configuration: update views.view.manage_offers. [7.64 sec, 156.49 MB] [notice] Synchronized configuration: update views.view.manage_organization_channel_mappings. [7.65 sec, 157.2 MB] [notice] Synchronized configuration: update views.view.manage_organizations. [7.68 sec, 160.52 MB] [notice] Synchronized configuration: update views.view.manage_poe_uploads. [7.69 sec, 161.13 MB] [notice] Synchronized configuration: update views.view.manage_pricing_promotion_announcements. [7.7 sec, 161.81 MB] [notice] Synchronized configuration: update views.view.manage_regions. [7.72 sec, 153.73 MB] [notice] Synchronized configuration: update views.view.manage_sbs_banners. [7.73 sec, 154.38 MB] [notice] Synchronized configuration: update views.view.manage_sub_offers. [7.77 sec, 155.81 MB] [notice] Synchronized configuration: update views.view.manage_telus_employees. [7.78 sec, 156.94 MB] [notice] Synchronized configuration: update views.view.manage_webstore_urls. [7.81 sec, 154.35 MB] [notice] Synchronized configuration: update views.view.message. [7.82 sec, 155.16 MB] [notice] Synchronized configuration: update views.view.offer_sub_offers. [7.85 sec, 153.86 MB] [notice] Synchronized configuration: update views.view.pricing_announcement. [7.86 sec, 154.78 MB] [notice] Synchronized configuration: update views.view.redirect. [7.87 sec, 155.66 MB] [notice] Synchronized configuration: update views.view.system_notifications. [7.91 sec, 155.07 MB] [notice] Synchronized configuration: update views.view.taxonomy_term. [7.92 sec, 155.56 MB] [notice] Synchronized configuration: update views.view.telus_channels. [7.92 sec, 156.22 MB] [notice] Synchronized configuration: update views.view.telus_store. [7.95 sec, 154.59 MB] [notice] Synchronized configuration: update views.view.telus_user_entity_reference. [7.98 sec, 154.55 MB] [notice] Synchronized configuration: update views.view.unconfigured_accounts_report. [7.99 sec, 155.32 MB] [notice] Synchronized configuration: update views.view.user_admin_people. [8.06 sec, 156.58 MB] [notice] Synchronized configuration: update views.view.view_all_epp_organizations. [8.09 sec, 153.86 MB] [notice] Synchronized configuration: update views.view.view_epp_organizations. [8.11 sec, 155.9 MB] [notice] Synchronized configuration: update views.view.track_announcement_acknowledgements. [8.14 sec, 154.3 MB] [notice] Synchronized configuration: update views.view.telus_request. [8.21 sec, 158.04 MB] [notice] Synchronized configuration: update views.view.splash_page_sbs_orgs. [8.22 sec, 158.43 MB] [notice] Synchronized configuration: update views.view.sbs_submissions_table. [8.48 sec, 175.01 MB] [notice] Synchronized configuration: update views.view.sbs_submissions_data_lake_report. [8.54 sec, 180.27 MB] [notice] Synchronized configuration: update views.view.sbs_data_lake_report. [8.58 sec, 184.39 MB] [notice] Synchronized configuration: update views.view.referral_results_dashboard. [8.67 sec, 160.15 MB] [notice] Synchronized configuration: update views.view.rebate_form_submissions. [8.73 sec, 156.97 MB] [notice] Synchronized configuration: update views.view.poe_audit. [8.76 sec, 160.05 MB] [notice] Synchronized configuration: update views.view.perks_campaign. [8.84 sec, 158.06 MB] [notice] Synchronized configuration: update views.view.migration_requests. [8.87 sec, 161.27 MB] [notice] Synchronized configuration: update views.view.master_event_report. [8.98 sec, 161.34 MB] [notice] Synchronized configuration: update views.view.manage_channel_mappings. [8.99 sec, 162.51 MB] [notice] Synchronized configuration: update views.view.lead_nurturing_for_dealers. [9.03 sec, 154.6 MB] [notice] Synchronized configuration: update views.view.hot_leads_reporting. [9.07 sec, 155.03 MB] [notice] Synchronized configuration: update views.view.evs_error_submissions. [9.11 sec, 155.97 MB] [notice] Synchronized configuration: update views.view.evs_customer_events. [9.16 sec, 155.95 MB] [notice] Synchronized configuration: update views.view.event_submission_report. [9.26 sec, 160.18 MB] [notice] Synchronized configuration: update views.view.event_form_submissions. [9.32 sec, 156.51 MB] [notice] Synchronized configuration: update views.view.epp_data_lake_report. [9.42 sec, 159.7 MB] [notice] Synchronized configuration: update views.view.dealer_traffic_reporting. [9.44 sec, 160.82 MB] [notice] Synchronized configuration: update views.view.customer_sbs_vip_referral_report. [9.55 sec, 160.18 MB] [notice] Synchronized configuration: update views.view.customer_sbs_referral_report. [9.69 sec, 162.86 MB] [notice] Synchronized configuration: update views.view.customer_iq_submissions. [9.73 sec, 166.37 MB] [notice] Synchronized configuration: update views.view.customer_event_tracking. [9.74 sec, 167.45 MB] [notice] Synchronized configuration: update views.view.customer_consumer_vip_referral_report. [9.82 sec, 158.14 MB] [notice] Synchronized configuration: update views.view.customers_multiply. [9.88 sec, 156.37 MB] [notice] Synchronized configuration: update views.view.customers_affiliates. [10.06 sec, 165.89 MB] [notice] Synchronized configuration: update views.view.customers. [10.35 sec, 177.81 MB] [notice] Synchronized configuration: update views.view.bib_request_queue. [10.45 sec, 159.49 MB] [notice] Synchronized configuration: update views.view.all_customer_codes. [10.62 sec, 166.13 MB] [notice] Synchronized configuration: update views.view.affiliates_data_lake_report. [10.67 sec, 170.31 MB] [notice] Synchronized configuration: update views.view.admin_form_submissions. [10.73 sec, 155.47 MB]
- π¬π§United Kingdom alexpott πͺπΊπ
Finally we get around to fixing what @fabianx pointed out in #2294569-10: Determine cause of high memory consumption β
- π¬π§United Kingdom alexpott πͺπΊπ
While this fix will be most pronounced on config import - it will also benefit every config save where we don't set the trusted flag to true. So that's config forms, some post updates, hooks - lots of places.
I don't have a test site with a ton of views, or at least, enough views, to validate the memory usage reduction. I tried installing umami, tweaking the views there, and running config import, but the numbers were not noticeably different.
That said, these changes make sense to me, and I validated that the schemaWrapper does not get used outside of save()/castValue() and is safe to set to NULL.
RTBC +1. Holding off moving in case someone wants to validate the memory leak fix.
- π¬π§United Kingdom alexpott πͺπΊπ
@godotislate you can easily see this with Umami...
Before
[notice] Synchronized configuration: update views.view.archive. [1.04 sec, 53.45 MB] [notice] Synchronized configuration: update views.view.articles_aside. [1.05 sec, 54.3 MB] [notice] Synchronized configuration: update views.view.block_content. [1.07 sec, 55.4 MB] [notice] Synchronized configuration: update views.view.content. [1.1 sec, 56.36 MB] [notice] Synchronized configuration: update views.view.content_recent. [1.11 sec, 57.29 MB] [notice] Synchronized configuration: update views.view.featured_articles. [1.12 sec, 58.13 MB] [notice] Synchronized configuration: update views.view.files. [1.16 sec, 60.13 MB] [notice] Synchronized configuration: update views.view.frontpage. [1.18 sec, 60.92 MB] [notice] Synchronized configuration: update views.view.glossary. [1.2 sec, 62.38 MB] [notice] Synchronized configuration: update views.view.media. [1.24 sec, 62.76 MB] [notice] Synchronized configuration: update views.view.media_library. [1.29 sec, 64.87 MB] [notice] Synchronized configuration: update views.view.moderated_content. [1.32 sec, 65.27 MB] [notice] Synchronized configuration: update views.view.promoted_items. [1.34 sec, 66.98 MB] [notice] Synchronized configuration: update views.view.recipe_collections. [1.36 sec, 66.65 MB] [notice] Synchronized configuration: update views.view.recipes. [1.37 sec, 67.49 MB] [notice] Synchronized configuration: update views.view.related_recipes. [1.38 sec, 68.44 MB] [notice] Synchronized configuration: update views.view.taxonomy_term. [1.4 sec, 67.96 MB] [notice] Synchronized configuration: update views.view.user_admin_people. [1.43 sec, 69.54 MB] [notice] Synchronized configuration: update views.view.watchdog. [1.46 sec, 71.66 MB] [notice] Synchronized configuration: update views.view.who_s_new. [1.47 sec, 71.16 MB] [notice] Synchronized configuration: update views.view.who_s_online. [1.48 sec, 71.87 MB]
After
[notice] Synchronized configuration: update views.view.archive. [1.16 sec, 53.45 MB] [notice] Synchronized configuration: update views.view.articles_aside. [1.17 sec, 54.3 MB] [notice] Synchronized configuration: update views.view.block_content. [1.19 sec, 54.55 MB] [notice] Synchronized configuration: update views.view.content. [1.22 sec, 53.74 MB] [notice] Synchronized configuration: update views.view.content_recent. [1.23 sec, 54.67 MB] [notice] Synchronized configuration: update views.view.featured_articles. [1.24 sec, 55.51 MB] [notice] Synchronized configuration: update views.view.files. [1.28 sec, 56.5 MB] [notice] Synchronized configuration: update views.view.frontpage. [1.3 sec, 55.66 MB] [notice] Synchronized configuration: update views.view.glossary. [1.32 sec, 57.12 MB] [notice] Synchronized configuration: update views.view.media. [1.35 sec, 56.18 MB] [notice] Synchronized configuration: update views.view.media_library. [1.41 sec, 55.13 MB] [notice] Synchronized configuration: update views.view.moderated_content. [1.45 sec, 54.43 MB] [notice] Synchronized configuration: update views.view.promoted_items. [1.47 sec, 56.14 MB] [notice] Synchronized configuration: update views.view.recipe_collections. [1.48 sec, 54.95 MB] [notice] Synchronized configuration: update views.view.recipes. [1.51 sec, 55.79 MB] [notice] Synchronized configuration: update views.view.related_recipes. [1.52 sec, 56.74 MB] [notice] Synchronized configuration: update views.view.taxonomy_term. [1.54 sec, 55.03 MB] [notice] Synchronized configuration: update views.view.user_admin_people. [1.58 sec, 54.87 MB] [notice] Synchronized configuration: update views.view.watchdog. [1.6 sec, 56.75 MB] [notice] Synchronized configuration: update views.view.who_s_new. [1.61 sec, 55.3 MB] [notice] Synchronized configuration: update views.view.who_s_online. [1.62 sec, 56.01 MB]
Here's a script to change something meaningless on each of umami's views....
foreach (\Drupal\views\Entity\View::loadMultiple() as $view) { $view->set('description', $view->get('description') . '1')->save(); }
Thanks, @alexpott. The way I tried testing before was:
- install umami
- export config
- copy views config yml to a separate folder
- install umami again
- drush cim --partial --source={folder with views files exported before}
Which wasn't showing a lot of difference in memory usage.
But now, testing with:
- install umami
- export config
- Run script to change views config per #8
- drush cim
I'm seeing:
Before
[notice] Synchronized configuration: update views.view.archive. [1.3 sec, 26.3 MB]
[notice] Synchronized configuration: update views.view.articles_aside. [1.3 sec, 27.16 MB]
[notice] Synchronized configuration: update views.view.block_content. [1.31 sec, 28.65 MB]
[notice] Synchronized configuration: update views.view.content. [1.33 sec, 29.9 MB]
[notice] Synchronized configuration: update views.view.content_recent. [1.34 sec, 30.83 MB]
[notice] Synchronized configuration: update views.view.featured_articles. [1.35 sec, 30.05 MB]
[notice] Synchronized configuration: update views.view.files. [1.37 sec, 32.6 MB]
[notice] Synchronized configuration: update views.view.frontpage. [1.38 sec, 33.78 MB]
[notice] Synchronized configuration: update views.view.glossary. [1.39 sec, 33.74 MB]
[notice] Synchronized configuration: update views.view.media. [1.41 sec, 35.61 MB]
[notice] Synchronized configuration: update views.view.media_library. [1.44 sec, 38.83 MB]
[notice] Synchronized configuration: update views.view.moderated_content. [1.46 sec, 39.92 MB]
[notice] Synchronized configuration: update views.view.promoted_items. [1.48 sec, 39.83 MB]
[notice] Synchronized configuration: update views.view.recipe_collections. [1.48 sec, 40.52 MB]
[notice] Synchronized configuration: update views.view.recipes. [1.5 sec, 39.25 MB]
[notice] Synchronized configuration: update views.view.related_recipes. [1.51 sec, 40.2 MB]
[notice] Synchronized configuration: update views.view.taxonomy_term. [1.51 sec, 41.2 MB]
[notice] Synchronized configuration: update views.view.user_admin_people. [1.53 sec, 42.85 MB]
[notice] Synchronized configuration: update views.view.watchdog. [1.55 sec, 44.52 MB]
[notice] Synchronized configuration: update views.view.who_s_new. [1.55 sec, 45.15 MB]
[notice] Synchronized configuration: update views.view.who_s_online. [1.56 sec, 43.57 MB]
[notice] Finalizing configuration synchronization. [1.65 sec, 44.03 MB]
[success] The configuration was imported successfully. [1.66 sec, 44.03 MB]After
[notice] Synchronized configuration: update views.view.archive. [1.07 sec, 26.37 MB]
[notice] Synchronized configuration: update views.view.articles_aside. [1.08 sec, 27.22 MB]
[notice] Synchronized configuration: update views.view.block_content. [1.09 sec, 28.71 MB]
[notice] Synchronized configuration: update views.view.content. [1.1 sec, 28.36 MB]
[notice] Synchronized configuration: update views.view.content_recent. [1.11 sec, 29.29 MB]
[notice] Synchronized configuration: update views.view.featured_articles. [1.12 sec, 27.03 MB]
[notice] Synchronized configuration: update views.view.files. [1.15 sec, 27.41 MB]
[notice] Synchronized configuration: update views.view.frontpage. [1.16 sec, 28.58 MB]
[notice] Synchronized configuration: update views.view.glossary. [1.17 sec, 27.96 MB]
[notice] Synchronized configuration: update views.view.media. [1.19 sec, 29.1 MB]
[notice] Synchronized configuration: update views.view.media_library. [1.22 sec, 31.11 MB]
[notice] Synchronized configuration: update views.view.moderated_content. [1.24 sec, 30.25 MB]
[notice] Synchronized configuration: update views.view.promoted_items. [1.25 sec, 29.06 MB]
[notice] Synchronized configuration: update views.view.recipe_collections. [1.26 sec, 29.74 MB]
[notice] Synchronized configuration: update views.view.recipes. [1.27 sec, 26.86 MB]
[notice] Synchronized configuration: update views.view.related_recipes. [1.28 sec, 27.81 MB]
[notice] Synchronized configuration: update views.view.taxonomy_term. [1.28 sec, 28.81 MB]
[notice] Synchronized configuration: update views.view.user_admin_people. [1.3 sec, 29.49 MB]
[notice] Synchronized configuration: update views.view.watchdog. [1.32 sec, 29.67 MB]
[notice] Synchronized configuration: update views.view.who_s_new. [1.32 sec, 30.3 MB]
[notice] Synchronized configuration: update views.view.who_s_online. [1.33 sec, 27.11 MB]
[notice] Finalizing configuration synchronization. [1.36 sec, 27.57 MB]
[success] The configuration was imported successfully. [1.36 sec, 27.57 MB]Seeing a significant improvement! Moving to RTBC.
- π¬π§United Kingdom alexpott πͺπΊπ
We could deprecate the
\Drupal\Core\Config\StorableConfigBase::$schemaWrapper
property and add it as a third argument to\Drupal\Core\Config\StorableConfigBase::castValue()
. But let's do that in a follow-up so we can backport this to 10.5.x and 11.2.x - π¬π§United Kingdom alexpott πͺπΊπ
- π¬π§United Kingdom catch
Cross-posted but I was going to say the same as #10 and add the needs follow-up tag which it now doesn't need because the follow-up exists.
- π¬π§United Kingdom catch
Committed/pushed to 11.x and cherry-picked to 11.2.x, thanks! Doesn't cherry-pick cleanly to 10.6.x
- π¬π§United Kingdom alexpott πͺπΊπ
Posted an MR against 10.5.x (that will also work for 10.6.x) as this is definitely worth back porting.
- π¬π§United Kingdom catch
Thanks for the 10.x MR, committed/pushed to 10.6.x and cherry-picked to 10.5.x too.