APCu requirement for 32M is checking wrong value

Created on 6 August 2024, 6 months ago
Updated 14 September 2024, 4 months ago

Problem/Motivation

After setting apc.shm_size to 32M, I was getting the warning:

PHP APCu caching. Enabled (32 MB)
Depending on your configuration, Drupal can run with a 32 MB APCu limit. However, a 32 MB APCu limit (the default) or above is recommended, especially if your site uses additional custom or contributed modules.

This is confusing because the site is set to the recommendation.

Steps to reproduce

Setting `apc.shm_size` in `php.ini` will allocate that memory to apcu. The amount of memory allocated to the segment size for apcu reported as `seg_size` from `apcu_sma_info(TRUE)`. However, depending on which version of PHP you’re using (and possibly other unknowns) the values aren’t the same.

$ drush ev 'print ini_get("apc.shm_size") . PHP_EOL;'
32M

According to [PHP's documentation on short values](https://www.php.net/manual/en/faq.using.php#faq.using.shorthandbytes), 32M should be 32*1048576 (32*1024*1024) which would be: `33554432`

$ drush ev 'print apcu_sma_info(TRUE)["seg_size"].PHP_EOL;'
33554304

This value being returned for seg_size is not quite the same because of how PHP/APC actually make the allocation. Since it happens to be 128 bytes smaller, it causes a warning.

This leads to a very small discrepancy, however with the changes made in πŸ› Status report wrongly warns of APCu memory limit when admin language is not English Fixed we check the value by converting to bytes and making a direct greater than comparison.

In addition, the APCu status check calculations do not take into account that more than one shared memory segment may be configured.

Proposed resolution

For the check against "recommendations", we should check the value of Bytes::toNumber(ini_get('apc.shm_size')) * ini_get('apc.shm_segments'). The calculations for % used are appropriate to use seg_size. In both cases, the seg_size and shm_size must be multiplied by the number of segments.

Release notes snippet

πŸ› Bug report
Status

Fixed

Version

10.3 ✨

Component
InstallΒ  β†’

Last updated 5 days ago

No maintainer
Created by

πŸ‡ΊπŸ‡ΈUnited States eporama

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @eporama
  • I have this problem, too, in Drupal 10.3

  • First commit to issue fork.
  • Pipeline finished with Success
    6 months ago
    Total: 487s
    #249638
  • Status changed to Needs review 6 months ago
  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco
  • πŸ‡ΊπŸ‡ΈUnited States bas123

    Drupal Version: 10.3.2
    PHP Version: 8.3.9 - Memory limit: 4G

    As others above, this error has been present through several recent updates of Drupal Core and other key modules, but always resolved after clearing caches, however since updating to D 10.3.2, it doesn't go away!

    I asked my hosting company to increase this limit to 64 MG and they complied with no hesitation.

    Their method was to add the following line to the document root (public_html) folder's .htaccess file.

    php_value apc.shm_size 64MB
    

    However, my Status Report still shows the same error:

    PHP APCu caching
    Enabled (32 MB)
    Depending on your configuration, Drupal can run with a 32 MB APCu limit. However, a 32 MB APCu limit (the default) or above is recommended, especially if your site uses additional custom or contributed modules.

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Clarify the proposed resolutions.

  • πŸ‡³πŸ‡±Netherlands dhendriks

    I have this problem as well:

    PHP APCu caching
    
    Enabled (32 MB)
    Depending on your configuration, Drupal can run with a 32 MB APCu limit. However, a 32 MB APCu limit (the default) or above is recommended, especially if your site uses additional custom or contributed modules.
    
  • Status changed to Needs work 6 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Small change but seems like something need test coverage for

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    It looks like the CI environment sets the APCu size to 3GB - whoa, that seems like a huge waste of memory, or could tests actually use that much APCu cache somehow? So, I don't think it will be easy to reproduce this exact bug there, unless there's a way for tests to modify the php config files, but we could add a test that verifies this case in test environments using the default config.

  • πŸ‡ΊπŸ‡ΈUnited States holljac

    I'm getting the same warning since updating to 10.3.2. I'm using php 8.3.6. My site is hosted on Bluehost. Is there any way to eliminate this warning?

    PHP APCu cachingEnabled (32 MB)
    Depending on your configuration, Drupal can run with a 32 MB APCu limit. However, a 32 MB APCu limit (the default) or above is recommended, especially if your site uses additional custom or contributed modules.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Re #11 I don't know the history of why it's configured like that, but given we run dozens of tests 24 at a time on a single machine it might be necessary to prevent apcu filling up and then fragmenting. Each test will get its own unique apcu prefix and there's no way to relive all prefixed items efficiently.

  • Status changed to Needs review 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Yes I can see how the APCu cache will fill up as tests are run (literally, I can see this happening locally). Although it might make sense to flush it somehow, but I guess that's easier said than done? Anyways, I added a functional test.

  • Pipeline finished with Success
    5 months ago
    Total: 785s
    #250954
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Is the gitlab setup reason the test-only feature could be passing?

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    @smustgrave yes the test-only will fail only on environments with the default php config, not the CI environment.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Cool will test locally in the morning

  • πŸ‡¬πŸ‡§United Kingdom catch

    Although it might make sense to flush it somehow, but I guess that's easier said than done?

    You can clear the entire cache, but because we run tests concurrently and tests can take anything from a few seconds to a couple of minutes each, flushing at the end of each test run would mean clearing the cache for all the other tests that are still running every few seconds. We could open an issue to see how performance and reliability looks like if we just set it a bit lower and let it fill up occasionally - went ahead and opened on here: #3467490: Investigate setting the apcu cache size lower β†’ .

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Yeah I meant flush the entries that can be flushed, pertaining to a discrete test run that is over, not flushing everything. But idk how that'd be done, since a testrunner is a cli process that doesn't have access to APCu..

  • Status changed to RTBC 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave
    1) Drupal\Tests\system\Functional\System\StatusTest::testStatusPage
    Failed asserting that true is false.
    

    Verifying test coverage locally.

  • Status changed to Needs work 5 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Needs work for that feedback.

  • Pipeline finished with Success
    5 months ago
    Total: 565s
    #252213
  • Status changed to Needs review 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Alternate resolution which seems less verbose

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco
  • Status changed to Needs work 5 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Unless I'm missing something that will re-introduce the original multilingual bug from πŸ› Status report wrongly warns of APCu memory limit when admin language is not English Fixed ?

  • Status changed to Needs review 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Back to needs review

  • Status changed to Needs work 5 months ago
  • πŸ‡«πŸ‡·France andypost

    The check and the value rendered in status report must be the same, otherwise no way to get a reason of warning

    See #3142928-69: Status report wrongly warns of APCu memory limit when admin language is not English β†’

  • Pipeline finished with Success
    5 months ago
    Total: 532s
    #253057
  • πŸ‡ΊπŸ‡ΈUnited States bas123

    UPDATE: My Case was resolved via the Hosting Provider, A2 Hosting (Advanced Tech Support

    Here's how:

    The apc.shm_size setting can only be adjusted in system-level ini files, so I had to create a special configuration file to update the setting:

    *SERVER*@az1-ls2 [~]# cat /etc/cl.php.d/alt-php83/custom.ini
    apc.shm_size = 64M

    https://YOURWEBSITE.net/info.php now shows 64MB of memory. Please let us know if there are any other issues we can help with.

    My Status Reports no longer show the error and actually reflect the correct APCU Settings

    PHP APCu available caching: Memory available: 45.48 MB.
    PHP APCu caching: Enabled (64 MB)

  • Status changed to Needs review 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    I don't see a problem in practice with displaying ByteSizeMarkup::create($memory_info['seg_size'] * $memory_info['num_seg']) to the end user, as it's generally going to be the same as displaying ByteSizeMarkup::create(Bytes::toNumber(ini_get('apc.shm_size'))). The difference between configured size and reported size should be tiny enough that ByteSizeMarkup doesn't care.

    Either way works for me, really. So if folks are adamant about displaying ByteSizeMarkup::create(Bytes::toNumber(ini_get('apc.shm_size'))), sounds good to me.

  • First commit to issue fork.
  • Pipeline finished with Canceled
    5 months ago
    Total: 266s
    #253074
  • Pipeline finished with Canceled
    5 months ago
    Total: 214s
    #253078
  • πŸ‡ΊπŸ‡¦Ukraine Taran2L Lviv

    @mfb thanks for applying the suggestions. I think the result will be different if someone has more than 1 segment for APCu.

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    Yes I guess this was a long-standing bug, that segments weren't taken into account. I'm updating the issue summary to also mention this bug.

  • Pipeline finished with Success
    5 months ago
    Total: 490s
    #253087
  • Status changed to RTBC 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Actually got hit by this on a client project and the MR does fix the issue.

    • catch β†’ committed 7bd49923 on 10.3.x
      Issue #3466399 by mfb, taran2l, eporama: APCu requirement for 32M is...
    • catch β†’ committed d7a02d8c on 10.4.x
      Issue #3466399 by mfb, taran2l, eporama: APCu requirement for 32M is...
    • catch β†’ committed a4d0e692 on 11.0.x
      Issue #3466399 by mfb, taran2l, eporama: APCu requirement for 32M is...
    • catch β†’ committed 8705b9ab on 11.x
      Issue #3466399 by mfb, taran2l, eporama: APCu requirement for 32M is...
  • πŸ‡¬πŸ‡§United Kingdom catch

    Committed/pushed to 11.x and cherry-picked back through to 10.3.x, thanks!

  • Status changed to Fixed 5 months ago
  • πŸ‡ΊπŸ‡ΈUnited States rraney

    Can someone please tell me how to implement a merge request? I'm used to patches. Is this a matter of updating Drupal itself? Thanks

  • @rraney Are you asking how to use the code changes that are in this issue before they are officially released?

  • πŸ‡ΊπŸ‡ΈUnited States rraney

    I just want to know how to fix this issue @cilefen
    I used the link to documentation provided by @solideogloria
    It provided info on how to get a diff/patch which I applied
    As for long-term fix, I assume it will be merged into future version of Drupal, correct?

  • Yes. Make sure you download the patch and put it in a project folder, e.g. "./patches/filename.patch". Don't use the URL for a Merge Request patch, as it can change/break over time.

  • @rraney That's correct. With Drupal, maintainers commit changes to supported development branches then later, release managers distribute tagged releases as new versions. Before the releases, you must patch.

  • πŸ‡¬πŸ‡§United Kingdom catch

    @rraney this will go out with the next patch release of Drupal 10.3 which will be the first Wednesday of September.

  • πŸ‡΅πŸ‡ΉPortugal joaomachado

    @bas123 #27 is correct, in order to increase the memory it has to be added to the Global php.ini file, but most important, make sure you rebuild PHP-FPM or the changes will not take effect, atleast no on a cPanel system.

  • Regarding #27, A2 Hosting told me they could not increase the size of APCu cache on shared hosting. It is stuck at 32MB

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • πŸ‡¨πŸ‡¦Canada paintingguy

    it's back after latest D10 update

Production build 0.71.5 2024