"Undefined array key" error from Drupal\Core\Cache\DatabaseBackend->get

Created on 22 June 2023, about 1 year ago
Updated 8 July 2023, 12 months ago

Drupal\Core\Cache\DatabaseBackend->get() can raise an "Undefined array key" error.

------ Original report ------
Sometimes I get the following in watchdog when saving a file to s3. The strange thing is that the file is uploaded.

Warning: Undefined array key "s3fs:uri:s3://attachments/test.pdf" in Drupal\Core\Cache\DatabaseBackend->getMultiple() (regel 121 van /data/code/project/www/core/lib/Drupal/Core/Cache/DatabaseBackend.php)
#0 /data/code/project/www/core/includes/bootstrap.inc(347): _drupal_error_handler_real(2, 'Undefined array...', '/data/code/jobb...', 121)
#1 /data/code/project/www/core/lib/Drupal/Core/Cache/DatabaseBackend.php(121): _drupal_error_handler(2, 'Undefined array...', '/data/code/jobb...', 121)
#2 /data/code/project/www/core/lib/Drupal/Core/Cache/DatabaseBackend.php(92): Drupal\Core\Cache\DatabaseBackend->getMultiple(Array, false)
#3 /data/code/project/www/modules/contrib/s3fs/src/StreamWrapper/S3fsStream.php(1391): Drupal\Core\Cache\DatabaseBackend->get('s3fs:uri:s3://a...')
#4 /data/code/project/www/modules/contrib/s3fs/src/StreamWrapper/S3fsStream.php(1362): Drupal\s3fs\StreamWrapper\S3fsStream->readCache('s3://attachment...')
#5 /data/code/project/www/modules/contrib/s3fs/src/StreamWrapper/S3fsStream.php(1283): Drupal\s3fs\StreamWrapper\S3fsStream->getS3fsObject('s3://attachment...')
#6 /data/code/project/www/modules/contrib/s3fs/src/StreamWrapper/S3fsStream.php(1098): Drupal\s3fs\StreamWrapper\S3fsStream->stat('s3://attachment...')
#7 [internal function]: Drupal\s3fs\StreamWrapper\S3fsStream->url_stat('s3://attachment...', 6)
#8 /data/code/project/www/core/lib/Drupal/Core/File/FileSystem.php(559): file_exists('s3://attachment...')
#9 /data/code/project/www/modules/contrib/s3fs/src/S3fsFileService.php(429): Drupal\Core\File\FileSystem->getDestinationFilename('s3://attachment...', 0)
#10 /data/code/project/www/core/modules/file/src/Upload/FileUploadHandler.php(193): Drupal\s3fs\S3fsFileService->getDestinationFilename('s3://attachment...', 0)
#11 /data/code/project/www/core/modules/file/file.module(846): Drupal\file\Upload\FileUploadHandler->handleFileUpload(Object(Drupal\file\Upload\FormUploadedFile), Array, 's3://attachment...', 0)
#12 /data/code/project/www/core/modules/file/file.module(722): file_save_upload('field_objection...', Array, 's3://attachment...', NULL, 0)
#13 /data/code/project/www/core/modules/file/file.module(1447): _file_save_upload_from_form(Array, Object(Drupal\Core\Form\FormState))
#14 /data/code/project/www/core/modules/file/src/Element/ManagedFile.php(76): file_managed_file_save_upload(Array, Object(Drupal\Core\Form\FormState))
#15 /data/code/project/www/modules/custom/project/src/Plugin/Field/FieldWidget/AdditionsFileWidget.php(324): Drupal\file\Element\ManagedFile::valueCallback(Array, Array, Object(Drupal\Core\Form\FormState))
#16 [internal function]: Drupal\project\Plugin\Field\FieldWidget\AdditionsFileWidget::value(Array, Array, Object(Drupal\Core\Form\FormState))
#17 /data/code/project/www/core/lib/Drupal/Core/Form/FormBuilder.php(1268): call_user_func_array(Array, Array)

Steps to reproduce

$cache = \Drupal::cache();
$cache->set('test1', 'lowercase');
$cache->get('test1'); //  Good here
$cache->get('test1 '); // The trailing space causes an issues because mysql ignores trailing spaces on primary keys and will return a result of cid 'test1' instead of 'test1 '
πŸ› Bug report
Status

Active

Version

11.0 πŸ”₯

Component
CacheΒ  β†’

Last updated about 8 hours ago

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @sboden
  • πŸ‡ΊπŸ‡ΈUnited States cmlara

    I'm going to send this over to Drupal Core as this error itself would appear to originate from the cache system.

    If we could see the file URI that would be helpful for the core developers.

    I was able to reproduce this with the following code (going off my knowledge of database primary key limitations for mysql) which may or may not be the cause of the original post

    $cache = \Drupal::cache();
    $cache->set('test1', 'lowercase');
    $cache->get('test1'); //  Good here
    $cache->get('test1 '); // The trailing space causes an issues because mysql ignores trailing spaces on primary keys and will return a result of cid 'test1' instead of 'test1 '
    

    https://dev.mysql.com/doc/refman/5.7/en/char.html for documentation on trailing pad characters and character primary keys.

    @sboden Note that even if this is space related that even if core fixes this we will still not support paths with trailing spaces until 4.x at the earliest, see πŸ› Issue with directories and trailing whitespaces Needs work . While we have work to do in s3fs, the cache system is suppose to handle converting strings into a form that can be stored/retrieved so a trailing slash in a cache::get() is not to my knowledge a fault in s3fs.

  • πŸ‡ΈπŸ‡ͺSweden TwoD Sweden

    We get the same type of error when page cache stores the results of a view with an exposed filter of some value, and you filter again for the same string with a different combination of uppercase/lowercase characters.

    Page cache generated a cache id with the first search string and stored that in the cache table via the database backend.
    The next time page cache asks for the same string (only case differs) the database backend performs a case-insensitive query and returns the original item, which has a different cache id compared to what it put in the mapping array.

    IIRC MariaDB, MySQL and SQLite all do case-insensitive queries by default, unsure about PostgreSQL, so maybe it would be easiest to make the database backend just lowercase the cids when normalizing them.
    Other cache backends, like memory, will of course treat these as different cache ids. The interface does not explicitly mention how this is intended to work so maybe we can get away with that?

  • πŸ‡ΈπŸ‡ͺSweden TwoD Sweden

    There is indeed a test which validates that all cache backends have case-sensitive cache ids in GenericCacheBackendUnitTestBase, but what makes it case-[in]sensitive is (obvious in hind-sight), the selected collation for the cid column.

    We were using utf8mb4_sv_0900_ai_ci, and switching to utf8mb4_sv_0900_as_cs for just that column on the cache tables fixes the issue with notices for us.

Production build 0.69.0 2024