Undefined constant "Drupal\node\Entity\DRUPAL_OPTIONAL" in functional tests

I'm getting this error intermittently in my setup whenever I save a change to a Paragraph via 'Edit paragraph' (not fields, display etc).
There doesn't seem to be much info around on it.

Error: Undefined constant "Drupal\node\Entity\DRUPAL_OPTIONAL" in Drupal\Core\Entity\EntityStorageBase->mapFromStorageRecords() (line 452 of /app/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php)

#0 /app/web/core/lib/Drupal/Core/Config/Entity/ConfigEntityStorage.php(182): Drupal\Core\Entity\EntityStorageBase->mapFromStorageRecords(Array)
#1 /app/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php(346): Drupal\Core\Config\Entity\ConfigEntityStorage->doLoadMultiple(NULL)
#2 /app/web/core/lib/Drupal/Core/Entity/EntityTypeBundleInfo.php(99): Drupal\Core\Entity\EntityStorageBase->loadMultiple()
#3 /app/web/core/lib/Drupal/Core/Entity/EntityTypeBundleInfo.php(80): Drupal\Core\Entity\EntityTypeBundleInfo->getAllBundleInfo()
#4 /app/web/core/lib/Drupal/Core/Entity/Plugin/DataType/Deriver/EntityDeriver.php(103): Drupal\Core\Entity\EntityTypeBundleInfo->getBundleInfo('block')
#5 /app/web/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(101): Drupal\Core\Entity\Plugin\DataType\Deriver\EntityDeriver->getDerivativeDefinitions(Array)
#6 /app/web/core/lib/Drupal/Component/Plugin/Discovery/DerivativeDiscoveryDecorator.php(87): Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDerivatives(Array)
#7 /app/web/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(291): Drupal\Component\Plugin\Discovery\DerivativeDiscoveryDecorator->getDefinitions()
#8 /app/web/core/lib/Drupal/Core/Plugin/DefaultPluginManager.php(181): Drupal\Core\Plugin\DefaultPluginManager->findDefinitions()
#9 /app/web/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryCachedTrait.php(22): Drupal\Core\Plugin\DefaultPluginManager->getDefinitions()
#10 /app/web/core/lib/Drupal/Core/TypedData/DataDefinition.php(195): Drupal\Core\Plugin\DefaultPluginManager->getDefinition('field_item:bool...')
#11 /app/web/core/lib/Drupal/Core/Field/BaseFieldDefinition.php(606): Drupal\Core\TypedData\DataDefinition->getClass()
#12 /app/web/core/lib/Drupal/Core/Entity/Query/Sql/Tables.php(106): Drupal\Core\Field\BaseFieldDefinition->getMainPropertyName()
#13 /app/web/core/lib/Drupal/Core/Entity/Query/Sql/Condition.php(58): Drupal\Core\Entity\Query\Sql\Tables->addField('default_langcod...', 'INNER', NULL)
#14 /app/web/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(177): Drupal\Core\Entity\Query\Sql\Condition->compile(Object(Drupal\mysql\Driver\Database\mysql\Select))
#15 /app/web/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(81): Drupal\Core\Entity\Query\Sql\Query->compile()
#16 /app/web/core/lib/Drupal/Core/Entity/EntityStorageBase.php(640): Drupal\Core\Entity\Query\Sql\Query->execute()
#17 /app/web/core/modules/basic_auth/src/Authentication/Provider/BasicAuth.php(93): Drupal\Core\Entity\EntityStorageBase->loadByProperties(Array)
#18 /app/web/modules/composer/shield/src/ShieldMiddleware.php(343): Drupal\basic_auth\Authentication\Provider\BasicAuth->authenticate(Object(Symfony\Component\HttpFoundation\Request))
#19 /app/web/modules/composer/shield/src/ShieldMiddleware.php(263): Drupal\shield\ShieldMiddleware->basicAuthRequestAuthenticate(Object(Symfony\Component\HttpFoundation\Request))
#20 /app/web/modules/composer/shield/src/ShieldMiddleware.php(161): Drupal\shield\ShieldMiddleware->bypass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#21 /app/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\shield\ShieldMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#22 /app/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#23 /app/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#24 /app/web/core/lib/Drupal/Core/DrupalKernel.php(718): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#25 /app/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#26 {main}

@timfletcher we're also seeing this same error. For me it consistently occurs when creating a new paragraph type.
I've narrowed it down to an incompatibility between the basic_auth core module and the contrib shield module.

Disabling one of those two modules 'fixes' the issue in my testing. From your backtrace it looks like you're also using both the shield module and basic_auth. Can you try uninstalling one at a time and confirm whether the error still occurs? Most likely this is a fault of the shield module as they have two open issues about compatibility with basic_auth module.

The site that I saw this problem on was also using shield module but not basic_auth.

I'm moving this over to the shield module and re-opening as it doesn't sound like a core issue and more of a shield issue. Feel free to move it back if I'm mistaken.

I guess the next thing to do is have a set of reproduction steps so it's easier to start debugging from a fresh install.

We have the same problem with module alert_message on the same code position: calling an entity-query in handle() in the middleware.
We can reproduce after saving an entity (via structure menu) in the backend while alter_message is enabled.

I guess after clearing caches there is not the complete drupal code available for the middleware stack. And shield calls via `handle()` > `bypass()` > `basicAuthRequestAuthenticate()` the module `basic_auth` which calls an entity-query.

In alert_message is the same problem here: https://git.drupalcode.org/project/alert_message/-/blob/1.0.x/src/StackM...

So i think it could be more a conceptional problem with the middleware and core; or - if intended - the middleware must avoid entity queries.

Hi @all,

Thanks for piging me on this @jan kellermann.

I confirm I was aware of this issue (and it is due to the middleware of alert_message), dunno if there is a similar case on shield module (but I see there is a middleware implementation also). I have this issue when running behat tests, every time I clear the caches.

Because this issue does not happen on prod (but only in our tests) I admit I didn't try to tackle it.

If there is a way to reproduce this using the backend (and not running tests) it would be nice to share it, so I can try investigating this. What is the workflow to reproduce it outside of tests?

Maybe we can play with the middleware weight, trying to get all the necessary before running an entity query?

So ... for now, the only decent solution I've found is to anticipate the load of the "system" module:
\Drupal::moduleHandler()->load('system');

The issue is that whatever we do, during bootstrap phase, Drupal loads all module files in core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php::handle(). This middleware has a priority of 100 to "Prepares the environment after page caching ran". Until we try to run a query on a middleware with a higher priority (in my case 210, just before page cache middleware), there is no way to have the const DRUPAL_OPTIONAL defined.

I still don't get how we are able to get a value for this constant (before clearing cache or triggering the error) and we might rely on some extra cache, like op cache, though I've investigated this and op_cache is fully loaded and includes "web/core/modules/system/system.module" even when the error happens.

This wouldn't do a thing:

$op_cache_status = opcache_get_status();
if (opcache_is_script_cached('/core/modules/system/system.module')) {
   return $this->httpKernel->handle($request, $type, $catch);
}
if (defined('DRUPAL_OPTIONAL')) {
   $defined = TRUE;
}

DRUPAL_OPTIONAL is NEVER defined as the file is still not loaded by KernelPreHandle.

So, to mitigate the change, early loading the system module prevent the bug to happen and is then catched in core/lib/Drupal/Core/Extension/ModuleHandler.php:123

public function load($name) {
    if (isset($this->loadedFiles[$name])) {
      return TRUE; // system is already loaded, we do not load it a second time.
    }
....
}

so we don't load system twice. I will try to run it on all my tests and see if I see some failure. Have no idea about potential side effects for now.

It maybe sounds strange but for me it was that I need to rehash the password for the basic auth user.

I had a similar issue with basic_auth. Resetting the user's password fix the issue for me. Happened after updating core from 10.0.x to 10.2.x.

@smulvih2 thanks a lot! You're post fixed my issue. Changing the user's password worked. I post the error bellow in case somebody else is having issues.

Error: Undefined constant "Drupal\Core\Entity\SAVED_UPDATED" in Drupal\Core\Entity\ContentEntityStorageBase->doSave() (line 703 of my_website_stage_5.0/docroot/core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php) 
#0 my_website_stage_5.0/docroot/core/lib/Drupal/Core/Entity/EntityStorageBase.php(486): Drupal\Core\Entity\ContentEntityStorageBase->doSave()
#1 my_website_stage_5.0/docroot/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(806): Drupal\Core\Entity\EntityStorageBase->save()
#2 my_website_stage_5.0/docroot/core/lib/Drupal/Core/Entity/EntityBase.php(339): Drupal\Core\Entity\Sql\SqlContentEntityStorage->save()
#3 my_website_stage_5.0/docroot/core/modules/user/src/UserAuth.php(57): Drupal\Core\Entity\EntityBase->save()
#4 my_website_stage_5.0/docroot/core/modules/basic_auth/src/Authentication/Provider/BasicAuth.php(110): Drupal\user\UserAuth->authenticate()
#5 my_website_stage_5.0/docroot/modules/contrib/shield/src/ShieldMiddleware.php(343): Drupal\basic_auth\Authentication\Provider\BasicAuth->authenticate()
#6 my_website_stage_5.0/docroot/modules/contrib/shield/src/ShieldMiddleware.php(263): Drupal\shield\ShieldMiddleware->basicAuthRequestAuthenticate()
#7 my_website_stage_5.0/docroot/modules/contrib/shield/src/ShieldMiddleware.php(137): Drupal\shield\ShieldMiddleware->bypass()
#8 my_website_stage_5.0/docroot/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\shield\ShieldMiddleware->handle()
#9 my_website_stage_5.0/docroot/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle()
#10 my_website_stage_5.0/docroot/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle()
#11 my_website_stage_5.0/docroot/core/lib/Drupal/Core/DrupalKernel.php(704): Drupal\Core\StackMiddleware\StackedHttpKernel->handle()
#12 my_website_stage_5.0/docroot/index.php(19): Drupal\Core\DrupalKernel->handle()

Same problem here.
Drupal 10.2.6

basic_auth enabled.
shield (8.x-1.7) enabled.

I edited a media entity and this error appears.
No core update happened.

I needed to disable shield, and it worked again.

Adjusting the passwords is very time-consuming as this has to be done on many instances. And what guarantees that it won't happen again? Is there a possibility of a bugfix on the Shield module?

I deactivated shield (not uninstalled), but error show up again if I want to grab some data from the instance.

Error: Undefined constant "Drupal\node\Entity\DRUPAL_OPTIONAL" in Drupal\Core\Entity\EntityStorageBase->mapFromStorageRecords() (Zeile 418 in /core/lib/Drupal/Core/Entity/EntityStorageBase.php)

#0 /core/lib/Drupal/Core/Config/Entity/ConfigEntityStorage.php(186): Drupal\Core\Entity\EntityStorageBase->mapFromStorageRecords(Array)
#1 /core/lib/Drupal/Core/Entity/EntityStorageBase.php(312): Drupal\Core\Config\Entity\ConfigEntityStorage->doLoadMultiple(NULL)
#2 /core/lib/Drupal/Core/Entity/EntityTypeBundleInfo.php(99): Drupal\Core\Entity\EntityStorageBase->loadMultiple()
#3 /core/lib/Drupal/Core/Entity/EntityTypeBundleInfo.php(80): Drupal\Core\Entity\EntityTypeBundleInfo->getAllBundleInfo()
#4 /core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php(200): Drupal\Core\Entity\EntityTypeBundleInfo->getBundleInfo('user')
#5 /core/lib/Drupal/Core/Entity/EntityFieldManager.php(384): Drupal\Core\Entity\ContentEntityStorageBase->getEntityClass('user')
#6 /core/lib/Drupal/Core/Entity/EntityFieldManager.php(353): Drupal\Core\Entity\EntityFieldManager->buildBundleFieldDefinitions('user', 'user', Array)
#7 /core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(1219): Drupal\Core\Entity\EntityFieldManager->getFieldDefinitions('user', 'user')
#8 /core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(503): Drupal\Core\Entity\Sql\SqlContentEntityStorage->loadFromDedicatedTables(Array, false)
#9 /core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(428): Drupal\Core\Entity\Sql\SqlContentEntityStorage->mapFromStorageRecords(Array)
#10 /core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(394): Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage(Array)
#11 /core/lib/Drupal/Core/Entity/EntityStorageBase.php(312): Drupal\Core\Entity\Sql\SqlContentEntityStorage->doLoadMultiple(Array)
#12 /core/lib/Drupal/Core/Entity/EntityStorageBase.php(608): Drupal\Core\Entity\EntityStorageBase->loadMultiple(Array)
#13 /core/modules/basic_auth/src/Authentication/Provider/BasicAuth.php(93): Drupal\Core\Entity\EntityStorageBase->loadByProperties(Array)
#14 /modules/contrib/shield/src/ShieldMiddleware.php(343): Drupal\basic_auth\Authentication\Provider\BasicAuth->authenticate(Object(Symfony\Component\HttpFoundation\Request))
#15 /modules/contrib/shield/src/ShieldMiddleware.php(263): Drupal\shield\ShieldMiddleware->basicAuthRequestAuthenticate(Object(Symfony\Component\HttpFoundation\Request))
#16 /modules/contrib/shield/src/ShieldMiddleware.php(137): Drupal\shield\ShieldMiddleware->bypass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#17 /core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\shield\ShieldMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#18 /core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#19 /core/lib/Drupal/Core/StackMiddleware/AjaxPageState.php(36): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#20 /core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\AjaxPageState->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#21 /core/lib/Drupal/Core/DrupalKernel.php(704): Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#22 /index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#23 {main}

Hi @handkerchief,

I'm attaching a patch based on this ticket finding and alert_message finding.

Basically, this happens when trying to use the entityTypeManager in the stackmiddleware because Drupal is not fully bootstrapped yet.

Early loading system module might help, see #10 and 🐛 Undefined constant "Drupal\node\Entity\DRUPAL_OPTIONAL" RTBC .

I have not tested my patch, but it has been helping people on alert_message for a while, and the issue is the same. Though, it is hard to provide tests as described in the ticket description because it's all happening before Drupal is fully bootstrapped and related to module orders. Playing with the weight of the middleware is also a solution, but sometimes you might want to run your code early (as in alert message and I assume in the shield module).

Hi Aporie, thank you very much for your information and your efforts.
I have tested the patch, rebuilt caches etc. Unfortunately still the same error as in #15. Maybe someone else can test this, it didn't work for me.

A strange phenomenon. After testing the patch, I can no longer uninstall the Shield module, otherwise it will discard the entire website. The design no longer works correctly and on every page it says (access denied), if I install the shield module again, the site works correctly again. I have now updated to shield 1.8.0, and it runs without a patch. But still the same. Only applies to the shield module. I didn't do anything except apply the patch once, tested it and then I wanted to uninstall shield again. Any idea what's going on?

The patch resolves this problem for me, though I haven't tested other issues yet.