Deep serialization for content entities to make an exact snapshot of an entity object's structure based on its current state

Created on 1 November 2016, about 8 years ago
Updated 6 May 2023, over 1 year ago

Problem/Motivation

Currently in ContentEntityBase::__sleep we remove the initialized fields which means that if an entity reference is contained in some of these fields and it was changed after serializing/unserializing the parent entity the entity reference will be loaded in the original version with the changes lost.

The core currently does not need this so it should not be the default behaviour, but we could make it possible so that if contrib/custom for some reason have to serialize the whole structure then they should be able to. I am currently working on an autosave solution so this is the first approach I am currently evaluating and I do need to serialize the referenced entities for the case when we have nested entity forms.

The issues which are blocked by this feature:
🐛 [PP-1] Serializing content entity form objects should deeply serialize the entity in case of nested/inline entity forms Postponed
🐛 Inconsistent form build between initial form generation and the first ajax request Needs work

When the form is cached for multi step forms then the form entity is cached as part of it and on each Ajax request for e.g. adding new fields the entity with the new field on it is cached as part of the cached form. Now imagine you do the same, but on the third level of a nested entity form - in this case the added field on the entity on the third level is lost because we would only cache the ID of the referenced entities and not the whole entity like the parent one. Here we have the first inconsistency between the parent and the child entities in a nested inline form structure.

From the performance point of view: consider that on each form rebuild you might have to load a lot of entities separately currently but if you use deep serialization you will not need to load not a single entity anymore. So in the case of nested entity inline forms we have PHP computation time to serialize 100 entities at once and on form rebuild do 0 entity loads versus serialize only the parent and on form rebuild do 99 separate entity loads. Not having measured the impact I would say the second is slower. It is better to retrieve e.g. 500 KB with a single query instead 500 KB with 99 queries.

Caching the parent entity prevents that during concurrent editing the changed made from another user and saved will suddenly be mixed into the entity that another user is still editing and on form rebuild when he adds a fields suddenly the values might get reordered or some be deleted or new ones present. As a summary a lot of unexpected things happen when the entity is not being serialized no matter if parent or reference. A similar case where we got into troubles is #2675010: Cloned entity will point to the same field objects if the clone was created after an entity translation has been initialized . In that issue you might see what happens if suddenly you have an modified entity in the form instead the original used one to create the form.

Deep serilization will not be active by default. It is there only for widgets that will require it. For this see 🐛 [PP-1] Serializing content entity form objects should deeply serialize the entity in case of nested/inline entity forms Postponed

Proposed resolution

  1. Set a flag "deepSerialization" through the method ContentEntityInterface::setDeepSerialization() in order to perform deep serialization when calling serialize on the entity object.
  2. Introduce FieldItemListInterface::getSerializationValue($deep_serialization) and let ContentEntityBase::__sleep call this method instead of the ::getValue method, which by default will just call ::getValue, but will have the knowledge if deep serialization is running or not.
  3. EntityReferenceFieldItemList::getSerializationValue($deep_serialization) should flag the referenced entities for deep serialization if deep serialization is currently running in order to deeply serialize the whole entity structure.

Remaining tasks

Answer #141-144 to see if points in 141 are needed.
Review & Commit.

User interface changes

None.

API changes

New methods:

  1. ContentEntityInterface::setDeepSerialization()
  2. FieldItemListInterface::getSerializationValue($deep_serialization)

Data model changes

None.

Feature request
Status

Needs work

Version

10.1

Component
Entity 

Last updated about 4 hours ago

Created by

🇩🇪Germany hchonov 🇪🇺🇩🇪🇧🇬

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • 🇮🇳India Akhil Yadav

    Added patch against #158 in 10.1 version

  • 🇮🇳India bhanu951

    Setting as need review for tests to run.

  • 🇮🇳India bhanu951

    Missing core/tests/Drupal/KernelTests/Core/Entity/ContentEntitySerializationTest.php
    in patch #162 which was present in #158.

    Hiding patch #162.

  • 🇮🇳India pooja saraah Chennai

    Fixed failed commands on #162
    Addressed the comment on #164
    Attached patch against Drupal 10.1.x

  • The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

    Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

  • 🇮🇳India Madhu Kumar M E

    Verified the patch #167 and tested it on Drupal version 10.1.x. The patch works fine in 10.1.x also and I have added the screenshots for reference.

  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    Patch Failed to Apply
  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    Patch Failed to Apply
  • Status changed to Needs work over 1 year ago
  • 🇫🇷France andypost
    1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
      @@ -184,6 +184,18 @@
      +   * @var bool
      ...
      +  protected $deepSerialization = FALSE;
      

      add type to new property

    2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
      @@ -525,12 +538,26 @@ public function __sleep() {
      +  public function setDeepSerialization($deep_serialization) {
      
      +++ b/core/lib/Drupal/Core/Entity/FieldableEntityInterface.php
      @@ -236,4 +236,34 @@ public function isValidationRequired();
      +   * @param bool $deep_serialization
      +   *   TRUE if deep serialization should be performed when the entity is being
      +   *   serialized, FALSE otherwise.
      ...
      +  public function setDeepSerialization($deep_serialization);
      
      +++ b/core/lib/Drupal/Core/Field/EntityReferenceFieldItemList.php
      @@ -13,6 +13,27 @@ class EntityReferenceFieldItemList extends FieldItemList implements EntityRefere
      +  public function getSerializationValue($deep_serialization) {
      
      +++ b/core/lib/Drupal/Core/Field/FieldItemList.php
      @@ -110,6 +110,13 @@ public function setValue($values, $notify = TRUE) {
      +  public function getSerializationValue($deep_serialization) {
      
      +++ b/core/lib/Drupal/Core/Field/FieldItemListInterface.php
      @@ -297,4 +297,19 @@ public function equals(FieldItemListInterface $list_to_compare);
      +  public function getSerializationValue($deep_serialization);
      

      also need to add bool type to function arguments

  • Status changed to Needs review over 1 year ago
  • Open in Jenkins → Open on Drupal.org →
    Environment: PHP 8.1 & MySQL 5.7
    last update over 1 year ago
    29,304 pass
  • 🇮🇳India bhanu951

    Review Comments from #171 addressed.

  • Status changed to Needs work over 1 year ago
  • 🇺🇸United States smustgrave

    Seems there were some key points being discussed in 141-144 but not a final decision made. Are 141.2 or 141.3 needed or not?

    Updated remaining task to include this.

Production build 0.71.5 2024