Entity Form: Build Entity can't process unlimited values

OK, I can confirm this. I've also tried with a field that allows multiple, e.g. 5, values instead of unlimited. That works as expected too. So, the issue is with the unlimited cardinality. I'll see if I can find something.

Done a few hours of further debugging, and I can't help but think that there is something odd in Drupal core. Especially because we get the correct result when using multi-value fields with a distinct number of values, but when we use unlimited cardinality, we get the wrong result.

Note: we only get the wrong value for the validate form event, but we can get the correct value for the submit form event. That's another indicator that something within core makes a difference. Just hard to find out, if we had to prepare the form somehow in order to get the correct result at an earlier stage.

Here are some more technical details: the essential code in ECA is in \Drupal\eca_form\Plugin\Action\FormBuildEntity::execute where we call this at the very end:

$entity = $can_build ? $form_object->buildEntity($form, $form_state) : clone $form_object->getEntity();

Both variations, $form_object->buildEntity($form, $form_state) and $form_object->getEntity() are from Drupal core and both provide the same result: when looking into $entity->values["field_test"]["x-default"], we do get an array of 3 items (I have submitted the form with 3 items) but each of those 3 items contains an empty array.

I'm not deep enough into the form API to get any idea on what would have to be done to get core to properly create the entity from the form. Or, it could even be a core bug, but then I have no idea where and how to report that.

Maybe someone's around who could move that forward based on this starting point?

I don't have the skills to delve into php code, so I'm trying to probe this problem from the outside with Drupal and ECA.

For unlimited value fields, adding values beyond the first requires that you add field instances to the form. So one possible explanation is that "build entity" is working with the form as it exists when the page loads, and is not able to make use of fields that are added to the DOM after the fact. Evidence supporting this idea: If you edit a previously-saved entity with a single value in an unlimited values field, the edit form appears with the previously saved value, plus an additional (empty) field instance. Data entered into that second field instance does appear in the built entity, but if you add a third field instance, it will be ignored. (This holds true for any number of values; if the saved entity has four values, a fifth field instance appears and is functional, but an added sixth instance is not.)

Looking for other ways to test this idea -- that fields added to an existing form are a not accessible to "build entity" -- I used "Form: add text field" and "Form field: set submitted value" to add and populate a field after the form is loaded. Using "Form field: get submitted value" on the newly-added field gives the proper result, but "Entity form: build entity" shows no value for that same field.

Thanks @brian.barry@utexas.edu, this is purely a PHP problem, and we need to find someone, ideally from the core maintainers, who is willing to have a look and find out, where this is going wrong.

Understood, @jurgenhaas. I have one last bit of amateur sleuthing to pass along and then I'll leave it to the experts.

In the ECA code you referenced in \Drupal\eca_form\Plugin\Action\FormBuildEntity::execute, where $form is being fed into buildEntity(), $form is defined in this way:

$form = &$this->getCurrentForm();

This is followed by some logic to determine whether $needs_manual_build should be triggered. If it is triggered, $form is redefined (as is $form_state). I can't say what is being accounted for in that process, but I can say that negating $needs_manual_build, and thereby negating the redefinition of $form, seems to fix the issue with unlimited values, by which I mean that field instances added in the form are correctly represented in the built entity.

So in my (admittedly simplistic) analysis, something is lost when $form is redefined.

Wow, this is an excellent observation @brian.barry@utexas.edu - this is far far away from "amateur sleuthing" !!!

Thank you so much for that hint. I'll follow-up on this with more tests and maybe some more knowledge soon. For now, let's just quote the comment from the developer who brought that in:

    // For incomplete submissions, simulate a complete form build. Only that
    // way we are able to receive normalized form input values, and then we
    // are finally able to build an entity from user input.

Unfortunately, I don't know exactly what that means and why this is now impacting exactly this use-case. But with your tip I can certainly give this another try.