Calculate field and component dependencies on save and store them in an easy to retrieve format

Note that the 'field union json' approach on ✨ JSON-based data storage proposal for component-based page building Active would probably solve this problem for field types (the field union type would be in a relational varchar column even if the values are in JSON, allowing us to check dependencies via that), but I don't think it would solve it for components, which whatever we do almost certainly need to be in a nested JSON tree structure.

@catch

1. Would you agree that this could be a fallback solution if 📌 Ensure querying JSON nested values when parent keys are unknown is possible in all supported databases Active ends up not working, i.e. an escape hatch?
2. Would you agree that 📌 Ensure querying JSON nested values when parent keys are unknown is possible in all supported databases Active is preferable?

@Wim Leers - short answer, maybe once it's been proved to scale to tens or hundreds of thousands of complex entities with dozens of revisions each on all supported databases? But if we don't build something that does that prior to a stable (or really, beta) release, then we are looking at having to update tens of hundreds of thousands of entities and their revisions to add the denormalization after the fact, which would not be good.

Good point. 🤔

Your thinking is: it's gonna be easier to drop these denormalizations in the future, if we end up not needing them. Is that fair?

That's one reason, but also:

Let's assume that mysql, sqlite and pgsql can all run the queries needed in 📌 Ensure querying JSON nested values when parent keys are unknown is possible in all supported databases Active . Even if they can, they each have different syntax for this. They also have different ways of handling indexes on JSON fields.

The database API does not have an abstraction layer for all of this, there is ✨ Add "json" as core data type to Schema and Database API Needs work but I don't know if that can handle nested values where the parent is unknown per the title of the other issue. Even if it can do that in the database layer, that concept currently doesn't exist in the entity query API (which is what we should be using for query entities) so would have to be added there too somehow.

So that's two levels of abstraction for JSON queries that currently does not exist either in core or contrib. Equally, XB should not be hardcoding JSON query syntax itself, whether mysql-only, or switching on pgsql or sqlite, and it's not a good use case for backend-overridable either - because these are entity queries not some custom storage layer.

Additionally, XB is currently the only use-case we have for adding arbitrary nesting support for JSON fields to entity query, but once it's in there we can't take it out again easily. So if all of that has to happen before it can be proven that it will scale to a medium-sized Drupal site (let alone a massive one), then it turns out it can't, that's an incredible amount of work wasted - and then we have a new API in core that we either have to support because other people started using it, or deprecate and remove again.

I also feel like doing all of this is a slippery slope towards providing views support for arbitrary nested JSON values and other things that have been discussed in the JSON storage issue but personally I would very much like to avoid.

@catch, I'm experiencing some permission issues with git access. However, based on my findings, I suggest the following approach:

In the widget, we can define the class as:

class ExpressionFieldWidget extends WidgetBase {

/**
* {@inheritdoc}
*/
public function formElement(FieldItemListInterface $items, $delta, array $element, array &$form, FormStateInterface $form_state) {
$element['value'] = [
'#type' => 'textfield',
'#title' => $this->t('Expression'),
'#default_value' => isset($items[$delta]->value) ? $items[$delta]->value : NULL,
];

return $element;
}
}

Then, we can define the field storage:

function experience_builder_install() {
// Define the field storage for the expression field.
\Drupal::service('entity_type.manager')->getStorage('field_storage_config')->create([
'field_name' => 'field_expressions',
'entity_type' => 'node',
'type' => 'expression_field',
'cardinality' => FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED,
])->save();
}

For querying the new field, we can use:

$query = \Drupal::database()->select('node__field_expressions', 'e')
->fields('e', ['entity_id'])
->condition('e.value', '%' . $field_expression . '%', 'LIKE');
$result = $query->execute();

By transitioning to a standard Field API field that accepts multiple values, we can avoid database-specific JSON manipulation issues and ensure the code is compatible across different database systems.

Please let me know your thoughts. Once I get the access issue resolved, I will apply these changes if everything looks good to you.

I don't really understand storing the field expressions or needing a widget - it ought to be possible to store just the field type itself that is in use, and this can be calculated and put into the field value in entity presave.

There will also need to be a field and appropriate logic for storing the components that are in use too.

Re: #8, see #3452756-18: Ensure querying JSON nested values when parent keys are unknown is possible in all supported databases → for some notes on the JSON querying side of this.