Use OptionsProviderInterface::getPossibleOptions() for allowed field values (anyOf / oneOf)

Created on 27 October 2023, almost 2 years ago

Problem/Motivation

Currently the DataDefinitionNormalizer class has some logic in it to add anyOf / oneOf enumerations for fields that define allowed_values. This is very helpful to API consumers who need to know what values are allowed. If disallowed values are submitted a validation error will prevent the entity from being saved.

I also needed support for allowed_values_function, so I opened an issue to add support for that specifically: ✨ Support allowed_values_function Closed: won't fix

But, now I'm looking into adding support for state fields, provided by the State Machine β†’ module, which also have a set of "allowed states". State Machine fields do not use allowed_values or allowed_values_function so they need a different solution.

As I was digging into this, I discovered that there is an interface for fields that define possible options called OptionsProviderInterface, and it has a method called getPossibleOptions().

https://git.drupalcode.org/project/drupal/-/blob/6a68463ce9a5a540ef25e1c...

This interface is implemented by both the State Machine module's StateItem field, as well as Drupal core's ListItemBase class, which is used for core's ListStringItem, ListFloatItem, and ListIntegerItem (which all support allowed_values and allowed_values_function settings). It is also used in core's BooleanItem and LanguageItem field types.

The getPossibleOptions() method implementation in ListItemBase has its own logic for using allowed_values or allowed_values_function:

https://git.drupalcode.org/project/drupal/-/blob/6a68463ce9a5a540ef25e1c...

(which ultimately calls options_allowed_values(): https://git.drupalcode.org/project/drupal/-/blob/6a68463ce9a5a540ef25e1c...)

So, if we could use OptionsProviderInterface::getPossibleOptions(), then not only would it cover both allowed_values (currently handled) and allowed_values_function (making this other issue unnecessary: ✨ Support allowed_values_function Closed: won't fix ), but it would also cover StateItem fields provided by State Machine, as well as some other core field types like BooleanItem.

Proposed resolution

Replace existing logic for handling allowed_values with more generalize logic that uses OptionsProviderInterface::getPossibleOptions().

Remaining tasks

- Open a MR

User interface changes

None.

API changes

Any field type that implements OptionsProviderInterface would automatically get anyOf / oneOf enumerations in JSON Schema endpoints.

Data model changes

None.

✨ Feature request
Status

Active

Version

1.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States m.stenta

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @m.stenta
  • Open on Drupal.org β†’
    Core: 10.1.4 + Environment: PHP 8.1 & MySQL 8
    last update almost 2 years ago
    Waiting for branch to pass
  • Status changed to Needs review almost 2 years ago
  • Open on Drupal.org β†’
    Core: 10.1.4 + Environment: PHP 8.1 & MySQL 8
    last update almost 2 years ago
    Waiting for branch to pass
  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    I managed to get this working, but I'd like to get some more eyes on it to see if it makes sense. I opened an MR, and also attached a patch.

    This required one additional change outside of DataDefinitionNormalizer. Here is why...

    Within the context of DataDefinitionNormalizer::normalize(), when it is normalizing a field that implements OptionsProviderInterface, we have access to the following method arguments:

    • $entity - An object of type \Drupal\Core\TypedData\DataDefinition, which represents the low-level data value of the field. (This is not an "entity" in the Drupal sense, despite its name.)
    • $format - This is always schema_json to indicate that this is being normalized into JSON Schema.
    • $context - An array with two keyed item: name (the data name, usually "value" in this context) and parent (the FieldItemDataDefinition representing the field itself).

    With these as our available input, we can get the field storage definition from $context['parent'], and then call getOptionsProvider(), which gives us the instantiated field item class object itself (eg: ListStringItem), which we can then call getPossibleOptions() on.

    The catch is: getOptionsProvider() requires a Drupal entity be passed into it. This makes sense, because in theory the allowed values may depend on the specific entity in question. However, in this context, we are building a general JSON Schema for all entities of a certain bundle. We don't have a specific entity.

    So, my patch creates a mock entity. I can't see any way around this, but would love to know if there's a better way!

    Now... even creating a mock entity is a little bit tricky because in the relevant context we're working in (DataDefinitionNormalizer::normalize()), we can't actually figure out what bundle we're dealing with! I tried getting that information from the field definition, with $context['parent']->getFieldDefinition()->getTargetBundle() but this always seems to return NULL, so it was a non-starter.

    This brings me to the "one additional change outside of DataDefinitionNormalizer" I mentioned above...

    In JsonApiSchemaController::addFieldsSchema(), currently the only thing being added to $context is $context['name'] (further down the chain, in ComplexDataDefinitionNormalizer, $context['parent'] gets added too). Another argument that is available to us in JsonApiSchemaController::addFieldsSchema() is \Drupal\jsonapi\ResourceType\ResourceType $resource_type. If we add that to $context['resource_type'] it becomes available downstream in DataDefinitionNormalizer::normalize(), and we can do $resource_type->getEntityTypeId() and $resource_type->getBundle() to get the entity type ID and bundle.

    With these two pieces of information, we can create a mock entity of the correct bundle, pass that into getOptionsProvider(), and then call getPossibleOptions().

    After that, we use the existing code to plug those allowed values into anyOf / oneOf enumerations in the normalized JSON schema. Voila! :-D

    Curious what others think! Or if there is a better way to do any of this. Setting to "Needs review"...

  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    Also worth noting, I came across this recent issue from @joachim, which feels very related: #3332360: Add possible values for data types β†’

    The MR in that issue *also* uses getPossibleValues(), but it seems to be for a different case than this one. It does not handle fields with allowed_values, etc. Nor does it conflict with this MR. I found that with both, it added even more enumerations that are currently missing, so we may actually want to merge both of them. I also wonder if there's a way we could combine their logic together, so they handle all cases, but that was a bit beyond my current means, so I didn't dig into it. Curious if @joachim has any thoughts on this one! I will comment on the other issue to bring attention to this one.

  • Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    Waiting for branch to pass
  • Open on Drupal.org β†’
    Core: 10.2.1 + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    Waiting for branch to pass
  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    It is also used in core's BooleanItem

    A user of this patch reported a bug today... tl;dr: we should omit BooleanItem fields from this logic.

    On BooleanItem fields, the oneOf values that are included in the schema are 1 and 0. This makes sense because that's what Drupal / PHP expects for boolean values, but JSON expects true and false.

    json-schema.org explicitly states:

    Note that values that evaluate to true or false, such as 1 and 0, are not accepted by the schema.

    https://json-schema.org/understanding-json-schema/reference/boolean

    This means that it's not possible to validate boolean fields with this patch, because it only allows values (1 and 0) that are unacceptable.

    I think the easiest solution to this is to simply omit BooleanItem fields from being included in this logic, so that we do not add the oneOf property to them.

    I've added a commit to the MR to this effect, and attached an updated patch.

  • Status changed to Needs work 3 months ago
  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    Rebased this on top of latest 8.x-1.x, which now includes automated tests (via #3257911). Let's see if this breaks anything in the current tests, and then add new tests for this functionality specifically.

  • πŸ‡ΊπŸ‡ΈUnited States m.stenta
  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    I've added tests for this. In doing so, I found that this patch was only working for base fields, not for config fields. It was a simple tweak to support both: instead of checking if the field storage definition is an instanceof ListDataDefinitionInterface, now it checks if the field definition is. Tests are included for both base and config field types now.

    I think the easiest solution to this is to simply omit BooleanItem fields from being included in this logic, so that we do not add the oneOf property to them.

    I also split the allowed values logic out (and streamlined/documented it) into an allowedValues() method on the DataDefinitionNormalizer class, and then subclassed that for boolean fields to disable allowed values. This removes the need to reference BooleanItem directly in DataDefinitionNormalizer, and it will allow field types that extend boolean to inherit the same default behavior, or override it if they want to.

  • πŸ‡ΊπŸ‡ΈUnited States m.stenta

    I think this is ready to merge. All tests are green, and all the context and reasoning is documented thoroughly above and in code comments.

    I reached out to @joachim regarding #3332360: Add possible values for data types β†’ to see if we can incorporate that too. If I don't hear back I may move forward with this regardless and we can tackle other cases later.

    • m.stenta β†’ committed 4a0d05dd on 8.x-1.x
      Issue #3397275: Use OptionsProviderInterface::getPossibleOptions() for...
    • m.stenta β†’ committed b4b66bfb on 8.x-1.x
      Issue #3397275: Omit BooleanItem fields from OptionsProviderInterface::...
    • m.stenta β†’ committed 3ebe7e50 on 8.x-1.x
      Issue #3397275: Use OptionsProviderInterface::getPossibleOptions() for...
  • πŸ‡ΊπŸ‡ΈUnited States m.stenta
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024