Problem/Motivation
First, a little bit of historical context
Even before shipping Drupal 8.0.0, and before the Drupal 8 ecosystem started adding functionality to Drupal core's REST/API-First support, it already was clear that the Symfony Serialization component that we use, is severely flawed in Drupal's plug-and-play model:
#2575761: Discuss a better system for discovering and selecting normalizers →
.
When Drupal 8's REST module was at its height of initial development, in 2013, the HAL normalization seemed to be the future. 5 years later, it's at IETF spec draft (iteration 8), last updated in May 2016.
Besides HAL, we also support a "default" normalization, which is mostly the same, but without any hypermedia metadata.
Since then, JSON API has risen in popularity, reaching 1.0 in mid 2015 … i.e. after Drupal 8 was frozen, and only critical bugfixes could go in. A contrib module has been developed, and it's rising in popularity quickly:
https://www.drupal.org/project/jsonapi →
. It offers a far better DX than core's REST module, because it solves many more problems (including collections, filtering, sorting, pagination, sparse fieldsets).
How does this relate to the issue title?
For many field types (@FieldType plugins), the normalizations of fields of those types at the time Drupal 8 shipped were either:
- incomplete, e.g. for
@FieldType=text fields, the processed text wasn't available.
- not-so-great DX, e.g. for
@FieldType=timestamp fields, a UNIX timestamp was returned, which led to bug reports about how to interpret them
We worked on addressing those problems. We followed the example of the existing normalizers, and hence ended up doing for example
#2768651: Let TimestampItem (de)normalize to/from RFC3339 timestamps, not UNIX timestamps, for better DX →
(all others are still in progress). Problem solved!
Different normalizations structure their normalizations differently
This may sound obvious, but it is not.
- For the "default" normalization (provided by the
serialization module for the json and xml formats), no top-level metadata keys are added the the normalization.
- For the "HAL" normalization (
hal module for the hal_json format), top-level _links and _embedded keys are added.
- For the "JSON API" normalization (
jsonapi module for the api_json format), everything is shifted two levels under a data and then attributes key, except for the UUID, which is relabeled to id and sits below data, next to attributes, and entity reference fields are shifted to a relationships key under data, next to attributes.
As you can imagine, this requires that the field-level normalization is implemented differently.
Which finally brings us to the problem that this issue aims to solve: the normalizer added in
#2768651: Let TimestampItem (de)normalize to/from RFC3339 timestamps, not UNIX timestamps, for better DX →
works for both the "default" normalization and the "HAL" normalization … because those happen to be similar enough. It doesn't work for the "JSON API" normalization. Which means that a contrib module developer would have to do the work:
- once for "default" + "HAL" (likely to happen because in core)
- again for "JSON API" (less likely to happen because in contrib)
- again for new contrib/custom normalizations (same)
Nobody had ever given this some thought. In fact, I even blamed the JSON API module for this at
#2860350-14: Document why JSON API only supports @DataType-level normalizers →
.
Strengthening API-First Drupal
If we continue along the path we're on right now, contrib modules keep adding @FieldType-level normalizers with the problems described above. Which means that new/contrib normalizations are always going to be lagging very far behind core's normalizations. Which means some data will simply not be accessible.
Now, if we instead would implement @DataType-level normalizers (in our example: \Drupal\Core\TypedData\Plugin\DataType\Timestamp aka @DataType=timestamp instead of \Drupal\Core\Field\Plugin\Field\FieldType\TimestampItem aka @FieldType=timestamp), then the normalizations of non-core normalizations would automatically support those.
99% of all field items (@FieldType-level) behave the same, it's only their property values (@DataType-level) that really need normalizers. The primary exception to that rule is @FieldType=entity_reference, because it's what determines relationships/hyperlinks, and almost every normalization has its own way of dealing with those.
IOW:
- have format-specific high-level normalizers (at entity, field item list, field item levels)
- have generic (formatless)
@DataType normalizers (at property value level)
Proposed resolution
- Never implement
@FieldType-level normalizers, only implement @DataType-level normalizers.
- Enforce this via test coverage: make the test discover all normalizer services, check the classes/interfaces it supports, those cannot be field types.
- Exempt the entity reference field type.
- This test should run for both core and contrib modules, which means some contrib modules' automated tests could start failing. This is necessary to nudge them to transition to
@DataType-level normalizers.
Remaining tasks
See
#62
📌
Discourage @FieldType-level normalizers, encourage @DataType-level normalizers, to strengthen the API-First ecosystem
Needs work
User interface changes
None.
API changes
None.
Data model changes
None.
Release notes snippet