Refactor the XB field type to be multi-valued, to de-jsonify the tree, and to reference the field_union type of the prop values

Created on 27 September 2024, about 2 months ago

Overview

Currently, the XB field type is a single-item field with two columns: tree and props, each defined as a JSON structure.

šŸ“Œ [PP-1] Consider not storing the ComponentTreeStructure data type as a JSON blob Active proposes to store the tree relationally, not as a JSON value.

Option #3 in the proposed resolution of āœØ JSON-based data storage proposal for component-based page building Active proposes to associate a field_union type with each component instance's static prop values in order to facilitate code outside of XB being able to use the information in the corresponding field_union config entities to implement Views integrations, migration tooling, configuring search indexes or other special view modes, etc.

This issue proposes to combine the de-jsonifying of the tree and the addition of the field_union type reference into a single refactor.

Proposed resolution

  • Change the field type from single-item to multi-valued. Each item would be for a single component instance.
  • Order (the delta column of) the items the same as how they appear in the left sidebar's Layers panel. This corresponds to how the component instances are ordered in the HTML when the page is rendered except where components render their slots in a different order than the order in which those slots are defined in the SDC's YAML.
  • Define the following columns (properties) in the field type:
    • instance_id (string)
    • component (string): Reference to the component config entity that defines the component that this is an instance of.
    • parent (string): The instance ID of the parent component in the tree. NULL for component instances that are at the top-level in the tree.
    • slot (string): The parent's slot that this component is in. NULL for component instances that are at the top-level in the tree or are in the default/unnamed slot of their parent (if in the future we decide to add support for default/unnamed slots).
    • data_sources (json): The sourceTypes and expression portion of what's currently in the props column (prior to this proposed refactoring) for this component instance. For example:
      {
        "prop1": {
          "sourceType": "dynamic",
          "expression": "ā„¹ļøŽāœentity:node:articleātitleāžāŸvalue"
        },
        "prop2": {
          "sourceType": "static:field_item:string",
          "expression": "ā„¹ļøŽstringāŸvalue"
        }
      }
      
    • static_values (json): The value portion of what's currently in the props column (prior to this proposed refactoring) for this component instance. For example, given the above example of data_sources, this could be:
      {
        "prop2": "Hello, world!"
      }
      

      The above example uses XB's current optimization of omitting the column/property name within the sourceType's field type if it's the sole property being used and it corresponds to the field type's mainPropertyName(). Given this issue's proposal to add a field_union reference (see below), we should evaluate if it would be better for people using that reference if we always explicitly included the column/property name, in which case the above would be:

      {
        "prop2": {
          "value": "Hello, world!"
        }
      }
      
    • field_union (string):
      Reference to the field_union config entity that defines the union of field types for this component. Alternatively, we could omit this from here and instead add the field_union reference to the component config entity. Since the component instance references the component config entity, this would just then be one more hop to get to the field_union config entity, but denormalizing the field_union reference into here might help with querying since Drupal doesn't have great support for JOINing on config entities (though that support could be improved if Drupal core refactored the config table to store as JSON instead of serialized PHP).


      Whether the field_union reference is in this field type directly, or only indirectly via component, it can be NULL in cases where static_values can't be, or wouldn't benefit from being, conformed to a field_union definition. Note that a field_union can be a union of fields that are of any type, including a JSON field, so the "wouldn't benefit from being" is more likely to be the case than "can't be". An example of this might be the values for block settings that we either can't or choose not to define field_union types for.

User interface changes

None

Risks

I ran this proposal by @Wim Leers before writing it up, and he pointed out that there could be nodes with hundreds of component instances on them, so this proposal creates the possibility of a multi-valued field item with more items in it than we're typically used to in Drupal, and we don't know what performance or memory issues FieldItemList and its related PHP objects will encounter at that scale. This is something we'll need to keep our eyes on, but I think there's two things that mitigate the risk:

  • I hope people don't actually put hundreds of component instances on a node. That wouldn't lead to a good authoring experience. A good design system, even if it includes some small components (atoms), should also include larger components (molecules, organisms) that content authors work with, so that content authors aren't in practice putting every atom one-by-one on a page.
  • If we need to, we can implement a list_class for the XB field type that's more suitable to very large lists than FieldItemList is.
šŸ“Œ Task
Status

Active

Version

0.0

Component

Data model

Created by

šŸ‡ŗšŸ‡øUnited States effulgentsia

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @effulgentsia
  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    Crediting @joachim who proposed de-jsonifying the tree back in #3440578-51: JSON-based data storage proposal for component-based page building ā†’ .

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    Alternatively, we could omit [the field_union column] and instead add the field_union reference to the component config entity.

    A big advantage of this would be that it would allow the field_union module to be an optional dependency. XB could add the field_union config entities, and reference them from the component config entities, when the field_union module is enabled, and not do that when that module is not enabled, without that affecting the schema of the XB field type.

    denormalizing the field_union reference into [the XB field type] might help with querying since Drupal doesn't have great support for JOINing on config entities (though that support could be improved if Drupal core refactored the config table to store as JSON instead of serialized PHP)

    Given the advantage above of letting the field_union module be an optional dependency if we keep the field type normalized and only access an item's field_union config entity via its component config entity, I recommend doing that, and solving the querying use case by changing core's config table from serialized PHP to json.

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    Added a Caveats section to the issue summary.

  • šŸ‡¬šŸ‡§United Kingdom catch

    Thanks for writing this up! I'm still digesting the proposed schema. Two minor things:

    so this proposal creates the possibility of a multi-valued field item with more items in it than we're typically used to in Drupal

    So I haven't actually used paragraphs, but I assume there's a single paragraph reference field that can have dozens if not hundreds of deltas in it referencing the paragraph entities. If so then we already have an equivalent example in the wild (except no extra entities for every row here). Also having written the last couple of sentences, I wonder if this starts to make an actual data migration from paragraphs more feasible.

    I recommend doing that, and solving the querying use case by changing core's config table from serialized PHP to json.

    We started discussing that in one of the JSON database support issues, it would allow us to remove the key value config stuff (which supports some limited querying now).

    However, I think we could workaround not having that yet, just by running extra queries. e.g. if we want to find out whether a field type is used, we can get a list of field unions that use it, then a list of components that use those field unions, then run an IN(). Given the main current use-case for that sort of querying is auditing, it should be OK.

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    XB could add the field_union config entities, and reference them from the component config entities, when the field_union module is enabled, and not do that when that module is not enabled

    I realized after writing this that the reason this is true is because the component config entities have essentially all of the information that would need to be in the field_union config entity, which is what would let us generate the field_union config entity at any time that we needed to.

    Given that, I wonder if making the field_union module a hard dependency wouldn't actually be that bad. It would let us take a bunch of stuff out of the component config entity and instead move that information to the field_union config entity.

  • šŸ‡ŖšŸ‡øSpain Carlitus

    Hi, I just wanted to comment on this:

    I hope people don't actually put hundreds of component instances on a node. That wouldn't lead to a good authoring experience. A good design system, even if it includes some small components (atoms), should also include larger components (molecules, organisms) that content authors work with, so that content authors aren't in practice putting every atom one-by-one on a page.

    We use a lot of low-level elements on a page, so we have a lot of freedom. Yes, we also have some elements like molecules, but we usually do that with templates that we can then modify. And this templates are a group a single atoms.

    And a landing, por example, can be very, very, very long.

    So actually the hundreds of components that @Wim Leers was talking about can be real in a lot of cases.

  • šŸ‡¬šŸ‡§United Kingdom catch

    It would let us take a bunch of stuff out of the component config entity and instead move that information to the field_union config entity.

    In general that sounds like a great idea, it would mean the component config entity only needs to hold the things that are unique to the concept.

    I had wondered whether we actually need two config entity types at all - i.e. could field union directly use a component config entity type instead of using its own, or could XB directly use field unions without an extra entity type in-between, but... no idea whether that would even be desirable even if it's possible.

  • šŸ‡«šŸ‡®Finland lauriii Finland

    @catch thinks it's needed to support #3462219: [META] Support alternative renderings of prop data added for the 'full' view mode such as for search indexing or newsletters, but @lauriii thinks those use cases could be solved in better ways by XB directly.

    Are there use cases outside of the use cases that have been already identified that this would help with? So far I've not heard compelling reasons to do this. I'm pretty strongly -1 to supporting the workflow proposed in šŸŒ± [META] Support alternative renderings of prop data added for the 'full' view mode such as for search indexing or newsletters Active out of the box because at least as I understand it, it would result in a extremely convoluted UX. As a fairly technical user, I'm having hard time imagining working with several lists of components and figuring myself how to build anything meaningful out of it. I believe there should be an easier way for managing the challenges related to the search indexing.

    Unless we can define what's the value we get out of this, I don't see why we would prioritize working on this over other work, especially because it sounds like that there's risk associated to introducing this. If I also understand correctly, this also means that there's additional complexity going forward because we support multiple data models out of the box (one for config, one for content).

    I hope people don't actually put hundreds of component instances on a node.

    I checked a sample front page I had built on another page builder and I had 129 components/elements on that page. This was still a fairly simple page using a mix atoms and organisms. I would have to do some more research to define what a reasonable upper bound would be, but it seems that the architecture should definitely be able to handle at least some hundreds of components.

    Change the field type from single-item to multi-valued. Each item would be for a single component instance.

    If we move from JSON structure to a multi-valued field (where each delta represents a component), how do we handle scenarios where there are overrides on top of the desktop breakpoint (e.g. for the mobile breakpoint)? This is requirement #20 from the original product requirements for Experience Builder.

    Example scenario would be that I want larger margin and padding on desktop than on mobile and I want to display a block recommending to install an app on mobile.

    How would this be represented in this data model? Would this still all be stored in the single list or would we have separate lists for different breakpoints?

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    If we move from JSON structure to a multi-valued field (where each delta represents a component), how do we handle scenarios where there are overrides on top of the desktop breakpoint (e.g. for the mobile breakpoint)?

    Is the thinking here that any prop could be overridden by breakpoint? For example, text content such as the quote for a testimonial component could be changed by breakpoint? Or only certain props, identified by the SDC creator as "style" props as opposed to "content" props?

  • šŸ‡§šŸ‡ŖBelgium wim leers Ghent šŸ‡§šŸ‡ŖšŸ‡ŖšŸ‡ŗ

    RE: issue summary

    Alternatively, we could omit this from here and instead add the field_union reference to the component config entity.

    It should definitely be present in the Component config entity, because that'll allow a config dependency on the FieldUnion šŸ‘

    RE: @catch in #7

    I assume there's a single paragraph reference field that can have dozens if not hundreds of deltas in it referencing the paragraph entities. If so then we already have an equivalent example in the wild (except no extra entities for every row here).

    Very good point! šŸ‘

    šŸ¤” Does anybody in this issue have access to a Paragraphs-heavy complex site with good performance, so that we can get some statistics? šŸ¤“šŸ¤ž

    @effulgentsia in #8

    [ā€¦] the component config entities have essentially all of the information that would need to be in the field_union config entity [ā€¦]

    Exactly!

    XB currently basically implements the functionality that Field Union provides (I wrote that previously ~4 months ago at the top of #3440578-52: JSON-based data storage proposal for component-based page building ā†’ ), but without another config entity; the metadata that defines what field types to use (just like a FieldUnion config entity does) is captured by the Component config entity type. See how similar these are (the most notable difference being the absence/presence of config validation):

    (It's rather unlikely that two SDCs would point to the exact same FieldUnion, because even just naming the SDC's props differently would result in different FieldUnions.)

    Given that, I wonder if making the field_union module a hard dependency wouldn't actually be that bad. It would let us take a bunch of stuff out of the component config entity and instead move that information to the field_union config entity.

    • Theoretically: absolutely.
    • Practically/devil's advocate: what would the benefit be?

    @catch in #10

    I had wondered whether we actually need two config entity types at all - i.e. could field union directly use a component config entity type instead of using its own, or could XB directly use field unions without an extra entity type in-between, but... no idea whether that would even be desirable even if it's possible.

    HAH! šŸ˜…

    Maybe ā€¦ maybe the existing Component config entity with its (validated) config schema (see links above) actually sufficiently addresses all those needs already then? šŸ˜„ Back when we were actively going back-and-forth on #3440578 (~4 months ago), a lot of this was much vaguer, less fleshed out. Now it is in a more complete (but nowhere near done!) state. If you look at the <dl> above ā€¦ does that already do what you're suggesting/thinking here? šŸ˜Š šŸ¤ž

    @lauriii in #11

    If we move from JSON structure to a multi-valued field (where each delta represents a component), how do we handle scenarios where there are overrides on top of the desktop breakpoint (e.g. for the mobile breakpoint)? This is requirement #20 from the original product requirements for Experience Builder.

    Hm, requirement #20's story says , which requires storing additional values, not merely conditionally displaying (which is yes/no based on some condition, whereas the example in the story is more nuanced) ā€¦ not sure. None of that is specced out nor estimated though. It sounds more like a "CSS-per-component instance" thing than a "prop value per component instance" thing though.

    That's why I doubt the delta/multi-value change this issue proposes would affect this product requirement; it'd likely be a new style or css field property on the field type, to allow per-component-instance (responsive aka media query) styles.

  • šŸ‡§šŸ‡ŖBelgium wim leers Ghent šŸ‡§šŸ‡ŖšŸ‡ŖšŸ‡ŗ

    In my prior comment I caught up on the issue and replied to things that stood out. @effulgentsia captured my first concern at the bottom of the issue summary ("what about hundreds of component instances"). @lauriii confirmed this must be supported. @catch suggested that Paragraphs likely already hits that scale. We still need numbers to gain confidence.

    In this comment, I point out concerns that have not been raised before.

    I'll use the same example in both concerns: suppose a PROVIDER:heading SDC that contain 2 props: a string (title) and a heading level (enum integer). Then a matching field union (first see the docs at https://git.drupalcode.org/project/field_union/blob/8.x-1.x/Readme.md) would be:

    $union = FieldUnion::create([
      'id' => 'xb_component_PROVIDER_heading',
      'fields' => [
        'text' => [
          'field_type' => 'string',
          'label' => 'Heading title',
          'name' => 'heading_title',
          'required' => TRUE,
          'translatable' => FALSE,
          'default_value' => ['whatever the example value is in the *.component.yml file'],
          'settings' => [
            'max_length' => 255,
          ],
        ],
        'level' => [
          'field_type' => 'list_integer',
          'label' => 'Heading level',
          'name' => 'heading_level',
          'required' => TRUE,
          'translatable' => FALSE,
          'default_value' => ['whatever the example value is in the *.component.yml file'],
          'settings' => [
            'allowed_values' => [
              ['value' => 1, 'label' => '<h1>'],
              ['value' => 2, 'label' => '<h2>'],
              ['value' => 3, 'label' => '<h3>'],
              ['value' => 4, 'label' => '<h4>'],
              ['value' => 5, 'label' => '<h5>'],
              ['value' => 6, 'label' => '<h6>'],
            ],
          ],
        ],
      ],
    ]);
    

    Concern 2: product requirement 7.1 Tokens aka Reusing values in the host entity's base/bundle fields

    What if I want to populate one of the SDC props using a value from a host entity base/bundle field?

    (This is called a dynamic prop source in current XB terminology because its value is dynamic: the value changes when the host entity's field values change. This is in contrast with a static prop source, where the value is manually/explicitly entered by the Content Creator, where the value that was entered is static: it will always evaluate to the same result. See XB terminology docs.)

    For example, I want to populate a component instance that uses the heading SDC in part with the label of the single-cardinality "Category" taxonomy term reference of my host entity type+bundle "News item". (Or, simpler example: the "News item" entity's "Title" field .)

    So, my heading component instance would be claiming to be using this xb_component_PROVIDER_heading FieldUnion, but ā€¦ actually only the level prop would be populated by the field union, the text SDC prop would be populated by the "Category" taxonomy term reference!

    This is what I was referring to in #3440578-30: JSON-based data storage proposal for component-based page building ā†’ . That's what product requirement 7.1 Tokens refers to.

    (The above interpretation AFAICT accurately/reasonably interprets the product requirement. @lauriii, please correct me if I'm wrong.)

    Concern 3: How will this work for SDC props that themselves are type: object-shaped?

    An SDC's props is always type: object. But what if some prop foo also is type: object-shaped?

    This is not yet supported in XB yet (issue: šŸ“Œ [later phase] Support `{type: object, ā€¦}` prop shapes that require *multiple* field types ā€” also: nested components/component reuse Postponed ), but I know/I'm confident it's possible.

    This is a common need, and a number of SDCs in https://www.drupal.org/project/demo_design_system ā†’ had to be refactored to not use that because #3467890 is not yet fixed! šŸ˜…

    An example in the XB codebase itself is the shoe_details component, which contains:

    ā€¦
    props:
    ā€¦
        expand_icon:
          title: Expand icon
          $ref: json-schema-definitions://experience_builder.module/shoe-icon
          # @todo slot prop on the icon should always be expand-icon
          type: object
        collapse_icon:
          title: Collapse icon
          $ref: json-schema-definitions://experience_builder.module/shoe-icon
          # @todo slot prop on the icon should always be collapse-icon
          type: object
    ā€¦
    

    I've found #3170831: Support nested union fields ā†’ in the field_union issue queue. I have no idea yet how much work it'd be to support that. I bet @larowlan can speak to that šŸ¤“

    But this would make concern 2 above more complicated: what if it's the "title" of the expand_icon that you want to populate using a base/bundle field value? Then it'd be a token that needs to be resolved for one of a nested field union.

  • šŸ‡¬šŸ‡§United Kingdom catch

    Maybe ā€¦ maybe the existing Component config entity with its (validated) config schema (see links above) actually sufficiently addresses all those needs already then?

    I think to answer this, we would need to figure out what addressing šŸŒ± [META] Support alternative renderings of prop data added for the 'full' view mode such as for search indexing or newsletters Active looks like with field_unions + components vs. just components or at least enough direction and agreement on the use-cases to be able to talk about it with a common understanding of the need, currently that does not seem to be the case.

    The big difference with a field_union is that it results in field data that can be used outside XB (i.e. regular manage display), or which could be configured as the optional source for 'dynamic props' inside XB, for a different view mode. Just this morning I tried to type up some thoughts on how a component-only solution to that issue theoretically possibly could work to have something to compare to.

    (This is called a dynamic prop source in current XB terminology because its value is dynamic: the value changes when the host entity's field values change. This is in contrast with a static prop source, where the value is manually/explicitly entered by the Content Creator, where the value that was entered is static: it will always evaluate to the same result.

    But if we used field unions, then everything (or at least everything that looks like field data using field types) would be dynamic - the difference would instead be whether field deltas can be created directly within the XB interface or not.

  • šŸ‡§šŸ‡ŖBelgium wim leers Ghent šŸ‡§šŸ‡ŖšŸ‡ŖšŸ‡ŗ

    I just saw @lauriii's DrupalCon presentation, where he walked through designs that show the 7.1 Token functionality I mentioned in concern 2 in #14. He showed DrupalCon before everyone else šŸ˜„

    That enables me to actually illustrate the problem:

    šŸ‘† That shows the props form for an PROVIDER:event_card SDC. It has 9 props:

    1. Variation aka āœØ Introduce component variants to SDC Active
    2. Primary title
    3. Logo
    4. Secondary title
    5. Body
    6. Address
    7. Price
    8. Image
    9. Button

    For each of the 9, I drew an arrow on the screenshot of the design:

    • 3 big blue arrows: static information stored for this component instance, "in the field union" if you will.
    • 6 small green arrows: for these, no information is stored, only an expression that describes how to retrieve this information from the host entity's bundle/base fields

    How do you map that onto a Field Union? The Field Union metadata would still be relevant (allowing "regular manage display" as you say), but the field data itself would empty for 6/9=66% of the fields in the field union.

    But if we used field unions, then everything (or at least everything that looks like field data using field types) would be dynamic - the difference would instead be whether field deltas can be created/re-ordered/edited directly within the XB interface or not.

    I don't understand this paragraph in two ways:

    • : I've been using "Dynamic" in the sense that what we store is a token/expression, and its value is dynamically retrieved.
    • ā†’ no idea what this refers to šŸ™ˆ

    Could you rephrase that? šŸ™

    (Could be me I've had a terrible night with our RCBO/GFCI interrupting twice in the middle of the night šŸ˜¬, so I'm not at 100% brain capacity.)

  • šŸ‡¬šŸ‡§United Kingdom catch

    I don't understand this paragraph in two ways:

    everything would be dynamic: I've been using "Dynamic" in the sense that what we store is a token/expression, and its value is dynamically retrieved.

    Currently dynamic is 'referenced from entity fields' and static is 'stored directly in the xb field', with field union, everything becomes a field reference/dynamic.

    the difference would instead be whether field deltas can be created/re-ordered/edited directly within the XB interface or not.

    We discussed this during the Barcelona meeting - let's say you have five images + description components back by field unions, when re-ordering them, we might want to re-order the field union deltas too so that the XB order and the field order stays in sync. This is opposed to say a single standard description field which doesn't have a delta order.

    How do you map that onto a Field Union? The Field Union metadata would still be relevant (allowing "regular manage display" as you say), but the field data itself would empty for 6/9=66% of the fields in the field union.

    Even with the diagram I still don't understand what's going on here unfortunately. Personally it seems odd to me that you would place a single component then have to individually map what comes from there in it. Why is the content editor making all those 7-9 decisions about each component they add?

  • šŸ‡§šŸ‡ŖBelgium wim leers Ghent šŸ‡§šŸ‡ŖšŸ‡ŖšŸ‡ŗ

    with field union, everything becomes a field reference/dynamic.

    šŸ¤” Aha! Maybe I have fundamentally misunderstood how Field Unions work. This is not mentioned in https://git.drupalcode.org/project/field_union/blob/8.x-1.x/Readme.md. I'll read the underlying code instead.

    Personally it seems odd to me that you would place a single component then have to individually map what comes from there in it. Why is the content editor making all those 7-9 decisions about each component they add?

    That is a very fair point! šŸ‘

    I wonder if this is solely because we don't have šŸŒ± [later phase] [META] 7. Content type templates ā€” aka "default layouts" ā€” affects the tree+props data model Active built yet (where it'd indeed be a Site Builder decision), or whether @lauriii truly intends for Content Creators to (be able to) decide this.

    That'd definitely change this conversation!

  • šŸ‡¬šŸ‡§United Kingdom catch

    šŸ¤” Aha! Maybe I have fundamentally misunderstood how Field Unions work. This is not mentioned in https://git.drupalcode.org/project/field_union/blob/8.x-1.x/Readme.md. I'll read the underlying code instead.

    Sorry I may not be doing a good job explaining what I mean, it should not be necessary to look at the current field union module.

    If we store XB-entered data in Field Union, then XB will be writing that data to field union field values. This means that the field union data is field data, same as any other field (except the extra stuff it adds on top).

    XB's static vs. dynamic distinction is for field API vs. non field API data.

    If all (or all*) data entered via XB is field API data, then I would assume that XB would switch to treating the field union data as field data and referencing it the same way that it does other field types.

    If that doesn't clarify things, we should grab each other on slack, figure out the disconnect, then report back here.

    I wonder if this is solely because we don't have #3455629: [later phase] [META] 7. Content type templates ā€” aka "default layouts" ā€” affects the tree+props data model built yet (where it'd indeed be a Site Builder decision), or whether @lauriii truly intends for Content Creators to (be able to) decide this.

    OK let's please try to clarify that asap. If content creators cannot do this manual mapping, (I agree it's something site builders might do with bundle-level fields similar to layout builder view mode config now), then each component added within XB by site editors will be 'coherent' in that its data will come from the same place and no need for 'partial field unions' which I agree would be weird.

  • šŸ‡§šŸ‡ŖBelgium wim leers Ghent šŸ‡§šŸ‡ŖšŸ‡ŖšŸ‡ŗ

    OK let's please try to clarify that asap. If content creators cannot do this manual mapping, (I agree it's something site builders might do with bundle-level fields similar to layout builder view mode config now), then each component added within XB by site editors will be 'coherent' in that its data will come from the same place and no need for 'partial field unions' which I agree would be weird.

    šŸ’Æ ā€” made @lauriii aware in a meeting an hour ago šŸ‘

  • šŸ‡«šŸ‡®Finland lauriii Finland

    I wonder if this is solely because we don't have #3455629: [later phase] [META] 7. Content type templates ā€” aka "default layouts" ā€” affects the tree+props data model built yet (where it'd indeed be a Site Builder decision), or whether @lauriii truly intends for Content Creators to (be able to) decide this.

    The main goal isn't necessarily to make content creators do the mapping themself because the task of mapping fields to properties would be quite challenging for most content creators to manage (as @catch is arguing in #17). That said, the aim has been to include this capability consistently across the system for site builders to utilize. This would also enable us to build capabilities where site builders could pre-configure mappings to components, which would make it easier for content creators to utilize this capability.

    Something to note is that the field mappings are not conceptually supposed to be restricted to Drupal fields. The plan is to eventually allow site builders to connect components to data from external APIs and other pre-configured integrations (e.g., Shopify, Zapier, Airtable, etc).

  • šŸ‡¬šŸ‡§United Kingdom catch

    I think if I'm understanding #21 correctly, you could have a component where the site builder has pre-configured mappings from different Drupal fields (or elsewhere), for say 3/5 of the sources, and then 2/5 are entered by the content editor. But then in that case, you could have a field union with two field types providing those two values and it would still be internally consistent. Can't think of a good concrete example but something along those lines?

    The other case that maybe applies here is a component where the fields are mapped for everything (maybe with a hard-coded string value on the bundle layout level), but the content editor can provide overrides on a specific content item. For example a section heading that is the same on 90% of entities but can be customized for the other 10% of entities.

    But in that case, the override can be an optional field value anyway, and the override just depends on whether it has content or not. And that would still be internally consistent.

  • šŸ‡«šŸ‡®Finland lauriii Finland

    But it sounds like it's not necessary to support a use case where the site builder sets up a component, and the content creator can unilaterally remap where things come from arbitrarily.

    A content creator wouldn't necessarily map properties to fields but a site builder could do this even in the context of a single page. This is also one way in which the site builder could build these pre-defined mappings in the first place.

    The goal is for the framework to behave similarly regardless if you're editing a component or a page. This way you get a consistent experience across the system, and can for example start building a component while you are building a page. This is a workflow that tools like Figma have popularized.

  • šŸ‡¬šŸ‡§United Kingdom catch

    This way you get a consistent experience across the system, and can for example start building a component while you are building a page.

    OK but if you do that, then the component that you're building would eventually get saved as config, and then if there is mappings involved, the source for those mappings would eventually get saved as field values - so you would still not have a page-unique field mapping? Or would you?

  • šŸ‡ŗšŸ‡øUnited States tedbow Ithaca, NY, USA

    #19 @catch

    If we store XB-entered data in Field Union, then XB will be writing that data to field union field values. This means that the field union data is field data, same as any other field (except the extra stuff it adds on top).

    So this is different from what is proposed in the summary of this issue, correct?

    In the summary there is static_values with the example

    {
      "prop2": "Hello, world!"
    }
    

    But in what you wrote in #19 it seems like this would be written to the field_union field values instead. Otherwise you wouldn't get the benefit of using it in manage display or views.

  • šŸ‡¬šŸ‡§United Kingdom catch

    Yes what effulgentsia wrote in the issue summary is not the same as what I was suggesting in āœØ JSON-based data storage proposal for component-based page building Active option 3.

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    Did I misunderstand option 3 from that issue? In it @catch wrote:

    The field table would store the usual field columns, plus the 'field union type' to identify which field union is stored, plus a single 'values' JSON column holding the actual entered field values.

    Isn't that what this issue's summary is also suggesting, except renaming values to static_values?

    For me, the key difference between this issue and how I understood option 3 from that issue is that in this issue I'm suggesting that in addition to static_values, we also have the other columns, in particular parent and slot, so that each item has all of the information about the component instance: both its "Field union JSON" value and its location in the tree.

  • šŸ‡¬šŸ‡§United Kingdom catch

    @effulgentsia I think it might stem from:

    "The field table.." in that paragraph.

    What I mean here is;

    "The field union table (as distinct from current field union which does not use JSON) would store the field values, distinct/independent from the XB storage.

    Instead of:

    "The XB field table" would store field unions.

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    Would the following reconcile #28 with this issue's current summary?

    The field union table

    What do you mean by the field union table? Currently, field_union defines a field type (via its deriver) for every field_union config entity. I imagine a "field union json" concept would be implemented as a new field type: mixed_field_union, where the properties/columns of this field type are: type and values, where type is a reference to the field_union config entity for that item, and values is the JSON.

    So that's basically the same as this issue's proposed last 2 columns. However, for XB, we also need the first 5 columns. We could add those additional columns in one of two ways:

    • Define the XB field type as a subclass of mixed_field_union and add the extra columns. Just like how in core FileItem subclasses EntityReferenceItem.
    • Define the XB field type as a field union of the first 5 columns plus a mixed_field_union. This might actually be nice in terms of making the component column a full-fledged EntityReferenceItem (sub)field in its own right.
  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    We could add those additional columns in one of two ways

    Given the choice of subclassing or aggregating, I think aggregating would fit the desired mental model better. FileItem subclasses EntityReferenceItem but that's because conceptually a file item is an entity reference item, that also has a description. However, the mental model of a component instance should not be that it's a field union that also has some other stuff; the mental model should be that a component instance is its own thing, where one of the things it has is static values and those static values can be modeled as a dynamic field union.

  • šŸ‡¬šŸ‡§United Kingdom catch

    Yes something like that.

    Define the XB field type as a subclass of dynamic_field_union and add the extra columns. Just like how in core FileItem subclasses EntityReferenceItem.
    Define the XB field type as a field union of the first 5 columns plus a dynamic_field_union. This might actually be nice in terms of making the component column a full-fledged EntityReferenceItem (sub)field in its own right.

    How I had it in my head is that the XB data would only reference the field union data (similar to how it does other fields on the entity), not incorporate it as such.

    he mental model should be that a component instance is its own thing, where one of the things that it has is static values and those static values can be modeled as a dynamic field union.

    Or that the component instance is its own thing, and it can reference dynamic values which happen to be in a field union.

  • šŸ‡ŗšŸ‡øUnited States effulgentsia

    XB data would only reference the field union data

    If the XB field is a multi-valued field of component instances, and there's a separate multi-valued dynamic_field_union field for the component instances' static values, then how would each component instance reference its corresponding dynamic_field_union item? Doing it with a numeric delta that gets re-ordered would be fragile. Each dynamic_field_union item would need a stable ID, which could be the same as the instance_id of the component instance, or it could be its own separate ID.

    Currently, a regular field_union doesn't have the concept of a stable item ID. Would we want to add that concept to dynamic_field_union without also adding it to field_union? Would we then be adding this to both field_union and dynamic_field_union solely for the XB use-case, or would a stable item ID serve other use cases as well? If it's only for the XB use case, then what makes this better than having the XB field either extend or aggregate the dynamic_field_union field?

  • šŸ‡¬šŸ‡§United Kingdom catch

    would a stable item ID serve other use cases as well

    I think it could help for translation, diff, and conflict resolution potentially? e.g. it would help to detect when field unions have been re-ordered as opposed to edited. If so, it seems as applicable to dynamic field unions as non-dynamic field unions.

    Field union for me is 'field collections or paragraphs (or custom blocks) without the extra entities', so if we give those a uuid or similar then it should cover any latent use cases where having an individual identifiable thing was relied upon.

    Another possible use case is for things like the featured image in an image gallery - if there's a way to select the featured image, and this is done by 'field union uuid' then that would persist across re-orderings. I would normally try to persuade someone that instead of selecting they should just automatically select the first delta instead, but if the requirements are specific it would enable use cases like that. I haven't personally used paragraphs (at least not for site building, I've seen it in performance audits...) but given it's theoretically possible to reference an individual paragraph entity now, I imagine a 'field union uuid' could be used to do similar things when they come up.

Production build 0.71.5 2024