I don't understand why e.g. the first authoring form in the diagram has a yellow "Add Component to Slot 1" on the right-hand side: shouldn't that be on the left-hand side of "the forms"? Or what is the meaning of the layout of the different boxes in each "Forms" box?
The form's aren't laid out in any specific order, it's just a collection of forms(based on my opinion) that show how different modeling approaches change the amount of options an author could have and how many forms they'd have to interact with to create a component.
I see 5 different approaches, but I don't see specified who defines them, nor where. For example, the second example component (labeled "Components for layout and content"): who defined at what time that there is an "Image component" in the yellow drop zone/slot? Is it the SDC author? Is it the Site Builder? Where is that information stored?
In this case the components shown beside the Authoring Forms show a fully authored component. So a user with authoring permissions authored a side by side component with those values using the forms associated to that model. How forms are composed, where they are composed, and how an SDC definition can change from SDC definition -> embedded in a component -> field mapping -> being displayed to an author is probably worthy of it's own diagram.
In our past discussions, we said that field formatters must be available early. If so, we must be able to express them. If so, exposing them as field blocks probably makes sense? It’d make the upgrade path easy too. OTOH, doesn’t that inherit all scalability problems? We could also make field formatters a distinct “component type” or “renderable” (name TBD), which seems to make things simpler?
I'd be interested to understand more about the motivation here. Field formatters as we know them in Drupal(ones that output html) go against the concept of components only being passed structured values that they then place in their html.
What if there's a prop that you don't want content creators to be able to modify?
The internals of the component should not be editable when the component is being used. When the component is being used, it's only possible to modify props of the component, and insert components to the slots of the component.
This warrants more discussion. IMO components composed of other components should definitely be able to compose their forms by passing thru props and customizing what props are available on the authoring form otherwise we end up in the same situation various Drupal approaches and Acquia Site Studio have where every usage of a component in another component requires redefining fields on the parent form so they can be mapped from the parent to the child.
Back to the main question for this issue component vs element vs pattern. +1 to @laurii's take that elements are just a type of simple component. Expanding on the similar but different concepts touched on in the issue description for how components will be created/placed on a page. I've seen 3 main types of components that authors want to place:
- : These are the components that will most commonly be placed in XB by authors, you place them on a given page and that's the only place they'll ever need to be rendered.
- : These are components that are globally available, managed by site builders or by privileged authors they exist so that the same content can be placed on many different pages and when the content changes all the pages using the component show the newest content. This concept is implemented in core by blocks where reusable blocks can be placed in layout builder and in paragraphs through the paragraphs_library submodule.
- : These are what are being called patterns in this issue(not a huge fan of the patterns name btw). This is a set of components that have been authored either on an XB page or a separate pattern creation page and saved by a site builder or author to either the site or author's personal library of "patterns". These saved patterns are then available for the author to place when creating a page with XB but unlike reusable components that is a self contained reference to a component pattern components end up on the page as a preconfigured set of standalone components that have no concept of what their parent pattern is. Tossing out alternative names to pattern these could also be called stencils, templates, or blueprints. This is implemented for layout builder by https://www.drupal.org/project/section_library → and not sure if paragraphs has an equivalent.
@larowlan one of the benefits of using structurizr's DSL from is that it's really easy to generate and keep in sync smaller diagrams that are easier to digest and if you put the code in the previewer at https://structurizr.com/dsl it's really easy to zoom in/out on specific areas. I'm not particularly attached to structurizr but am a fan of the concepts behind c4 that encourage diagramming systems at multiple levels of detail so people can more easily onboard.
@wimleers I know these diagrams are early drafts but it looks like XB.dsl is missing Senior Developers as a person and Code Defined components that live alongside Config Defined components.
Redid the diagrams and updated the issue description to include @laurii's feedback and other feedback from Drupalcon discussions. Also with talk of this being referenced from the project page I changed the color scheme to be more accessible.
Changes
- Added the concept of an open, restricted, and closed slot. Restricted slots can be restricted by types of components that can be placed or the total number of components that can be placed within the slot. Closed slots help to clarify the functionality of embedded slots. When using a two column component for layout the side by side component may want to populate one column slot with known components through the side by side form(closing the slot) while leaving the other column free for any component to be placed in(open slot)
- Added a few more variations of how the side by side component could be built with slots.
- Added more descriptive text around the models and what "flexibility" means in the context of the diagram.
- Split the diagram into two images one for models and another for forms.
- Removed the different types of form approaches since the direction for XB appears to be using a sidebar. These were replaced with my ideas of what forms would be needed based on how a component is modeled.
Another possible home for the ADRs is the project's gitlab wiki. I think it's available to our poejcts but not sure if our unique contrib flow would work well with them. Gitlab does create a new repo just for the wiki, all changes to wiki are recorded as commits, and it supports markdown so overall sounds like it could work?
Wim Leers → credited ctrlADel → .
2.How can we load the Media Library within the Experience Builder (which is a React SPA)? (Frontend)
I've managed to decouple(ish) Entity Browser in the RJSF module, it still depends on some drupalSettings values so it's not fully decoupled. Never explored doing it for Media Library because Entity Browser did a good enough job for my needs and could be used for any entity type. Linking in case it's helpful. https://git.drupalcode.org/project/rjsf/-/tree/1.0.x/modules/rjsf_entity...
griffynh → credited ctrlADel → .
@pdureau this branch/MR was created before there was a 0.x and there were some previous components pulled from SDC at the top level which it looks like you reviewed. I just rebased the MR onto 0.x so now the only example components are the ones in the experience_builder_example_components submodule, would be great to get your thoughts on those.
Knowing that XB will likely require some changes to SDC schemas and that there are many features that need to be built on top of the schemas the overall goal here is to provide a diverse set of example implementation approaches to ensure that XB doesn't unintentionally limit how components can be built/used. A smaller set of example components also has the benefit of being easier to update and adjust as extra settings are added/removed from the schema. The full design system implementations you mentioned will be great to use as a more thorough test once XB is a bit better defined.
Would be interested to know If there are any common component structures/approaches you use that are missing from the examples and what would be a good example we could use to add them in.
ctrlADel → changed the visibility of the branch experience_builder-3446722-2 to hidden.
I wrote the https://www.drupal.org/project/rjsf → module to do this and was successful using it to implement component based authoring for several sites but after talking with Wim at Drupalcon there are a number of limitations that made me think it wouldn't be the best path forward for XB.
When it comes to forms there's really 3 different parts, the data model(schema), the view(form display), and the current data. JSON Schema does a really good job at defining the data model but has no concepts to define how a prop should display in a form. All the various JSON Schema form generators add on their own layer of properties and keys to the schema to allow defining how props should be displayed which potentially makes the XB JSON Schema less portable to other JSON Schema tools. If we were to adopt a form generator for XB we'd need to consider:
- If the library is extensible enough to meet all our needs, including allowing contrib to provide an arbitrary number widgets, from my work with RJSF it's good but is complex to make extensible for contrib.
- If the library's license is compatible with Drupal. RJSF is licensed under Apache-2.0 which as far as I understand is not compatible with GPL making distributing it with core difficult
- How developers(and non developers) would discover the various plugin widgets available and how they map to schema structures
- When defining a component meant to be used across multiple sites how do you know which components are available or denote a dependency? What happens when a widget defined the schema isn't available?
- How difficult it would be to reimplement and maintain two versions of core and contrib field widgets. We'd need both a PHP version and a JS version.
- Which schema are we storing? The schema for the content being collected or the schema for the content being rendered. Often the content when authored has a different structure then the template needs to render. Think a button link component, the target URL for is often an entity reference where we need to collect and store the entity reference from the form but the component template expects a url string.
- How would validation work? Most of these form generators have a built in client side set of validation that runs against the schema but Drupal's validation is much more extensive and would likely have to happen server side then need to be sent back to the form in some way.
I'm a fan of generated forms from schema and think that XB should have some amount of functionality to allow authoring of a component straight from the schema without additional configuration but overall after seeing that it's possible I'm a much bigger fan of reusing our existing field system and mapping known/validated content field structures to json schema props. Even if it means we have relatively ugly fields and forms to begin with. This approach lets JSON Schema do what it's good at which is define the structure of the props needed to render the component while allowing Drupal to do one of the things it's best at which is collecting and providing structured content to templates.
griffynh → credited ctrlADel → .
tekNorah → credited ctrlADel → .
Can you give it another try? I pushed a fix for the width issue, looks like something I broke while doing clean up before opening the MR.
ctrlADel → created an issue.
Just a note on the components in 📌 Create an example set of SDC components Active . They are very much intended as a place to test and experiment with the config/data structures experience builder will need. They may serve as a good starting point for some of the default components but definitely are not meant to be included as defaults.
ctrlADel → created an issue.
ctrlADel → created an issue.
lauriii → credited ctrlADel → .
Overall in favor of moving layout builder field storage from serialized php to json. Being able to easily read field values and query them would be great. Since the layout builder value is already exported to yaml for display mode configs it seems like it should be an easy jump to store it as json in the database.
An additional approach to the ones mentioned in #15:
Plain old plugin blocks
In this approach each component is represented as a plugin block in layout builder using the block's form to author the block within layout builder and then stored in the layout builder field along with the block's plugin id. This avoids the revisioning and proliferation of entities that come with paragraphs or content blocks. Downside is that the block and authoring form need to be defined in code(currently) with Form API. Since we have to use form API this results in losing access to an interface to configure the form/display and you can't use many use contrib modules that only provide fields and not form elements.
I've had pretty good success with this approach using an in house SDC like schema definition system and generated forms from schema using RJSF →
That works if you have one image gallery with multiple images. What happens if you want two image galleries?
With a component first approach you'd define two components an Image Gallery component to store multiple images and then a Two Gallery component with two slots where each slot would contain a Simple Gallery.
Other thoughts:
Perhaps a bit off topic for this issue but since the discussion is happening here I feel like there could be room for a whole new component entity type in addition to config and content entities. The definining characteristics of a component entity would be
- built in support for props aka fields
- built in support for slots aka drop zones/sections/regions that contain other components
- built in support to template/layout props and slots
- provides a way to restrict slots to only allow certain components
- is stored as a singular json blob
- Either one or both of:
- can gracefully handle when a saved json blob does not match the entities prop/slot structure
- leverages a working entity usage system so update hooks can reliably update json blobs when the component structure changes
- isn't a standalone authorable entity but is instead assumed to live within a config or content entity
- provides a way to transform prop values before passing them to a template(a reference field stores the entity id but in the component you always need a value from the entity like the title)
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
volkswagenchick → credited ctrlADel → .
I helped with Matt's globbing at Drupal's second best camp
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
This was included in the Alpha 2 release.
- Facet settings were added to the endpoint response using an event subscriber
- Facet settings were reviewed and some support for settings that apply to all facets was added
- Facet configuration from the API is preferred over the facet config provided by attributes, unless preferAttributes is present in a components attributes
Would this also postpone nice to have features for using cookie based auth to make requests? I agree in a fully decoupled scenario JWTs make a lot more sense but when looking to progressively decouple part of the admin interface cookie support is nice. Since a user logs in via Drupal to access the admin portion of the site anyways they typically will already have a cookie that can be used.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
ctrlADel → created an issue.
I had some free time and inspiration since opening the issue and put together https://www.drupal.org/project/search_web_components → . I would say it's a different take to the existing client/ui so not sure how much of a drop in alternative it really is.
Still debating if the module will support the ui submodule or not. With everything web component based it seems like Drupal's existing layout tools may be sufficient.
froboy → credited ctrlADel → .
ctrlADel → created an issue.
ctrlADel → created an issue.
This is done.
froboy → credited ctrlADel → .
Confirming me and John worked on this at NEDCamp contrib day.
johnpicozzi → credited ctrlADel → .
johnpicozzi → credited ctrlADel → .
Opened a MR with a tweak to the getExternalUrl() in the StreamWrapper to ensure animate is part of the effects applied. Not 100% sure this is the right, or only, place for this change but it does fix the issue on our site.
ctrlADel → created an issue.
ctrlADel → created an issue.
volkswagenchick → credited ctrlADel → .
Another confirmation that our site ran in to this while upgrading. Took a long time to track down that this was the cause so I can't confirm that the patch in #13 solves the issue when first encountered but I can confirm that running the code after the delete in the update hook fixed our issue.
drush eval "\Drupal::service('config.installer')->installDefaultConfig('module', 'moderation_dashboard')"
Patch applied and Enqueue content is working as expected.
Applied the patch in #12 and everything appears to be in working order on our sites. Setting RTBC.
Opened a MR that does two things
- Enables fuzzy type casting for arrays and objects during validation using Constraint::CHECK_MODE_TYPE_CAST from the validation library https://github.com/justinrainbow/json-schema
- Check the error messages for a NULL found error and append additional context that points people to the most likely problem/solution which is that the empty data value and the populated data value have different types. I wasn't able to find an easy way to check the schema against the props which is why I checked the error message text instead of a more thorough approach.
After talking with e0ipso it seems like documenting the way to correctly type empty values is the path forward. There are two main cases that we are likely to encounter
- Array found expecting Object
- Null found expecting Anything(string, number, array, etc)
In both cases the solution is to use the |default() filter to set the correct empty value.
Another thought was to try and capture this type of error in the validator and in the error message link to the fix on drupal.org.
ctrlADel → created an issue.
ctrlADel → created an issue.
volkswagenchick → credited ctrlADel → .
We think it's not a good idea for core to support significant alterations to discovery and definition. This is because it increments the API surface of SDC and the maintenance of it in core. Additionally having disparate discovery and definition in different sites goes against the standarization goal. It will be difficult for a new front-end developer to understand their site when the documentation and examples look so different from what they see in their site. Contributed modules, however, can alter this behavior.
The problem I see with this is that regardless of what we think best practice should be devs are going to tweak, change, and extend how SDC works because components sit at the very core of modern component based authoring experiences. The question is will they be able to do it in a way that is easy and supported or will they be stuck replacing parts of a core module? If different changes require replacing the same parts of SDC that also creates a compatibility problem in the contrib ecosystem.
A few use cases I can think of:
1. I have a design system from a client that contains components with twig/react/etc templates and ~75% of the definition SDC components need but the definitions are in a single json file, the missing 25% can be inferred from the 75%. How could a custom module create SDC components from that source?
2. Some of the npm libraries I require via composer and install to the libraries directory have standard yaml SDC components in them. How do I get SDC to see them?
3. I add a third party setting to my SDC components in a multisite environment that lists which sites a component should be available on. How do I filter the list of available components based on this prop?
We also believe it's not a good idea to alter how a component is rendered. This is to avoid some of the current confusion that themers experience when several alters change how their template is rendered, or change their variables, etc. When that happens in different modules and base themes, finding your way around is quite hard. When everything can happen everywhere at any time, we can't have good documentation, we can't provide relatable examples, we can't give good support in Slack, newcomers can't state their problems correctly, ... We want to avoid that in components. The principle is that everything that affects a component, is in the component's folder.
While I agree it'll be best practice to not preprocess components there's a valid argument that some times you just need to do the same preprocessing step across many components based on their values or context. Perhaps SDC shouldn't include a way to do preprocessing but it should include a way for contrib to add render preprocessing to SDC. With the current SDC approach how would a module add a render preprocessing system?
Pushing all the preprocessing in to twig with custom functions/filters is also less than ideal for a few reasons
- It's likely all the functions and filters will end up in one file or even worse spread across multiple modules and classes. Taking twig tweak's class as a sample of what could be needed on a project https://git.drupalcode.org/project/twig_tweak/-/blob/3.x/src/TwigTweakEx..., I wouldn't call it super approachable for a front end dev.
- Pulling content directly in to twig via custom functions/filters has security, caching, and access control nuances that would make it a very hard feature for front end devs to use correctly. For example see #2660002: Allow explicit bubbling of cacheability metadata inside Twig template (when accessing data from instead of rendering render arrays) →
- Custom functions and filters reduce the portability of twig templates(especially if the custom functions load in additional content or processing logic via Drupal services). The custom functions/filters have to be stubbed or fully reimplemented to render the template in a system other then Drupal. I'm mainly thinking if you're trying to render the template in a js framework or Storybook using something like twig.js.
I took a pass at splitting the API and render parts of SDC into separate modules to see what it would look like. The result can be found here https://gitlab.com/ctrladel/sdc-component-api/-/tree/1.x/component_api.
The result looks very similar to Layout API and Layout Builder. An API to define, validate, and discover a data structure and an implementation that makes the data structure actually usable in Drupal.
I’m gonna start my response by quoting a comment I left in #3346022 →
After spending some time in the code, if SDC were released today I'm not sure I'd adopt it. At the individual component level I would love to use them but if the only way to leverage components is through the current discovery/processing approach then I'm not sure. For a system that sits at the center of a site build it doesn't feel like I'd have enough control to make it my own and make changes to function in a way that works for my team or my clients.
I’m as excited as everyone else about the potential of a standard component definition for the community to start converging on instead of all doing our own thing. My concern is that SDC contains a good definition of a component but is not a low level system, it’s a fully featured system built upon the approach used in cl_components, which has ~600 reported installs, not the approach in ui_patterns which has the much mentioned 4k+ users.
The system introduced by SDC is opinionated, rigid, and with no affordances for contrib to build on top of it. Any attempt to add reasonable extension points has been met with a “we don’t think this is needed and are trying to limit API surface area” See:
✨
Allow different methods of discovery for SDC components
Fixed
✨
Have a way to implement the a preprocess function per each SDC component (ideally in the same folder)
Active
🐛
Add alter hook and cache backend to component plugin manager for SDC
Fixed
The assurances that we’d be able to use a SDC Extras module to accomplish the iteration and maturing SDC needs is questionable in my mind, none of the three linked issues can be addressed without overriding/replacing significant pieces of the current SDC code.
Specific areas of concern
UI Patterns
There has been much talk about how widely used UI Patterns is and how much SDC would benefit the ecosystem but as far as I could track down there hasn’t been any work to prove out SDC in it’s current state is compatible with UI Patterns. There is mention of potentially using it on the roadmap for 2.x 🌱 [2.0.x] Roadmap Active but that’s it.
Template resolution
SDC introduces a new way for resolving templates in twig
{{ include('sdc_examples:my-button', {......}) }} that would not be available in other systems that use twig. This approach makes it so Drupal SDC components can’t be rendered in other php applications or in frontend tooling like Storybook. It wouldn’t be impossible to fix, there are paths to adding SDC’s template resolving to other applications but it creates a significant barrier. Twig.js for example doesn't have a documented way to add this https://github.com/twigjs/twig.js/wiki/Extending-twig.js. There is always the approach used by cl_components which is to connect Storybook to a live Drupal backend for rendering but that comes with its own set of drawbacks.
Using twig’s native namespace functionality for template resolution would make SDC components much more compatible with other systems.
Component discovery
SDC only supports discovering components from *.component.yml files that are located in a module or theme but as mentioned earlier in the comments the concept of a component is not unique to Drupal and they can take many shapes in the wider frontend world. Even in Drupal we already have multiple ways to define components like UI Patterns, Component → , Single File Components → . Only supporting *.components.yml discovery is limiting and will make it difficult for other contrib modules to adopt SDC components. #3346022 → aimed to address this by allowing anyone to add a component discovery method.
A potential path forward?
If we’re pushing hard to include the concept of a component in core, I think it'd be a good idea to split the component definition and component system parts of SDC into separate modules. This follows what was done with Layout API and Layout Builder which has proven to be a successful pattern since there are multiple widely used modules dependent on Layout API, namely Layout Builder and Layout Paragraphs. "SDC Component" containing the schema definition/validation could be included in core. Then contrib modules, including "SDC System", would be able to build on top of that truly low level system.
Overall the definition and structure of a component in SDC is solid and I’d happily use it. The system and approaches that have been built up around that component definition I’m less confident in. The question for me is if I have to fight or significantly patch/override/replace parts of SDC to make it work for my project why not just build my own system?
@e0ipso is there an extension point that would allow another module to add preprocess hooks or the preprocess approach I described to SDC? Perhaps that'd be a decent compromise.
I don't see custom twig functions and filters as being a good enough stand in for preprocessing on the backend.
- It's not any more approachable than plugins to a frontend developer. In some ways it's worse since all the functions and filters will likely end up in one file. Taking twig tweaks class as an sample of what could be needed on a project https://git.drupalcode.org/project/twig_tweak/-/blob/3.x/src/TwigTweakEx..., I wouldn't call this super approachable for a non backend dev.
- As I mentioned in #3346715-5 🌱 Make twig sandboxed again, and empower themers by injecting services Closed: won't fix direct access of values in during render creates a lot of nuance and possibility for mistakes.
- Custom functions and filters reduce the portability of twig templates. The custom functions/filters have to be stubbed or fully reimplemented to render the template in a system other then Drupal. I'm mainly thinking if you're trying to render the template in a js tool like Storybook.
ctrlADel → created an issue.
volkswagenchick → credited ctrlADel → .
Looked like the reroll was missing the tests that were in the patch from #25 so pushed those to both the 9.5 and 10 MRs
ctrlADel → made their first commit to this issue’s fork.
ctrlADel → made their first commit to this issue’s fork.