Unify & simplify render & theme system: component-based rendering (enables pattern library, style guides, interface previews, client-side re-rendering)

I wonder where this fits now that we have SDC, and Experience Builder is in early development?

The Ideas project is being deprecated. This issue is moved to the Drupal project. Check that the selected component is correct. Also, add the relevant tags, especially any 'needs manager review' tags.

I wonder where this fits now that we have SDC, and Experience Builder is in early development?

We still need to work on this because:

• SDC is addressing only UI components, an important part of a modern Render API but not everything
• Experience Builder is a display builder, so an higher level tool. We need to address the lower levels

Nearly 5 years after my previous comment, here we are.

Current status

Let's not forget the Render API is great:

• Declarative: Easy to type. (de)Serializable if clean.
• Easy nesting: The Virtual DOM of Drupal. We are building a tree.
• Data bubbling: Declare locally, impact globally
• Asset libraries management: Our beloved libraries.yml
• Clever caching: Context, tags, keys…

❤️ Building display by assembling configurable plugins returning renderables.

However, the renderables themselves are an issue.

They are too many of them and they are of 3 kinds:

• 35 render elements in Core:
  

  $ grep -hoEr "#type' => '(\S+)'" core/ --exclude-dir tests | sort | uniq -c | sort -nr
  138 #type' => 'details'
    94 #type' => 'container'
    93 #type' => 'link'
    68 #type' => 'table'
    47 #type' => 'actions'
    36 #type' => 'inline_template'
    27 #type' => 'fieldset'
    23 #type' => 'html_tag'
    21 #type' => 'status_messages'
    14 #type' => 'pager'

• 99 theme hooks in Core:
  

  $ grep -hoEr "#theme' => '(\S+)'" core/ --exclude-dir tests | grep -v "__" | sort | uniq -c | sort -nr
  84 #theme' => 'item_list'
  21 #theme' => 'username'
  16 #theme' => 'image'
    7 #theme' => 'status_messages'
    7 #theme' => 'image_style'
    6 #theme' => 'table'
    6 #theme' => 'links'
    5 #theme' => 'update_version'
    5 #theme' => 'indentation'
    5 #theme' => 'file_upload_help'
    …
  

• 2 special ones:
  

     1016 '#markup'
         85 '#plain_text'
  

😱 135 renderables to learn/use? This is the issue.

What can we remove?

We have SDC for UI components, so:

• #type => component can replace a lot of hook_theme, the one describing UI components: breadcrumb, progress_bar, links...
• We can also replace the 34 render elements which are wrapper around hook_theme (because what they do, adding asset libraries, variable typing... is already done by SDC) we removed : table, status_messages, pager...

The hook_theme not suitable for for conversion to SDC are often annoying wrappers we can get rid of anyway : block, node, view, field…

The few remaining render elements can't be expressed as UI components:

• some must be kept because they represent other design systems artefacts (icons...) or low level bricks (HTML elements, placeholders...)
• some may (not found in my today look) be shortcut to Drupal callables returning renderables, it is better to explicitly call the callable (service, function, method..) instead

Deprecation of ThemeManager::render()

If all template file based renderable are managed by SDC and because SDC is not passing through the ThemeManager, we dont need the template loading feature of the ThemeManager and we get remove it to get roi of the old confusing stuff:

• Theme wrappers: Confusing and useless. You can always use an upper level instead.
• Templates suggestions: Not discoverable. Messy. Blur the business / view separation.
• Preprocess hooks: Risky. Unfriendly. Blur the business / view separation. Unpredictable order of execution.

Conclusion

We can have an equally useful Render API with less than 10 renderables (not counting Form API):

• Design systems artefacts:
  • #type => component: for UI components
  • #type => icon: for UI components
• Lower level bricks:
  • #type => inline_template
  • #type => html_tag : for HTML elements
  • #type => link
  • #markup
  • #plain_text
  • #lazy_builder

  Keeping the beloved universal render properties: #attached, #cache... and adding some related to design systems: #styles, #tokens, #mode...

  And now we have an purely design/UI based Render API!

  If we do that, step by step, deprecating slowly, not breaking anything, the display building tools from Core and Contrib will have a simpler implementation and will have an easier time introducing new features.

  To play safe, we can also introduce a new Render API alongside the existing, using @ instead of # for example:

  • #type => component, #component= "foo:bar" ⮕ @component => "foo:bar"
  • #type => icon ⮕ @icon => []
  • #type => inline_template, #template => "foo" ⮕ @template => "foo"
  • #type => html_tag, #tag => "foo" ⮕
    

    @element => "foo"</li>
        <li><code>#type =>link

    ⮕ @link

  • #markup ⮕ @markup</li>
  • #plain_text ⮕ @plain_text
  • #lazy_builder ⮕ @lazy_builder

I'm going to take the optimism path and assume that moving this from "ideas" to a real core issue is a promotion which increases its visibility and chances for success. 😅

I don't fully understand #type => component or how to entirely get rid of preprocess via SDCs yet (would be great to get separate issues for those opened maybe?), but everything else in #99 I'm 100% agreed with, and I would love to get rid of preprocess and as many (all?) theme hooks as possible.

Adding three concrete active issues that could use reviews/help.

There are also a lot of open child issues of #1804614: [meta] Consolidate similar twig templates and properly use theme suggestions in core → still.

how to entirely get rid of preprocess via SDCs yet

I'm more likely to implement a twig extension instead of a preprocess function nowadays. Maybe it would be enough to just improve the DX around twig extensions. E.g., attributes:

#[TwigFilter('without')]
public function withoutFilter($element, ...$args) {
// [...]
}

#[TwigFunction('url', options: ['is_safe_callback' => Foo::isUrlGenerationSafe])]
public function getUrl($name, $parameters = [], $options = []) {
// [...]
}

but everything else in #99 I'm 100% agreed

@catch in #106 ? ;)

I don't fully understand #type => component or how to entirely get rid of preprocess via SDCs yet

In UI Suite community (not only the contributors in drupal.org, but also the agencies doing projects I am interacting with), we are not using preprocess hooks anymore, for many years, except for forms. I don't really know why it is a less an issue for us. Maybe because we fully switched to a design system + display builder paradigm. It is worth investigating and writing that down.

Adding three concrete active issues that could use reviews/help.

Thanks you. I will have a look after Atlanta.

I'm more likely to implement a twig extension instead of a preprocess function nowadays.

Twig extensions have a high risk of being harmful:

• Written in PHP, they may hide some business logic specific to a specific project, or calls to CMS API. A SDC component is just a "dumb" piece of UI logic receiving already resolved data.
• Written in PHP, they may introduce PHP in a Drupal theme (which we cant to avoid, a theme must be a front dev friendly place) or a dependency to module.
• Instead of fixing the root cause (why this data is not OK to be injected in my template? We are moving the problematic mechanism elsewhere

In UI Suite community (not only the contributors in drupal.org, but also the agencies doing projects I am interacting with), we are not using preprocess hooks anymore, for many years, except for forms. I don't really know why it is a less an issue for us.

The projects might not be using them directly, but there are dozens/hundreds of preprocess hooks running on those projects via core and contrib modules. https://api.drupal.org/api/drupal/core%21modules%21node%21node.module/fu... is a monster just by itself. Removing just one variable is non-trivial, although at least we have deprecation support now.

The projects might not be using them directly, but there are dozens/hundreds of preprocess hooks running on those projects via core and contrib modules.

Of course.

Also, it may be the opportunity to challenge the various additions Drupal made to Twig last 10 years.

Today, in my very personal and humble opinion, I see only 5 additions which will stay relevant if we move to SDC, decoupling between UI logic and business logic, and design system philosophy:

• the PrintNode visitor to send all printed variable to the renderer service. That's the core of Drupal Twig rendering, it works very well and must be kept.
• Attribute object and the related create_attribute() function, but I hope this issue will allow us to get rid of it one of those days: ✨ HTML attributes as Twig mappings instead of PHP obejcts Active
• add_class() & set_attribute() filters, they make sense used with the SDC slots.
• the new icon() function for the Icon API. It follows Twig philosophy of using functions to generate printable data
• the translations helpers (t function, trans tag). Important feature for a CMS. Why not proposing this to upstream Twig?

So, other Drupal additions (clean_id(), attach_library(), link(), |add_suggestion()...) seems OK to be deprecated to me, once we have "cleaned" the Render API. I would happy to discuss this.