[Meta] High-performance images (nearly) out of the box

Thanks for this issue! As the maintainer of the Easy Responsive Images module, I have struggled with this before. When working with responsive images in core, the following issues were the most important reasons we created the Easy Responsive Images module:

• You need to create a lot of image styles, most of them are similar. Maintaining them is quite troublesome as the site continues to grow.
• Configuring responsive image styles is quite complex, you need to first create all the image styles, then the responsive styles for all breakpoints, and then configure them on a formatter for your fields. This is not very intuitive.
• The picture element selects images based on the browser width, not the width of the image container. This is a problem for component-based designs where a block can be used in a 2/3/4 column grid for example.

You need to create a lot of image styles / Configuring responsive image styles is quite complex
Since the amount of use cases is huge, it is very hard to come up with a solution that is easy to use, yet supports everything. It would already greatly help if all the configuration for responsive images can be done in 1 place, letting it generate the image styles it needs in the background. As long as it cleans up/updates generated styles when you change things (maybe even lock generated styles and hide them), they won't complicate the interface "regular" image styles.

The responsive image styles could then be an interface similar to regular image styles, just with breakpoints.

The picture element selects images based on the browser width, not the width of the image container
This would definitely need a JavaScript-based solution at the moment. There is awesome work being done to make the sizes attribute for images support the "auto" value. That would be amazing, and it could be worth it to wait for this.

Output WebP for image by default
This is currently done using the WebP module → or ImageAPI Optimize WebP → . I agree, it would be nice if core would just do this. All major browsers support it, so not sure if there is a good reason not to? It could be a checkbox on the responsive image style and image style entity forms to opt-in or opt-out?

I love the ideas expressed here. I even proposed it as an effort that _needs_ get done. However it wasn't sponsored at the time. Anything we can do to improve the UX/DX in this space would be GREAT.

If this is a meta/plan issue, we should start to link more items here. To that end, I've re-tagged as a plan and added references to:

• webp has been part of drupal core since 9.2. See https://www.drupal.org/node/3171135 →
• It is the default now for core's install profile as well, see 📌 Add webp image conversion to core's install profile's image style Fixed .
• More work is ongoing to support webp in ✨ Allow user-uploaded WebP images everywhere in Drupal by default: image fields, media library, text editors Needs work .
• And AVIF support is slowly making its way into this space as well over in ✨ Let GDToolkit support AVIF image format Needs work .

The responsive image configuration issues discussed in #2 are just about in the purview of field UX (given you end up selecting responsive images in field formatter configuration), so adding that tag.

https://github.com/whatwg/html/pull/8008 seems like it's starting to happen already, so we should definitely open an issue to support that, then we've got a chance of it working when the support is more widespread.

I had a crack at something similar in contrib, yonks ago: https://www.drupal.org/project/responsive_image_automatic →

Glad to see this problem is being raised here. I've been in a several discussions where this has come up. Drupal Core Responsive Image module is really powerful but at the same it can be time consuming and complicated to configure it properly. I'm happy to support the efforts where needed to the extent I can. 😊

I've personally really enjoyed using Next.js Image plugin. It makes setting up and configuring responsive images really easy. It doesn't provide all of the flexibility that Drupal Responsive Images module provides, but maybe it's worth revisiting if we need all of that flexibility.

webp has been part of drupal core since 9.2

What the webp support in core lacks is having an automatic jpeg/png fallback in picture tags for browsers that don't support WebP. (And the same logic could also apply to AVIF.)
The webp contrib module does that and ✨ Add fallback format support to responsive images Needs work discusses adding this functionality to core.

The fallback is less necessary for webp specifically because browser support is pretty much universal now, the last holdout I knew of was pre-2015 Macs, which also can't be detected via usage stats due to some weird decisions by the osx team. We decided that was an anomaly as far as 📌 Add webp image conversion to core's install profile's image style Fixed was concerned so implemented default webp in the standard profile for 10.2.x

However, ✨ Add fallback format support to responsive images Needs work will be useful for avif or any other new format that arises, so we should still try to add that.

In a perfect world the resulting image format should depend on image, webp vs avif - mostly always the later wins

@seanB, @GaëlG :

> To do this in Drupal, I have to use the webp contrib module (or some other contrib/custom module).
> - @GaëlG

> Output WebP for image by default
> This is currently done using the WebP module or ImageAPI Optimize WebP. I agree, it would be nice if core would just do this.
> - @seanB

As of Drupal 9.2 according to @heddn in #3 above, Core is able to provide WebP support out-of-the-box for images styles. It is hidden away out of sight, so I do agree it is hard to find. To enable it, you have to add the 'Convert' action to an image style, and you will be able to select WebP as the desired format. I think that the fallback issue should still be solved in Core, but for now it seems that the https://www.drupal.org/project/wpf → (WebP Fallback Image) module is able to handle that in a nice way without the need for additional configuration or setup. What I like about this approach is that the site will be serving WebP images by default, and will only need to do additional processing for the fallback images that are required.

I do agree that it is pretty time-consuming nowadays to setup high-performance using Core. With more and more different images sizes, pixel-densities (2x is considered 'default' by now, 3x is already being used by modern iDevices but not often implemented, and its just a matter of time before the 4X multiplier will be introduced...) . If you would want to support all that for a full-width hero-image (which is a common use-case), also providing efficient images for use on mobile, you will already need quite some configuration and setup. Having something that makes this easier (e.g. the module mentioned/created by @seanB) would be much appreciated.

Dropping the notes from my research of the 2 modules I've come across that try to help in this space:

• responsive_image_automatic was written 7 years ago and is feeling its age. the ideas in there are GREAT, but it doesn't add any 1.25, 1.5, 2x, 3x multipliers. maybe with a bit of work, it could be a real contender if it were modernized.
• easy_responsive_images is a recent addition, assumes use of contrib webp support (not core), and uses a resizer javascript helper script

Are there other options to review? While rough, I really liked the simplicity of responsive_image_automatic. With a little bit of dusting off, I think it could really do a lot of things here.

Thanks for the research. I've updated responsive_image_automatic module so I can test it in Drupal 10.1: https://github.com/miloskroulik/responsive_image_automatic.

Apart from missing multipliers, it (probably obviously) doesn't support lazy-loading of responsive images, which has been introduced in D 10.1 → .

Following the work in that github repo.

There's also another module - https://www.drupal.org/project/drimage → , that seems to be very similar to https://www.drupal.org/project/easy_responsive_images: →

• it needs a JS code to work
• it integrates with focal_point and imageapi_optimize_webp

But it provides an image field formatter, which makes the setup very quick.

I've done some basic testing - it seems to be working fine on the frontend, but JS file is currently not loading in the CKeditor ( https://www.drupal.org/project/drimage/issues/3079190 ✨ embeds in wysiwyg not displaying media Closed: works as designed - but it looks like there's some workaround).

re #14: that looks pretty nice. I wish it would do its thing without js/css though. And it would support native/core webp instead of requiring a contrib module.

It looks like there's a tradeoff - either you can generate images based on a static configuration (increases in viewport width), like responsive_image_automatic does or you need to use a JS to scale the image based on the dimensions of the container.

But I wonder, if it would make sense to generate the image styles and responsive image mappings automatically based on a "template" image style (which could for example define the webp conversion) and the theme.breakpoints.yml file.

From a predictability perspective, I'd want a starting place and some sizing parameters defined in the field formatter. I also want to be able to set all of this in the Node view mode (not in media view mode). It feels a little scary to have a module that automatically generates image styles without any limits. Could easily lead to a DDoS and filling up the disk.

I also want 1.5x, 2x, 3x and (soon) 4x image styles that increase the size of the image but also reduce the image quality to 50% or 30%.

All of this should put the experience of building styles for responsive images front and center. Maybe there could be some integration w/ https://ausi.github.io/respimagelint/ to instantly test the built styles?

For configuration on the node view mode we'd need ✨ Responsive image format for media Needs work .

For #16 something like that seems like a good idea. I think what you're suggesting is you start with one manually created image styles, and then responsive image module when you're creating a responsive style gives you the option to generate variations based on the same aspect ratios. It would allow for what srcset and sizes attributes can do. Completely manual configuration with different aspect ratios is still there either for different aspect ratios/cropping rules or just fine tuning.

How about an interface where instead of generating image styles, you can define aspect ratios to use (e.g. 1/1, 4/3, 16/9, etc). Then, for each aspect ratio, you can (manually) define the widths for which to generate image styles (e.g. 120px, 320px, 640px, 1200px, etc). Additionally, you could enter / select a range of multipliers (1x, 1.5x, 2x, 3x, 4x) to use. All that combined will result in a matrix that can generate all required image styles beforehand without the need for Javascript during runtime. Added benefit is that because the aspect ratio is the same, the image styles for the larger breakpoints can also be injected into the smaller breakpoint (e.g 2x 640x480 can also be used as 1x 1280x960).

This does allow the freedom to decide which breakpoints and image sizes to generate.

I do like #19, but I have concerns about re-using image styles that are 1x in one place for 2x, 3x, etc in other places. The image quality for 2x and 3x doesn't need to be 80%+. A quality of 30-50% is sufficient. Because while mobile devices can render greater pixel density, the human eye cannot tell the difference between 30% 2x and 80% 2x and the size saving for a 2x 30% is significant enough to justify making a different image style.

I hadn’t thought about that, seems to be a fair point (although I’m not sure if I would go as low as 30%, but that’s a concern for later).

While my suggestion from #19 is not yet what I would call ‘out of the box’, it would be much easier to configure for a large amount of image sizes/ratios. Of course, it would be possible to supply some (optional) default configuration, that e.g. provides the 4/3 ratio by default with some common used widths (320px … 1200px … 2500px?) in 1x and 2x multipliers, in WebP?

Hello there, thank you all for the great feedback on this issue! It looks like many people are concerned and have ideas! :)

It may be the good time to start organizing a bit the various problems and ideas discussed here. It's look like there are 3 main questions.

1) What should be the decent/recommended/optimized/default way to display an image in HTML?

picture, source, srcset, sizes, image file formats, multipliers, image quality, breakpoints,... This issue in somewhat Drupal-agnostic, I guess we can get answers elsewhere on the web.

Avenues in comments #1, #4, #17, #20

See also:
https://www.stefanjudis.com/notes/should-responsive-images-work-with-con...
https://github.com/whatwg/html/pull/8008
https://ausi.github.io/respimagelint/docs.html

2) Which technical way to share/pool the creation and also the updates of many Drupal image derivatives needed for a same responsive image display, to get a better admin UX?

ImageStyleInterface should not extend ConfigEntityInterface? Or we should use config at the field formatter level and get rid of the Drupal "image style" concept?
Wizard à la Views?

I believe we need a clearer view in question 1 before going further with this.

Avenues in comments #1, #16, #17, #18, #19, #21

See also:
https://www.drupal.org/project/easy_responsive_images →
https://www.drupal.org/project/responsive_image_automatic →
https://www.drupal.org/project/drimage →
https://www.drupal.org/project/image_styles_builder →
https://www.drupal.org/project/image_style_generate →
✨ Responsive image format for media Needs work
🌱 [META] Usability of Responsive Image in Core Active

3) Webp and avif conversions by default in core, with fallbacks (at least for avif which is less supported for now)

This seems to be more a task than a question, as there is some consensus on what should be done (maybe not that much on how in should be done?). This is already a work in progress independently from this issue. There could be a follow-up to question 2 to handle question 3 in the new UX paradigm.

See also:
https://www.drupal.org/node/3171135 →
✨ Add fallback format support to responsive images Needs work
📌 Add webp image conversion to core's install profile's image style Fixed
✨ Allow user-uploaded WebP images everywhere in Drupal by default: image fields, media library, text editors Needs work
✨ Let GDToolkit support AVIF image format Needs work
https://www.drupal.org/project/wpf →
and comment #9

So, do we make this issue a meta and create 3 children issues?

picture, source, srcset, sizes

I don't think that needs revisiting here. Core responsive images module uses the picture element which has a superset of all the other methods. So they may be cases where it's overkill, but we'd always need to provide it unless browsers deprecate it. We could potentially look at alternative formatters/formatter configuration options when picture isn't needed but that doesn't feel necessary.

Great summary in #22. I think bullet 2 is key and will make/break responsive images. They are too hard to setup and manage, so they simply aren't used nearly enough.

Great ideas and discussions! Thanks everyone. I think it's definitely a good idea to start using this as a meta issue. We can discuss, share and track possible improvements here.

Some ideas for issues to start improving things:

• Automatically generate a WebP version for image styles and use WebP by default.
  All major browsers support it. Since AVIF is not supported by Edge, we need to check the HTTP_ACCEPT header which makes it harder to cache pages, so I suggest postponing that and add it later.
• Improve the UI for configuring responsive images.
  We could use 🌱 [META] Usability of Responsive Image in Core Active for this one (one idea to improve this can be found in #2)
• Add a responsive image formatter using an img tag with the srcset / sizes attributes
  I definitely think we should consider supporting an img element with srcset / sizes attributes. All major browsers support it. My reasons for this are the following:
  1. It is a more lightweight option.
  2. The extra picture element can make it harder to provide consistent theming and JavaScript.
  3. Once browsers have implemented the sizes="auto" feature. This would allow lazy-loaded images to load images based on their actual size, not the viewport size.
  4. There is a difference, but in core I don't think we are using any features of picture that srcset / sizes don't cover.

  For backwards compatibility, I like the idea of additional formatters for this.

• Implement the sizes="auto" feature for lazy-loaded images in the new responsive image formatter.
  The spec has landed (https://github.com/whatwg/html/pull/8008). We should postpone this until all major browsers have implemented the feature, but this can definitely improve performance when entities might be rendered in different 1/2/3 column layouts (like block content or paragraphs).
  https://bugs.chromium.org/p/chromium/issues/detail?id=1359051
  https://bugzilla.mozilla.org/show_bug.cgi?id=1819581
  https://bugs.webkit.org/show_bug.cgi?id=253143

Since AVIF is not supported by Edge, we need to check the HTTP_ACCEPT header which makes it harder to cache pages

You don't need to do this. If you generate a picture tag with multiple formats, the browser will only load the AVIF file if it supports this format: https://avif.io/blog/tutorials/html/#implementation

@catch #23: Sorry, I wasn't clear enough. Yes, picture/source can be useful because they allow for file type fallbacks, media queries (notably for art direction),... And even if a picture tag with only one child source tag can be simplified into an img tag, it's not necessary.

I meant: what is a bit more controversial is: most of the time, to display an image (usually a photo), should we use:
- sizes?
- and/or media queries?
- and/or multipliers?
If we use srcset, which widths should be provided? Should we have a different image quality (I mean compression in %, not size in px) depending on the multiplier?

In other words: if I have a source image file and a desired aspect ratio, what should be the resulting HTML, by default in core? My current answer is in the issue summary, but I'm not sure it's the right one. I did not know about sizes=auto but it looks great, as it prevents the need to set the right sizes value according to CSS, which can be difficult and can't be computed as a default.

@heddn #24: Yes, the big need is point 2. But it's easier to discuss if we agree on point 1 before.

@seanB #25: This is now a meta issue :) But before creating most children issues for Drupal implementation tasks (I like your ideas), I think we need to agree on point 1.

@prudloff #26: Yup, there are two ways to serve image formats with fallbacks. One is to look at HTTP_ACCEPT header and serve a different HTML according to it. The other is to use a picture tag. The second one seems better to me because it eases caching. But this is one of the questions on which we should agree before we can set a default system.

Just referencing this blog post which may help with step 1 of comment #22: https://mariohernandez.io/blog/responsive-images-in-drupal-a-guide/

I also talked to @nod_ which is Provisional Frontend Framework Manager and may help take a decision on step 1.

Great to see this.

This is how Wagtail is currently doing this:
https://wagtail.org/sustainability/
https://wagtail.org/blog/wagtail-greener-and-leaner/

Also an idea for relative CO2 impacts (all estimates of course):
https://sustainablescreen.org/unlocking-sustainable-speed-optimising-ima...

Thank you @mgifford. I can see that in Wagtail, they seem to have no default value for the sizes and srcset values. So it's up to the "site builder" to define the right values, which can be difficult. I guess in Drupal we should have a decent default, if possible.

https://docs.wagtail.org/en/stable/reference/jinja2.html#picture

One idea I've posted over in the #sustainability-team channel on the Drupal Slack, and @catch suggested there to repost it in this issue as well.

There is the following linting tool: https://github.com/ausi/respimagelint which provides recommendations about the used image sizes as well as the syntax of img srcset and the picture element. I wonder for a while now if it wouldn't be possible to adjust the algorithm of respimagelint to enable Drupal Core to auto generate a list of image styles for the installed default theme? For the default theme you know all the breakpoints across viewports and the syntax for img srcset and the picture elements, shouldn't it then be possible, instead of just optimizing image sizes, to auto generate a list of image styles from the ground up? But I am not a developer therefore i am uncertain if that might be a reasonable approach?

I just read about the Starshot project. This issue could be related?

Yes I think so too, it would be a good thing to add alongside a theme builder.

I also opened #3446582: Responsive images recipe → a couple of days ago, to ship some default image styles matching Olivero's break points to start with and linked this issue from there too.