text_summary() should output valid HTML and Unicode text

@larowlan Re #259

In D 10.2.3, text_summary (trimmed) output is still much shorter than the selected number of characters in "Trimmed limit." It appears that it still counts markup within the character count.

E.g. I set a Trimmed limit to 1500 characters but get this for trimmed output:

This is only 900 characters.

If the node contains something with more markup, such as bullet points, the trimmed value is even shorter. e.g. with the same filler text put into a numbered list, like this:

Meeting recording:
[video embed]

Your questions -- answered!

This is some filler text that I am using to test the trimmed text function.

1. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
2. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
3. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
4. This is some filler text that I am using to test the trimmed text function.
5. This is some filler text that I am using to test the trimmed text function.
6. This is some filler text that I am using to test the trimmed text function.

And continues on with more paragraphs of text here.

Then the trimmed output looks like this:

With the same text as in the first screenshot, this time the trimmed value only has 139 characters.

Thanks @leeksoup - can you update the issue summary with remaining tasks etc?

@larowlan - Do the remaining items need to be split off into a new / separate issue?

I think this issue is fine, thank you for updating the issue summary!

Should this be split into two issues? One for the valid HTML and one for excluding markup from character count? I think the character count part of it can't really be called a bug since it is explicitly tested to work that way, meaning it is intentional behaviour. I do agree that it makes sense to exclude it but that seems like a feature request.

I also think this would need to be opt-in for existing sites because it will change what is displayed for some sites.

Updated IS to be a bit more clear.

@pameeela I agree the focus should be to fix the broken HTML. Not counting the HTML characters can be a less relevant follow-up feature!

Updating this issue to reflect that the bug reported was about invalid markup. However, I'm unable to reproduce it on D11, so I think maybe it's fixed in CKE5? I can't reproduce it using basic or full HTML regardless of whether 'Correct faulty and chopped off HTML' is enabled.

Marking postponed in case I'm missing something obvious. I will create a separate issue for excluding tags from trimming, as already noted.

Actually, I just noticed #3067116: text_summary() returns malformed (not normalized) HTML for basic_html and other formats that use filter_html instead of filter_htmlcorrector → so I think it was fixed there. but by then this issue had already expanded to include the trimming. So going to close this one.

Created ✨ Strip tags from formatted text before trimming Active