text_summary() should output valid HTML and Unicode text, and not count markup characters as part of the text length

@larowlan Re #259

In D 10.2.3, text_summary (trimmed) output is still much shorter than the selected number of characters in "Trimmed limit." It appears that it still counts markup within the character count.

E.g. I set a Trimmed limit to 1500 characters but get this for trimmed output:

This is only 900 characters.

If the node contains something with more markup, such as bullet points, the trimmed value is even shorter. e.g. with the same filler text put into a numbered list, like this:

Meeting recording:
[video embed]

Your questions -- answered!

This is some filler text that I am using to test the trimmed text function.

1. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
2. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
3. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function. This is some filler text that I am using to test the trimmed text function.
4. This is some filler text that I am using to test the trimmed text function.
5. This is some filler text that I am using to test the trimmed text function.
6. This is some filler text that I am using to test the trimmed text function.

And continues on with more paragraphs of text here.

Then the trimmed output looks like this:

With the same text as in the first screenshot, this time the trimmed value only has 139 characters.

Thanks @leeksoup - can you update the issue summary with remaining tasks etc?

@larowlan - Do the remaining items need to be split off into a new / separate issue?

I think this issue is fine, thank you for updating the issue summary!

Should this be split into two issues? One for the valid HTML and one for excluding markup from character count? I think the character count part of it can't really be called a bug since it is explicitly tested to work that way, meaning it is intentional behaviour. I do agree that it makes sense to exclude it but that seems like a feature request.

I also think this would need to be opt-in for existing sites because it will change what is displayed for some sites.

Updated IS to be a bit more clear.