HTML API: Reported Issues related to HTML Parsing

[dmsnell]

HTML issues | Refactors | ↑ HTML API Roadmap

General filtering

• Improper ampersand escaping Core-60191
• Post-title to slug conversion isn't sound Core-44958, Core-59691
• Broken autop behavior when mixing with shortcodes Core-58366
• wp_xmlrpc_server pingback code performs funky HTML parsing to "Work around bug in strip_tags()"
• convert_invalid_entities() is transforming the C1 controls but only for decimal numeric character references. also, it's removing the five non-transformed code points instead of leaving them in, which could change HTML syntax.

Casting HTML and character sets vs. parsing+printing

• Core-33073 Translations including character references show plaintext entities instead of converted entities in email.

Questions

• Should Core detect utf8mb3 encoding and rewrite four-byte-in-utf8 character to avoid crashing? Core-60362.

esc_attr(), esc_html(), and friends

• esc_html() returns empty string with special character at end Core-60295
• Core-57336
• Core-51159
• Greater-than-sign in attribute values cause other special characters to be encoded as HTML entities inside block's render php #56597
• Core-61378 (new code)

kses

• data prefix to image content in a data uri is stripped. Core-60243
• Long data uri contents crash the parser. Core-25851
• "breaking" text fragments in links. Core-60347
  • not yet clear what kind of breakage is going on here.
  • the example link includes the "text match" syntax: <a href="#:~:text=highlight>Link</a>
• Tailwind classes break attributes
  • Ensure valid HTML is properly processed by refining regex handling wordpress-develop#5697
  • Core-57381 Texturize breaks brackets
  • Core-45387 Mangled HTML on the frontend
  • Core-43457 HTML split broken on attributes

Excerpts

• Inconsistent handling of line breaks in excerpts #15117

"Balanced tags"

• wporg_support_wrap_standalone_li_tags_in_ul
• force_balance_tags() - does this even need to exist? should we do it on save?

To investigate

These may or may not be related to HTML parsing, but there is at least superficial overlap in terms of text-encoding or parsing issues.

• Shrug icon crashes post save Core-60156
• Search doesn't search shortcodes Core-60200
  • this probably stems from searching the database itself and not the rendered pages, but it would be good to verify that existing search knows the shortcode boundaries.

Tangential issues

• API uploads not decoding URL-encoded filenames. Core-60349