Nikolai ShcheglovandAlexandre Alapetite acde1bfbf5 Fix non-ASCII characters in scripts of fetched HTML pages (#9384) ...

* Fix non-ASCII characters in scripts of fetched HTML pages

To add a <base> to a fetched HTML page, enforceHtmlBase() re-serialises it
with libxml2 saveHTML(). For a whole document, libxml2 2.9 only knows the
charset from <meta http-equiv="Content-Type">: with <meta charset="UTF-8">
or a byte order mark, it writes every non-ASCII character as an HTML entity.
In text that is harmless, but inside <script> entities are not decoded, so
e.g. the JSON read by HTML + XPath + JSON feeds gets "Wall Street&rsquo;s"
instead of "Wall Street’s", which then shows in titles.

UTF-8 documents are now serialised node by node, which writes UTF-8.
Other charsets keep the previous output.

* Clarify comment about the byte order mark

* Remove comment about the byte order mark

* Minor test syntax preference

---------

Co-authored-by: Alexandre Alapetite <alexandre@alapetite.fr>

2026-10-31 14:48:37 +01:00

..

Exceptions

Fix muting of feeds gone with HTTP 410 (#9391)

2026-10-03 09:59:51 +02:00

Models

Replace bcrypt challenge auth with direct password comparison (#9358)

2026-10-30 22:21:26 +01:00

Utils

Fix non-ASCII characters in scripts of fetched HTML pages (#9384)

2026-10-31 14:48:37 +01:00

Views

Preserve explicit sort overrides when searching (#9467)

2026-10-19 20:27:22 +02:00