{"id":1830,"date":"2016-03-10T00:00:00","date_gmt":"2016-03-09T23:00:00","guid":{"rendered":"https:\/\/wwwneu.strehle.de\/tim\/weblog\/archives\/2016\/03\/10\/1590-2\/"},"modified":"2025-07-31T21:57:40","modified_gmt":"2025-07-31T19:57:40","slug":"1590-2","status":"publish","type":"post","link":"https:\/\/www.strehle.de\/tim\/weblog\/archives\/2016\/03\/10\/1590-2\/","title":{"rendered":"Turn HTML into plain text with proper whitespace (in XSLT and PHP)"},"content":{"rendered":"\n

Turning HTML into (unformatted) plain text seems simple at first: PHP has <a href=\"http:\/\/www.php.net\/strip_tags\">strip_tags()<\/a><\/code>, XSLT has <a href=\"https:\/\/www.w3.org\/TR\/xslt#value-of\">xsl:value-of<\/a><\/code>. In practice, though, you\u2019ll frequently find that words are glued together which should have whitespace between them.<\/p>\n\n\n\n

Take this example \u2013 extra weirdly-formatted to get the point across:<\/p>\n\n\n\n

\"\"\/<\/figure>\n\n\n\n

<\/p>\n\n\n\n

If you select and copy this text in the browser, the result will look similar to the following:<\/p>\n\n\n\n

Hello\nWorld\n\nFirst line\nSecond line.\n\n    1\n    2\n<\/code><\/pre>\n\n\n\n
Now look what we get if we feed the same HTML source code into strip_tags()<\/code> or xsl:value-of<\/code>:<\/p>\n\n\n\n
HelloWorld\nFirst lineSecond line.\n12\n<\/code><\/pre>\n\n\n\n
Words (\u201cHelloWorld\u201d instead of \u201cHello World\u201d) and lines are glued together! To understand why this happens, pay attention to the (perfectly valid) missing spaces between HTML tags in my example\u2019s source (I\u2019m using XHTML to be able to process it via XSLT):<\/p>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<html xmlns=\"http:\/\/www.w3.org\/1999\/xhtml\">\n<body>\n<h1>Hello<\/h1><h2>World<\/h2>\n<p class=\"c\"><span class=\"u\">F<\/span>irst l<a href=\"#\">in<\/a>e<br \/><b>S<\/b><del>e<\/del><em>c<\/em><i>o<\/i><ins>n<\/ins><mark>d<\/mark> <strike>l<\/strike><strong>i<\/strong><sub>n<\/sub><sup>e<\/sup><u>.<\/u><\/p>\n<ul><li>1<\/li><li>2<\/li><\/ul>\n<\/body>\n<\/html>\n<\/code><\/pre>\n\n\n\n
PHP and the XSLT processor simply remove all the angle-bracketed HTML tags. This is fine for inline elements; <span>4<\/span>2<\/code> is supposed to become 42<\/code>. But it\u2019s wrong for block-level elements<\/a> (\u201ctypically displayed with a newline both before and after the element by browsers\u201d) \u2013 <p>4<\/p><p>2<\/p><\/code> means 4 2<\/code> not 42<\/code>. Since the PHP and XSLT functions have no knowledge of block-level elements in HTML, they have no way to correctly distinguish between them and inline elements.<\/p>\n\n\n\n
In our application, we\u2019ve implemented a slightly hacky workaround that solves that problem. In essence, we\u2019re maintaining a list of inline elements (currently: a<\/code>, b<\/code>, del<\/code>, em<\/code>, i<\/code>, ins<\/code>, mark<\/code>, span<\/code>, strike<\/code>, strong<\/code>, sub<\/code>, sup<\/code>, u<\/code>) which are simply going to be removed. All other elements are supposed to be block-level and will be appended a newline. We\u2019ve done two implementations, in both XSLT and PHP.<\/p>\n\n\n\n
Here\u2019s our XSLT solution \u2013 certainly not the most elegant one (I\u2019m no XSLT guru):<\/p>\n\n\n\n
<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<xsl:stylesheet version=\"1.0\"\n    xmlns:xhtml=\"http:\/\/www.w3.org\/1999\/xhtml\"\n    xmlns:xsl=\"http:\/\/www.w3.org\/1999\/XSL\/Transform\"\n    >\n\n  <xsl:output method=\"xml\" encoding=\"UTF-8\" indent=\"yes\"\/>\n\n  <xsl:template match=\"\/xhtml:html\">\n    <plaintext>\n      <wrong>\n        <xsl:value-of select=\"xhtml:body\"\/>\n      <\/wrong>\n      <correct>\n        <xsl:apply-templates select=\"xhtml:body\" mode=\"xhtml_to_plaintext\"\/>\n      <\/correct>\n    <\/plaintext>\n  <\/xsl:template>\n  \n  <xsl:template match=\"xhtml:*\" mode=\"xhtml_to_plaintext\">\n    <xsl:choose>\n      <xsl:when test=\"self::text()\">\n        <xsl:value-of select=\".\"\/>\n      <\/xsl:when>\n      <xsl:otherwise>\n        <xsl:apply-templates mode=\"xhtml_to_plaintext\"\/>\n        <xsl:variable name=\"html_element_type\">\n          <xsl:call-template name=\"get_html_element_type\">\n            <xsl:with-param name=\"e\" select=\"local-name()\"\/>\n          <\/xsl:call-template>\n        <\/xsl:variable>\n        <xsl:if test=\"$html_element_type = 'block'\">\n          <xsl:text>\n<\/xsl:text>\n        <\/xsl:if>\n      <\/xsl:otherwise>\n    <\/xsl:choose>\n  <\/xsl:template>  \n\n  <xsl:template name=\"get_html_element_type\">\n    <xsl:param name=\"e\"\/>\n    <xsl:choose>     \n      <xsl:when test=\"$e='a' or $e='b' or $e='del' or $e='em' or $e='i' or $e='ins' or $e='mark' or $e='span' or $e='strike' or $e='strong' or $e='sub' or $e='sup' or $e='u'\">\n        <xsl:text>inline<\/xsl:text>\n      <\/xsl:when>\n      <xsl:otherwise>\n        <xsl:text>block<\/xsl:text>\n      <\/xsl:otherwise>\n    <\/xsl:choose>\n  <\/xsl:template>\n\n<\/xsl:stylesheet>\n<\/code><\/pre>\n\n\n\n
And here\u2019s the same thing in PHP:<\/p>\n\n\n\n
<?php\n\nfunction xhtmlToPlaintext($html)\n{\n    \/\/ Add line breaks after block elements\n    $html = preg_replace_callback('\/(<.*?>)\/', 'replaceCallback', $html);\n\n    \/\/ Remove HTML tags with strip_tags()\n    $plaintext = strip_tags($html);\n    \n    \/\/ Replace multiple spaces with a single space\n    $plaintext = trim(preg_replace('\/ +\/s', ' ', $plaintext));\n    \n    \/\/ Decode HTML entities like &quot;\n    return html_entity_decode($plaintext, ENT_QUOTES, 'UTF-8');\n}\n\nfunction replaceCallback($matches)\n{\n    $inline_elements =\n    [\n        'a', 'b', 'del', 'em', 'i', 'ins', 'mark', 'span', \n        'strike', 'strong', 'sub', 'sup', 'u' \n    ];\n\n    \/\/ <span class=\"x\"> => span\n    \n    $replace = [ '<' => '', '>' => '', '\/' => '' ];\n    $parts = explode(' ', trim(strtr($matches[ 1 ], $replace)));\n    \n    $e = $parts[ 0 ];\n\n    $result = $matches[ 0 ];\n\n    if (! in_array($e, $inline_elements))\n    {\n        $result .= \"\\n\";\n    }\n\n    return $result;\n}\n\n$html = file_get_contents('xhtml-in.html');\n\necho \"Wrong:\\n\";\nvar_dump(trim(strip_tags($html)));\n\necho \"Correct:\\n\";\nvar_dump(xhtmlToPlaintext($html));\n<\/code><\/pre>\n\n\n\n
Suggestions for improvement are welcome!<\/p>\n","protected":false},"excerpt":{"rendered":"
Turning HTML into (unformatted) plain text seems simple at first: PHP has <a href=“http:\/\/www.php.net\/strip_tags“>strip_tags()<\/a>, XSLT has <a href=“https:\/\/www.w3.org\/TR\/xslt#value-of“>xsl:value-of<\/a>. In practice, though, you\u2019ll frequently find that words are glued together which should have whitespace between them. Take this example \u2013 extra weirdly-formatted to get the point across: If you select and copy this text in the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_share_on_mastodon":"0"},"categories":[1],"tags":[],"class_list":["post-1830","post","type-post","status-publish","format-standard","hentry","category-weblog"],"share_on_mastodon":{"url":"","error":""},"_links":{"self":[{"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/posts\/1830","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/comments?post=1830"}],"version-history":[{"count":1,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/posts\/1830\/revisions"}],"predecessor-version":[{"id":1912,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/posts\/1830\/revisions\/1912"}],"wp:attachment":[{"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/media?parent=1830"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/categories?post=1830"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.strehle.de\/tim\/wp-json\/wp\/v2\/tags?post=1830"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}