Reputation: 170
Google text-to-speech API has a quota per request - 5000 characters. So we have an HTML page that needs to be splitted into parts, not more than 5000 characters with the preservation of words and html tags.
Here is an example of the input HTML (simplified for example):
<div id="myID">
<span class="test">
Links in PHP are a means of accessing the contents of one variable under different names.
</span>
<span>
They are not like pointers in C and are not aliases for the symbol table.
</span>
</div>
<p>
In PHP, the name of a variable and its contents are different things, so one content can have different names.
</p>
Suppose we divide the text (just the text) into fragments of 70 characters with preserving the markup and without breaking the words, as a result we get:
Part 1
<div id="myID">
<span class="test">
Links in PHP are a means of accessing the contents of one variable under
</span>
</div>
Part 2
<div id="myID">
<span class="test">
different names.
</span>
<span>
They are not like pointers in C and are not aliases for the symbol table.
</span>
</div>
Part 3
<p>
In PHP, the name of a variable and its contents are different things, so one
</p>
Part 4
<p>
content can have different names.
</p>
There is a great solution that has long been walking around the internet:
/**
* Truncates text.
*
* Cuts a string to the length of $length and replaces the last characters
* with the ending if the text is longer than length.
*
* @param string $text String to truncate.
* @param integer $length Length of returned string, including ellipsis.
* @param string $ending Ending to be appended to the trimmed string.
* @param boolean $exact If true, $text will not be cut mid-word
* @param boolean $considerHtml If true, HTML tags would be handled correctly
* @return string Trimmed string.
*/
function str_truncate($text, $length = 100, $ending = '...', $exact = true, $considerHtml = false) {
if ($considerHtml) {
// if the plain text is shorter than the maximum length, return the whole text
if (strlen(preg_replace('/<.*?>/', '', $text)) <= $length) {
return $text;
}
// splits all html-tags to scanable lines
preg_match_all('/(<.+?>)?([^<>]*)/s', $text, $lines, PREG_SET_ORDER);
$total_length = strlen($ending);
$open_tags = array();
$truncate = '';
foreach ($lines as $line_matchings) {
// if there is any html-tag in this line, handle it and add it (uncounted) to the output
if (!empty($line_matchings[1])) {
// if it's an "empty element" with or without xhtml-conform closing slash (f.e. <br/>)
if (preg_match('/^<(\s*.+?\/\s*|\s*(img|br|input|hr|area|base|basefont|col|frame|isindex|link|meta|param)(\s.+?)?)>$/is', $line_matchings[1])) {
// do nothing
// if tag is a closing tag (f.e. </b>)
} else if (preg_match('/^<\s*\/([^\s]+?)\s*>$/s', $line_matchings[1], $tag_matchings)) {
// delete tag from $open_tags list
$pos = array_search($tag_matchings[1], $open_tags);
if ($pos !== false) {
unset($open_tags[$pos]);
}
// if tag is an opening tag (f.e. <b>)
} else if (preg_match('/^<\s*([^\s>!]+).*?>$/s', $line_matchings[1], $tag_matchings)) {
// add tag to the beginning of $open_tags list
array_unshift($open_tags, strtolower($tag_matchings[1]));
}
// add html-tag to $truncate'd text
$truncate .= $line_matchings[1];
}
// calculate the length of the plain text part of the line; handle entities as one character
$content_length = strlen(preg_replace('/&[0-9a-z]{2,8};|&#[0-9]{1,7};|&#x[0-9a-f]{1,6};/i', ' ', $line_matchings[2]));
if ($total_length+$content_length> $length) {
// the number of characters which are left
$left = $length - $total_length;
$entities_length = 0;
// search for html entities
if (preg_match_all('/&[0-9a-z]{2,8};|&#[0-9]{1,7};|&#x[0-9a-f]{1,6};/i', $line_matchings[2], $entities, PREG_OFFSET_CAPTURE)) {
// calculate the real length of all entities in the legal range
foreach ($entities[0] as $entity) {
if ($entity[1]+1-$entities_length <= $left) {
$left--;
$entities_length += strlen($entity[0]);
} else {
// no more characters left
break;
}
}
}
$truncate .= substr($line_matchings[2], 0, $left+$entities_length);
// maximum lenght is reached, so get off the loop
break;
} else {
$truncate .= $line_matchings[2];
$total_length += $content_length;
}
// if the maximum length is reached, get off the loop
if($total_length>= $length) {
break;
}
}
} else {
if (strlen($text) <= $length)
return $text;
else
$truncate = substr($text, 0, $length - strlen($ending));
}
// if the words shouldn't be cut in the middle...
if ($exact) {
// ...search the last occurance of a space...
$spacepos = strrpos($truncate, ' ');
if (isset($spacepos)) {
// ...and cut the text in this position
$truncate = substr($truncate, 0, $spacepos);
}
}
// add the defined ending to the text
$truncate .= $ending;
if($considerHtml) {
// close all unclosed html-tags
foreach ($open_tags as $tag)
$truncate .= '</' . $tag . '>';
}
return $truncate;
}
Its only drawback is that we get only the first part of the Html. If it was possible to get not only the first part but also the end - it would be ideal.
I would appreciate any clue where to dig.
Upvotes: 4
Views: 996
Reputation: 1694
I have faced with the same task and I have a solution for splitting the HTML into 2 parts taking into account the word boundaries and the HTML tags. You can extend it into multiple columns with a bit more effort, like calculating the size of a chunk and getting the next chunk till you reach the end end of the string.
// splitHtmlTextIntoTwoEqualColumnsTrait.php
<?php
/**
* TCPDF doesn't support to have a 2 columns text where the length of the text is limited and the height of the 2 columns are equal.
*
* This trait calculates the middle of the text, split it into 2 parts and returns with them
* Keeps the word boundaries and takes care of the HTML tags too! There is no broken HTML tag after the split.
*/
trait splitHtmlTextIntoTwoEqualColumnsTrait
{
protected function splitHtmlTextIntoTwoEqualColumns(string $htmlText): array
{
// removes unnecessary characters and HTML tags
$htmlText = str_replace("\xc2\xa0", ' ', $htmlText);
$htmlText = html_entity_decode($htmlText);
$pureText = $this->getPureText($htmlText);
// calculates the length of the text
$fullLength = strlen($pureText);
$halfLength = ceil($fullLength / 2);
$words = explode(' ', $pureText);
// finds the word which is in the middle of the text
$middleWordPosition = $this->getPositionOfMiddleWord($words, $halfLength);
// iterates through the HTML and split the text into 2 parts when it reaches the middle word.
$columns = $this->splitHtmlStringInto2Strings($htmlText, $middleWordPosition);
return $this->closeUnclosedHtmlTags($columns, $halfLength * 2);
}
private function getPureText(string $htmlText): string
{
$pureText = strip_tags($htmlText);
$pureText = preg_replace('/[\x00-\x1F\x7F]/', '', $pureText);
return str_replace(["\r\n", "\r", "\n"], ['', '', ''], $pureText);
}
/**
* finds the word which is in the middle of the text
*/
private function getPositionOfMiddleWord(array $words, int $halfLength): int
{
$wordPosition = 0;
$stringLength = 0;
for ($p = 0; $p < count($words); $p++) {
$stringLength += mb_strlen($words[$p], 'UTF-8') + 1;
if ($stringLength > $halfLength) {
$wordPosition = $p;
break;
}
}
return $wordPosition;
}
/**
* iterates through the HTML and split the text into 2 parts when it reaches the middle word.
*/
private function splitHtmlStringInto2Strings(string $htmlText, int $wordPosition): array
{
$columns = [
1 => '',
2 => '',
];
$columnId = 1;
$wordCounter = 0;
$inHtmlTag = false;
for ($s = 0; $s <= strlen($htmlText) - 1; $s++) {
if ($inHtmlTag === false && $htmlText[$s] === '<') {
$inHtmlTag = true;
}
if ($inHtmlTag === true) {
$columns[$columnId] .= $htmlText[$s];
if ($htmlText[$s] === '>') {
$inHtmlTag = false;
}
} else {
if ($htmlText[$s] === ' ') {
$wordCounter++;
}
if ($wordCounter > $wordPosition && $columnId < 2) {
$columnId++;
$wordCounter = 0;
}
$columns[$columnId] .= $htmlText[$s];
}
}
return array_map('trim', $columns);
}
private function closeUnclosedHtmlTags(array $columns, int $maxLength): array
{
$column1 = $columns[1];
$unclosedTags = $this->getUnclosedHtmlTags($columns[1], $maxLength);
foreach (array_reverse($unclosedTags) as $tag) {
$column1 .= '</' . $tag . '>';
}
$column2 = '';
foreach ($unclosedTags as $tag) {
$column2 .= '<' . $tag . '>';
}
$column2 .= $columns[2];
return [$column1, $column2];
}
/**
* https://stackoverflow.com/a/26175271/5356216
*/
private function getUnclosedHtmlTags(string $html, int $maxLength = 250): array
{
$htmlLength = strlen($html);
$unclosed = [];
$counter = 0;
$i = 0;
while (($i < $htmlLength) && ($counter < $maxLength)) {
if ($html[$i] == "<") {
$currentTag = "";
$i++;
if (($i < $htmlLength) && ($html[$i] != "/")) {
while (($i < $htmlLength) && ($html[$i] != ">") && ($html[$i] != "/")) {
$currentTag .= $html[$i];
$i++;
}
if ($html[$i] == "/") {
do {
$i++;
} while (($i < $htmlLength) && ($html[$i] != ">"));
} else {
$currentTag = explode(" ", $currentTag);
$unclosed[] = $currentTag[0];
}
} elseif ($html[$i] == "/") {
array_pop($unclosed);
do {
$i++;
} while (($i < $htmlLength) && ($html[$i] != ">"));
}
} else {
$counter++;
}
$i++;
}
return $unclosed;
}
}
how to use it:
// yourClass.php
<?php
declare(strict_types=1);
class yourClass
{
use splitHtmlTextIntoTwoEqualColumnsTrait;
public function do()
{
// your logic
$htmlString = '';
[$column1, $column2] = $this->splitHtmlTextIntoTwoEqualColumns($htmlString);
}
}
Upvotes: 2