CODEWITHSUNDEEP
php
John
John

Reputation: 4944

Removing punctuation from words that occur in a string

The code below works great. It creates a list of words occurring in the string $commentstring, ranked by number of occurrences. It omits numbers and anything in the $blacklist.

It treats words ending in punctuation as unique. So american, american., and american, are all considered different. How can I make words the same whether or not they end with punctuation?

$words = explode(" ", $commentstring);      
$result = array();
arsort($words);

foreach($words as $word) {
    if(!is_numeric($word)){
        $result[$word]++;
        arsort($result);
    }
}

echo "<table>";

$blacklist = array($submission, 'DESPITE', 'FARE', 'DECENT', 'AMAZING', 'WOULD', 'DISLIKE', 'HATE', 'OKAY', 'JUST', 'NOTHING', 'CURRENTLY', 'BASICALLY', 'BIT', 'COME', 'WANT', 'TOO', 'HERE', 'EATING', 'EAT', 'WAS', 'TRIED', 'TRY', 'MAKES', 'HAS', 'EVEN', 'THINK', 'BETTER', 'YET', 'MORE', 'LOVE', 'WHILE', 'WHERE', 'WRONG', 'FIND', 'EVER', 'RIGHT', 'BEST', 'HAVE', 'WE', 'WAY', 'GREAT', 'NICE', 'HOW', 'RESTAURANTS', 'RESTAURANT', 'EXCELLENT', 'FORGET', 'THEY', 'REALLY', 'MISS', 'VERY', 'LOOKING', 'YOU\'LL', 'CAN\'T', 'WON\'T', 'PLACE', 'ABOUT', 'FOR', 'MOST', 'GOOD', 'CAN', 'GET', 'THING', 'DON\'T', 'BY', 'YOUR', 'BE', 'YOU', 'BRING', 'THAT\'S', 'LITTLE', 'OTHER', 'MANAGES', 'ATE', 'ATE', 'EAT', 'SO', 'SOMEHOW', 'MAKE', 'ALL', 'UP', 'THEM', 'AS', 'THEM', 'YOU\'RE', 'WILL', 'ONLY', 'IF', 'GO', 'DO', 'I\'VE', 'HAD', 'TO', 'SOME', 'FOOD', 'THIS','DOES', 'NOT', 'IT.', 'IT,', 'SEEM', 'END', 'THERE\'S', 'WHETHER', 'DOUBT', 'WHAT', 'WHICH', 'RECOMMEND', 'THE', 'IS', 'A', 'IT\'S', 'OUT', 'JAN', 'IT', 'IT', 'IT', 'LIKE', 'THAN', 'WITH', 'SEEMS', 'WHICH', 'THAT', 'SAY', 'AT', 'ON', 'AN', 'BUT', 'APART', 'STILL', 'ARE', 'OR', 'TEST', 'IN', 'IT', 'AND', 'SET', 'TO', 'NO', 'OF', '', 'A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', 'AA', 'BB', 'CC', 'DD', 'EE', 'FF', 'GG', 'HH', 'II', 'JJ', 'KK', 'LL', 'MM', 'NN', 'OO', 'PP', 'QQ', 'RR', 'SS', 'TT', 'UU', 'VV', 'WW', 'XX', 'YY', 'ZZ');

foreach($result as $word => $count1)
{
    if (in_array($word, $blacklist)) continue;

    echo '<tr>';    
    echo '<td>';
    echo "$word";
    echo '</td>';

    echo '<td>';
    echo "$count1 ";
    echo '</td>';

    echo '</tr>';
}

echo "</table>";

Upvotes: 0

Views: 446

Answers (3)

Dan
Dan

Reputation: 1888

An alternative to freefaller's solution would be just removing all punctuations you would like to get rid of before you tokenize the string.

$text = str_replace(array('.',',',':'), '', $commentstring);
$words = explode(' ', $text);

Upvotes: 1

Mr. Llama
Mr. Llama

Reputation: 20919

For simplicity, it'd be easiest to remove any leading and trailing punctuation from a word, then compare it to the list of "blacklisted" words.

After you explode the words, insert the following:

foreach ($words as &$w) { $w = preg_replace('/\W/', '', $w); }

It iterates across all entries in $words and modifies them by removing any non-word characters. A "word" character is A-Za-z0-9 and underscore (if I remember correctly).

Also, for efficiency, move arsort($result); to just outside it's foreach loop. It guarantees it'll run once, which is better than possibly running once for each word you process.

Upvotes: 0

freefaller
freefaller

Reputation: 19963

Have you considered using regular expressions? Look at the preg functions (such as preg_match_all) and a pattern along the lines of \b[\w]*\b

Upvotes: 0

Related Questions