how to properly remove banned words?

Question

I have a line from which I want to remove all words beginning with the symbol @, I do not fully understand how to do it expressively. It is clear that you could write something like this:

1. Split the string into words

2. Use the list filter to weed out unnecessary words

But I guess I don't understand how to break lines, because in addition to the space, there are such characters as \t and \n, besides, I will lose them and can not restore the original text.

An example of what I want to get:

original string:

haha lala\n@delete_me all-ok

expected result:

haha lala\nall-ok

Answer 1

Another way to look at the problem is that we want to delete strings of non-spaces that begin with an at sign @, as well as any following spaces. We don’t want to treat line breaks or other characters specially at all. That can be expressed with a simple recursive function using span / break and dropWhile:

censor :: String -> String

censor "" = ""

censor text0 = spaces ++ nonspaces ++ censor rest
  where

    (spaces, text1) = span isSpace text0

    (word, text2) = break isSpace text1

    (nonspaces, rest)

      | banned word
      = ("", trim text2)

      | otherwise
      = (word, text2)

banned :: String -> Bool
banned ('@' : _) = True
banned _ = False

trim :: String -> String
trim = dropWhile isSpace  

Consider an example:

1. censor " send @beans money to sam@example.com"
2. span returns " " and "send @beans…"
3. break returns "send" and " @beans…"
4. banned returns false for "send", so we will keep it
5. We recursively call censor " @beans money…"
6. span returns " " and "@beans money…"
7. break returns "@beans" and " money…"
8. Now banned returns true for "@beans", so we drop it and trim the rest
9. We recursively call censor "money…"
10. We keep all the remaining substrings, including sam@example.com, since it is not banned
11. Finally, we reach the end of the string and censor "" returns ""

The end result is this expression:

"  " ++ "send" ++ " " ++ "" ++ "money" ++ " " ++ "to" ++ " " ++ "sam@example.com" ++ ""

Notice that we use a series of updates to the input string resulting in a series of variables text0, text1, text2, rest for the intermediate states. Consider how you could express this pattern using State instead.

Answer 2

You might want to use Data.List.Split.split with Data.List.Split.oneOf.

It returns split words including separators, so you can rebuild text with them.

split (oneOf "xyz") "aazbxyzcxd" == ["aa","z","b","x","","y","","z","c","x","d"]