CODEWITHSUNDEEP
pythonhtmljinja2web.py
Paxwell
Paxwell

Reputation: 758

Rendering newlines in user-submitted content (Python web app)

I have a web.py app that takes input from a textarea and inputs it to a database. I can get the information from the database and post it to the page but the NEWLINES are gone. How do I preserver newlines when posting back into HTML? The data does have \r\n in it but that isn't rendered as NEWLINES in HTML. Any thoughts? Here is a small example:

(2, u'Title', u'content here...hey\r\nthis\r\nhas\r\nbreaks in it....?', 
    datetime.datetime(2012, 7, 5, 21, 5, 14, 354516))

That is my return from the data base. I need the \r\n to represent a <br /> and if there is two a <p> would be awesome. Any direction would be very much appreciated.

Also is there a library for this? I have heard of markdown and mark up but I can find no examples of how to post html data from python strings?

Upvotes: 4

Views: 9742

Answers (4)

Michel M&#252;ller
Michel M&#252;ller

Reputation: 5705

Newlines not being rendered as newlines is an intended behavior in html. What you want is to insert either <br> or parse your input into text paragraphs and wrap those with <p>..</p>

I would do a mixture of both: Use <br> for single newlines and <p>..</p> for double newlines. You could do this parsing either at the time you save into your database or at the time you get it out.

Edit: I've made the following graphic for you. There are dozens of ways to look at a parser. Personally I'd like to think of them as a form of state machine. In order to implement it, you should read the input string in a streaming way, for example using http://docs.python.org/library/stringio.html.

enter image description here Edit2: Changed description "pushdown automaton" to "state machine". Pushdown automaton would be correct but not precise and it doesn't match the graph well - I mixed the two up.

Edit3: Here's some sudo code on how to implement a state machine parser in code, using a while loop, a switch case and if statements for the state transitions.

state = 'plainState'
streamer = get_stream_reader_from_input()
buffer = ''
while true {
  nextchar = streamer.readchar()
  if (nextchar == null) { //EOS
   print(buffer)
   exit
  }
  switch (state) {
    case('plainState') {
      if (nextchar == '\n') {
        state = 'singleBreakState' 
      }
      else if (nextchar == '\r') {
        state = 'CRState'
      }
      else {
        buffer += nextchar
      }
    }
    case('singleBreakState') {
      if (nextchar == '\n') {
        state = 'doubleBreakState' 
      }
      else if (nextchar == '\r') {
        state = 'CRState2'
      }
      else {
        state = 'plainState'
        buffer += '<br>' + nextchar
      }
    }
    //...
  }
}

Upvotes: 6

Burhan Khalid
Burhan Khalid

Reputation: 174662

Two main ways to do this. The easiest one is to wrap the output in <pre></pre> which will format it as entered.

Or, you can replace newlies with <br /> (and not with <p>) as the characters represent a line break and not a paragraph.

For the second option, this is one approach:

>>> s
'hello\nthere\r\nthis\n\ris a test'
>>> r = '<br />'
>>> s.replace('\r\n',r).replace('\n\r',r).replace('\r',r).replace('\n',r)
'hello<br />there<br />this<br />is a test'
>>>

Or the third option - which is to use one of the many text entry libraries/formats and render the content through them (as mentioned by others - like markdown).

However, that would be overkill if all you want to do is a simple replace.

Upvotes: 12

Helgi
Helgi

Reputation: 5436

You need some basic understanding of what's going on before coming up with a solution. If you don't understand the problem, the “throw in another library” approach either won't work at all (better), or will backfire soon (worse).

@MichelMüller is correct in stating that \ns in the HTML source are not rendered as such in the browser. See this tutorial (caveat, HTML 2.0 described) for a more detailed explanation of this behaviour. Now, to put a line break in HTML, you use <br>; to put a new paragraph, <p>.

You can do a lot of things to achieve this, but what's the problem you're solving? What is this user-submitted content? Who submits it? Two aspects to think about are:

  1. Formatting. Is it a comment on a public site, or a post prepared by the website staff, or UGC on a site like Stack Overflow?
  2. Security. Is it posted by a stranger, or by a user having full trust, or setting in-between?

Possible solutions:

  • The most direct approach is to run text.replace('\r\n', '<br>') before outputting it to the template formatter. In won't work if you don't put a { text | safe } in the template, because Jinja should not escape <br>s you generated. However, the text itself should not have full trust, so you have to escape < and & (at least) before you replace the newlines.

  • Take a look at MarkupSafe for a less ad-hoc approach of dealing with HTML escapes. It is employed by Jinja, by the way.

  • For formatting of unstructured content (i.e., user-submitted comments a la YouTube), take a look at the PottyMouth library.

  • If your content is more prepared (posts on a blogging platform or Stack Overflow-like site), try Markdown, as suggested by @BernhardKircher. It has some learning curve, so it works best if the users are willing to invest some time writing posts. Remember to configure the parser correctly, because core Markdown does not escape HTML.

  • For staff-prepared content, you can use Markdown or something more sophisticated. It really depends on the staff's background. Here, unescaped HTML might be a blessing, not a curse.

Upvotes: 3

Bernhard Kircher
Bernhard Kircher

Reputation: 4182

Instead of using using the inserted text which is not html and therefore will not be displayed as html (\r\n is not a "p" tag etc), you could use a formatting language like markdown (as you mentioned).

Otherwise you will need to replace/parse the entered text manually (which I think is not a good idea, because languages like markdown have been invented for this).

There are some good python libraries to convert markdown (the data you store in database) to html, like python-markdown2.

It's really easy to use, see the examples from the python-markdown link:

>>> import markdown2
>>> markdown2.markdown("*boo!*")  # or use `html = markdown_path(PATH)`
u'<p><em>boo!</em></p>\n'

>>> markdowner = Markdown()
>>> markdowner.convert("*boo!*")
u'<p><em>boo!</em></p>\n'
>>> markdowner.convert("**boom!**")
u'<p><strong>boom!</strong></p>\n'

This forces you to enter the content using markdown syntax (or whatever format you use). To make this easier, you could use an wysiwyg-editor that creates the markdown for you (like the editor here on Stackoverflow). I think Stackoverflow uses wmd but there are a lot of other markdown wysiwyg editors.

Example for wmd:

<html>
    <head>
        <title>WMD Example using jquery</title>
        <link rel="stylesheet" type="text/css" href="wmd.css"/>

        <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"></script>
        <script type="text/javascript" src="jquery.wmd.min.js"></script>
    </head>
    <body>
        <h1>WMD Example using jquery</h1>
        <div>
            <textarea id="notes"/>
        </div>

        <script type="text/javascript">
            $().ready(function() {
               $("#notes").wmd(); 
            });
        </script>
    </body>
</html>

Upvotes: 2

Related Questions