Flask URL Parameters with % Are Not Properly Handled

Question

EDIT2: I apologize for the lack of clarity. I will provide several values. The first is the URL that I call using my frontend app. The second is the value before calling urllib.unquote. The third is the value after calling urlib.unquote.

frontend:

console.log('http://localhost:8080/v1/' + encodeURIComponent(name))

backend:

def f(param=''):
    print('*', param)
    param = urllib.unquote(param)
    print('**', param)

Ex.

http://localhost:8080/v1/https%3A%2F%2Fgoogle.com
* https:%2F%2Fgoogle.com
** https://google.com

Ex2.

http://localhost:8080/v1/foo%2520bar
* foo%20bar
** foo bar

Ex3.

http://localhost:8080/v1/foo%20bar
* foo bar
** foo bar

Thank you for being patient and helping me with this. I apologize for being unclear in my original post.

EDIT: In short, if I call /v1/%2520, param is equal to " " at the end of the function, instead of "%20", at the beginning of the function it is equal to "%20" and not "%2520".

I'm currently working on a Flask App using Python 2.7.

I'm trying to create a function which can handle URL parameters.

@app.route('/v1/<param>', methods=['DELETE'])
def f(param=''):
    param = urllib.unquote(param)

On my frontend application, I call this function by encoding the param. However, if I pass "foo bar" and "foo%20bar" to the function, the param is resolved to the same value -- "foo bar", when really "foo bar" should be "foo bar" and "foo%20bar" should be "foo%20bar".

Due to this bug, I am unable to delete the "foo%20bar" entry. If I try to delete it, it will delete "foo bar", and once "foo bar" is deleted, the "foo%20bar"entry will never be deleted.

I believe this is because "%20" is not equal to "%2520" even though that is what the param is. When I print this value before calling urllib.unquote(param) it is already equal to "%20". Then, when I call urllib.unquote(param) the value is changed to " ".

I'm not really sure if this is a bug in Flask/Werkzeug, but it is causing my application to not work.

Do you have any suggestions for fixing this issue? Thanks!

Answer 1

No, Flask is usually handling percent encoding exactly right. Parameters in a URL are percent encoded, and these are decoded for you when the WSGI environment is set up. Flask then passes this on to your route when matching.

You do not need to decode the parameter value again, remove your urllib.unquote() call.

Your browser will actually encode spaces in the URL to %20 for you, even though the location bar will show a space. The location bar decodes percent-encoded components to make it possible to read international characters (so %E3%81%A9%E3%81%86%E3%82%82%E3%81%82%E3%82%8A%E3%81%8C%E3%81%A8%E3%81%86 is shown as どうもありがとう, for example).

If you are having issues with encoded slashes (/, %2F), then see issue #900, there are edge cases with Apache directives (and other WSGI servers) to consider. You would need to use a <path:param> component to match those, because the default string parameter type will not match slashes.

If I use the following test script, named routetest.py:

from flask import Flask
try:
    from urllib.parse import unquote  # PY3
except ImportError:
    from urllib import unquote  # PY2

app = Flask(__name__)

@app.route('/v1/<path:param>')  # NOTE: <path:param> is required to match /
def f(param=''):
    return (
        f"param: {param}\ndecoded param: {unquote(param)}\n",
        200,
        {'content-type': 'text/plain'}
    )

use FLASK_APP=routetest flask run to launch this script on localhost:5000, then I can't reproduce your issues:

$ curl http://localhost:5000/v1/https%3A%2F%2Fgoogle.com
param: https://google.com
decoded param: https://google.com
$ curl http://localhost:5000/v1/foo%2520bar
param: foo%20bar
decoded param: foo bar
$ curl http://localhost:5000/v1/foo%20bar
param: foo bar
decoded param: foo bar

which can only mean that you have a WSGI server that is mishandling quoting in paths.