CODEWITHSUNDEEP
pythongoogle-app-enginecharacter-encoding
David S.
David S.

Reputation: 11200

Strange GAE Python encoding issue

I am trying to get tweets from Twitter and send it through GTalk using the xmpp API in GAE. I got a very strange issue.

I have successfully retrieved the data from Twitter. The error occurs when I send them using the XMPP API. According to the trace information, the error occurred in the GAE code, instead of mine own. I have tested the specific tweet with Python2.5, and it is able to process the tweet text correctly, and that tweet contains only English characters.

ERROR    2011-12-08 14:29:54,200 dev_appserver.py:2700] Exception encountered handling request
Traceback (most recent call last):
  File "/home/google_appengine/google/appengine/tools/dev_appserver.py", line 2641, in _HandleRequest
    self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
  File "/home/google_appengine/google/appengine/tools/dev_appserver.py", line 2528, in _Dispatch
    base_env_dict=env_dict)
  File "/home/google_appengine/google/appengine/tools/dev_appserver.py", line 616, in Dispatch
    base_env_dict=base_env_dict)
  File "/home/google_appengine/google/appengine/tools/dev_appserver.py", line 1592, in Dispatch
    self._module_dict)
  File "/home/google_appengine/google/appengine/tools/dev_appserver.py", line 1517, in ExecuteCGI
    logservice_stub._flush_logs_buffer()
  File "/home/google_appengine/google/appengine/api/logservice/logservice_stub.py", line 71, in _flush_logs_buffer
    logservice.logs_buffer().flush()
  File "/home/google_appengine/google/appengine/api/logservice/logservice.py", line 228, in flush
    self._lock_and_call(self._flush)
  File "/home/google_appengine/google/appengine/api/logservice/logservice.py", line 112, in _lock_and_call
    return method(*args)
  File "/home/google_appengine/google/appengine/api/logservice/logservice.py", line 260, in _flush
    apiproxy_stub_map.MakeSyncCall('logservice', 'Flush', request, response)
  File "/home/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 94, in MakeSyncCall
    return stubmap.MakeSyncCall(service, call, request, response)
  File "/home/google_appengine/google/appengine/api/apiproxy_stub_map.py", line 308, in MakeSyncCall
    rpc.CheckSuccess()
  File "/home/google_appengine/google/appengine/api/apiproxy_rpc.py", line 156, in _WaitImpl
    self.request, self.response)
  File "/home/google_appengine/google/appengine/api/apiproxy_stub.py", line 87, in MakeSyncCall
    method(request, response)
  File "/home/google_appengine/google/appengine/api/logservice/logservice_stub.py", line 309, in _Dynamic_Flush
    new_app_logs = self.put_log_lines(group.log_line_list())
  File "/home/google_appengine/google/appengine/api/logservice/logservice_stub.py", line 321, in put_log_lines
    return _run_in_namespace(self._put_log_lines, lines)
  File "/home/google_appengine/google/appengine/api/logservice/logservice_stub.py", line 93, in _run_in_namespace
    return f(*args)
  File "/home/google_appengine/google/appengine/api/logservice/logservice_stub.py", line 330, in _put_log_lines
    message=app_log.message())
  File "/home/google_appengine/google/appengine/ext/db/__init__.py", line 945, in __init__
    prop.__set__(self, value)
  File "/home/google_appengine/google/appengine/ext/db/__init__.py", line 599, in __set__
    value = self.validate(value)
  File "/home/google_appengine/google/appengine/ext/db/__init__.py", line 2696, in validate
    value = self.data_type(value)
  File "/home/google_appengine/google/appengine/api/datastore_types.py", line 1138, in __new__
    return super(Text, cls).__new__(cls, arg, encoding)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xca in position 135: ordinal not in range(128)

Upvotes: 2

Views: 538

Answers (2)

David S.
David S.

Reputation: 11200

Sorry guys...false alarm :(

The cause of the issue is that the Google logging service only accepts 'ascii' codec, and when I testing the XMPP service on local, it uses the logging service to display the message, and the message I send is in Unicode, hence the error occurs.

The solution is to use str.encode('ascii', 'ignore') while testing, or simply ignore that error.

Upvotes: 1

Drew Sears
Drew Sears

Reputation: 12838

This error means that you tried to store a multi-byte character in a 7-bit ASCII string. Pass the HTTP response body around as unicode to avoid this. If you need more specific advice, post your code.

Upvotes: 1

Related Questions