UnicodeDecodeError for uncaught exception

[carlosfunk]

I had an uncaught IOError exception in my Django project and the notification made it to New Relic but nothing appeared in Sentry. I've got the json from the logs and can't see any weird characters in it. (I would have to sanitize it a bit before I could post it anywhere public.)

This is the traceback I found in the logs:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/raven/transport/threaded.py", line 162, in send_sync
    super(ThreadedHTTPTransport, self).send(data, headers)
  File "/usr/local/lib/python2.7/dist-packages/raven/transport/http.py", line 47, in send
    ca_certs=self.ca_certs,
  File "/usr/local/lib/python2.7/dist-packages/raven/utils/http.py", line 62, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/local/lib/python2.7/dist-packages/newrelic-2.54.0.41/newrelic/hooks/external_urllib2.py", line 15, in _nr_wrapper_opener_director_open_
    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/dist-packages/raven/utils/http.py", line 46, in https_open
    return self.do_open(ValidHTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
    self.endheaders(body)
  File "/usr/local/lib/python2.7/dist-packages/newrelic-2.54.0.41/newrelic/hooks/external_httplib.py", line 35, in httplib_endheaders_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 833, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)

[Jyggafey]

After upgrading to raven==5.6.0 i can reproduce this with python manage.py raven test
a76101f appears to be related at a glance.

Traceback (most recent call last):
  File "/srv/oscar/oscarenv/local/lib/python2.7/site-packages/raven/transport/threaded.py", line 162, in send_sync
    super(ThreadedHTTPTransport, self).send(data, headers)
  File "/srv/oscar/oscarenv/local/lib/python2.7/site-packages/raven/transport/http.py", line 47, in send
    ca_certs=self.ca_certs,
  File "/srv/oscar/oscarenv/local/lib/python2.7/site-packages/raven/utils/http.py", line 62, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1222, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 979, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1013, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 975, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 833, in _send_output
    msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128)
[u'This is a test message generated using ``raven test``']

[Jyggafey]

possibly fixed here: #651

[illagrenan]

I have similar problem. If I run python manage.py raven test, everything seems OK:

Client configuration:
  base_url       : http://80.xxx.xxx.xx:9000
  project        : 13
  public_key     : 4ec95b99ff1b4b6xxxxxxxxxx
  secret_key     : xxx7c456097645xxxxxxxxx

Sending a test message... Event ID was '352d379750a94554972xxxxxxxx'

But in logfile I can see:

Sentry responded with an error: 'ascii' codec can't decode byte 0x9c in position 1: ordinal not in range(128) (url: xxx/store/)

This happened after upgrading to raven==5.6.0. I also tried raven==5.5.0 but same problem appeared. With raven>5.0.0,<5.5.0 (it is resolved to 5.4.4) everything works fine.

[dcramer]

I believe this is related to the gzip changes, but need to investigate further.

[lionel-panhaleux]

I had the exact same issue last week, fixed it by making sure the dsn setting for raven is of type str in python 2, not unicode.

[nourspace]

@lionel-panhaleux thank you!
I use from __future__ import unicode_literals in top of all my files and as you suggested I just wrapped the dsn value in my settings.py with str() it helped. I hate that fact that I don't know why it did though.

[lionel-panhaleux]

on recent versions, Raven is sending binary data (deflated dict, not base64-encoded anymore). If your URL is unicode, you're mixing strings with binary and get the error.

[artemiy-rodionov]

@lionel-panhaleux Thanks a lot. Changing to str in python2 worked for me

[hheimbuerger]

Same issue here (the DSN was a unicode string due to from __future__ import unicode_literals).

Can we have this fixed? Is there any reason the DSN can't be a unicode string?

[dcramer]

I'm not entirely sure how to reproduce this (unless we fixed it in master and I've forgotten about it).

I attempted to just Client(u'https://public:secret@example.com/1') but that seemed to work fine.

[hheimbuerger]

@dcramer For what it's worth, I had this on 5.7.2 by setting the SENTRY_DSN in the Django settings.py to a unicode string. Not sure whether that is entirely the same, or maybe there's some subtle difference to setting it directly on the Client constructor.

I haven't tested with master yet.

[dcramer]

@hheimbuerger what version of Python? Assuming 2.7.

[hheimbuerger]

@dcramer Yep, that was on 2.7.10.

[dcramer]

Alright the issue is an invalid unicode value sent to urllib. Will fix in 5.8