Skip to content
This repository has been archived by the owner on Oct 5, 2020. It is now read-only.

fix timezone info UnicodeDecodeError #227

Closed
wants to merge 2 commits into from

Conversation

wangyanbn
Copy link

ref cloudify-cosmo/cloudify-agent#112:
In agent script_runner.tasks.run ,if an agent locale is not of ascii codeset (such as "Chinese (Simplified)_People's Republic of China", '936'), the timezone info is not utf-8 encoded.This will cause exception in logging (amqp_client.publish_message in cloudify/logs.py) , which will cause exception in local logging with json.dumps.
This method can fix it: when celery app starts, set default locale, and when getting timezone info in logging ,use the correct codeset from locale.

Here is the Traceback:
[2016-05-09 10:08:15,540: ERROR/MainProcess] Task script_runner.tasks.run[84982083-9ef8-4879-add3-da53d623330a] raised unexpected: UnicodeDecodeError('utf8', '2016-05-09 10:08:15.481\xd6\xd0\xb9\xfa\xb1\xea\xd7\xbc\xca\xb1\xbc\xe4', 23, 24, 'invalid continuation byte')
Traceback (most recent call last):
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(args, kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\celery\app\trace.py", line 438, in protected_call
return self.run(args, *kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\decorators.py", line 126, in wrapper
ctx.task_name, exc_info=True)
File "C:\Python27.11\Lib\logging__init
_.py", line 1185, in error
self.log(ERROR, msg, args, *kwargs)
File "C:\Python27.11\Lib\logging__init
.py", line 1278, in log
self.handle(record)
File "C:\Python27.11\Lib\logging__init_.py", line 1288, in handle
self.callHandlers(record)
File "C:\Python27.11\Lib\logging__init__.py", line 1328, in callHandlers
hdlr.handle(record)
File "C:\Python27.11\Lib\logging__init__.py", line 751, in handle
self.emit(record)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 119, in emit
self.out_func(log)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 342, in amqp_log_out
publish_message(log, 'log', logging.getLogger('cloudify_logs'))
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 322, in wrapper
func(client, args, *kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 379, in publish_message
'message={1}, log={2}]'
File "C:\Python27.11\Lib\json__init.py", line 244, in dumps
return _default_encoder.encode(obj)
File "C:\Python27.11\Lib\json\encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Python27.11\Lib\json\encoder.py", line 270, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd6 in position 23: invalid continuation byte

wangyanbn added 2 commits May 9, 2016 14:57
ref cloudify-cosmo/cloudify-agent#112:
In agent script_runner.tasks.run ,if an agent locale is not of ascii codeset (such as "Chinese (Simplified)_People's Republic of China", '936'), the timezone info is not utf-8 encoded.This will cause exception in logging (amqp_client.publish_message in cloudify/logs.py) , which will cause exception in local logging with json.dumps.
This method can fix it: when celery app starts, set default locale, and when getting timezone info in logging ,use the correct codeset from locale.

Here is the Traceback:
[2016-05-09 10:08:15,540: ERROR/MainProcess] Task script_runner.tasks.run[84982083-9ef8-4879-add3-da53d623330a] raised unexpected: UnicodeDecodeError('utf8', '2016-05-09 10:08:15.481\xd6\xd0\xb9\xfa\xb1\xea\xd7\xbc\xca\xb1\xbc\xe4', 23, 24, 'invalid continuation byte')
Traceback (most recent call last):
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\celery\app\trace.py", line 240, in trace_task
R = retval = fun(args, *kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\celery\app\trace.py", line 438, in protected_call
return self.run(args, *kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\decorators.py", line 126, in wrapper
ctx.task_name, exc_info=True)
File "C:\Python27.11\Lib\logging__init__.py", line 1185, in error
self.log(ERROR, msg, args, **kwargs)
File "C:\Python27.11\Lib\logging__init_.py", line 1278, in log
self.handle(record)
File "C:\Python27.11\Lib\logging__init_.py", line 1288, in handle
self.callHandlers(record)
File "C:\Python27.11\Lib\logging__init__.py", line 1328, in callHandlers
hdlr.handle(record)
File "C:\Python27.11\Lib\logging__init__.py", line 751, in handle
self.emit(record)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 119, in emit
self.out_func(log)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 342, in amqp_log_out
publish_message(log, 'log', logging.getLogger('cloudify_logs'))
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 322, in wrapper
func(client, args, *kwargs)
File "c:\users\sys\test_windows_host_2d6ed\env\lib\site-packages\cloudify\logs.py", line 379, in publish_message
'message={1}, log={2}]'
File "C:\Python27.11\Lib\json__init.py", line 244, in dumps
return _default_encoder.encode(obj)
File "C:\Python27.11\Lib\json\encoder.py", line 207, in encode
chunks = self.iterencode(o, _one_shot=True)
File "C:\Python27.11\Lib\json\encoder.py", line 270, in iterencode
return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd6 in position 23: invalid continuation byte
E265 block comment should start with '# '
E231 missing whitespace after ','
@geokala
Copy link
Contributor

geokala commented Feb 1, 2019

Closed: As this PR is more than six months old without having been merged. Please reopen (and rebase if necessary) if it is still relevant.

@geokala geokala closed this Feb 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants