Commit messages containing unicode characters causes submission failure

[samgalen]

A student in intro to security ran into this issue (names & traceback modified for privacy) when submitting trying to submit an assignment.

SUBMISSION FOR ASSIGNMENT project1 (Project1)
---------------------------------------------
 
This is an INDIVIDUAL submission for Student
 
You have already submitted assignment project1
 
You submitted the following commit on <Time>:
 
      Commit: <Commit>
        Date: <Time>
     Message: submitting project1
      Author: <Student @ Student's machine>
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
IF YOU CONTINUE, THE ABOVE SUBMISSION FOR project1 (Project1) WILL BE CANCELLED.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
If you continue, your submission will instead point to the latest commit in your repository:
 
      Commit: <Different Commit>
        Date: <Different Time>
ERROR: Unexpected exception
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/chisubmit/cli/common.py", line 92, in new_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/chisubmit/cli/common.py", line 61, in new_func
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/chisubmit/cli/common.py", line 82, in new_func
    return ctx.invoke(f, ctx.obj["course_obj"], *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/chisubmit/cli/student/assignment.py", line 359, in student_assignment_submit
    print_commit(commit)
  File "/usr/local/lib/python2.7/dist-packages/chisubmit/cli/student/assignment.py", line 206, in print_commit
    print "     Message: %s" % commit.message
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u2026' in position 91: ordinal not in range(256)

Looking at the commit message for , the message does include u'\u2026', or the triple dot ellipsis.

I have not been able find whether or not git commit messages should be able to contain all unicode characters, so it is possible this is not a bug because not all unicode characters should be supported for commit messages. In either case, it is low priority issue as we can easily instruct students to avoid including non-latin-1 characters in their commit messages.

[borjasotomayor]

chisubmit will (hopefully) be migrated to Python 3 before the next school year (we attempted this a year ago, but there were still a couple dependencies that were available only in Python 2; it seems like all those dependencies are finally available for Python 3). At that point, I'm hoping all unicode issues will go away on their own. So, we'll probably not fix this issue specifically until then (and, as you point out, you can just ask students to not use non-ascii characters in their commit messages). However, I'll leave the issue open until we do the Python 3 migration and can verify it's been resolved.

[borjasotomayor]

chisubmit was migrated to Python 3 a few months ago, so hopefully this issue (and other unicode-related issues) have been dealt with automatically. I will reopen this issue if this reoccurs.

[erijohnt]

Issue persists on chisubmit running on CSIL machines. Commits with the ellipsis character '\u2026' will cause an unexpected exception and fail to submit.

I think git will automatically truncate any messages that are more than 50 characters long and add an ellipsis to make the header fit, then put the entirety of the commit message in the body. Therefore anyone who writes extremely eloquent commits like "turns out you'll get a null ptr exception if you try to copy stuff into an array initialized as null. fascinating!" and tries to submit it will hit this issue. Successfully submitted the assignment after making a new commit with the comment "touch".

$ chisubmit student assignment submit hw0
SUBMISSION FOR ASSIGNMENT hw0 (HW0)
-----------------------------------

This is an INDIVIDUAL submission for <me>

The latest commit in your repository is the following:

      Commit: <commit>
        Date: <time>
ERROR: Unexpected exception
Traceback (most recent call last):
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/chisubmit/cli/common.py", line 93, in new_func
    return f(*args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/chisubmit/cli/common.py", line 62, in new_func
    return f(*args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/chisubmit/cli/common.py", line 83, in new_func
    return ctx.invoke(f, ctx.obj["course_obj"], *args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/chisubmit/cli/student/assignment.py", line 361, in student_assignment_submit
    print_commit(commit)
  File "/usr/local/chisubmit_client/lib/python3.5/site-packages/chisubmit/cli/student/assignment.py", line 208, in print_commit
    print("     Message: %s" % commit.message)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2026' in position 94: ordinal not in range(256)

$ chisubmit --version
chisubmit, version 2.0.0

[borjasotomayor]

Could you provide a link to the commit on mit.cs.uchicago.edu? (I'd like to experiment further on this)

[erijohnt]

Sure thing.
https://mit.cs.uchicago.edu/cmsc23500-win-19/erijohnt/commit/6155d773e02486b13923e5b7c427c2f980a5c1a0