Git 'hooks/post_receive.py': UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte

Git 'hooks/post_receive.py': UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte

[Thomas Schwinge]

Hi!

The following is not an actual problem for me or GCC/Rust; just for your
information:

I've just pushed to GCC devel/rust/master branch
Git commits cc23831ec66..74913f718b0, which 'hooks/post_receive.py' threw
up upon:

    $ git push upstream github/Rust-GCC/gccrs/master:devel/rust/master
    Enumerating objects: 326, done.
    Counting objects: 100% (326/326), done.
    Delta compression using up to 12 threads
    Compressing objects: 100% (143/143), done.
    Writing objects: 100% (224/224), 32.72 KiB | 6.54 MiB/s, done.
    Total 224 (delta 191), reused 106 (delta 79)
    remote: Resolving deltas: 100% (191/191), completed with 86 local objects.
    remote: Traceback (most recent call last):
    remote:   File "hooks/post_receive.py", line 118, in <module>
    remote:     post_receive(refs_data, args.submitter_email)
    remote:   File "hooks/post_receive.py", line 65, in post_receive
    remote:     submitter_email)
    remote:   File "hooks/post_receive.py", line 47, in post_receive_one
    remote:     update.send_email_notifications()
    remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 189, in send_email_notifications
    remote:     self.__email_new_commits()
    remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 1031, in __email_new_commits
    remote:     commit, self.get_standard_commit_email(commit))
    remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 1011, in __send_commit_email
    remote:     default_diff=email.diff)
    remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 946, in __maybe_get_email_custom_contents
    remote:     hook_input=json.dumps(hooks_data),
    remote:   File "/usr/lib64/python2.7/json/__init__.py", line 244, in dumps
    remote:     return _default_encoder.encode(obj)
    remote:   File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
    remote:     chunks = self.iterencode(o, _one_shot=True)
    remote:   File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
    remote:     return _iterencode(o, 0)
    remote: UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte
    To git+ssh://gcc.gnu.org/git/gcc.git
       cc23831ec66..74913f718b0  github/Rust-GCC/gccrs/master -> devel/rust/master

I very much suppose this "byte 0xff" is
commit 92d6dc8443acbd273333cab411bfd68eaca56a89
"Implemented UTF-8 checking for include_str!()" adding
'gcc/testsuite/rust/compile/invalid_utf8':

    $ od -x gcc/testsuite/rust/compile/invalid_utf8
    0000000 0aff
    0000002

Due to that (I suppose), Git commit emails to <gcc-cvs@gcc.gnu.org> have
not been sent.


Grüße
 Thomas
-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

Re: Git 'hooks/post_receive.py': UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte

[Martin Liška]

On 2/16/23 13:26, Thomas Schwinge wrote:
> Hi!
> 
> The following is not an actual problem for me or GCC/Rust; just for your
> information:

Hello.

Thanks for letting me know.

> 
> I've just pushed to GCC devel/rust/master branch
> Git commits cc23831ec66..74913f718b0, which 'hooks/post_receive.py' threw
> up upon:
> 
>     $ git push upstream github/Rust-GCC/gccrs/master:devel/rust/master
>     Enumerating objects: 326, done.
>     Counting objects: 100% (326/326), done.
>     Delta compression using up to 12 threads
>     Compressing objects: 100% (143/143), done.
>     Writing objects: 100% (224/224), 32.72 KiB | 6.54 MiB/s, done.
>     Total 224 (delta 191), reused 106 (delta 79)
>     remote: Resolving deltas: 100% (191/191), completed with 86 local objects.
>     remote: Traceback (most recent call last):
>     remote:   File "hooks/post_receive.py", line 118, in <module>
>     remote:     post_receive(refs_data, args.submitter_email)
>     remote:   File "hooks/post_receive.py", line 65, in post_receive
>     remote:     submitter_email)
>     remote:   File "hooks/post_receive.py", line 47, in post_receive_one
>     remote:     update.send_email_notifications()
>     remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 189, in send_email_notifications
>     remote:     self.__email_new_commits()
>     remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 1031, in __email_new_commits
>     remote:     commit, self.get_standard_commit_email(commit))
>     remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 1011, in __send_commit_email
>     remote:     default_diff=email.diff)
>     remote:   File "/sourceware1/projects/src-home/git-hooks/hooks/updates/__init__.py", line 946, in __maybe_get_email_custom_contents
>     remote:     hook_input=json.dumps(hooks_data),
>     remote:   File "/usr/lib64/python2.7/json/__init__.py", line 244, in dumps
>     remote:     return _default_encoder.encode(obj)
>     remote:   File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
>     remote:     chunks = self.iterencode(o, _one_shot=True)
>     remote:   File "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
>     remote:     return _iterencode(o, 0)
>     remote: UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte
>     To git+ssh://gcc.gnu.org/git/gcc.git
>        cc23831ec66..74913f718b0  github/Rust-GCC/gccrs/master -> devel/rust/master

Well, the https://github.com/AdaCore/git-hooks were ported to Python 3 some time ago and I thought we've been
using the updated version. But it seems we're still on python2.7 :((

Joseph, can we update it, please?

> 
> I very much suppose this "byte 0xff" is
> commit 92d6dc8443acbd273333cab411bfd68eaca56a89
> "Implemented UTF-8 checking for include_str!()" adding
> 'gcc/testsuite/rust/compile/invalid_utf8':
> 
>     $ od -x gcc/testsuite/rust/compile/invalid_utf8
>     0000000 0aff
>     0000002
> 
> Due to that (I suppose), Git commit emails to <gcc-cvs@gcc.gnu.org> have
> not been sent.

Dunno if that will be fixed or not, but Python 3 should make our life easier.

Martin

> 
> 
> Grüße
>  Thomas
> -----------------
> Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

Re: Git 'hooks/post_receive.py': UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 2766: invalid start byte

[Joseph Myers]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Thu, 16 Feb 2023, Martin Liška wrote:

> Well, the https://github.com/AdaCore/git-hooks were ported to Python 3 
> some time ago and I thought we've been using the updated version. But it 
> seems we're still on python2.7 :((
> 
> Joseph, can we update it, please?

If someone wishes to update them I'd encourage doing so (this particular 
error might be https://github.com/AdaCore/git-hooks/issues/19).  The 
update would need testing first (using the gcc-reposurgeon-8 repository 
which we've used for such test purposes) and there would need to be a 
check for any changes to configuration semantics or defaults since the 
version we're using that indicate a need to update 
refs/meta/config:project.config.

-- 
Joseph S. Myers
joseph@codesourcery.com