AW: m68k: Simple loop compiles into boundless recursion with -O2

["Stefan Franke" <s.franke@bebbosoft.de>]

> -----Ursprüngliche Nachricht-----
> Von: Gcc-help <gcc-help-bounces@gcc.gnu.org> Im Auftrag von Fredrik
> Noring
> Gesendet: Mittwoch, 13. Januar 2021 17:02
> An: gcc-help@gcc.gnu.org
> Betreff: m68k: Simple loop compiles into boundless recursion with -O2
> 
> Hi,
> 
> Compiler used is GCC m68k-elf version 10.2.0. A variant of the classic memset
> 
> 	void *memset2(void *s, int c, unsigned int n)
> 	{
> 		char *b = s;
> 		for (unsigned int i = 0; i < n; i++)
> 			b[i] = c;
> 		return s;
> 	}
> 
> compiles into boundless recursion with O2 optimisation and the m68k-elf
> target. This will, of course, exhaust the stack and crash badly.
> 
> The commands
> 
> 	m68k-elf-gcc -O2 -march=68000 -c -o memset2.o memset2.c
> 	m68k-elf-objdump -d memset2.o
> 
> produce
> 
> 	00000000 <memset2>:
> 	   0:	2f02           	movel %d2,%sp@-
> 	   2:	242f 0008      	movel %sp@(8),%d2
> 	   6:	202f 0010      	movel %sp@(16),%d0
> 	   a:	6718           	beqs 24 <memset2+0x24>
> 	   c:	2f00           	movel %d0,%sp@-
> 	   e:	102f 0013      	moveb %sp@(19),%d0
> 	  12:	4880           	extw %d0
> 	  14:	3040           	moveaw %d0,%a0
> 	  16:	2f08           	movel %a0,%sp@-
> 	  18:	2f02           	movel %d2,%sp@-
> 	  1a:	4eb9 0000 0000 	jsr 0 <memset2>   /* <<<--- recursion
> */
> 	  20:	4fef 000c      	lea %sp@(12),%sp
> 	  24:	2002           	movel %d2,%d0
> 	  26:	241f           	movel %sp@+,%d2
> 	  28:	4e75           	rts
> 
> O1 optimisation is more reasonable, as it instead produces
> 
> 	00000000 <memset2>:
> 	   0:	2f02           	movel %d2,%sp@-
> 	   2:	202f 0008      	movel %sp@(8),%d0
> 	   6:	242f 000c      	movel %sp@(12),%d2
> 	   a:	4aaf 0010      	tstl %sp@(16)
> 	   e:	670e           	beqs 1e <memset2+0x1e>
> 	  10:	2040           	moveal %d0,%a0
> 	  12:	222f 0010      	movel %sp@(16),%d1
> 	  16:	d280           	addl %d0,%d1
> 	  18:	10c2           	moveb %d2,%a0@+
> 	  1a:	b288           	cmpl %a0,%d1
> 	  1c:	66fa           	bnes 18 <memset2+0x18>
> 	  1e:	241f           	movel %sp@+,%d2
> 	  20:	4e75           	rts
> 
> The machine code with O2 looks like a plain compiler bug to me.
> 
> What to do?
> 
> Fredrik

I guess that the label here

> 1a:	4eb9 0000 0000 	jsr 0 <memset2>   /* <<<--- recursion

is printed incorrectly and instead a call to memset is done.

objdump is known to use the first label it finds for an offset, and also often uses the wrong section... You should look at the assembly or run objdump with `-dr` to also print relocations.

You may also use the compiler explorer here https://franke.ms/cex/  to view the results for some different m68k gcc versions. (note that the % for registers is omitted and labels have an underscore).

Here is the asm output of gcc-10.2.0-elf:

_memset2:
        move.l d2,-(sp)
        move.l 8(sp),d2
        move.l 16(sp),d0
        jeq .L4
        move.l d0,-(sp)
        move.b 19(sp),d0
        ext.w d0
        move.w d0,a0
        move.l a0,-(sp)
        move.l d2,-(sp)
        jsr _memset
        lea (12,sp),sp
.L4:
        move.l d2,d0
        move.l (sp)+,d2
        rts


/cheers

Stefan