m68k: Simple loop compiles into boundless recursion with -O2

[Fredrik Noring <noring@nocrew.org>]

Hi,

Compiler used is GCC m68k-elf version 10.2.0. A variant of the classic
memset

	void *memset2(void *s, int c, unsigned int n)
	{
		char *b = s;
		for (unsigned int i = 0; i < n; i++)
			b[i] = c;
		return s;
	}

compiles into boundless recursion with O2 optimisation and the m68k-elf
target. This will, of course, exhaust the stack and crash badly.

The commands

	m68k-elf-gcc -O2 -march=68000 -c -o memset2.o memset2.c
	m68k-elf-objdump -d memset2.o

produce

	00000000 <memset2>:
	   0:	2f02           	movel %d2,%sp@-
	   2:	242f 0008      	movel %sp@(8),%d2
	   6:	202f 0010      	movel %sp@(16),%d0
	   a:	6718           	beqs 24 <memset2+0x24>
	   c:	2f00           	movel %d0,%sp@-
	   e:	102f 0013      	moveb %sp@(19),%d0
	  12:	4880           	extw %d0
	  14:	3040           	moveaw %d0,%a0
	  16:	2f08           	movel %a0,%sp@-
	  18:	2f02           	movel %d2,%sp@-
	  1a:	4eb9 0000 0000 	jsr 0 <memset2>   /* <<<--- recursion */
	  20:	4fef 000c      	lea %sp@(12),%sp
	  24:	2002           	movel %d2,%d0
	  26:	241f           	movel %sp@+,%d2
	  28:	4e75           	rts

O1 optimisation is more reasonable, as it instead produces

	00000000 <memset2>:
	   0:	2f02           	movel %d2,%sp@-
	   2:	202f 0008      	movel %sp@(8),%d0
	   6:	242f 000c      	movel %sp@(12),%d2
	   a:	4aaf 0010      	tstl %sp@(16)
	   e:	670e           	beqs 1e <memset2+0x1e>
	  10:	2040           	moveal %d0,%a0
	  12:	222f 0010      	movel %sp@(16),%d1
	  16:	d280           	addl %d0,%d1
	  18:	10c2           	moveb %d2,%a0@+
	  1a:	b288           	cmpl %a0,%d1
	  1c:	66fa           	bnes 18 <memset2+0x18>
	  1e:	241f           	movel %sp@+,%d2
	  20:	4e75           	rts

The machine code with O2 looks like a plain compiler bug to me.

What to do?

Fredrik