* ubsan: undefined shift in frag_align_code
@ 2022-06-02 7:38 Alan Modra
2022-06-03 13:23 ` Alan Modra
0 siblings, 1 reply; 2+ messages in thread
From: Alan Modra @ 2022-06-02 7:38 UTC (permalink / raw)
To: binutils
* frags.c (MAX_MEM_FOR_RS_ALIGN_CODE): Avoid signed integer
overflow.
diff --git a/gas/frags.c b/gas/frags.c
index f5afe1a49b5..8e88496f37c 100644
--- a/gas/frags.c
+++ b/gas/frags.c
@@ -367,7 +367,7 @@ frag_align_pattern (int alignment, const char *fill_pattern,
# ifndef HANDLE_ALIGN
# define MAX_MEM_FOR_RS_ALIGN_CODE 1
# else
-# define MAX_MEM_FOR_RS_ALIGN_CODE ((1 << alignment) - 1)
+# define MAX_MEM_FOR_RS_ALIGN_CODE (((size_t) 1 << alignment) - 1)
# endif
#endif
--
Alan Modra
Australia Development Lab, IBM
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: ubsan: undefined shift in frag_align_code
2022-06-02 7:38 ubsan: undefined shift in frag_align_code Alan Modra
@ 2022-06-03 13:23 ` Alan Modra
0 siblings, 0 replies; 2+ messages in thread
From: Alan Modra @ 2022-06-03 13:23 UTC (permalink / raw)
To: binutils
This one needs the same fix too.
* config/tc-i386.h (MAX_MEM_FOR_RS_ALIGN_CODE): Avoid signed
integer overflow.
diff --git a/gas/config/tc-i386.h b/gas/config/tc-i386.h
index d7c6b33c778..817539ea560 100644
--- a/gas/config/tc-i386.h
+++ b/gas/config/tc-i386.h
@@ -207,7 +207,8 @@ if ((n) \
goto around; \
}
-#define MAX_MEM_FOR_RS_ALIGN_CODE (alignment ? ((1 << alignment) - 1) : 1)
+#define MAX_MEM_FOR_RS_ALIGN_CODE \
+ (alignment ? ((size_t) 1 << alignment) - 1 : (size_t) 1)
extern void i386_cons_align (int);
#define md_cons_align(nbytes) i386_cons_align (nbytes)
--
Alan Modra
Australia Development Lab, IBM
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2022-06-03 13:23 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-02 7:38 ubsan: undefined shift in frag_align_code Alan Modra
2022-06-03 13:23 ` Alan Modra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).