public inbox for gcc@gcc.gnu.org
 help / color / mirror / Atom feed
* [GSoC] A bunch of questions about sm-malloc behavior, and proposition of a GSoC subject.
@ 2023-03-29 22:50 Benjamin Priour
  2023-03-30  0:04 ` David Malcolm
  0 siblings, 1 reply; 2+ messages in thread
From: Benjamin Priour @ 2023-03-29 22:50 UTC (permalink / raw)
  To: David Malcolm; +Cc: gcc

[-- Attachment #1: Type: text/plain, Size: 4404 bytes --]

Hi David,

I've been playing around with sm-malloc on C++ samples.
I've noticed double delete don't go under -Wanalyzer-double-free but within
-Wanalyzer-use-after-free scope.

With the reproducer ...
> struct A {};
>
> int main() {
>     A* a = new A();
>     delete a;
>     delete a;
>     return 0;
> }

... the analyzer diagnoses:
build/gcc/xg++ -Bbuild/gcc -S -fanalyzer
-Wanalyzer-mismatching-deallocation -Wanalyzer-use-after-free
-Wanalyzer-double-free -Wanalyzer-malloc-leak  double_delete_test.cpp
double_delete_test.cpp: In function ‘int main()’:
double_delete_test.cpp:7:1: warning: use after ‘delete’ of ‘a’ [CWE-416]
[-Wanalyzer-use-after-free]
    7 | }
      | ^
  ‘int main()’: events 1-8
    |
    |    3 |     A* a = new A();
    |      |                  ^
    |      |                  |
    |      |                  (1) allocated here
    |      |                  (2) assuming ‘operator new(8)’ is non-NULL
    |    4 |     delete a;
    |      |     ~~~~~~~~
    |      |     |      |
    |      |     |      (4) ...to here
    |      |     |      (5) deleted here
    |      |     (3) following ‘true’ branch...
    |    5 |     delete a;
    |      |     ~~~~~~~~
    |      |     |      |
    |      |     |      (7) ...to here
    |      |     (6) following ‘true’ branch...
    |    6 |     return 0;
    |    7 | }
    |      | ~
    |      | |
    |      | (8) use after ‘delete’ of ‘a’; deleted at (5)
    |
double_delete_test.cpp:3:18: warning: dereference of possibly-NULL
‘operator new(8)’ [CWE-690] [-Wanalyzer-possible-null-dereference]
    3 |     A* a = new A();
      |                  ^
  ‘int main()’: events 1-2
    |
    |    3 |     A* a = new A();
    |      |                  ^
    |      |                  |
    |      |                  (1) this call could return NULL
    |      |                  (2) ‘operator new(8)’ could be NULL:
unchecked value from (1)
    |


Debugging read out two things:
- During second call of 'on_deallocator_call' on delete, the state would be
'stop' instead of the expected 'freed'
- The call to set_next_state transitions malloc instead of delete from
'nonnull' to 'freed'.
I'm still trying to come up with why these two behaviors happens.

By the way, in the first call to 'on_deallocator_call' on delete, the state
is set to 'nonnull',
which conforms to C++ behavior for new. However, a
-Wanalyzer-possible-null-dereference is emitted at the expression 'new'.
I haven't yet figured out why, but I'm looking into it.


Another observation was in the distinction between delete and free in the
case of mixing them.
With 'a' being allocated with malloc:
> A* a = (A*) malloc(sizeof(A));
> free(a);
> delete a; // -Wanalyzer-use-after-free, OK, might expect warning for
double free though ?

but with allocation through new and inversion of free/delete
> A* a = new A();
> delete a;
> free(a); // No diagnostics at all from the analyzer, got
-Wmismatched-new-delete from the front-end though.

I believe this might come from a similar issue as above, but I still have
to investigate on that front.

I just noticed another unexpected behavior. Let's consider
> struct A {int x; int y;};
> void* foo() { A* a = (A*) __builtin_malloc(sizeof(A)); return a; }
> int main() {
>     A* a2 = (A*) __builtin_malloc(sizeof(A));
>     foo();
>     return 0;
> }

Then a -Wanalyzer-malloc-leak is correctly ensued for allocation in foo(),
but none is emitted for the leak in main(), although I checked the exploded
graph it is correctly marked as unchecked(free).

Should I file those on bugzilla ?


Also I have taken interest in PR106388 - Support for use-after-move in
-fanalyzer -.
The prerequisite PR106386 - Reuse libstdc++ assertions - would also be of
great help in extending the support of -Wanalyzer-out-of-bounds,
as we could detect out-of-bounds on vectors through
__glibcxx_requires_subscript used in the definition of operator[].

I already thought about a few ideas to implement that, but I'll develop
them more and try to come up with some proof of concept before sending them
to you.
Hopefully by tomorrow I'll update on this, I'll preferably hop to bed now
haha.
Do you think this could make a suitable GSoC subject ?


Best,
Benjamin.

^ permalink raw reply	[flat|nested] 2+ messages in thread
end of thread, other threads:[~2023-03-30  0:04 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-29 22:50 [GSoC] A bunch of questions about sm-malloc behavior, and proposition of a GSoC subject Benjamin Priour
2023-03-30  0:04 ` David Malcolm

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).