More relaxed -Wvla option (allowing VLA pointers)

More relaxed -Wvla option (allowing VLA pointers)

[Elias Åström]

Hi!

Some projects (most notably the linux-kernel) choose to use -Wvla to
warn on
usage of variable length array.  The primary reason is usually that it
can
cause stack overflows if the programmer isn't careful, which introduces
a
security vulnerability.

I think an overlooked advantage of VLA's is the ability to create
indexable
multi-dimensional arrays with run-time sizes.  These arrays doesn't have
to be
allocated on the stack, they can also be pointers to head-allocated
memory.
VLA's can also be useful to document preconditions on array-sizes in
function-prototypes.

I would like to suggest an option to warn against creating VLA objects
on the
stack, but not for VLA-pointers.  When that is enabled the following
scenarios
will not emit warnings:

Scenario 1. Using decayed pointers from VLA's in function parameters

    int sum(int n, int a[n]);

Scenario 2. Using pointers and manipulating pointers to VLA objects

    float (*matrix1)[n][m] = malloc(sizeof *matrix);
    float (*matrix2)[m] = malloc(n * sizeof *matrix);

While creating a VLA object on stack would still emit a warning:

    int f(int n) {
        int a[n]; // warning for creating a VLA object on the stack
    }

Also note that scenario 1 can occur when including header files from a
library
that uses VLA's in function prototypes to annotate dynamic array sizes. 
I
personally would like to see more libraries use C99 VLA's to annotate
array
sizes in function prototypes, but it would be annoying if that makes
them
unusable with "-Wvla".

I think this could either be an entirely new option, maybe called
"-Wvla-stack".  Or an option to use in combination with -Wvla, maybe
called
"-Wno-vla-pointer" which has the same behavior when used like
"-Wvla -Wno-vla-pointer".

Just thought this would preserve many of the advantages of VLA's for
software
projects hesitant to enable VLAs completely to avoid stack smashing. 
You may
implement it if you want, but I would be glad if you would consider it.

Regards,
Elias Åström