other/6312: libiberty and vasprintf() sigsegv's on some platforms

[gp@qnx.com]

>Number:         6312
>Category:       other
>Synopsis:       libiberty and vasprintf() sigsegv's on some platforms
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    unassigned
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Apr 15 12:36:12 PDT 2002
>Closed-Date:
>Last-Modified:
>Originator:     Graeme Peterson
>Release:        unknown-1.0
>Organization:
>Environment:
I observed this with the head branch of gdb hosted on i386-qnx6 targetting ppc-qnx6, but it is a generic issue in libiberty.
>Description:
The exact bug is in vasprintf() in libiberty/vasprintf.c.

Consider the following example. All looks good, but on the PPC (ppc-qnx6) it won't print out "2". See below for an explanation.

==================
#include <stdio.h>
#include <stdarg.h>

void
handle_foo(char *fmt, va_list *pva) {
    printf("%d\n", va_arg(*pva, int));
}

void
vfoo(char *fmt, va_list va) {
    handle_foo(fmt, &va);
}

void
foo(char *fmt, ...) {
    va_list va;

    va_start(va, fmt);
    vfoo(fmt, va);
    va_end(va);
}

int
main() {
    foo("", 2);
    return 0;
}
===============

The problem is that sometimes the va_list type is an array (as on the PPC) and sometimes not (X86, etc). The C standard says that prototypes such as vfoo() have the array type silently coerced to be a pointer to a base type. This makes things work when you pass an array object to the function. An array-typed expression
is converted to a pointer to the first element when used in an rvalued context, the coercion in the function makes everybody happy. The problem comes when you then pass the address of the va_list parameter to another function. It's expecting a pointer to the array, but what it _really_ gets is a pointer to a pointer (because of the original conversion).

Any use of the va_list in the second function won't get the right data.

Here's the example modified so that it works in all cases:

=======================
#include <stdio.h>
#include <stdarg.h>

void
handle_foo(char *fmt, va_list *pva) {
    printf("%d\n", va_arg(*pva, int));
}

void
vfoo(char *fmt, va_list va) {
    va_list temp;

    va_copy(temp, va);
    handle_foo(fmt, &temp);
}

void
foo(char *fmt, ...) {
    va_list va;

    va_start(va, fmt);
    vfoo(fmt, va);
    va_end(va);
}

int
main() {
    foo("", 2);
    return 0;
}
=============

The use of the va_copy() 'undoes' the coercion that happens in the parameter list, so that the handle_foo() function now gets the proper data.
>How-To-Repeat:
See libiberty/vasprintf.c vasprintf().
>Fix:
Change vasprintf() from:

int
vasprintf (result, format, args)
     char **result;
     const char *format;
#if defined (_BSD_VA_LIST_) && defined (__FreeBSD__)
     _BSD_VA_LIST_ args;
#else
     va_list args;
#endif
{
  return int_vasprintf (result, format, &args);
}

to:

int
vasprintf (result, format, args)
     char **result;
     const char *format;
#if defined (_BSD_VA_LIST_) && defined (__FreeBSD__)
     _BSD_VA_LIST_ args;
#else
     va_list args;
#endif
{
  /* Handle both array and non-array va_list types. */
  va_list temp;
  va_copy(temp, args);
  return int_vasprintf (result, format, &temp);
}
>Release-Note:
>Audit-Trail:
>Unformatted: