Re: [PATCH] gcov: Add __gcov_info_to_gdca()

["Martin Liška" <mliska@suse.cz>]

On 11/20/20 11:11 AM, Sebastian Huber wrote:
> On 20/11/2020 10:49, Martin Liška wrote:
> 
>> On 11/20/20 10:25 AM, Sebastian Huber wrote:
>>> On 20/11/2020 09:37, Martin Liška wrote:
>>>
>>>> On 11/17/20 10:57 AM, Sebastian Huber wrote:
>>>>> This is a proposal to get the gcda data for a gcda info in a free-standing
>>>>> environment.  It is intended to be used with the -fprofile-info-section option.
>>>>> A crude test program which doesn't use a linker script is:
>>>>
>>>> Hello.
>>>>
>>>> I'm not pretty sure how this set up is going to work. Can you please explain me that?
>>>>
>>>> I was thinking about your needs and I can imagine various techniques how to generate
>>>> gcda files format:
>>>>
>>>> 1) embedded system can override fopen, fwrite, fseek to a functions that do a remote
>>>> write-related functions
>>> Yes, this is one option, however, the inhibit_libc disables quite a lot of libgcov functionality if Newlib is used for example.
>>
>> I see. Btw do you have available Newlib in the embedded environment? If so, what I/O functionality is provided?
> Yes, I use Newlib with the RTEMS real-time operating system. Newlib provides the standard C library I/O functions (fopen, etc.). However, having Newlib available doesn't mean that every application uses its. Applications are statically linked with the operating system and Newlib. They only use what is required. Some applications cannot use the standard C library I/O since they use a lot of infrastructure and memory. You can do a lot of things with just a couple of KiBs available.

I see.

>>
>>>>
>>>> 2) - use -fprofile-info-section
>>>>    - run an app on an embedded system and do a memory dump to a terminal/console
>>>>    - take the memory dump to a host system (with IO), run __gcov_init_from_memory_dump (...)
>>>>      and then do a normal __gcov_dump
>>>
>>> I am not sure if a plain memory dump really simplifies things. You have to get the filename separately since it is only referenced in gcov_info and not included in the structure:
>>>
>>> struct gcov_info
>>> {
>>> [...]
>>>    const char *filename;        /* output file name */
>>> [...]
>>> #ifndef IN_GCOV_TOOL
>>>    const struct gcov_fn_info *const *functions; /* pointer to pointers
>>>                                                    to function information  */
>>> [...]
>>> #endif /* !IN_GCOV_TOOL */
>>> };
>>
>> I see!
>>
>>>
>>> Also the gcov_fn_info is not embedded in the gcov_info structure. If you do a plain memory dump, then you dump also pointers and how do you deal with these pointers on the host? You would need some extra information to describe the memory dump. So, why not use the gcda format for this? It is also more compact since zero value counters are skipped. Serial lines are slow, so less data to transfer is good.
>>>
>>> /* Convert the gcov information to a gcda data stream.  The first callback is
>>>     called exactly once with the filename associated with the gcov information.
>>>     The filename may be NULL.  Afterwards, the second callback is subsequently
>>>     called with chunks (the begin and length of the chunk are passed as the
>>>     first two arguments) of the gcda data stream.  The fourth parameter is a
>>>     user-provided argument passed as the last argument to the callback
>>>     functions.  */
>>>
>>> extern void __gcov_info_to_gcda (const struct gcov_info *gi_ptr,
>>>                   void (*filename) (const char *name, void *arg),
>>>                   void (*dump) (const void *begin, unsigned size, void *arg),
>>>
>>>                   void *arg);
>>>
>>> If __gcov_info_to_gcda() is correctly implemented, then this should give you directly gcda files if you use something like this:
>>>
>>> #include <gcov.h>
>>> #include <stdio.h>
>>>
>>> extern const struct gcov_info *__gcov_info_start[];
>>> extern const struct gcov_info *__gcov_info_end[];
>>>
>>> static void
>>> filename (const char *f, void *arg)
>>> {
>>>    FILE **file = arg;
>>>    *file = fopen(f, "rb");
>>> }
>>>
>>> static void
>>> dump (const void *d, unsigned n, void *arg)
>>> {
>>>    FILE **file = arg;
>>>    fwrite(d, n, 1, *file);
>>> }
>>>
>>> static void
>>> dump_gcov_info (void)
>>> {
>>>    const struct gcov_info **info = __gcov_info_start;
>>>    const struct gcov_info **end = __gcov_info_end;
>>>
>>>    /* Obfuscate variable to prevent compiler optimizations.  */
>>>    __asm__ ("" : "+r" (end));
>>>
>>>    while (info != end)
>>>    {
>>>      FILE *file = NULL;
>>>      __gcov_info_to_gcda (*info, filename, dump, &file);
>>>      fclose(file);
>>>      ++info;
>>>    }
>>> }
>>>
>>> int
>>> main()
>>> {
>>>    dump_gcov_info();
>>>    return 0;
>>> }
>>>
>>> The callback functions give the user the full control how the data of the gcda file is encoded for the transfer to a host. No gcov internals are exposed.
>>>
>>
>> All right. Btw. how will you implement these 2 callbacks on the embedded target?
> 
> One options is to convert the gcov info to YAML:
> 
> gcov-info:
> 
> - file: filename1
> 
>    data: <... base64 encoded data from __gcov_info_to_gcda ... >
> 
> - file: filename2
> 
>    data: ...
> 
> Then send the data to the host via a serial line. On the host read the data, parse the YAML, and create the gcda files. The __gcov_info_to_gcda() needs about 408 bytes of ARM Thumb-2 code and no data. You need a polled character output function, the linker set iteration and two callbacks. So, you can easily dump the gcov information with about 1KiB of code and no data except a small stack.
> 
>> Apart from these 2 hooks, I bet you will also need gcov_position and gcov_seek functions,
>> can be seen in my sent patch.
> For what do I need them?
> 

I prefer the way with the 2 extra hooks.
Can you please prepare a patch where the newly added functions __gcov_info_to_gcda and __gcov_fn_info_to_gcda
will be used in libgcov (with the hooks equal to fopen and fwrite?

Thanks,
Martin