Re: [PATCH] Fail if dwz fails.

Re: [PATCH] Fail if dwz fails.

[Mark Wielaard]

Hi Martin,

On Thu, 2021-05-13 at 10:01 +0200, Martin Liška wrote:
> PING^1

Sorry I missed this earlier.

> On 4/22/21 1:11 PM, Martin Liška wrote:
> > Right now, dwz return code is 0 or a small integer value (<= 3)
> > for situations like:
> > 
> > - dwz: Too few files for multifile optimization
> > - dwz: Multi-file optimization not allowed for different pointer sizes or endianity
> > 
> > These are not fatal errors and the script should continue. On the other
> > hand abort or segfault error values should cause failure of the script.
> > 
> > Let's reserve values 1-16 for a recoverable dwz exit codes.

With "recoverable" you mean that dwz did do compression, but could not
deliver optimal compression?

It would be good to have an ack from the dwz developers that 1 to 16
are "recoverable" errors (I added dwz@sourceware.org to CC).

Would it be possible to tweak the find-debuginfo.sh script to avoid
them? e.g. Could we see how many arguments we have so that we only use
-m when there are 1+ debug files? And/Or detect debug files using
different endianess and ptr sizes so they are processed in different
batches?

Ideally an error code from dwz means something went terribly wrong and
we abort find-debuginfo.sh. IMHO.

Cheers,

Mark

Re: [PATCH] Fail if dwz fails.

[Martin Liška]

On 5/13/21 4:07 PM, Mark Wielaard wrote:
> Hi Martin,
> 
> On Thu, 2021-05-13 at 10:01 +0200, Martin Liška wrote:
>> PING^1
> 
> Sorry I missed this earlier.
> 
>> On 4/22/21 1:11 PM, Martin Liška wrote:
>>> Right now, dwz return code is 0 or a small integer value (<= 3)
>>> for situations like:
>>>
>>> - dwz: Too few files for multifile optimization
>>> - dwz: Multi-file optimization not allowed for different pointer sizes or endianity
>>>
>>> These are not fatal errors and the script should continue. On the other
>>> hand abort or segfault error values should cause failure of the script.
>>>
>>> Let's reserve values 1-16 for a recoverable dwz exit codes.
> 
> With "recoverable" you mean that dwz did do compression, but could not
> deliver optimal compression?

No, I speak about cases, where compression cannot be done
(different endianess, ptr sizes, not beneficial, any other reason).

> 
> It would be good to have an ack from the dwz developers that 1 to 16
> are "recoverable" errors (I added dwz@sourceware.org to CC).

Yes. That's why I CCed Tom.

> 
> Would it be possible to tweak the find-debuginfo.sh script to avoid
> them? e.g. Could we see how many arguments we have so that we only use
> -m when there are 1+ debug files? And/Or detect debug files using
> different endianess and ptr sizes so they are processed in different
> batches?

No, I would leave it to dwz. There may be other reasons.

> 
> Ideally an error code from dwz means something went terribly wrong and
> we abort find-debuginfo.sh. IMHO.

I would allow the mentioned "soft" error codes.

Martin

> 
> Cheers,
> 
> Mark
> 

Re: [PATCH] Fail if dwz fails.

[Mark Wielaard]

Hi Martin,

On Fri, May 14, 2021 at 12:46:15PM +0200, Martin Liška wrote:
> On 5/13/21 4:07 PM, Mark Wielaard wrote:
> > > On 4/22/21 1:11 PM, Martin Liška wrote:
> > > > Right now, dwz return code is 0 or a small integer value (<= 3)
> > > > for situations like:
> > > > 
> > > > - dwz: Too few files for multifile optimization
> > > > - dwz: Multi-file optimization not allowed for different pointer sizes or endianity
> > > > 
> > > > These are not fatal errors and the script should continue. On the other
> > > > hand abort or segfault error values should cause failure of the script.
> > > > 
> > > > Let's reserve values 1-16 for a recoverable dwz exit codes.
> > 
> > With "recoverable" you mean that dwz did do compression, but could not
> > deliver optimal compression?
> 
> No, I speak about cases, where compression cannot be done
> (different endianess, ptr sizes, not beneficial, any other reason).

I think we are talking about somewhat the same thing. In those cases
dwz does something. The result is just not optimal.

> > It would be good to have an ack from the dwz developers that 1 to 16
> > are "recoverable" errors (I added dwz@sourceware.org to CC).
> 
> Yes. That's why I CCed Tom.
> 
> > 
> > Would it be possible to tweak the find-debuginfo.sh script to avoid
> > them? e.g. Could we see how many arguments we have so that we only use
> > -m when there are 1+ debug files? And/Or detect debug files using
> > different endianess and ptr sizes so they are processed in different
> > batches?
> 
> No, I would leave it to dwz. There may be other reasons.

But for the reasons we do know about we could improve
find-debuginfo.sh to do the right thing (not use -m if there is only
one file, sort files by endianess/ptrsize). Ideally you call
find-debuginfo.sh and it does the right thing even if you only have
one ELF file or a mix of ELF files.

> > Ideally an error code from dwz means something went terribly wrong and
> > we abort find-debuginfo.sh. IMHO.
> 
> I would allow the mentioned "soft" error codes.

Couldn't optimize is an "soft" error indeed. I don't know about the
others.

Cheers,

Mark

Re: [PATCH] Fail if dwz fails.

[Tom de Vries]

On 5/16/21 10:14 PM, Mark Wielaard wrote:
> Hi Martin,
> 
> On Fri, May 14, 2021 at 12:46:15PM +0200, Martin Liška wrote:
>> On 5/13/21 4:07 PM, Mark Wielaard wrote:
>>>> On 4/22/21 1:11 PM, Martin Liška wrote:
>>>>> Right now, dwz return code is 0 or a small integer value (<= 3)
>>>>> for situations like:
>>>>>
>>>>> - dwz: Too few files for multifile optimization
>>>>> - dwz: Multi-file optimization not allowed for different pointer sizes or endianity
>>>>>
>>>>> These are not fatal errors and the script should continue. On the other
>>>>> hand abort or segfault error values should cause failure of the script.
>>>>>
>>>>> Let's reserve values 1-16 for a recoverable dwz exit codes.
>>>
>>> With "recoverable" you mean that dwz did do compression, but could not
>>> deliver optimal compression?
>>
>> No, I speak about cases, where compression cannot be done
>> (different endianess, ptr sizes, not beneficial, any other reason).
> 
> I think we are talking about somewhat the same thing. In those cases
> dwz does something. The result is just not optimal.
> 
>>> It would be good to have an ack from the dwz developers that 1 to 16
>>> are "recoverable" errors (I added dwz@sourceware.org to CC).
>>
>> Yes. That's why I CCed Tom.
>>
>>>

AFAIK, the exit codes are either 0 or 1 (excluding abort/assert).

FWIW, also invalid input dwarf results in exit code 1.

>>> Would it be possible to tweak the find-debuginfo.sh script to avoid
>>> them? e.g. Could we see how many arguments we have so that we only use
>>> -m when there are 1+ debug files? And/Or detect debug files using
>>> different endianess and ptr sizes so they are processed in different
>>> batches?
>>
>> No, I would leave it to dwz. There may be other reasons.
> 
> But for the reasons we do know about we could improve
> find-debuginfo.sh to do the right thing (not use -m if there is only
> one file, sort files by endianess/ptrsize). Ideally you call
> find-debuginfo.sh and it does the right thing even if you only have
> one ELF file or a mix of ELF files.
> 
>>> Ideally an error code from dwz means something went terribly wrong and
>>> we abort find-debuginfo.sh. IMHO.
>>
>> I would allow the mentioned "soft" error codes.
> 
> Couldn't optimize is an "soft" error indeed. I don't know about the
> others.

Couldn't optimize is not an error:
...
$ ./dwz hello ; echo $?
0
$ ./dwz hello ; echo $?
./dwz: hello: DWARF compression not beneficial - old size 3372 new size 3372
0
$ ...

OTOH, failing to create a file specified with -o is:
...
$ ./dwz hello -o hello.1; echo $?
0
$ ./dwz hello.1 -o hello.2; echo $?
./dwz: hello.1: DWARF compression not beneficial - old size 3372 new
size 3372
1
$
...

FWIW, I don't have a strong opinion on this patch.

As dwz developer, more bug reports means a better dwz, so this is good
for me.

I just wonder whether this is a good choice for package developers.
Isn't a dwz crash for them just a case of "no debuginfo compression"?

Thanks,
- Tom