Re: Adding binutils to the GNU Toolchain buildbot on sourceware

[Jan Beulich <jbeulich@suse.com>]

On 26.04.2022 14:27, Nick Clifton wrote:
>> For gas/testsuite/gas/i386/rept I did already suggest [1] to simply purge
>> the test as unreliable. I also put under question the purpose that it was
>> originally added for.
>>
>> [1] https://sourceware.org/pipermail/binutils/2022-April/120339.html
> 
> Hmm, I seem to have missed this one.  In fact the entire patch series.  Sorry.
> The series looks good to me, so please go ahead and apply it in its entirety

No problem. With my new powers I had committed it already.

> As for disabling the rept because it is so memory expensive: I think that we
> used to.have an environment variable called something like RUN_EXPENSIVE_TESTS
> that had to be set before certain tests were run.  Checking the sources though
> I cannot find it, so maybe I am imagining things.  I still might be a good idea
> though.  If we use it consistently in the binutils testsuites for the big tests
> then users will probably appreciate the facility.

But a test which is run by almost nobody is more likely to break. Also I
think "expensive" has multiple dimensions (memory and time at least), and
depending on the system one may want to run (or suppress) one but not the
other kind. For the test in question, it is both memory and cycles hungry,
so there the distinction may not matter.

But I'd like to raise the question again: Is what the test was added for
actually a useful thing to test, at the risk of the test failing simply
because there's too little memory available? Iirc the problem was non-
graceful error handling. But the test does not check that the error in
question now is handled gracefully; it expects that there be no error.

Jan