Re: Ada policy

[Robert Dewar <dewar@gnat.com>]

Mike Stump wrote:
> On Aug 30, 2004, at 2:21 PM, Richard Kenner wrote:
> 
>> second that nearly all of the test cases are proprietary code.
> 
> 
> Yes, and this was true of quite a few of the C/C++ testcases too...  
> but, this doesn't relieve the obligation to add a test case.
> 
> Don't start fixing a testcase until after you've trimmed the testcase 
> down, then you will find that all bugs you fix will have cut down test 
> cases.  A few bug fixes will never have testcases, that is fine.  A good 
> example is a memory smasher that randomly goes in and out of workingness 
> as the testcase is changed.  These will not usually have a testcase.  
> However the vast majority of the rest can have a minimal testcase 
> produced.  A few will need 12 hours to cut down the testcase, the bulk 
> of them should be substantially easier.

It's simply a matter of not having the resources to follow this
desirable approach.

> Sometimes there will be areas where the testing infrastructure doesn't 
> support testing, for which you'll just skip adding tests, this too is 
> fine to a degree.  In C/C++, this is done with stuff like cross module 
> or cross shared library issues...  Not perfect, but the best we can do 
> at the moment.  In time, we'll have a driver even for that.
> 
> We can describe the procedure to transform a proprietary testcase in 
> detail into one that isn't, if you want.  There is an algorithm.  
> Roughly, get testcase, remove every function, variable, block, 
> statement, argument that can be that doesn't affect the testcase.  
> Understand why what's left is minimal as you fix the bug, then try and 
> recode the testcase to make the things that happen to be true, to be 
> true.  For example, need register pressure, insert canonical code to 
> increase pressure...  Mostly works for me.  It is greedy, and has near 
> linear cost.  Further, at times, people automate it in various ways 
> (delta, slice)...  If it is a hardship, pick an automation to handle the 
> grunt work, or an intern...  Even a language agnostic one that just runs 
> the algorithm line by line might be enough for you...  One that is done 
> change all variable/function/type names and then examine code to be sure 
> that no trade secrets or original structure or original intent, use or 
> meaning is left that the customer would care about, if in doubt, just 
> ask them directly.

Again, we don't need to know how this is done, we know perfectly
well. We simply don't have the resources to do it (even keeping the
trees syncrhonized is a huge amount of work).

> So, I guess I am confused as to, do you just not want to do the work to 
> produce the testcases, or do you lack the understanding of how to do it?

It is not a matter of what we want, it is a matter of what we have the
resources to provide.

> As for the past, I'd say that while it would be nice to get more of the 
> Ada testcase that you have into the the FSF testsuite, I think it is 
> reasonable to merely require this of _new_ bugfixes to new 
> bugs/regressions.

But as I mentioned before, the practical effect of this requirement, if
you make it a requirement is that fixes will simply not get transferred
to the FSF tree in many cases. Mike, I encourage you to examine, in
detail, typical front end fixes, which is what we are talking about
here. I think the concern is more theoretical than actual.