Re: [PATCH] define auto_vec copy ctor and assignment (PR 90904)

[Richard Biener <richard.guenther@gmail.com>]

On Wed, Jun 30, 2021 at 3:46 AM Martin Sebor <msebor@gmail.com> wrote:
>
> On 6/29/21 4:58 AM, Richard Biener wrote:
> > On Mon, Jun 28, 2021 at 8:07 PM Martin Sebor <msebor@gmail.com> wrote:
> >>
> >> On 6/28/21 2:07 AM, Richard Biener wrote:
> >>> On Sat, Jun 26, 2021 at 12:36 AM Martin Sebor <msebor@gmail.com> wrote:
> >>>>
> >>>> On 6/25/21 4:11 PM, Jason Merrill wrote:
> >>>>> On 6/25/21 4:51 PM, Martin Sebor wrote:
> >>>>>> On 6/1/21 3:38 PM, Jason Merrill wrote:
> >>>>>>> On 6/1/21 3:56 PM, Martin Sebor wrote:
> >>>>>>>> On 5/27/21 2:53 PM, Jason Merrill wrote:
> >>>>>>>>> On 4/27/21 11:52 AM, Martin Sebor via Gcc-patches wrote:
> >>>>>>>>>> On 4/27/21 8:04 AM, Richard Biener wrote:
> >>>>>>>>>>> On Tue, Apr 27, 2021 at 3:59 PM Martin Sebor <msebor@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 4/27/21 1:58 AM, Richard Biener wrote:
> >>>>>>>>>>>>> On Tue, Apr 27, 2021 at 2:46 AM Martin Sebor via Gcc-patches
> >>>>>>>>>>>>> <gcc-patches@gcc.gnu.org> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> PR 90904 notes that auto_vec is unsafe to copy and assign because
> >>>>>>>>>>>>>> the class manages its own memory but doesn't define (or delete)
> >>>>>>>>>>>>>> either special function.  Since I first ran into the problem,
> >>>>>>>>>>>>>> auto_vec has grown a move ctor and move assignment from
> >>>>>>>>>>>>>> a dynamically-allocated vec but still no copy ctor or copy
> >>>>>>>>>>>>>> assignment operator.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The attached patch adds the two special functions to auto_vec
> >>>>>>>>>>>>>> along
> >>>>>>>>>>>>>> with a few simple tests.  It makes auto_vec safe to use in
> >>>>>>>>>>>>>> containers
> >>>>>>>>>>>>>> that expect copyable and assignable element types and passes
> >>>>>>>>>>>>>> bootstrap
> >>>>>>>>>>>>>> and regression testing on x86_64-linux.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> The question is whether we want such uses to appear since those
> >>>>>>>>>>>>> can be quite inefficient?  Thus the option is to delete those
> >>>>>>>>>>>>> operators?
> >>>>>>>>>>>>
> >>>>>>>>>>>> I would strongly prefer the generic vector class to have the
> >>>>>>>>>>>> properties
> >>>>>>>>>>>> expected of any other generic container: copyable and
> >>>>>>>>>>>> assignable.  If
> >>>>>>>>>>>> we also want another vector type with this restriction I suggest
> >>>>>>>>>>>> to add
> >>>>>>>>>>>> another "noncopyable" type and make that property explicit in
> >>>>>>>>>>>> its name.
> >>>>>>>>>>>> I can submit one in a followup patch if you think we need one.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure (and not strictly against the copy and assign).
> >>>>>>>>>>> Looking around
> >>>>>>>>>>> I see that vec<> does not do deep copying.  Making auto_vec<> do it
> >>>>>>>>>>> might be surprising (I added the move capability to match how vec<>
> >>>>>>>>>>> is used - as "reference" to a vector)
> >>>>>>>>>>
> >>>>>>>>>> The vec base classes are special: they have no ctors at all (because
> >>>>>>>>>> of their use in unions).  That's something we might have to live with
> >>>>>>>>>> but it's not a model to follow in ordinary containers.
> >>>>>>>>>
> >>>>>>>>> I don't think we have to live with it anymore, now that we're
> >>>>>>>>> writing C++11.
> >>>>>>>>>
> >>>>>>>>>> The auto_vec class was introduced to fill the need for a conventional
> >>>>>>>>>> sequence container with a ctor and dtor.  The missing copy ctor and
> >>>>>>>>>> assignment operators were an oversight, not a deliberate feature.
> >>>>>>>>>> This change fixes that oversight.
> >>>>>>>>>>
> >>>>>>>>>> The revised patch also adds a copy ctor/assignment to the auto_vec
> >>>>>>>>>> primary template (that's also missing it).  In addition, it adds
> >>>>>>>>>> a new class called auto_vec_ncopy that disables copying and
> >>>>>>>>>> assignment as you prefer.
> >>>>>>>>>
> >>>>>>>>> Hmm, adding another class doesn't really help with the confusion
> >>>>>>>>> richi mentions.  And many uses of auto_vec will pass them as vec,
> >>>>>>>>> which will still do a shallow copy.  I think it's probably better
> >>>>>>>>> to disable the copy special members for auto_vec until we fix vec<>.
> >>>>>>>>
> >>>>>>>> There are at least a couple of problems that get in the way of fixing
> >>>>>>>> all of vec to act like a well-behaved C++ container:
> >>>>>>>>
> >>>>>>>> 1) The embedded vec has a trailing "flexible" array member with its
> >>>>>>>> instances having different size.  They're initialized by memset and
> >>>>>>>> copied by memcpy.  The class can't have copy ctors or assignments
> >>>>>>>> but it should disable/delete them instead.
> >>>>>>>>
> >>>>>>>> 2) The heap-based vec is used throughout GCC with the assumption of
> >>>>>>>> shallow copy semantics (not just as function arguments but also as
> >>>>>>>> members of other such POD classes).  This can be changed by providing
> >>>>>>>> copy and move ctors and assignment operators for it, and also for
> >>>>>>>> some of the classes in which it's a member and that are used with
> >>>>>>>> the same assumption.
> >>>>>>>>
> >>>>>>>> 3) The heap-based vec::block_remove() assumes its elements are PODs.
> >>>>>>>> That breaks in VEC_ORDERED_REMOVE_IF (used in gcc/dwarf2cfi.c:2862
> >>>>>>>> and tree-vect-patterns.c).
> >>>>>>>>
> >>>>>>>> I took a stab at both and while (1) is easy, (2) is shaping up to
> >>>>>>>> be a big and tricky project.  Tricky because it involves using
> >>>>>>>> std::move in places where what's moved is subsequently still used.
> >>>>>>>> I can keep plugging away at it but it won't change the fact that
> >>>>>>>> the embedded and heap-based vecs have different requirements.
> >>>>>>>>
> >>>>>>>> It doesn't seem to me that having a safely copyable auto_vec needs
> >>>>>>>> to be put on hold until the rats nest above is untangled.  It won't
> >>>>>>>> make anything worse than it is.  (I have a project that depends on
> >>>>>>>> a sane auto_vec working).
> >>>>>>>>
> >>>>>>>> A couple of alternatives to solving this are to use std::vector or
> >>>>>>>> write an equivalent vector class just for GCC.
> >>>>>>>
> >>>>>>> It occurs to me that another way to work around the issue of passing
> >>>>>>> an auto_vec by value as a vec, and thus doing a shallow copy, would
> >>>>>>> be to add a vec ctor taking an auto_vec, and delete that.  This would
> >>>>>>> mean if you want to pass an auto_vec to a vec interface, it needs to
> >>>>>>> be by reference.  We might as well do the same for operator=, though
> >>>>>>> that isn't as important.
> >>>>>>
> >>>>>> Thanks, that sounds like a good idea.  Attached is an implementation
> >>>>>> of this change.  Since the auto_vec copy ctor and assignment have
> >>>>>> been deleted by someone else in the interim, this patch doesn't
> >>>>>> reverse that.  I will propose it separately after these changes
> >>>>>> are finalized.
> >>>>>>
> >>>>>> My approach was to 1) disable the auto_vec to vec conversion,
> >>>>>> 2) introduce an auto_vec::to_vec() to make the conversion possible
> >>>>>> explicitly, and 3) resolve compilation errors by either changing
> >>>>>> APIs to take a vec by reference or callers to convert auto_vec to
> >>>>>> vec explicitly by to_vec().  In (3) I tried to minimize churn while
> >>>>>> improving the const-correctness of the APIs.
> >>>>>
> >>>>> What did you base the choice between reference or to_vec on?  For
> >>>>> instance, it seems like c_parser_declaration_or_fndef could use a
> >>>>> reference, but you changed the callers instead.
> >>>>
> >>>> I went with a reference whenever I could.  That doesn't work when
> >>>> there are callers that pass in a vNULL, so there, and in assignments,
> >>>> I used to_vec().
> >>>
> >>> Is there a way to "fix" the ugliness with vNULL?  All those functions
> >>> should be able to use const vec<>& as otherwise they'd leak memory?
> >>> Can't we pass vNULL to a const vec<>&?
> >>
> >> vNULL can bind to a const vec& (via the vec conversion ctor) but
> >> not to vec&.  The three functions that in the patch are passed
> >> vNULL modify the argument when it's not vNULL but not otherwise.
> >> An alternate design is to have them take a vec* and pass in
> >> a plain NULL (or nullptr) instead of vNULL.  That would require
> >> some surgery on the function bodies that I've been trying to
> >> avoid in the first pass.
> >
> > But I wonder if since you now identified them they could be massaged
> > prior to doing the change.
> >
> > I do hope we end up not needing .to_vec () after all, if no users remain ;)
>
> I'd be happy to if none remained.  I see how to eliminate those in
> calls to functions like c_parser_declaration_or_fndef() (done in
> the attached revision of the patch), but no easy way to get rid
> of those that replace other implicit conversions, like all those
> assignments to the vec members of the ipa_call_arg_values ctor.
> If it's appropriate to std::move those then that would get rid
> of the .to_vec () call.  I'm not familiar with the code but I
> have the impression it might be meant more as a reference to
> some "remote" object (an instance of ipa_auto_call_arg_values?)
> If that's right then making the vec members auto_vec references
> (or pointers) would be one way to "fix" this.

I think ipa_call_arg_values is just a temporary container used to
pass a collection of vec<>s along API boundaries.  I'm not sure
whether it's default CTOR is ever used but it's definitely an
optimization avoiding extra indirection (when changing the vec<>
members to vec<> * or references, in case the default CTOR is
not used).  It _might_ be that the vecs are all just read and never
written to in the APIs using this abstract type but then it's
likely the vector are always appropriately pre-allocated.

Maybe using array_slice instead of vec<> members would work,
but they'd pack less efficient (but I guess not an issue for this
aggregate which should be only used temporarily for argument
passing).

I suppose Martin can answer most of the above ...

>
> >> Functions that don't leak memory now shouldn't leak with these
> >> changes, and conversely, those that do will still leak.  The patch
> >> doesn't change that (as far as I know).
> >
> > It just occurs to me those cases could pass auto_vec<>() by reference instead
> > of vNULL?  So if the vector is modified then it's released afterwards?
> > That would fix the memleak.
>
> I see what you mean.  A function that modified the unnamed vec
> temporary constructed from vNULL then the modified vector would
> leak.  I don't think the functions the patch touches do that but
> I've removed the vNULL conversion from all of them.  There are
> many others that pass vNULL to a vec arguments that that the patch
> doesn't touch but those would be worth a closer look at some point.

Yeah.

> Attached is a revised patch with these changes (a superset of
> those I sent in response to Jason's question), tested on x86_64.
>
> Martin
>
> >
> >> Going forward I think it's possible to replace most uses of vNULL
> >> in GCC with direct initialization (e.g., vec<T> v{ }).  Those that
> >> can't be readily replaced are the ones where vNULL is passed as
> >> an argument to functions taking a vec by value.  Those could be
> >> changed to avoid vNULL too, but it would take a different approach
> >> and more effort.  I'm not against it but I'd rather decouple those
> >> changes from this already sizeable patch.
> >>
> >> Martin
> >>
> >>>
> >>> Richard.
> >>>
> >>>>
> >>>> Martin
> >>>>
> >>
>