Re: [PATCH 1/2] c++: factor out TYPENAME_TYPE substitution

[Patrick Palka <ppalka@redhat.com>]

On Wed, 15 Feb 2023, Jason Merrill wrote:

> On 2/15/23 09:21, Patrick Palka wrote:
> > On Tue, 14 Feb 2023, Jason Merrill wrote:
> > 
> > > On 2/13/23 09:23, Patrick Palka wrote:
> > > > [N.B. this is a corrected version of
> > > > https://gcc.gnu.org/pipermail/gcc-patches/2022-November/607443.html ]
> > > > 
> > > > This patch factors out the TYPENAME_TYPE case of tsubst into a separate
> > > > function tsubst_typename_type.  It also factors out the two tsubst flags
> > > > controlling TYPENAME_TYPE substitution, tf_keep_type_decl and tf_tst_ok,
> > > > into distinct boolean parameters of this new function (and of
> > > > make_typename_type).  Consequently, callers which used to pass tf_tst_ok
> > > > to tsubst now instead must directly call tsubst_typename_type when
> > > > appropriate.
> > > 
> > > Hmm, I don't love how that turns 4 lines into 8 more complex lines in each
> > > caller.  And the previous approach of saying "a CTAD placeholder is OK"
> > > seem
> > > like better abstraction than repeating the specific TYPENAME_TYPE handling
> > > in
> > > each place.
> > 
> > Ah yeah, I see what you mean.  I was thinking since tf_tst_ok is
> > specific to TYPENAME_TYPE handling and isn't propagated (i.e. it only
> > affects top-level TYPENAME_TYPEs), it seemed cleaner to encode the flag
> > as a bool parameter "template_ok" of tsubst_typename_type instead of as
> > a global tsubst_flag that gets propagated freely.
> > 
> > > 
> > > > In a subsequent patch we'll add another flag to
> > > > tsubst_typename_type controlling whether we want to ignore non-types
> > > > during the qualified lookup.
> > 
> > As mentioned above, the second patch in this series would just add
> > another flag "type_only" alongside "template_ok", since this flag will
> > also only affects top-level TYPENAME_TYPEs and doesn't need to propagate
> > like tsubst_flags.
> > 
> > Except, it turns it, this new flag _does_ need to propagate, namely when
> > expanding a variadic using:
> > 
> >    using typename Ts::type::m...; // from typename25a.C below
> > 
> > Here we have a USING_DECL whose USING_DECL_SCOPE is a
> > TYPE_PACK_EXPANSION over TYPENAME_TYPE.  In order to correctly
> > substitute this TYPENAME_TYPE, the USING_DECL case of tsubst_decl needs
> > to pass an appropriate tsubst_flag to tsubst_pack_expansion to be
> > propagated to tsubst (to be propagated to make_typename_type).
> > 
> > So in light of this case it seems adding a new tsubst_flag is the
> > way to go, which means we can avoid this refactoring patch entirely.
> > 
> > Like so?  Bootstrapped and regtested on x86_64-pc-linux-gnu.
> 
> OK, though I still wonder about adding a tsubst_scope function that would add
> the tf_qualifying_scope.

Hmm, but we need to add tf_qualifying_scope to two tsubst_copy calls,
one tsubst call and one tsubst_aggr_type call (with entering_scope=true).
Would tsubst_scope call tsubst, tsubst_copy or tsubst_aggr_type?

> 
> > -- >8 --
> > 
> > Subject: [PATCH] c++: TYPENAME_TYPE lookup ignoring non-types [PR107773]
> > 
> > Currently when resolving a TYPENAME_TYPE for 'typename T::m' via
> > make_typename_type, we consider only type bindings of 'm' and ignore
> > non-type ones.  But [temp.res.general]/3 says, in a note, "the usual
> > qualified name lookup ([basic.lookup.qual]) applies even in the presence
> > of 'typename'", and qualified name lookup doesn't discriminate between
> > type and non-type bindings.  So when resolving such a TYPENAME_TYPE
> > we want the lookup to consider all bindings.
> > 
> > An exception is when we have a TYPENAME_TYPE corresponding to the
> > qualifying scope of the :: scope resolution operator, such as
> > 'T::type' in 'typename T::type::m'.  In that case, [basic.lookup.qual]/1
> > applies, and lookup for such a TYPENAME_TYPE must ignore non-type bindings.
> > So in order to correctly handle all cases, make_typename_type needs an
> > additional flag controlling whether lookup should ignore non-types or not.
> > 
> > To that end this patch adds a new tsubst flag tf_qualifying_scope to
> > communicate to make_typename_type whether we want to ignore non-type
> > bindings during the lookup (by default we don't want to ignore them).
> > In contexts where we do want to ignore non-types (when substituting
> > into the scope of TYPENAME_TYPE, SCOPE_REF or USING_DECL) we simply
> > pass tf_qualifying_scope to the relevant tsubst / tsubst_copy call.
> > This flag is intended to apply only to top-level TYPENAME_TYPEs so
> > we must be careful to clear the flag to avoid propagating it during
> > substitution of sub-trees.
> > 
> > 	PR c++/107773
> > 
> > gcc/cp/ChangeLog:
> > 
> > 	* cp-tree.h (enum tsubst_flags): New flag tf_qualifying_scope.
> > 	* decl.cc (make_typename_type): Use lookup_member instead of
> > 	lookup_field.  If tf_qualifying_scope is set, pass want_type=true
> > 	instead of =false to lookup_member.  Generalize format specifier
> > 	in diagnostic to handle both type and non-type bindings.
> > 	* pt.cc (tsubst_aggr_type_1): Clear tf_qualifying_scope.  Tidy
> > 	the function.
> > 	(tsubst_decl) <case USING_DECL>: Set tf_qualifying_scope when
> > 	substituting USING_DECL_SCOPE.
> > 	(tsubst): Clear tf_qualifying_scope right away and remember if
> > 	it was set.  Do the same for tf_tst_ok sooner.
> > 	<case TYPENAME_TYPE>: Set tf_qualifying_scope when substituting
> > 	TYPE_CONTEXT.  Pass tf_qualifying_scope to make_typename_type
> > 	if it was set.
> > 	(tsubst_qualified_id): Set tf_qualifying_scope when substituting
> > 	the scope.
> > 	(tsubst_copy): Clear tf_qualifying_scope and remember if it was
> > 	set.
> > 	<case SCOPE_REF>: Set tf_qualifying_scope when substituting the
> > 	scope.
> > 	<case *_TYPE>: Pass tf_qualifying_scope to tsubst if it was set.
> > 	* search.cc (lookup_member): Document default argument.
> > 
> > gcc/testsuite/ChangeLog:
> > 
> > 	* g++.dg/template/typename24.C: New test.
> > 	* g++.dg/template/typename25.C: New test.
> > 	* g++.dg/template/typename25a.C: New test.
> > 	* g++.dg/template/typename26.C: New test.
> > ---
> >   gcc/cp/cp-tree.h                            |  3 ++
> >   gcc/cp/decl.cc                              |  9 ++--
> >   gcc/cp/pt.cc                                | 58 ++++++++++++---------
> >   gcc/cp/search.cc                            |  2 +-
> >   gcc/testsuite/g++.dg/template/typename24.C  | 18 +++++++
> >   gcc/testsuite/g++.dg/template/typename25.C  | 33 ++++++++++++
> >   gcc/testsuite/g++.dg/template/typename25a.C | 37 +++++++++++++
> >   gcc/testsuite/g++.dg/template/typename26.C  | 20 +++++++
> >   8 files changed, 150 insertions(+), 30 deletions(-)
> >   create mode 100644 gcc/testsuite/g++.dg/template/typename24.C
> >   create mode 100644 gcc/testsuite/g++.dg/template/typename25.C
> >   create mode 100644 gcc/testsuite/g++.dg/template/typename25a.C
> >   create mode 100644 gcc/testsuite/g++.dg/template/typename26.C
> > 
> > diff --git a/gcc/cp/cp-tree.h b/gcc/cp/cp-tree.h
> > index be8775ed0f8..891fcf521a8 100644
> > --- a/gcc/cp/cp-tree.h
> > +++ b/gcc/cp/cp-tree.h
> > @@ -5597,6 +5597,9 @@ enum tsubst_flags {
> >     tf_tst_ok = 1 << 12,		 /* Allow a typename-specifier to name
> >   				    a template (C++17 or later).  */
> >     tf_dguide = 1 << 13,		/* Building a deduction guide from a
> > ctor.  */
> > +  tf_qualifying_scope = 1 << 14, /* Substituting the LHS of the ::
> > operator.
> > +				    Controls TYPENAME_TYPE resolution from
> > +				    make_typename_type.  */
> >     /* Convenient substitution flags combinations.  */
> >     tf_warning_or_error = tf_warning | tf_error
> >   };
> > diff --git a/gcc/cp/decl.cc b/gcc/cp/decl.cc
> > index d606b31d7a7..2f6412d04e6 100644
> > --- a/gcc/cp/decl.cc
> > +++ b/gcc/cp/decl.cc
> > @@ -4305,9 +4305,10 @@ make_typename_type (tree context, tree name, enum
> > tag_types tag_type,
> >        member of the current instantiation or a non-dependent base;
> >        lookup will stop when we hit a dependent base.  */
> >     if (!dependent_scope_p (context))
> > -    /* We should only set WANT_TYPE when we're a nested typename type.
> > -       Then we can give better diagnostics if we find a non-type.  */
> > -    t = lookup_field (context, name, 2, /*want_type=*/true);
> > +    {
> > +      bool want_type = (complain & tf_qualifying_scope);
> > +      t = lookup_member (context, name, /*protect=*/2, want_type,
> > complain);
> > +    }
> >     else
> >       t = NULL_TREE;
> >   @@ -4359,7 +4360,7 @@ make_typename_type (tree context, tree name, enum
> > tag_types tag_type,
> >         else
> >   	{
> >   	  if (complain & tf_error)
> > -	    error ("%<typename %T::%D%> names %q#T, which is not a type",
> > +	    error ("%<typename %T::%D%> names %q#D, which is not a type",
> >   		   context, name, t);
> >   	  return error_mark_node;
> >   	}
> > diff --git a/gcc/cp/pt.cc b/gcc/cp/pt.cc
> > index e89dbf47097..d11d540ab44 100644
> > --- a/gcc/cp/pt.cc
> > +++ b/gcc/cp/pt.cc
> > @@ -13919,8 +13919,7 @@ tsubst_aggr_type_1 (tree t,
> >   {
> >     if (TYPE_TEMPLATE_INFO (t) && uses_template_parms (t))
> >       {
> > -      tree argvec;
> > -      tree r;
> > +      complain &= ~tf_qualifying_scope;
> >           /* Figure out what arguments are appropriate for the
> >   	 type we are trying to find.  For example, given:
> > @@ -13931,18 +13930,14 @@ tsubst_aggr_type_1 (tree t,
> >   	 and supposing that we are instantiating f<int, double>,
> >   	 then our ARGS will be {int, double}, but, when looking up
> >   	 S we only want {double}.  */
> > -      argvec = tsubst_template_args (TYPE_TI_ARGS (t), args,
> > -				     complain, in_decl);
> > +      tree argvec = tsubst_template_args (TYPE_TI_ARGS (t), args,
> > +					  complain, in_decl);
> >         if (argvec == error_mark_node)
> > -	r = error_mark_node;
> > -      else
> > -	{
> > -	  r = lookup_template_class (t, argvec, in_decl, NULL_TREE,
> > -				     entering_scope, complain);
> > -	  r = cp_build_qualified_type (r, cp_type_quals (t), complain);
> > -	}
> > +	return error_mark_node;
> >   -      return r;
> > +      tree r = lookup_template_class (t, argvec, in_decl, NULL_TREE,
> > +				      entering_scope, complain);
> > +      return cp_build_qualified_type (r, cp_type_quals (t), complain);
> >       }
> >     else
> >       /* This is not a template type, so there's nothing to do.  */
> > @@ -15003,11 +14998,15 @@ tsubst_decl (tree t, tree args, tsubst_flags_t
> > complain)
> >   	  tree scope = USING_DECL_SCOPE (t);
> >   	  if (PACK_EXPANSION_P (scope))
> >   	    {
> > -	      scope = tsubst_pack_expansion (scope, args, complain, in_decl);
> > +	      scope = tsubst_pack_expansion (scope, args,
> > +					     complain | tf_qualifying_scope,
> > +					     in_decl);
> >   	      variadic_p = true;
> >   	    }
> >   	  else
> > -	    scope = tsubst_copy (scope, args, complain, in_decl);
> > +	    scope = tsubst_copy (scope, args,
> > +				 complain | tf_qualifying_scope,
> > +				 in_decl);
> >     	  tree name = DECL_NAME (t);
> >   	  if (IDENTIFIER_CONV_OP_P (name)
> > @@ -15821,6 +15820,12 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
> > tree in_decl)
> >         || TREE_CODE (t) == TRANSLATION_UNIT_DECL)
> >       return t;
> >   +  tsubst_flags_t tst_ok_flag = (complain & tf_tst_ok);
> > +  complain &= ~tf_tst_ok;
> > +
> > +  tsubst_flags_t qualifying_scope_flag = (complain & tf_qualifying_scope);
> > +  complain &= ~tf_qualifying_scope;
> > +
> >     if (DECL_P (t))
> >       return tsubst_decl (t, args, complain);
> >   @@ -15889,9 +15894,6 @@ tsubst (tree t, tree args, tsubst_flags_t
> > complain, tree in_decl)
> >     bool fndecl_type = (complain & tf_fndecl_type);
> >     complain &= ~tf_fndecl_type;
> >   -  bool tst_ok = (complain & tf_tst_ok);
> > -  complain &= ~tf_tst_ok;
> > -
> >     if (type
> >         && code != TYPENAME_TYPE
> >         && code != TEMPLATE_TYPE_PARM
> > @@ -16428,7 +16430,9 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
> > tree in_decl)
> >   	tree ctx = TYPE_CONTEXT (t);
> >   	if (TREE_CODE (ctx) == TYPE_PACK_EXPANSION)
> >   	  {
> > -	    ctx = tsubst_pack_expansion (ctx, args, complain, in_decl);
> > +	    ctx = tsubst_pack_expansion (ctx, args,
> > +					 complain | tf_qualifying_scope,
> > +					 in_decl);
> >   	    if (ctx == error_mark_node
> >   		|| TREE_VEC_LENGTH (ctx) > 1)
> >   	      return error_mark_node;
> > @@ -16442,8 +16446,9 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
> > tree in_decl)
> >   	    ctx = TREE_VEC_ELT (ctx, 0);
> >   	  }
> >   	else
> > -	  ctx = tsubst_aggr_type (ctx, args, complain, in_decl,
> > -				  /*entering_scope=*/1);
> > +	  ctx = tsubst_aggr_type (ctx, args,
> > +				  complain | tf_qualifying_scope,
> > +				  in_decl, /*entering_scope=*/1);
> >   	if (ctx == error_mark_node)
> >   	  return error_mark_node;
> >   @@ -16473,8 +16478,7 @@ tsubst (tree t, tree args, tsubst_flags_t
> > complain, tree in_decl)
> >   	  }
> >     	tsubst_flags_t tcomplain = complain | tf_keep_type_decl;
> > -	if (tst_ok)
> > -	  tcomplain |= tf_tst_ok;
> > +	tcomplain |= tst_ok_flag | qualifying_scope_flag;
> >   	f = make_typename_type (ctx, f, typename_type, tcomplain);
> >   	if (f == error_mark_node)
> >   	  return f;
> > @@ -16879,7 +16883,7 @@ tsubst_qualified_id (tree qualified_id, tree args,
> >     scope = TREE_OPERAND (qualified_id, 0);
> >     if (args)
> >       {
> > -      scope = tsubst (scope, args, complain, in_decl);
> > +      scope = tsubst (scope, args, complain | tf_qualifying_scope,
> > in_decl);
> >         expr = tsubst_copy (name, args, complain, in_decl);
> >       }
> >     else
> > @@ -17125,6 +17129,9 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
> > complain, tree in_decl)
> >     if (t == NULL_TREE || t == error_mark_node || args == NULL_TREE)
> >       return t;
> >   +  tsubst_flags_t qualifying_scope_flag = (complain &
> > tf_qualifying_scope);
> > +  complain &= ~tf_qualifying_scope;
> > +
> >     if (tree d = maybe_dependent_member_ref (t, args, complain, in_decl))
> >       return d;
> >   @@ -17598,7 +17605,8 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
> > complain, tree in_decl)
> >         case SCOPE_REF:
> >         {
> > -	tree op0 = tsubst_copy (TREE_OPERAND (t, 0), args, complain, in_decl);
> > +	tree op0 = tsubst_copy (TREE_OPERAND (t, 0), args,
> > +				complain | tf_qualifying_scope, in_decl);
> >   	tree op1 = tsubst_copy (TREE_OPERAND (t, 1), args, complain, in_decl);
> >   	return build_qualified_name (/*type=*/NULL_TREE, op0, op1,
> >   				     QUALIFIED_NAME_IS_TEMPLATE (t));
> > @@ -17713,7 +17721,7 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
> > complain, tree in_decl)
> >       case TYPEOF_TYPE:
> >       case DECLTYPE_TYPE:
> >       case TYPE_DECL:
> > -      return tsubst (t, args, complain, in_decl);
> > +      return tsubst (t, args, complain | qualifying_scope_flag, in_decl);
> >         case USING_DECL:
> >         t = DECL_NAME (t);
> > diff --git a/gcc/cp/search.cc b/gcc/cp/search.cc
> > index f3f19cafec6..e472a97679d 100644
> > --- a/gcc/cp/search.cc
> > +++ b/gcc/cp/search.cc
> > @@ -1109,7 +1109,7 @@ build_baselink (tree binfo, tree access_binfo, tree
> > functions, tree optype)
> >     tree
> >   lookup_member (tree xbasetype, tree name, int protect, bool want_type,
> > -	       tsubst_flags_t complain, access_failure_info *afi)
> > +	       tsubst_flags_t complain, access_failure_info *afi /* = NULL */)
> >   {
> >     tree rval, rval_binfo = NULL_TREE;
> >     tree type = NULL_TREE, basetype_path = NULL_TREE;
> > diff --git a/gcc/testsuite/g++.dg/template/typename24.C
> > b/gcc/testsuite/g++.dg/template/typename24.C
> > new file mode 100644
> > index 00000000000..8b2b3718442
> > --- /dev/null
> > +++ b/gcc/testsuite/g++.dg/template/typename24.C
> > @@ -0,0 +1,18 @@
> > +// PR c++/107773
> > +// Verify lookup for a non-neste TYPENAME_TYPE correctly considers
> > +// non-types.
> > +
> > +struct a {
> > +  typedef void get;
> > +};
> > +
> > +struct b : a {
> > +  int get(int i) const;
> > +};
> > +
> > +template<class T>
> > +void f() {
> > +  typedef typename T::get type; // { dg-error "'int b::get\\(int\\) const',
> > which is not a type" }
> > +}
> > +
> > +template void f<b>();
> > diff --git a/gcc/testsuite/g++.dg/template/typename25.C
> > b/gcc/testsuite/g++.dg/template/typename25.C
> > new file mode 100644
> > index 00000000000..04e48e11724
> > --- /dev/null
> > +++ b/gcc/testsuite/g++.dg/template/typename25.C
> > @@ -0,0 +1,33 @@
> > +// PR c++/107773
> > +// Verify lookup for TYPENAME_TYPE appearing to the left of the ::
> > +// scope resolution operator correctly ignores non-types.
> > +
> > +struct a {
> > +  typedef void type;
> > +};
> > +
> > +struct c {
> > +  struct b : a {
> > +    typedef b self;
> > +    static int m;
> > +  };
> > +  int b;
> > +};
> > +
> > +template<class T>
> > +void f() {
> > +  // A TYPENAME_TYPE whose TYPE_CONTEXT is a nested TYPENAME_TYPE.
> > +  typedef typename T::b::type type;
> > +  // A SCOPE_REF whose first operand is a TYPENAME_TYPE.
> > +  int m = T::b::m;
> > +}
> > +
> > +template void f<c>();
> > +
> > +template<class T>
> > +struct d : T::b::self {
> > +  // A USING_DECL whose USING_DECL_SCOPE is a TYPENAME_TYPE.
> > +  using typename T::b::type;
> > +};
> > +
> > +template struct d<c>;
> > diff --git a/gcc/testsuite/g++.dg/template/typename25a.C
> > b/gcc/testsuite/g++.dg/template/typename25a.C
> > new file mode 100644
> > index 00000000000..ecb34aada34
> > --- /dev/null
> > +++ b/gcc/testsuite/g++.dg/template/typename25a.C
> > @@ -0,0 +1,37 @@
> > +// PR c++/107773
> > +// A variadic version of typename25.C
> > +// { dg-do compile { target c++11 } }
> > +
> > +struct a {
> > +  typedef void type;
> > +};
> > +
> > +struct c {
> > +  struct b : a {
> > +    typedef b self;
> > +    static int m;
> > +  };
> > +  int b;
> > +};
> > +
> > +template<class...> void sink(...);
> > +
> > +template<class... Ts>
> > +void f() {
> > +  // A TYPENAME_TYPE whose TYPE_CONTEXT is a nested TYPENAME_TYPE.
> > +  sink<typename Ts::b::type...>();
> > +  // A SCOPE_REF whose first operand is a TYPENAME_TYPE.
> > +  sink(Ts::b::m...);
> > +}
> > +
> > +template void f<c>();
> > +
> > +template<class... Ts>
> > +struct d : Ts::b::self... {
> > +#if __cpp_variadic_using
> > +  // A USING_DECL whose USING_DECL_SCOPE is a TYPENAME_TYPE.
> > +  using typename Ts::b::type...;
> > +#endif
> > +};
> > +
> > +template struct d<c>;
> > diff --git a/gcc/testsuite/g++.dg/template/typename26.C
> > b/gcc/testsuite/g++.dg/template/typename26.C
> > new file mode 100644
> > index 00000000000..4e6b764a97b
> > --- /dev/null
> > +++ b/gcc/testsuite/g++.dg/template/typename26.C
> > @@ -0,0 +1,20 @@
> > +// Example 4 from [temp.res.general]/3.
> > +
> > +struct A {
> > +  struct X { };
> > +  int X;
> > +};
> > +struct B {
> > +  struct X { };
> > +};
> > +template<class T> void f(T t) {
> > +  typename T::X x; // { dg-error "'int A::X', which is not a type" }
> > +}
> > +void foo() {
> > +  A a;
> > +  B b;
> > +  f(b); // OK, T::X refers to B::X
> > +  // { dg-bogus "" "" { target *-*-* } .-1 }
> > +  f(a); // error: T::X refers to the data member A::X not the struct A::X
> > +  // { dg-message "required from here" "" { target *-*-* } .-1 }
> > +}
> 
>