Re: [PATCH 1/2] c++: factor out TYPENAME_TYPE substitution

[Jason Merrill <jason@redhat.com>]

On 2/15/23 12:11, Patrick Palka wrote:
> On Wed, 15 Feb 2023, Jason Merrill wrote:
> 
>> On 2/15/23 09:21, Patrick Palka wrote:
>>> On Tue, 14 Feb 2023, Jason Merrill wrote:
>>>
>>>> On 2/13/23 09:23, Patrick Palka wrote:
>>>>> [N.B. this is a corrected version of
>>>>> https://gcc.gnu.org/pipermail/gcc-patches/2022-November/607443.html ]
>>>>>
>>>>> This patch factors out the TYPENAME_TYPE case of tsubst into a separate
>>>>> function tsubst_typename_type.  It also factors out the two tsubst flags
>>>>> controlling TYPENAME_TYPE substitution, tf_keep_type_decl and tf_tst_ok,
>>>>> into distinct boolean parameters of this new function (and of
>>>>> make_typename_type).  Consequently, callers which used to pass tf_tst_ok
>>>>> to tsubst now instead must directly call tsubst_typename_type when
>>>>> appropriate.
>>>>
>>>> Hmm, I don't love how that turns 4 lines into 8 more complex lines in each
>>>> caller.  And the previous approach of saying "a CTAD placeholder is OK"
>>>> seem
>>>> like better abstraction than repeating the specific TYPENAME_TYPE handling
>>>> in
>>>> each place.
>>>
>>> Ah yeah, I see what you mean.  I was thinking since tf_tst_ok is
>>> specific to TYPENAME_TYPE handling and isn't propagated (i.e. it only
>>> affects top-level TYPENAME_TYPEs), it seemed cleaner to encode the flag
>>> as a bool parameter "template_ok" of tsubst_typename_type instead of as
>>> a global tsubst_flag that gets propagated freely.
>>>
>>>>
>>>>> In a subsequent patch we'll add another flag to
>>>>> tsubst_typename_type controlling whether we want to ignore non-types
>>>>> during the qualified lookup.
>>>
>>> As mentioned above, the second patch in this series would just add
>>> another flag "type_only" alongside "template_ok", since this flag will
>>> also only affects top-level TYPENAME_TYPEs and doesn't need to propagate
>>> like tsubst_flags.
>>>
>>> Except, it turns it, this new flag _does_ need to propagate, namely when
>>> expanding a variadic using:
>>>
>>>     using typename Ts::type::m...; // from typename25a.C below
>>>
>>> Here we have a USING_DECL whose USING_DECL_SCOPE is a
>>> TYPE_PACK_EXPANSION over TYPENAME_TYPE.  In order to correctly
>>> substitute this TYPENAME_TYPE, the USING_DECL case of tsubst_decl needs
>>> to pass an appropriate tsubst_flag to tsubst_pack_expansion to be
>>> propagated to tsubst (to be propagated to make_typename_type).
>>>
>>> So in light of this case it seems adding a new tsubst_flag is the
>>> way to go, which means we can avoid this refactoring patch entirely.
>>>
>>> Like so?  Bootstrapped and regtested on x86_64-pc-linux-gnu.
>>
>> OK, though I still wonder about adding a tsubst_scope function that would add
>> the tf_qualifying_scope.
> 
> Hmm, but we need to add tf_qualifying_scope to two tsubst_copy calls,
> one tsubst call and one tsubst_aggr_type call (with entering_scope=true).
> Would tsubst_scope call tsubst, tsubst_copy or tsubst_aggr_type?

In general it would call tsubst.

It's odd that anything is calling tsubst_copy with a type, that seems 
like a copy/paste error.  But it just hands off to tsubst anyway, so the 
effect is the same.

tsubst_aggr_type is needed when pushing into the scope of a declarator; 
I don't know offhand why we would need that when substituting the scope 
of a TYPENAME_TYPE.

I'd call tsubst, and leave the tsubst_aggr_type call alone for GCC 13.

>>> -- >8 --
>>>
>>> Subject: [PATCH] c++: TYPENAME_TYPE lookup ignoring non-types [PR107773]
>>>
>>> Currently when resolving a TYPENAME_TYPE for 'typename T::m' via
>>> make_typename_type, we consider only type bindings of 'm' and ignore
>>> non-type ones.  But [temp.res.general]/3 says, in a note, "the usual
>>> qualified name lookup ([basic.lookup.qual]) applies even in the presence
>>> of 'typename'", and qualified name lookup doesn't discriminate between
>>> type and non-type bindings.  So when resolving such a TYPENAME_TYPE
>>> we want the lookup to consider all bindings.
>>>
>>> An exception is when we have a TYPENAME_TYPE corresponding to the
>>> qualifying scope of the :: scope resolution operator, such as
>>> 'T::type' in 'typename T::type::m'.  In that case, [basic.lookup.qual]/1
>>> applies, and lookup for such a TYPENAME_TYPE must ignore non-type bindings.
>>> So in order to correctly handle all cases, make_typename_type needs an
>>> additional flag controlling whether lookup should ignore non-types or not.
>>>
>>> To that end this patch adds a new tsubst flag tf_qualifying_scope to
>>> communicate to make_typename_type whether we want to ignore non-type
>>> bindings during the lookup (by default we don't want to ignore them).
>>> In contexts where we do want to ignore non-types (when substituting
>>> into the scope of TYPENAME_TYPE, SCOPE_REF or USING_DECL) we simply
>>> pass tf_qualifying_scope to the relevant tsubst / tsubst_copy call.
>>> This flag is intended to apply only to top-level TYPENAME_TYPEs so
>>> we must be careful to clear the flag to avoid propagating it during
>>> substitution of sub-trees.
>>>
>>> 	PR c++/107773
>>>
>>> gcc/cp/ChangeLog:
>>>
>>> 	* cp-tree.h (enum tsubst_flags): New flag tf_qualifying_scope.
>>> 	* decl.cc (make_typename_type): Use lookup_member instead of
>>> 	lookup_field.  If tf_qualifying_scope is set, pass want_type=true
>>> 	instead of =false to lookup_member.  Generalize format specifier
>>> 	in diagnostic to handle both type and non-type bindings.
>>> 	* pt.cc (tsubst_aggr_type_1): Clear tf_qualifying_scope.  Tidy
>>> 	the function.
>>> 	(tsubst_decl) <case USING_DECL>: Set tf_qualifying_scope when
>>> 	substituting USING_DECL_SCOPE.
>>> 	(tsubst): Clear tf_qualifying_scope right away and remember if
>>> 	it was set.  Do the same for tf_tst_ok sooner.
>>> 	<case TYPENAME_TYPE>: Set tf_qualifying_scope when substituting
>>> 	TYPE_CONTEXT.  Pass tf_qualifying_scope to make_typename_type
>>> 	if it was set.
>>> 	(tsubst_qualified_id): Set tf_qualifying_scope when substituting
>>> 	the scope.
>>> 	(tsubst_copy): Clear tf_qualifying_scope and remember if it was
>>> 	set.
>>> 	<case SCOPE_REF>: Set tf_qualifying_scope when substituting the
>>> 	scope.
>>> 	<case *_TYPE>: Pass tf_qualifying_scope to tsubst if it was set.
>>> 	* search.cc (lookup_member): Document default argument.
>>>
>>> gcc/testsuite/ChangeLog:
>>>
>>> 	* g++.dg/template/typename24.C: New test.
>>> 	* g++.dg/template/typename25.C: New test.
>>> 	* g++.dg/template/typename25a.C: New test.
>>> 	* g++.dg/template/typename26.C: New test.
>>> ---
>>>    gcc/cp/cp-tree.h                            |  3 ++
>>>    gcc/cp/decl.cc                              |  9 ++--
>>>    gcc/cp/pt.cc                                | 58 ++++++++++++---------
>>>    gcc/cp/search.cc                            |  2 +-
>>>    gcc/testsuite/g++.dg/template/typename24.C  | 18 +++++++
>>>    gcc/testsuite/g++.dg/template/typename25.C  | 33 ++++++++++++
>>>    gcc/testsuite/g++.dg/template/typename25a.C | 37 +++++++++++++
>>>    gcc/testsuite/g++.dg/template/typename26.C  | 20 +++++++
>>>    8 files changed, 150 insertions(+), 30 deletions(-)
>>>    create mode 100644 gcc/testsuite/g++.dg/template/typename24.C
>>>    create mode 100644 gcc/testsuite/g++.dg/template/typename25.C
>>>    create mode 100644 gcc/testsuite/g++.dg/template/typename25a.C
>>>    create mode 100644 gcc/testsuite/g++.dg/template/typename26.C
>>>
>>> diff --git a/gcc/cp/cp-tree.h b/gcc/cp/cp-tree.h
>>> index be8775ed0f8..891fcf521a8 100644
>>> --- a/gcc/cp/cp-tree.h
>>> +++ b/gcc/cp/cp-tree.h
>>> @@ -5597,6 +5597,9 @@ enum tsubst_flags {
>>>      tf_tst_ok = 1 << 12,		 /* Allow a typename-specifier to name
>>>    				    a template (C++17 or later).  */
>>>      tf_dguide = 1 << 13,		/* Building a deduction guide from a
>>> ctor.  */
>>> +  tf_qualifying_scope = 1 << 14, /* Substituting the LHS of the ::
>>> operator.
>>> +				    Controls TYPENAME_TYPE resolution from
>>> +				    make_typename_type.  */
>>>      /* Convenient substitution flags combinations.  */
>>>      tf_warning_or_error = tf_warning | tf_error
>>>    };
>>> diff --git a/gcc/cp/decl.cc b/gcc/cp/decl.cc
>>> index d606b31d7a7..2f6412d04e6 100644
>>> --- a/gcc/cp/decl.cc
>>> +++ b/gcc/cp/decl.cc
>>> @@ -4305,9 +4305,10 @@ make_typename_type (tree context, tree name, enum
>>> tag_types tag_type,
>>>         member of the current instantiation or a non-dependent base;
>>>         lookup will stop when we hit a dependent base.  */
>>>      if (!dependent_scope_p (context))
>>> -    /* We should only set WANT_TYPE when we're a nested typename type.
>>> -       Then we can give better diagnostics if we find a non-type.  */
>>> -    t = lookup_field (context, name, 2, /*want_type=*/true);
>>> +    {
>>> +      bool want_type = (complain & tf_qualifying_scope);
>>> +      t = lookup_member (context, name, /*protect=*/2, want_type,
>>> complain);
>>> +    }
>>>      else
>>>        t = NULL_TREE;
>>>    @@ -4359,7 +4360,7 @@ make_typename_type (tree context, tree name, enum
>>> tag_types tag_type,
>>>          else
>>>    	{
>>>    	  if (complain & tf_error)
>>> -	    error ("%<typename %T::%D%> names %q#T, which is not a type",
>>> +	    error ("%<typename %T::%D%> names %q#D, which is not a type",
>>>    		   context, name, t);
>>>    	  return error_mark_node;
>>>    	}
>>> diff --git a/gcc/cp/pt.cc b/gcc/cp/pt.cc
>>> index e89dbf47097..d11d540ab44 100644
>>> --- a/gcc/cp/pt.cc
>>> +++ b/gcc/cp/pt.cc
>>> @@ -13919,8 +13919,7 @@ tsubst_aggr_type_1 (tree t,
>>>    {
>>>      if (TYPE_TEMPLATE_INFO (t) && uses_template_parms (t))
>>>        {
>>> -      tree argvec;
>>> -      tree r;
>>> +      complain &= ~tf_qualifying_scope;
>>>            /* Figure out what arguments are appropriate for the
>>>    	 type we are trying to find.  For example, given:
>>> @@ -13931,18 +13930,14 @@ tsubst_aggr_type_1 (tree t,
>>>    	 and supposing that we are instantiating f<int, double>,
>>>    	 then our ARGS will be {int, double}, but, when looking up
>>>    	 S we only want {double}.  */
>>> -      argvec = tsubst_template_args (TYPE_TI_ARGS (t), args,
>>> -				     complain, in_decl);
>>> +      tree argvec = tsubst_template_args (TYPE_TI_ARGS (t), args,
>>> +					  complain, in_decl);
>>>          if (argvec == error_mark_node)
>>> -	r = error_mark_node;
>>> -      else
>>> -	{
>>> -	  r = lookup_template_class (t, argvec, in_decl, NULL_TREE,
>>> -				     entering_scope, complain);
>>> -	  r = cp_build_qualified_type (r, cp_type_quals (t), complain);
>>> -	}
>>> +	return error_mark_node;
>>>    -      return r;
>>> +      tree r = lookup_template_class (t, argvec, in_decl, NULL_TREE,
>>> +				      entering_scope, complain);
>>> +      return cp_build_qualified_type (r, cp_type_quals (t), complain);
>>>        }
>>>      else
>>>        /* This is not a template type, so there's nothing to do.  */
>>> @@ -15003,11 +14998,15 @@ tsubst_decl (tree t, tree args, tsubst_flags_t
>>> complain)
>>>    	  tree scope = USING_DECL_SCOPE (t);
>>>    	  if (PACK_EXPANSION_P (scope))
>>>    	    {
>>> -	      scope = tsubst_pack_expansion (scope, args, complain, in_decl);
>>> +	      scope = tsubst_pack_expansion (scope, args,
>>> +					     complain | tf_qualifying_scope,
>>> +					     in_decl);
>>>    	      variadic_p = true;
>>>    	    }
>>>    	  else
>>> -	    scope = tsubst_copy (scope, args, complain, in_decl);
>>> +	    scope = tsubst_copy (scope, args,
>>> +				 complain | tf_qualifying_scope,
>>> +				 in_decl);
>>>      	  tree name = DECL_NAME (t);
>>>    	  if (IDENTIFIER_CONV_OP_P (name)
>>> @@ -15821,6 +15820,12 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
>>> tree in_decl)
>>>          || TREE_CODE (t) == TRANSLATION_UNIT_DECL)
>>>        return t;
>>>    +  tsubst_flags_t tst_ok_flag = (complain & tf_tst_ok);
>>> +  complain &= ~tf_tst_ok;
>>> +
>>> +  tsubst_flags_t qualifying_scope_flag = (complain & tf_qualifying_scope);
>>> +  complain &= ~tf_qualifying_scope;
>>> +
>>>      if (DECL_P (t))
>>>        return tsubst_decl (t, args, complain);
>>>    @@ -15889,9 +15894,6 @@ tsubst (tree t, tree args, tsubst_flags_t
>>> complain, tree in_decl)
>>>      bool fndecl_type = (complain & tf_fndecl_type);
>>>      complain &= ~tf_fndecl_type;
>>>    -  bool tst_ok = (complain & tf_tst_ok);
>>> -  complain &= ~tf_tst_ok;
>>> -
>>>      if (type
>>>          && code != TYPENAME_TYPE
>>>          && code != TEMPLATE_TYPE_PARM
>>> @@ -16428,7 +16430,9 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
>>> tree in_decl)
>>>    	tree ctx = TYPE_CONTEXT (t);
>>>    	if (TREE_CODE (ctx) == TYPE_PACK_EXPANSION)
>>>    	  {
>>> -	    ctx = tsubst_pack_expansion (ctx, args, complain, in_decl);
>>> +	    ctx = tsubst_pack_expansion (ctx, args,
>>> +					 complain | tf_qualifying_scope,
>>> +					 in_decl);
>>>    	    if (ctx == error_mark_node
>>>    		|| TREE_VEC_LENGTH (ctx) > 1)
>>>    	      return error_mark_node;
>>> @@ -16442,8 +16446,9 @@ tsubst (tree t, tree args, tsubst_flags_t complain,
>>> tree in_decl)
>>>    	    ctx = TREE_VEC_ELT (ctx, 0);
>>>    	  }
>>>    	else
>>> -	  ctx = tsubst_aggr_type (ctx, args, complain, in_decl,
>>> -				  /*entering_scope=*/1);
>>> +	  ctx = tsubst_aggr_type (ctx, args,
>>> +				  complain | tf_qualifying_scope,
>>> +				  in_decl, /*entering_scope=*/1);
>>>    	if (ctx == error_mark_node)
>>>    	  return error_mark_node;
>>>    @@ -16473,8 +16478,7 @@ tsubst (tree t, tree args, tsubst_flags_t
>>> complain, tree in_decl)
>>>    	  }
>>>      	tsubst_flags_t tcomplain = complain | tf_keep_type_decl;
>>> -	if (tst_ok)
>>> -	  tcomplain |= tf_tst_ok;
>>> +	tcomplain |= tst_ok_flag | qualifying_scope_flag;
>>>    	f = make_typename_type (ctx, f, typename_type, tcomplain);
>>>    	if (f == error_mark_node)
>>>    	  return f;
>>> @@ -16879,7 +16883,7 @@ tsubst_qualified_id (tree qualified_id, tree args,
>>>      scope = TREE_OPERAND (qualified_id, 0);
>>>      if (args)
>>>        {
>>> -      scope = tsubst (scope, args, complain, in_decl);
>>> +      scope = tsubst (scope, args, complain | tf_qualifying_scope,
>>> in_decl);
>>>          expr = tsubst_copy (name, args, complain, in_decl);
>>>        }
>>>      else
>>> @@ -17125,6 +17129,9 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
>>> complain, tree in_decl)
>>>      if (t == NULL_TREE || t == error_mark_node || args == NULL_TREE)
>>>        return t;
>>>    +  tsubst_flags_t qualifying_scope_flag = (complain &
>>> tf_qualifying_scope);
>>> +  complain &= ~tf_qualifying_scope;
>>> +
>>>      if (tree d = maybe_dependent_member_ref (t, args, complain, in_decl))
>>>        return d;
>>>    @@ -17598,7 +17605,8 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
>>> complain, tree in_decl)
>>>          case SCOPE_REF:
>>>          {
>>> -	tree op0 = tsubst_copy (TREE_OPERAND (t, 0), args, complain, in_decl);
>>> +	tree op0 = tsubst_copy (TREE_OPERAND (t, 0), args,
>>> +				complain | tf_qualifying_scope, in_decl);
>>>    	tree op1 = tsubst_copy (TREE_OPERAND (t, 1), args, complain, in_decl);
>>>    	return build_qualified_name (/*type=*/NULL_TREE, op0, op1,
>>>    				     QUALIFIED_NAME_IS_TEMPLATE (t));
>>> @@ -17713,7 +17721,7 @@ tsubst_copy (tree t, tree args, tsubst_flags_t
>>> complain, tree in_decl)
>>>        case TYPEOF_TYPE:
>>>        case DECLTYPE_TYPE:
>>>        case TYPE_DECL:
>>> -      return tsubst (t, args, complain, in_decl);
>>> +      return tsubst (t, args, complain | qualifying_scope_flag, in_decl);
>>>          case USING_DECL:
>>>          t = DECL_NAME (t);
>>> diff --git a/gcc/cp/search.cc b/gcc/cp/search.cc
>>> index f3f19cafec6..e472a97679d 100644
>>> --- a/gcc/cp/search.cc
>>> +++ b/gcc/cp/search.cc
>>> @@ -1109,7 +1109,7 @@ build_baselink (tree binfo, tree access_binfo, tree
>>> functions, tree optype)
>>>      tree
>>>    lookup_member (tree xbasetype, tree name, int protect, bool want_type,
>>> -	       tsubst_flags_t complain, access_failure_info *afi)
>>> +	       tsubst_flags_t complain, access_failure_info *afi /* = NULL */)
>>>    {
>>>      tree rval, rval_binfo = NULL_TREE;
>>>      tree type = NULL_TREE, basetype_path = NULL_TREE;
>>> diff --git a/gcc/testsuite/g++.dg/template/typename24.C
>>> b/gcc/testsuite/g++.dg/template/typename24.C
>>> new file mode 100644
>>> index 00000000000..8b2b3718442
>>> --- /dev/null
>>> +++ b/gcc/testsuite/g++.dg/template/typename24.C
>>> @@ -0,0 +1,18 @@
>>> +// PR c++/107773
>>> +// Verify lookup for a non-neste TYPENAME_TYPE correctly considers
>>> +// non-types.
>>> +
>>> +struct a {
>>> +  typedef void get;
>>> +};
>>> +
>>> +struct b : a {
>>> +  int get(int i) const;
>>> +};
>>> +
>>> +template<class T>
>>> +void f() {
>>> +  typedef typename T::get type; // { dg-error "'int b::get\\(int\\) const',
>>> which is not a type" }
>>> +}
>>> +
>>> +template void f<b>();
>>> diff --git a/gcc/testsuite/g++.dg/template/typename25.C
>>> b/gcc/testsuite/g++.dg/template/typename25.C
>>> new file mode 100644
>>> index 00000000000..04e48e11724
>>> --- /dev/null
>>> +++ b/gcc/testsuite/g++.dg/template/typename25.C
>>> @@ -0,0 +1,33 @@
>>> +// PR c++/107773
>>> +// Verify lookup for TYPENAME_TYPE appearing to the left of the ::
>>> +// scope resolution operator correctly ignores non-types.
>>> +
>>> +struct a {
>>> +  typedef void type;
>>> +};
>>> +
>>> +struct c {
>>> +  struct b : a {
>>> +    typedef b self;
>>> +    static int m;
>>> +  };
>>> +  int b;
>>> +};
>>> +
>>> +template<class T>
>>> +void f() {
>>> +  // A TYPENAME_TYPE whose TYPE_CONTEXT is a nested TYPENAME_TYPE.
>>> +  typedef typename T::b::type type;
>>> +  // A SCOPE_REF whose first operand is a TYPENAME_TYPE.
>>> +  int m = T::b::m;
>>> +}
>>> +
>>> +template void f<c>();
>>> +
>>> +template<class T>
>>> +struct d : T::b::self {
>>> +  // A USING_DECL whose USING_DECL_SCOPE is a TYPENAME_TYPE.
>>> +  using typename T::b::type;
>>> +};
>>> +
>>> +template struct d<c>;
>>> diff --git a/gcc/testsuite/g++.dg/template/typename25a.C
>>> b/gcc/testsuite/g++.dg/template/typename25a.C
>>> new file mode 100644
>>> index 00000000000..ecb34aada34
>>> --- /dev/null
>>> +++ b/gcc/testsuite/g++.dg/template/typename25a.C
>>> @@ -0,0 +1,37 @@
>>> +// PR c++/107773
>>> +// A variadic version of typename25.C
>>> +// { dg-do compile { target c++11 } }
>>> +
>>> +struct a {
>>> +  typedef void type;
>>> +};
>>> +
>>> +struct c {
>>> +  struct b : a {
>>> +    typedef b self;
>>> +    static int m;
>>> +  };
>>> +  int b;
>>> +};
>>> +
>>> +template<class...> void sink(...);
>>> +
>>> +template<class... Ts>
>>> +void f() {
>>> +  // A TYPENAME_TYPE whose TYPE_CONTEXT is a nested TYPENAME_TYPE.
>>> +  sink<typename Ts::b::type...>();
>>> +  // A SCOPE_REF whose first operand is a TYPENAME_TYPE.
>>> +  sink(Ts::b::m...);
>>> +}
>>> +
>>> +template void f<c>();
>>> +
>>> +template<class... Ts>
>>> +struct d : Ts::b::self... {
>>> +#if __cpp_variadic_using
>>> +  // A USING_DECL whose USING_DECL_SCOPE is a TYPENAME_TYPE.
>>> +  using typename Ts::b::type...;
>>> +#endif
>>> +};
>>> +
>>> +template struct d<c>;
>>> diff --git a/gcc/testsuite/g++.dg/template/typename26.C
>>> b/gcc/testsuite/g++.dg/template/typename26.C
>>> new file mode 100644
>>> index 00000000000..4e6b764a97b
>>> --- /dev/null
>>> +++ b/gcc/testsuite/g++.dg/template/typename26.C
>>> @@ -0,0 +1,20 @@
>>> +// Example 4 from [temp.res.general]/3.
>>> +
>>> +struct A {
>>> +  struct X { };
>>> +  int X;
>>> +};
>>> +struct B {
>>> +  struct X { };
>>> +};
>>> +template<class T> void f(T t) {
>>> +  typename T::X x; // { dg-error "'int A::X', which is not a type" }
>>> +}
>>> +void foo() {
>>> +  A a;
>>> +  B b;
>>> +  f(b); // OK, T::X refers to B::X
>>> +  // { dg-bogus "" "" { target *-*-* } .-1 }
>>> +  f(a); // error: T::X refers to the data member A::X not the struct A::X
>>> +  // { dg-message "required from here" "" { target *-*-* } .-1 }
>>> +}
>>
>>
>