RFA (hash-*): PATCH for c++/68309

RFA (hash-*): PATCH for c++/68309

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 641 bytes --]

The C++ front end uses a temporary hash table to remember 
specializations of local variables during template instantiations.  In a 
nested function such as a lambda or local class member function, we need 
to retain the elements from the enclosing function's 
local_specializations table; otherwise the testcase crashes because we 
don't find a local specialization for the non-captured use of 'args' in 
the decltype.

This patch addresses that by making a copy of the enclosing 
local_specializations table if it exists; to enable that I've added copy 
constructors to hash_table and hash_map.

Tested x86_64-pc-linux-gnu.  OK for trunk?

[-- Attachment #2: 68309.patch --]
[-- Type: text/x-patch, Size: 3811 bytes --]

commit 2dfc74fa638b7255658f194c82308e8bf8ada9f9
Author: Jason Merrill <jason@redhat.com>
Date:   Thu Dec 10 15:37:50 2015 -0500

    	PR c++/68309
    gcc/
    	* hash-table.h: Add copy constructor.
    	* hash-map.h: Add copy constructor.
    gcc/cp/
    	* pt.c (instantiate_decl): Copy local_specializations for nested
    	function.

diff --git a/gcc/cp/pt.c b/gcc/cp/pt.c
index 0e087a6..5db3a32 100644
--- a/gcc/cp/pt.c
+++ b/gcc/cp/pt.c
@@ -21725,8 +21725,13 @@ instantiate_decl (tree d, int defer_ok,
 	 template from within the body of another.  */
       saved_local_specializations = local_specializations;
 
-      /* Set up the list of local specializations.  */
-      local_specializations = new hash_map<tree, tree>;
+      /* Set up the list of local specializations, copying the current
+	 list if there is one.  */
+      if (local_specializations)
+	local_specializations
+	  = new hash_map<tree, tree> (*local_specializations);
+      else
+	local_specializations = new hash_map<tree, tree>;
 
       /* Set up context.  */
       if (DECL_OMP_DECLARE_REDUCTION_P (code_pattern)
diff --git a/gcc/hash-map.h b/gcc/hash-map.h
index b83708c..9b3d1e9 100644
--- a/gcc/hash-map.h
+++ b/gcc/hash-map.h
@@ -110,6 +110,11 @@ public:
 		     bool gather_mem_stats = true CXX_MEM_STAT_INFO)
     : m_table (n, ggc, gather_mem_stats, HASH_MAP_ORIGIN PASS_MEM_STAT) {}
 
+  hash_map (const hash_map &h, bool ggc = false,
+	    bool gather_mem_stats = true CXX_MEM_STAT_INFO)
+    : m_table (h.m_table, ggc, gather_mem_stats,
+	       HASH_MAP_ORIGIN PASS_MEM_STAT) {}
+
   /* Create a hash_map in ggc memory.  */
   static hash_map *create_ggc (size_t size, bool gather_mem_stats = true
 			       CXX_MEM_STAT_INFO)
diff --git a/gcc/hash-table.h b/gcc/hash-table.h
index 192be30..d172841 100644
--- a/gcc/hash-table.h
+++ b/gcc/hash-table.h
@@ -364,6 +364,10 @@ public:
   explicit hash_table (size_t, bool ggc = false, bool gather_mem_stats = true,
 		       mem_alloc_origin origin = HASH_TABLE_ORIGIN
 		       CXX_MEM_STAT_INFO);
+  hash_table (const hash_table &, bool ggc = false,
+	      bool gather_mem_stats = true,
+	      mem_alloc_origin origin = HASH_TABLE_ORIGIN
+	      CXX_MEM_STAT_INFO);
   ~hash_table ();
 
   /* Create a hash_table in gc memory.  */
@@ -580,6 +584,27 @@ hash_table<Descriptor, Allocator>::hash_table (size_t size, bool ggc, bool
 }
 
 template<typename Descriptor, template<typename Type> class Allocator>
+hash_table<Descriptor, Allocator>::hash_table (const hash_table &h, bool ggc,
+					       bool gather_mem_stats,
+					       mem_alloc_origin origin
+					       MEM_STAT_DECL) :
+    m_n_elements (h.m_n_elements), m_n_deleted (h.m_n_deleted),
+    m_searches (0), m_collisions (0),
+    m_ggc (ggc), m_gather_mem_stats (gather_mem_stats)
+{
+  size_t size = h.m_size;
+
+  if (m_gather_mem_stats)
+    hash_table_usage.register_descriptor (this, origin, ggc
+					  FINAL_PASS_MEM_STAT);
+
+  m_entries = alloc_entries (size PASS_MEM_STAT);
+  memcpy (m_entries, h.m_entries, size * sizeof (value_type));
+  m_size = size;
+  m_size_prime_index = h.m_size_prime_index;
+}
+
+template<typename Descriptor, template<typename Type> class Allocator>
 hash_table<Descriptor, Allocator>::~hash_table ()
 {
   for (size_t i = m_size - 1; i < m_size; i--)
diff --git a/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
new file mode 100644
index 0000000..76b6b3f
--- /dev/null
+++ b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
@@ -0,0 +1,9 @@
+// PR c++/68309
+// { dg-do compile { target c++11 } }
+
+template <class... Ts> void f(Ts...);
+template <class T> T g(T);
+template <typename... Ts> void print(Ts... args) {
+  [&] { f(g<decltype(args)>(args)...); }();
+}
+int main() { print(5.2); }

Re: RFA (hash-*): PATCH for c++/68309

[Richard Biener]

On Thu, Dec 10, 2015 at 11:03 PM, Jason Merrill <jason@redhat.com> wrote:
> The C++ front end uses a temporary hash table to remember specializations of
> local variables during template instantiations.  In a nested function such
> as a lambda or local class member function, we need to retain the elements
> from the enclosing function's local_specializations table; otherwise the
> testcase crashes because we don't find a local specialization for the
> non-captured use of 'args' in the decltype.
>
> This patch addresses that by making a copy of the enclosing
> local_specializations table if it exists; to enable that I've added copy
> constructors to hash_table and hash_map.
>
> Tested x86_64-pc-linux-gnu.  OK for trunk?

I don't think  you can copy the elements with memcpy they may be C++ classes
that are not copyable.  Also watch out for the bool gather_mem_stats = true
to bool gather_mem_stats = GATHER_STATISTICS change if that crosses your
change.

I also think copying hash tables should be discouraged ;)  I wonder if you
can get around the copying by adding a generation count (to easily "backtrack")
instead.

Richard.

Re: RFA (hash-*): PATCH for c++/68309

[Jason Merrill]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]

On 12/11/2015 05:10 AM, Richard Biener wrote:
> On Thu, Dec 10, 2015 at 11:03 PM, Jason Merrill <jason@redhat.com> wrote:
>> The C++ front end uses a temporary hash table to remember specializations of
>> local variables during template instantiations.  In a nested function such
>> as a lambda or local class member function, we need to retain the elements
>> from the enclosing function's local_specializations table; otherwise the
>> testcase crashes because we don't find a local specialization for the
>> non-captured use of 'args' in the decltype.
>>
>> This patch addresses that by making a copy of the enclosing
>> local_specializations table if it exists; to enable that I've added copy
>> constructors to hash_table and hash_map.
>>
>> Tested x86_64-pc-linux-gnu.  OK for trunk?
>
> I don't think  you can copy the elements with memcpy they may be C++ classes
> that are not copyable.

True.  Fixed thus:

> +  for (size_t i = 0; i < size; ++i)
> +    {
> +      value_type &entry = h.m_entries[i];
> +      if (is_deleted (entry))
> +       mark_deleted (m_entries[i]);
> +      else if (!is_empty (entry))
> +       m_entries[i] = entry;
> +    }

> Also watch out for the bool gather_mem_stats = true
> to bool gather_mem_stats = GATHER_STATISTICS change if that crosses your
> change.

OK.

> I also think copying hash tables should be discouraged ;)  I wonder if you
> can get around the copying by adding a generation count (to easily "backtrack")
> instead.

I considered having a chain of tables to check, to handle generations, 
but I figured that the tables in question were small enough (only 
containing local variables for a single function) that a simple copy was 
reasonable.

Jason


[-- Attachment #2: 68309.patch --]
[-- Type: text/x-patch, Size: 3956 bytes --]

commit ae8c23201f9a057516f081d4d9fde1fb3281153f
Author: Jason Merrill <jason@redhat.com>
Date:   Thu Dec 10 15:37:50 2015 -0500

    	PR c++/68309
    gcc/
    	* hash-table.h: Add copy constructor.
    	* hash-map.h: Add copy constructor.
    gcc/cp/
    	* pt.c (instantiate_decl): Copy local_specializations for nested
    	function.

diff --git a/gcc/cp/pt.c b/gcc/cp/pt.c
index 60cc94c..a45e6df 100644
--- a/gcc/cp/pt.c
+++ b/gcc/cp/pt.c
@@ -21725,8 +21725,13 @@ instantiate_decl (tree d, int defer_ok,
 	 template from within the body of another.  */
       saved_local_specializations = local_specializations;
 
-      /* Set up the list of local specializations.  */
-      local_specializations = new hash_map<tree, tree>;
+      /* Set up the list of local specializations, copying the current
+	 list if there is one.  */
+      if (local_specializations)
+	local_specializations
+	  = new hash_map<tree, tree> (*local_specializations);
+      else
+	local_specializations = new hash_map<tree, tree>;
 
       /* Set up context.  */
       if (DECL_OMP_DECLARE_REDUCTION_P (code_pattern)
diff --git a/gcc/hash-map.h b/gcc/hash-map.h
index b83708c..9b3d1e9 100644
--- a/gcc/hash-map.h
+++ b/gcc/hash-map.h
@@ -110,6 +110,11 @@ public:
 		     bool gather_mem_stats = true CXX_MEM_STAT_INFO)
     : m_table (n, ggc, gather_mem_stats, HASH_MAP_ORIGIN PASS_MEM_STAT) {}
 
+  hash_map (const hash_map &h, bool ggc = false,
+	    bool gather_mem_stats = true CXX_MEM_STAT_INFO)
+    : m_table (h.m_table, ggc, gather_mem_stats,
+	       HASH_MAP_ORIGIN PASS_MEM_STAT) {}
+
   /* Create a hash_map in ggc memory.  */
   static hash_map *create_ggc (size_t size, bool gather_mem_stats = true
 			       CXX_MEM_STAT_INFO)
diff --git a/gcc/hash-table.h b/gcc/hash-table.h
index 192be30..e3f9dbd 100644
--- a/gcc/hash-table.h
+++ b/gcc/hash-table.h
@@ -364,6 +364,10 @@ public:
   explicit hash_table (size_t, bool ggc = false, bool gather_mem_stats = true,
 		       mem_alloc_origin origin = HASH_TABLE_ORIGIN
 		       CXX_MEM_STAT_INFO);
+  hash_table (const hash_table &, bool ggc = false,
+	      bool gather_mem_stats = true,
+	      mem_alloc_origin origin = HASH_TABLE_ORIGIN
+	      CXX_MEM_STAT_INFO);
   ~hash_table ();
 
   /* Create a hash_table in gc memory.  */
@@ -580,6 +584,34 @@ hash_table<Descriptor, Allocator>::hash_table (size_t size, bool ggc, bool
 }
 
 template<typename Descriptor, template<typename Type> class Allocator>
+hash_table<Descriptor, Allocator>::hash_table (const hash_table &h, bool ggc,
+					       bool gather_mem_stats,
+					       mem_alloc_origin origin
+					       MEM_STAT_DECL) :
+  m_n_elements (h.m_n_elements), m_n_deleted (h.m_n_deleted),
+  m_searches (0), m_collisions (0), m_ggc (ggc),
+  m_gather_mem_stats (gather_mem_stats)
+{
+  size_t size = h.m_size;
+
+  if (m_gather_mem_stats)
+    hash_table_usage.register_descriptor (this, origin, ggc
+					  FINAL_PASS_MEM_STAT);
+
+  m_entries = alloc_entries (size PASS_MEM_STAT);
+  for (size_t i = 0; i < size; ++i)
+    {
+      value_type &entry = h.m_entries[i];
+      if (is_deleted (entry))
+	mark_deleted (m_entries[i]);
+      else if (!is_empty (entry))
+	m_entries[i] = entry;
+    }
+  m_size = size;
+  m_size_prime_index = h.m_size_prime_index;
+}
+
+template<typename Descriptor, template<typename Type> class Allocator>
 hash_table<Descriptor, Allocator>::~hash_table ()
 {
   for (size_t i = m_size - 1; i < m_size; i--)
diff --git a/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
new file mode 100644
index 0000000..76b6b3f
--- /dev/null
+++ b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
@@ -0,0 +1,9 @@
+// PR c++/68309
+// { dg-do compile { target c++11 } }
+
+template <class... Ts> void f(Ts...);
+template <class T> T g(T);
+template <typename... Ts> void print(Ts... args) {
+  [&] { f(g<decltype(args)>(args)...); }();
+}
+int main() { print(5.2); }

Re: RFA (hash-*): PATCH for c++/68309

[Richard Biener]

On Fri, Dec 11, 2015 at 8:05 PM, Jason Merrill <jason@redhat.com> wrote:
> On 12/11/2015 05:10 AM, Richard Biener wrote:
>>
>> On Thu, Dec 10, 2015 at 11:03 PM, Jason Merrill <jason@redhat.com> wrote:
>>>
>>> The C++ front end uses a temporary hash table to remember specializations
>>> of
>>> local variables during template instantiations.  In a nested function
>>> such
>>> as a lambda or local class member function, we need to retain the
>>> elements
>>> from the enclosing function's local_specializations table; otherwise the
>>> testcase crashes because we don't find a local specialization for the
>>> non-captured use of 'args' in the decltype.
>>>
>>> This patch addresses that by making a copy of the enclosing
>>> local_specializations table if it exists; to enable that I've added copy
>>> constructors to hash_table and hash_map.
>>>
>>> Tested x86_64-pc-linux-gnu.  OK for trunk?
>>
>>
>> I don't think  you can copy the elements with memcpy they may be C++
>> classes
>> that are not copyable.
>
>
> True.  Fixed thus:
>
>> +  for (size_t i = 0; i < size; ++i)
>> +    {
>> +      value_type &entry = h.m_entries[i];
>> +      if (is_deleted (entry))
>> +       mark_deleted (m_entries[i]);
>> +      else if (!is_empty (entry))
>> +       m_entries[i] = entry;
>> +    }
>
>
>> Also watch out for the bool gather_mem_stats = true
>> to bool gather_mem_stats = GATHER_STATISTICS change if that crosses your
>> change.
>
>
> OK.
>
>> I also think copying hash tables should be discouraged ;)  I wonder if you
>> can get around the copying by adding a generation count (to easily
>> "backtrack")
>> instead.
>
>
> I considered having a chain of tables to check, to handle generations, but I
> figured that the tables in question were small enough (only containing local
> variables for a single function) that a simple copy was reasonable.

Looks good to me now.  Needs adjustment to use GATHER_STATISTICS
as default-arg for gather_mem_stats now.

Thanks,
Richard.

> Jason
>

Re: RFA (hash-*): PATCH for c++/68309

[Trevor Saunders]

>  
> +  hash_map (const hash_map &h, bool ggc = false,
> +	    bool gather_mem_stats = true CXX_MEM_STAT_INFO)

sorry about the late response, but wouldn't it be better to make this
and the hash_table constructor explicit?  Its probably less important
than other constructors, but is there a reasonable use for taking or
return them by value?

Trev


> +    : m_table (h.m_table, ggc, gather_mem_stats,
> +	       HASH_MAP_ORIGIN PASS_MEM_STAT) {}
> +
>    /* Create a hash_map in ggc memory.  */
>    static hash_map *create_ggc (size_t size, bool gather_mem_stats = true
>  			       CXX_MEM_STAT_INFO)
> diff --git a/gcc/hash-table.h b/gcc/hash-table.h
> index 192be30..d172841 100644
> --- a/gcc/hash-table.h
> +++ b/gcc/hash-table.h
> @@ -364,6 +364,10 @@ public:
>    explicit hash_table (size_t, bool ggc = false, bool gather_mem_stats = true,
>  		       mem_alloc_origin origin = HASH_TABLE_ORIGIN
>  		       CXX_MEM_STAT_INFO);
> +  hash_table (const hash_table &, bool ggc = false,
> +	      bool gather_mem_stats = true,
> +	      mem_alloc_origin origin = HASH_TABLE_ORIGIN
> +	      CXX_MEM_STAT_INFO);
>    ~hash_table ();
>  
>    /* Create a hash_table in gc memory.  */
> @@ -580,6 +584,27 @@ hash_table<Descriptor, Allocator>::hash_table (size_t size, bool ggc, bool
>  }
>  
>  template<typename Descriptor, template<typename Type> class Allocator>
> +hash_table<Descriptor, Allocator>::hash_table (const hash_table &h, bool ggc,
> +					       bool gather_mem_stats,
> +					       mem_alloc_origin origin
> +					       MEM_STAT_DECL) :
> +    m_n_elements (h.m_n_elements), m_n_deleted (h.m_n_deleted),
> +    m_searches (0), m_collisions (0),
> +    m_ggc (ggc), m_gather_mem_stats (gather_mem_stats)
> +{
> +  size_t size = h.m_size;
> +
> +  if (m_gather_mem_stats)
> +    hash_table_usage.register_descriptor (this, origin, ggc
> +					  FINAL_PASS_MEM_STAT);
> +
> +  m_entries = alloc_entries (size PASS_MEM_STAT);
> +  memcpy (m_entries, h.m_entries, size * sizeof (value_type));
> +  m_size = size;
> +  m_size_prime_index = h.m_size_prime_index;
> +}
> +
> +template<typename Descriptor, template<typename Type> class Allocator>
>  hash_table<Descriptor, Allocator>::~hash_table ()
>  {
>    for (size_t i = m_size - 1; i < m_size; i--)
> diff --git a/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
> new file mode 100644
> index 0000000..76b6b3f
> --- /dev/null
> +++ b/gcc/testsuite/g++.dg/cpp0x/lambda/lambda-variadic3.C
> @@ -0,0 +1,9 @@
> +// PR c++/68309
> +// { dg-do compile { target c++11 } }
> +
> +template <class... Ts> void f(Ts...);
> +template <class T> T g(T);
> +template <typename... Ts> void print(Ts... args) {
> +  [&] { f(g<decltype(args)>(args)...); }();
> +}
> +int main() { print(5.2); }

Re: RFA (hash-*): PATCH for c++/68309

[Jason Merrill]

On 12/14/2015 07:49 PM, Trevor Saunders wrote:
>>
>> +  hash_map (const hash_map &h, bool ggc = false,
>> +	    bool gather_mem_stats = true CXX_MEM_STAT_INFO)
>
> sorry about the late response, but wouldn't it be better to make this
> and the hash_table constructor explicit?  Its probably less important
> than other constructors, but is there a reasonable use for taking or
> return them by value?

Makes sense; done.

Jason