[gcc/devel/rust/master] macro-substitute: Do not substitute non-repetition fragments in sub-maps

[gcc/devel/rust/master] macro-substitute: Do not substitute non-repetition fragments in sub-maps

[Thomas Schwinge]

https://gcc.gnu.org/g:a498b2c5d6d64c40955b3c13fd6ca8dc72a7bd67

commit a498b2c5d6d64c40955b3c13fd6ca8dc72a7bd67
Author: Arthur Cohen <arthur.cohen@embecosm.com>
Date:   Tue Mar 1 15:15:07 2022 +0100

    macro-substitute: Do not substitute non-repetition fragments in sub-maps
    
    When creating a sub-map for repetitions, we need to be weary of not
    accessing matched-fragments beyond the vector's size. For example, with
    the following *fragments*
    
    { "e": [1], "es": [2, 3, 10]},
    
    the sub-maps we want to create are the following:
    
    { "e": [1], "es": [2]},
    { "e": [1], "es": [3]},
    { "e": [1], "es": [10]},
    
    Up until this point however, we were trying to access the second index
    for the "e" metavar when creating the second submap, then the third, and
      so on... which is obviously outside of the vector's bounds.
    
    We can simply check if the metavar only has one match and expand that
    one in that case. We still need to work on checking that multiple
    metavars in the same transcriber repetition pattern have the same amount
    of matched fragments (ie the original vectors of matches in the
    original map have the same size)

Diff:
---
 gcc/rust/expand/rust-macro-substitute-ctx.cc | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
index 8542614fb60..3d07b484134 100644
--- a/gcc/rust/expand/rust-macro-substitute-ctx.cc
+++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
@@ -17,8 +17,8 @@ SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
   else
     {
       // Replace
-      // We only care about the vector when expanding repetitions. Just access
-      // the first element of the vector.
+      // We only care about the vector when expanding repetitions.
+      // Just access the first element of the vector.
       // FIXME: Clean this up so it makes more sense
       auto &frag = it->second[0];
       for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
@@ -103,7 +103,13 @@ SubstituteCtx::substitute_repetition (size_t pattern_start, size_t pattern_end)
       for (auto &kv_match : fragments)
 	{
 	  std::vector<MatchedFragment> sub_vec;
-	  sub_vec.emplace_back (kv_match.second[i]);
+
+	  // FIXME: Hack: If a fragment is not repeated, how does it fit in the
+	  // submap? Do we really want to expand it? Is this normal behavior?
+	  if (kv_match.second.size () == 1)
+	    sub_vec.emplace_back (kv_match.second[0]);
+	  else
+	    sub_vec.emplace_back (kv_match.second[i]);
 
 	  sub_map.insert ({kv_match.first, sub_vec});
 	}
@@ -177,6 +183,7 @@ std::vector<std::unique_ptr<AST::Token>>
 SubstituteCtx::substitute_tokens ()
 {
   std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
+  rust_debug ("expanding tokens");
 
   for (size_t i = 0; i < macro.size (); i++)
     {

[gcc/devel/rust/master] macro-substitute: Do not substitute non-repetition fragments in sub-maps

[Thomas Schwinge]

https://gcc.gnu.org/g:c8499831fd6c4adba5e72a0e042859a7c5a15893

commit c8499831fd6c4adba5e72a0e042859a7c5a15893
Author: Arthur Cohen <arthur.cohen@embecosm.com>
Date:   Tue Mar 1 15:15:07 2022 +0100

    macro-substitute: Do not substitute non-repetition fragments in sub-maps
    
    When creating a sub-map for repetitions, we need to be weary of not
    accessing matched-fragments beyond the vector's size. For example, with
    the following *fragments*
    
    { "e": [1], "es": [2, 3, 10]},
    
    the sub-maps we want to create are the following:
    
    { "e": [1], "es": [2]},
    { "e": [1], "es": [3]},
    { "e": [1], "es": [10]},
    
    Up until this point however, we were trying to access the second index
    for the "e" metavar when creating the second submap, then the third, and
      so on... which is obviously outside of the vector's bounds.
    
    We can simply check if the metavar only has one match and expand that
    one in that case. We still need to work on checking that multiple
    metavars in the same transcriber repetition pattern have the same amount
    of matched fragments (ie the original vectors of matches in the
    original map have the same size)

Diff:
---
 gcc/rust/expand/rust-macro-substitute-ctx.cc | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gcc/rust/expand/rust-macro-substitute-ctx.cc b/gcc/rust/expand/rust-macro-substitute-ctx.cc
index 8542614fb60..3d07b484134 100644
--- a/gcc/rust/expand/rust-macro-substitute-ctx.cc
+++ b/gcc/rust/expand/rust-macro-substitute-ctx.cc
@@ -17,8 +17,8 @@ SubstituteCtx::substitute_metavar (std::unique_ptr<AST::Token> &metavar)
   else
     {
       // Replace
-      // We only care about the vector when expanding repetitions. Just access
-      // the first element of the vector.
+      // We only care about the vector when expanding repetitions.
+      // Just access the first element of the vector.
       // FIXME: Clean this up so it makes more sense
       auto &frag = it->second[0];
       for (size_t offs = frag.token_offset_begin; offs < frag.token_offset_end;
@@ -103,7 +103,13 @@ SubstituteCtx::substitute_repetition (size_t pattern_start, size_t pattern_end)
       for (auto &kv_match : fragments)
 	{
 	  std::vector<MatchedFragment> sub_vec;
-	  sub_vec.emplace_back (kv_match.second[i]);
+
+	  // FIXME: Hack: If a fragment is not repeated, how does it fit in the
+	  // submap? Do we really want to expand it? Is this normal behavior?
+	  if (kv_match.second.size () == 1)
+	    sub_vec.emplace_back (kv_match.second[0]);
+	  else
+	    sub_vec.emplace_back (kv_match.second[i]);
 
 	  sub_map.insert ({kv_match.first, sub_vec});
 	}
@@ -177,6 +183,7 @@ std::vector<std::unique_ptr<AST::Token>>
 SubstituteCtx::substitute_tokens ()
 {
   std::vector<std::unique_ptr<AST::Token>> replaced_tokens;
+  rust_debug ("expanding tokens");
 
   for (size_t i = 0; i < macro.size (); i++)
     {