[Bug target/110059] New: When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[Bug target/110059] New: When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[d_vampile at 163 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

            Bug ID: 110059
           Summary: When SPEC is used to test the GCC (10.3.1), the test
                    result of subitem 548 fluctuates abnormally.
           Product: gcc
           Version: 10.3.1
            Status: UNCONFIRMED
          Severity: normal
          Priority: P3
         Component: target
          Assignee: unassigned at gcc dot gnu.org
          Reporter: d_vampile at 163 dot com
  Target Milestone: ---

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[d_vampile at 163 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

d_vampile <d_vampile at 163 dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Version|10.3.1                      |10.3.0
             Target|                            |aarch64
                URL|                            |https://gcc.gnu.org/piperma
                   |                            |il/gcc-patches/2019-Septemb
                   |                            |er/530304.html

--- Comment #1 from d_vampile <d_vampile at 163 dot com> ---
When the SPEC is used to test the GCC (10.3.0) performance, the test result of
subitem 548 fluctuates abnormally. 
Trace the cause of the exception. I analyze that the commit(r276140) is faulty.
Revert this commit(r276140), and the fluctuation of spec 548 test results for
gcc 10.3.0 is significantly reduced. The following is the test result.

GCC 10.3.0 test result:

Benchmarks          copies   runtime   ratio
548.exchange2_r     128        553        607  
548.exchange2_r     128        512        654 
548.exchange2_r     128        513        654  
548.exchange2_r     128        553        607  
548.exchange2_r     128        553        607  
548.exchange2_r     128        514        652  
548.exchange2_r     128        515        652  
548.exchange2_r     128        553        607  
548.exchange2_r     128        553        607  
548.exchange2_r     128        508        660  

Test result after revert this commit:

Benchmarks          copies   runtime   ratio
548.exchange2_r     128        520        645  
548.exchange2_r     128        680        494  
548.exchange2_r     128        527        637  
548.exchange2_r     128        679        494  
548.exchange2_r     128        524        640  
548.exchange2_r     128        679        494  
548.exchange2_r     128        524        640  
548.exchange2_r     128        524        640  
548.exchange2_r     128        520        645  
548.exchange2_r     128        521        644

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[pinskia at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

Andrew Pinski <pinskia at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
     Ever confirmed|0                           |1
             Status|UNCONFIRMED                 |WAITING
   Last reconfirmed|                            |2023-05-31

--- Comment #2 from Andrew Pinski <pinskia at gcc dot gnu.org> ---
What core is this on?
Second, if the benchmark is this noisy, then the issue is not the code that is
generated but rather how the code is executed.

Can you pinpoint the code that changes because of this change?

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[ktkachov at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

ktkachov at gcc dot gnu.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ktkachov at gcc dot gnu.org

--- Comment #3 from ktkachov at gcc dot gnu.org ---
548.exchange2_r was improved in GCC 12 after PR98782 was fixed. I'd suggest you
try out a later version of GCC

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[jakub at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

--- Comment #4 from Jakub Jelinek <jakub at gcc dot gnu.org> ---
I'd like to point out that doing performance testing for GCC 10 at this point
isn't a very good idea, GCC 10 is going out of support within a month or two,
and while some important wrong-code generation bugs can be still fixed there,
it is very unlikely any performance related bugs would be fixed there.  Of
course, if you find a performance problem with GCC 13 or even better trunk
compared to GCC 9 or older and then bisect it to a particular change, it is
helpful.

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

--- Comment #5 from Richard Biener <rguenth at gcc dot gnu.org> ---
The "after revert" numbers fluctuate more, so not sure what you are after.

[Bug target/110059] When SPEC is used to test the GCC (10.3.1), the test result of subitem 548 fluctuates abnormally.

[d_vampile at 163 dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=110059

d_vampile <d_vampile at 163 dot com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|---                         |FIXED