GSoC

GSoC

[shivam tiwari]

I have New Project  Idea for GSOC 2020 Where Can I discuss about  New
Project idea.OR Can I share my project idea Proposal on this mail.

Re: GSoC

[Richard Biener]

On Fri, Feb 21, 2020 at 9:18 AM shivam tiwari
<shivam.tiwari00021@gmail.com> wrote:
>
> I have New Project  Idea for GSOC 2020 Where Can I discuss about  New
> Project idea.OR Can I share my project idea Proposal on this mail.

You can share your proposal on this mailing-list, the existing project
ideas list
is not meant to be exhaustive (but ideas there are likely to have somebody
willing to mentor it).

Richard.

GSoC

[koushiki khobare]

[-- Attachment #1: Type: text/plain, Size: 1047 bytes --]

Respected sir,
              I am Koushiki Khobare from India and currently a first year student(Computer Science Student) studying in Pune Institute of Computer Technology, Pune, Maharashtra. Recently I heard about GSoC and got very excited to explore it. I will be very much thankful to you if you provide me some guidance and help me to explore more. I have learnt C programming language. So I visited your projects and they were all amazing and quite interesting to work upon. One of them I am thinking to explore is “Rust Front End”.So I will be grateful to work with all my mentors- Mr.Pierre-Emmaunel Patry, Mr. Philip Herron, Mr.Arthur Cohen, all being excellent in their works. I will be so much of grateful if I get to know what actual skills do you expect from me apart from mentioned on website and guide me with what all do I need to do in my proposal. I request you to provide me some guidance with my proposal.

Yours faithfully,
Koushiki Khobare

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows

Re: GSoC

[Martin Jambor]

On Sat, Mar 23 2024, koushiki khobare via Gcc wrote:
> Respected sir,
>
> I am Koushiki Khobare from India and currently a first year
> student(Computer Science Student) studying in Pune Institute of
> Computer Technology, Pune, Maharashtra. Recently I heard about GSoC
> and got very excited to explore it. I will be very much thankful to
> you if you provide me some guidance and help me to explore more. I
> have learnt C programming language. So I visited your projects and
> they were all amazing and quite interesting to work upon.

We are delighted you found contributing to GCC interesting.  

Please note that apart from C (and C++!) coding skills, most projects
require some rudimentary theoretical background in compilers.

> One of them
> I am thinking to explore is “Rust Front End”.So I will be grateful to
> work with all my mentors- Mr.Pierre-Emmaunel Patry, Mr. Philip Herron,
> Mr.Arthur Cohen, all being excellent in their works.

Please note that Rust-GCC projects are a bit special in the sense that
they are often discussed primarily on Zulip of the gcc-rust team:

https://gcc-rust.zulipchat.com/

So you may want to reach out to them there as well.

> I will be so much
> of grateful if I get to know what actual skills do you expect from me
> apart from mentioned on website and guide me with what all do I need
> to do in my proposal.

The skills mentioned on the website
(i.e. https://gcc.gnu.org/wiki/SummerOfCode ) should be quite
sufficient.  I can only advise that you look at the "Before you apply"
section and take (most of) the steps described there.

> I request you to provide me some guidance with my proposal.

You need to select a particular project and make at least a few first
steps (again, at look at those described at
https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply ) yourself.  Then
we can help you to polish things up.

Good luck!

Martin Jambor

GSoC

[Abhinav Gupta]

Dear GCC Community,
    I hope this email finds you well. My name is Abhinav Gupta. I am a
3rd-year student at IIT Tirupati pursuing a bachelor's degree in
computer science and engineering. I am writing to express my interest
in contributing to the GCC project under GSoC.
I am interested in two projects - Offloading to a separate process on
the same host, I am already working in parallel computing,
specifically parallelising tensor algorithms using various techniques
as part of my research project at IIT Tirupati. Although this is not
directly related to compilers, I will be able to get going with the
project quickly.
The second project is Rust Front-End - both the BIR location support
and rustc testsuite adapter are of interest to me, having worked on
compiler front ends as part of my college's compiler design course
combined with my experience in working with large libraries written in
C++ (such as CTF) I believe that these two projects are something that
I can do.

Proposed Timeline:
I can start working as soon as my end-semester exams finish, i.e. 9th
May 2024, and continue to work for however long it requires me to
complete the project.
Week 1-2 -> Knowing the existing code and understanding how it works.
Week 3-8 -> Working on the implementation of whichever project we
decide to move forward with
Week 9-12 -> Testing and creating understandable documentation for the same.

This is a very rough timeline, and I will refine it further as we
discuss the project idea. This email is more of a call for guidance
than an application, and I would appreciate any feedback you give me.

 Best Regards,
Abhinav Gupta,
ph - +917289953000

Re: GSoC

[Martin Jambor]

Hello,

On Wed, Mar 06 2024, Abhinav Gupta via Gcc wrote:
> Dear GCC Community,
>     I hope this email finds you well. My name is Abhinav Gupta. I am a
> 3rd-year student at IIT Tirupati pursuing a bachelor's degree in
> computer science and engineering. I am writing to express my interest
> in contributing to the GCC project under GSoC.

We are very happy that you find contributing to GCC interesting.

> I am interested in two projects - Offloading to a separate process on
> the same host, I am already working in parallel computing,
> specifically parallelising tensor algorithms using various techniques
> as part of my research project at IIT Tirupati. Although this is not
> directly related to compilers, I will be able to get going with the
> project quickly.

I'd personally very much like to see this project implemented.  There is
a lot of information on offloading at
https://gcc.gnu.org/wiki/Offloading

To give you a bit more context where the idea comes from, it was first
thought of in email thread starting with
https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603984.html and
the patch that was "scrubbed" from the email archive eventually became
commit
https://gcc.gnu.org/cgit/gcc/commit/?id=e4cba49413ca429dc82f6aa2e88129ecb3fdd943

I hope these pointers will further help help you find out where to look
when planning the project.  If you need more help, please feel free to
ask (I'm CCing Thomas who then is perhaps best placed to answer).

> The second project is Rust Front-End - both the BIR location support
> and rustc testsuite adapter are of interest to me,

Please note that Rust-GCC projects are a bit special in the sense that
they are often discussed primarily on Zulip of the gcc-rust team:

https://gcc-rust.zulipchat.com/

So you may want to reach out to them there as well.

> having worked on
> compiler front ends as part of my college's compiler design course
> combined with my experience in working with large libraries written in
> C++ (such as CTF) I believe that these two projects are something that
> I can do.

You seem to be quite ready!

>
> Proposed Timeline:
> I can start working as soon as my end-semester exams finish, i.e. 9th
> May 2024, and continue to work for however long it requires me to
> complete the project.
> Week 1-2 -> Knowing the existing code and understanding how it works.

Right, but please try to do a bit of this, at least on the high level,
also now when preparing the proposal.  There will be lots to learn in
the first weeks even so.  Mainly because...

> Week 3-8 -> Working on the implementation of whichever project we
> decide to move forward with
> Week 9-12 -> Testing and creating understandable documentation for the same.
>
> This is a very rough timeline,

...eventually the milestones in the application will have to be more
specific, mainly to demonstrate you have basic understanding of the
proposed project.

> and I will refine it further as we
> discuss the project idea. This email is more of a call for guidance
> than an application, and I would appreciate any feedback you give me.

This is a very good start.

Good luck!

Martin

Re: GSoC

[Martin Jambor]

Hello,

On Tue, Mar 12 2024, Abhinav Gupta wrote:
> Hi! Thank you for replying to my request!
>     I looked at all the links you provided in the reply and read the
> paper cited on the GCC page for GSoC. I also looked into the rust
> frontend project during this time, and the Offloading project
> interests me more, so I will focus solely on it in the remaining seven
> days before the deadline for GSoC application submission.

AFAIU, in five days (from now) the application period *opens*, the
deadline is 2nd April.  Still it is good idea not to lose any time.

> Are there other resources I can look at to better understand the whole
> process? Reading the git commit on the website is proving to be very
> slow. I think the git commit about Intel MIC would be like a
> "template" in loose terms to implement the host-ISA mode at least, but
> for the libgomp plugin, I need a better understanding.

You need to understand how OpenMP programs are internally represented.

Look at the following (hopefully correct, at least as long as you try it
on a system without any Offloading enabled) simple testcase for OpenMP
target construct:

----------------------------------------------------------------------
#include <stdio.h>

volatile int v = 1;

int main (int argc, char **argv)
{
  int i = v;

#pragma omp target map(to:i)
  {
    printf ("OpenMP target hello world, i is: %i\n", i);
  }

  return 0;
}
----------------------------------------------------------------------

and compile it with:

  gcc -fopenmp omptgt-hello.c -O2 -fdump-tree-optimized

and then look at the generated optimized dump.  This should give you an
idea how OpenMP regions are internally represented (as outlined
functions) and how calls into libgomp, the run-time library doing a lot
of the magic, look like.

You can try to do more similar exercises with more OpenMP testcase which
you can find in sub-directory libgomp/testsuite/libgomp.c of in the GCC
repository.

And then you should perhaps have a look into libgomp itself (you'll find
it in libgomp sub-directory) how GOMP_target_ext is implemented - though
don't worry if you don't understand many of the details, it is complex
stuff.  At this point hopefully more of the contents of the Offloading
wiki page will make sense.

Again, ask on the mailing list if you have any specific questions.

Good luck,

Martin

> Thanking you
> Abhinav
>
>
> On Thu, 7 Mar 2024 at 20:37, Martin Jambor <mjambor@suse.cz> wrote:
>>
>> Hello,
>>
>> On Wed, Mar 06 2024, Abhinav Gupta via Gcc wrote:
>> > Dear GCC Community,
>> >     I hope this email finds you well. My name is Abhinav Gupta. I am a
>> > 3rd-year student at IIT Tirupati pursuing a bachelor's degree in
>> > computer science and engineering. I am writing to express my interest
>> > in contributing to the GCC project under GSoC.
>>
>> We are very happy that you find contributing to GCC interesting.
>>
>> > I am interested in two projects - Offloading to a separate process on
>> > the same host, I am already working in parallel computing,
>> > specifically parallelising tensor algorithms using various techniques
>> > as part of my research project at IIT Tirupati. Although this is not
>> > directly related to compilers, I will be able to get going with the
>> > project quickly.
>>
>> I'd personally very much like to see this project implemented.  There is
>> a lot of information on offloading at
>> https://gcc.gnu.org/wiki/Offloading
>>
>> To give you a bit more context where the idea comes from, it was first
>> thought of in email thread starting with
>> https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603984.html and
>> the patch that was "scrubbed" from the email archive eventually became
>> commit
>> https://gcc.gnu.org/cgit/gcc/commit/?id=e4cba49413ca429dc82f6aa2e88129ecb3fdd943
>>
>> I hope these pointers will further help help you find out where to look
>> when planning the project.  If you need more help, please feel free to
>> ask (I'm CCing Thomas who then is perhaps best placed to answer).
>>
>> > The second project is Rust Front-End - both the BIR location support
>> > and rustc testsuite adapter are of interest to me,
>>
>> Please note that Rust-GCC projects are a bit special in the sense that
>> they are often discussed primarily on Zulip of the gcc-rust team:
>>
>> https://gcc-rust.zulipchat.com/
>>
>> So you may want to reach out to them there as well.
>>
>> > having worked on
>> > compiler front ends as part of my college's compiler design course
>> > combined with my experience in working with large libraries written in
>> > C++ (such as CTF) I believe that these two projects are something that
>> > I can do.
>>
>> You seem to be quite ready!
>>
>> >
>> > Proposed Timeline:
>> > I can start working as soon as my end-semester exams finish, i.e. 9th
>> > May 2024, and continue to work for however long it requires me to
>> > complete the project.
>> > Week 1-2 -> Knowing the existing code and understanding how it works.
>>
>> Right, but please try to do a bit of this, at least on the high level,
>> also now when preparing the proposal.  There will be lots to learn in
>> the first weeks even so.  Mainly because...
>>
>> > Week 3-8 -> Working on the implementation of whichever project we
>> > decide to move forward with
>> > Week 9-12 -> Testing and creating understandable documentation for the same.
>> >
>> > This is a very rough timeline,
>>
>> ...eventually the milestones in the application will have to be more
>> specific, mainly to demonstrate you have basic understanding of the
>> proposed project.
>>
>> > and I will refine it further as we
>> > discuss the project idea. This email is more of a call for guidance
>> > than an application, and I would appreciate any feedback you give me.
>>
>> This is a very good start.
>>
>> Good luck!
>>
>> Martin

Re: GSoC

[Thomas Schwinge]

Hi Abhinav!

Thanks for your interest in contributing to GCC, and thanks to Martin for
all the information you already provided.  Just a bit more, to get you
started on developing a proper project proposal:

On 2024-03-13T14:54:52+0100, Martin Jambor <mjambor@suse.cz> wrote:
> On Tue, Mar 12 2024, Abhinav Gupta wrote:
>>     I looked at all the links you provided in the reply and read the
>> paper cited on the GCC page for GSoC. I also looked into the rust
>> frontend project during this time, and the Offloading project
>> interests me more, so I will focus solely on it in the remaining seven
>> days before the deadline for GSoC application submission.
>
> AFAIU, in five days (from now) the application period *opens*, the
> deadline is 2nd April.  Still it is good idea not to lose any time.

Indeed, no rush yet.  :-)

>> Are there other resources I can look at to better understand the whole
>> process?

At some point, no too late, you should spend some time on learning how to
build GCC, and run the test suite.
<https://gcc.gnu.org/wiki/SummerOfCode> and
<https://gcc.gnu.org/wiki/#Getting_Started_with_GCC_Development> have
some pointers to get started.  If you have specific questions, we're
happy to help, of course.

>> Reading the git commit on the website is proving to be very
>> slow.

Yes, that's not going to be necessary.

>> I think the git commit about Intel MIC would be like a
>> "template" in loose terms

Right, that's in fact a very good description.  The idea here is not to
bring back the whole Intel MIC emulator, but something very much simpler.

>> to implement the host-ISA mode at least, but
>> for the libgomp plugin, I need a better understanding.

Find existing libgomp plugins as 'libgomp/plugin/plugin-*.c'.  The
'GOMP_OFFLOAD_[...]' functions implement the entry points, the offloading
plugin API.  Actually also the very simple 'libgomp/oacc-host.c' should
be helpful.  That's essentially the API you need to care about (for
OpenACC; but OpenMP 'target' also won't require much more, for a start).

Make some thoughts (or actual experiments) about how we could
use/implement a separate host process for code offloading.

And otherwise, as Martin said:

> You need to understand how OpenMP programs are internally represented.
>
> Look at the following (hopefully correct, at least as long as you try it
> on a system without any Offloading enabled) simple testcase for OpenMP
> target construct:
>
> ----------------------------------------------------------------------
> #include <stdio.h>
>
> volatile int v = 1;
>
> int main (int argc, char **argv)
> {
>   int i = v;
>
> #pragma omp target map(to:i)
>   {
>     printf ("OpenMP target hello world, i is: %i\n", i);
>   }
>
>   return 0;
> }
> ----------------------------------------------------------------------
>
> and compile it with:
>
>   gcc -fopenmp omptgt-hello.c -O2 -fdump-tree-optimized
>
> and then look at the generated optimized dump.  This should give you an
> idea how OpenMP regions are internally represented (as outlined
> functions) and how calls into libgomp, the run-time library doing a lot
> of the magic, look like.
>
> You can try to do more similar exercises with more OpenMP testcase which
> you can find in sub-directory libgomp/testsuite/libgomp.c of in the GCC
> repository.
>
> And then you should perhaps have a look into libgomp itself (you'll find
> it in libgomp sub-directory) how GOMP_target_ext is implemented - though
> don't worry if you don't understand many of the details, it is complex
> stuff.  At this point hopefully more of the contents of the Offloading
> wiki page will make sense.
>
> Again, ask on the mailing list if you have any specific questions.


Grüße
 Thomas


>> On Thu, 7 Mar 2024 at 20:37, Martin Jambor <mjambor@suse.cz> wrote:
>>>
>>> Hello,
>>>
>>> On Wed, Mar 06 2024, Abhinav Gupta via Gcc wrote:
>>> > Dear GCC Community,
>>> >     I hope this email finds you well. My name is Abhinav Gupta. I am a
>>> > 3rd-year student at IIT Tirupati pursuing a bachelor's degree in
>>> > computer science and engineering. I am writing to express my interest
>>> > in contributing to the GCC project under GSoC.
>>>
>>> We are very happy that you find contributing to GCC interesting.
>>>
>>> > I am interested in two projects - Offloading to a separate process on
>>> > the same host, I am already working in parallel computing,
>>> > specifically parallelising tensor algorithms using various techniques
>>> > as part of my research project at IIT Tirupati. Although this is not
>>> > directly related to compilers, I will be able to get going with the
>>> > project quickly.
>>>
>>> I'd personally very much like to see this project implemented.  There is
>>> a lot of information on offloading at
>>> https://gcc.gnu.org/wiki/Offloading
>>>
>>> To give you a bit more context where the idea comes from, it was first
>>> thought of in email thread starting with
>>> https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603984.html and
>>> the patch that was "scrubbed" from the email archive eventually became
>>> commit
>>> https://gcc.gnu.org/cgit/gcc/commit/?id=e4cba49413ca429dc82f6aa2e88129ecb3fdd943
>>>
>>> I hope these pointers will further help help you find out where to look
>>> when planning the project.  If you need more help, please feel free to
>>> ask (I'm CCing Thomas who then is perhaps best placed to answer).
>>>
>>> > The second project is Rust Front-End - both the BIR location support
>>> > and rustc testsuite adapter are of interest to me,
>>>
>>> Please note that Rust-GCC projects are a bit special in the sense that
>>> they are often discussed primarily on Zulip of the gcc-rust team:
>>>
>>> https://gcc-rust.zulipchat.com/
>>>
>>> So you may want to reach out to them there as well.
>>>
>>> > having worked on
>>> > compiler front ends as part of my college's compiler design course
>>> > combined with my experience in working with large libraries written in
>>> > C++ (such as CTF) I believe that these two projects are something that
>>> > I can do.
>>>
>>> You seem to be quite ready!
>>>
>>> >
>>> > Proposed Timeline:
>>> > I can start working as soon as my end-semester exams finish, i.e. 9th
>>> > May 2024, and continue to work for however long it requires me to
>>> > complete the project.
>>> > Week 1-2 -> Knowing the existing code and understanding how it works.
>>>
>>> Right, but please try to do a bit of this, at least on the high level,
>>> also now when preparing the proposal.  There will be lots to learn in
>>> the first weeks even so.  Mainly because...
>>>
>>> > Week 3-8 -> Working on the implementation of whichever project we
>>> > decide to move forward with
>>> > Week 9-12 -> Testing and creating understandable documentation for the same.
>>> >
>>> > This is a very rough timeline,
>>>
>>> ...eventually the milestones in the application will have to be more
>>> specific, mainly to demonstrate you have basic understanding of the
>>> proposed project.
>>>
>>> > and I will refine it further as we
>>> > discuss the project idea. This email is more of a call for guidance
>>> > than an application, and I would appreciate any feedback you give me.
>>>
>>> This is a very good start.
>>>
>>> Good luck!
>>>
>>> Martin

Re: GSoC

[Abhinav Gupta]

Hello GCC community,
     Since resuming work for the GSoC proposal recently, I have made
significant progress in understanding the code snippets provided by
Mr. Jambor and delved into the libgomp folder, compiling random
snippets in the examples folder and reading the IR. Building GCC from
source has also provided me with a deeper understanding of its inner
workings.
I've also looked at the GOMP_target_ext function and its
implementations for both the existing targets as pointed out by Mr.
Schwinge, these config folders have also given me an understanding of
what is actually expected for this project, however I need some
clarity regarding a few things-

The GOMP_* functions calls for different targets: I need further
assistance to understand how they should be implemented for the host
machine, I am aware that this is exactly what the project is, I am
just looking for a start point.

Host Fallback Mode: I came across mentions of a host fallback mode in
the offloading Wiki page and as you mentioned that there is already a
oacc-host.c api. Could you please provide more clarity on whether this
existing feature can be repurposed for offloading to a separate
process on the same host? If not, could you elaborate on what exactly
the host fallback mode is and how it differs from the proposed
offloading mechanism?

Thanking you
Abhinav

On Fri, 15 Mar 2024 at 03:54, Thomas Schwinge <tschwinge@baylibre.com> wrote:
>
> Hi Abhinav!
>
> Thanks for your interest in contributing to GCC, and thanks to Martin for
> all the information you already provided.  Just a bit more, to get you
> started on developing a proper project proposal:
>
> On 2024-03-13T14:54:52+0100, Martin Jambor <mjambor@suse.cz> wrote:
> > On Tue, Mar 12 2024, Abhinav Gupta wrote:
> >>     I looked at all the links you provided in the reply and read the
> >> paper cited on the GCC page for GSoC. I also looked into the rust
> >> frontend project during this time, and the Offloading project
> >> interests me more, so I will focus solely on it in the remaining seven
> >> days before the deadline for GSoC application submission.
> >
> > AFAIU, in five days (from now) the application period *opens*, the
> > deadline is 2nd April.  Still it is good idea not to lose any time.
>
> Indeed, no rush yet.  :-)
>
> >> Are there other resources I can look at to better understand the whole
> >> process?
>
> At some point, no too late, you should spend some time on learning how to
> build GCC, and run the test suite.
> <https://gcc.gnu.org/wiki/SummerOfCode> and
> <https://gcc.gnu.org/wiki/#Getting_Started_with_GCC_Development> have
> some pointers to get started.  If you have specific questions, we're
> happy to help, of course.
>
> >> Reading the git commit on the website is proving to be very
> >> slow.
>
> Yes, that's not going to be necessary.
>
> >> I think the git commit about Intel MIC would be like a
> >> "template" in loose terms
>
> Right, that's in fact a very good description.  The idea here is not to
> bring back the whole Intel MIC emulator, but something very much simpler.
>
> >> to implement the host-ISA mode at least, but
> >> for the libgomp plugin, I need a better understanding.
>
> Find existing libgomp plugins as 'libgomp/plugin/plugin-*.c'.  The
> 'GOMP_OFFLOAD_[...]' functions implement the entry points, the offloading
> plugin API.  Actually also the very simple 'libgomp/oacc-host.c' should
> be helpful.  That's essentially the API you need to care about (for
> OpenACC; but OpenMP 'target' also won't require much more, for a start).
>
> Make some thoughts (or actual experiments) about how we could
> use/implement a separate host process for code offloading.
>
> And otherwise, as Martin said:
>
> > You need to understand how OpenMP programs are internally represented.
> >
> > Look at the following (hopefully correct, at least as long as you try it
> > on a system without any Offloading enabled) simple testcase for OpenMP
> > target construct:
> >
> > ----------------------------------------------------------------------
> > #include <stdio.h>
> >
> > volatile int v = 1;
> >
> > int main (int argc, char **argv)
> > {
> >   int i = v;
> >
> > #pragma omp target map(to:i)
> >   {
> >     printf ("OpenMP target hello world, i is: %i\n", i);
> >   }
> >
> >   return 0;
> > }
> > ----------------------------------------------------------------------
> >
> > and compile it with:
> >
> >   gcc -fopenmp omptgt-hello.c -O2 -fdump-tree-optimized
> >
> > and then look at the generated optimized dump.  This should give you an
> > idea how OpenMP regions are internally represented (as outlined
> > functions) and how calls into libgomp, the run-time library doing a lot
> > of the magic, look like.
> >
> > You can try to do more similar exercises with more OpenMP testcase which
> > you can find in sub-directory libgomp/testsuite/libgomp.c of in the GCC
> > repository.
> >
> > And then you should perhaps have a look into libgomp itself (you'll find
> > it in libgomp sub-directory) how GOMP_target_ext is implemented - though
> > don't worry if you don't understand many of the details, it is complex
> > stuff.  At this point hopefully more of the contents of the Offloading
> > wiki page will make sense.
> >
> > Again, ask on the mailing list if you have any specific questions.
>
>
> Grüße
>  Thomas
>
>
> >> On Thu, 7 Mar 2024 at 20:37, Martin Jambor <mjambor@suse.cz> wrote:
> >>>
> >>> Hello,
> >>>
> >>> On Wed, Mar 06 2024, Abhinav Gupta via Gcc wrote:
> >>> > Dear GCC Community,
> >>> >     I hope this email finds you well. My name is Abhinav Gupta. I am a
> >>> > 3rd-year student at IIT Tirupati pursuing a bachelor's degree in
> >>> > computer science and engineering. I am writing to express my interest
> >>> > in contributing to the GCC project under GSoC.
> >>>
> >>> We are very happy that you find contributing to GCC interesting.
> >>>
> >>> > I am interested in two projects - Offloading to a separate process on
> >>> > the same host, I am already working in parallel computing,
> >>> > specifically parallelising tensor algorithms using various techniques
> >>> > as part of my research project at IIT Tirupati. Although this is not
> >>> > directly related to compilers, I will be able to get going with the
> >>> > project quickly.
> >>>
> >>> I'd personally very much like to see this project implemented.  There is
> >>> a lot of information on offloading at
> >>> https://gcc.gnu.org/wiki/Offloading
> >>>
> >>> To give you a bit more context where the idea comes from, it was first
> >>> thought of in email thread starting with
> >>> https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603984.html and
> >>> the patch that was "scrubbed" from the email archive eventually became
> >>> commit
> >>> https://gcc.gnu.org/cgit/gcc/commit/?id=e4cba49413ca429dc82f6aa2e88129ecb3fdd943
> >>>
> >>> I hope these pointers will further help help you find out where to look
> >>> when planning the project.  If you need more help, please feel free to
> >>> ask (I'm CCing Thomas who then is perhaps best placed to answer).
> >>>
> >>> > The second project is Rust Front-End - both the BIR location support
> >>> > and rustc testsuite adapter are of interest to me,
> >>>
> >>> Please note that Rust-GCC projects are a bit special in the sense that
> >>> they are often discussed primarily on Zulip of the gcc-rust team:
> >>>
> >>> https://gcc-rust.zulipchat.com/
> >>>
> >>> So you may want to reach out to them there as well.
> >>>
> >>> > having worked on
> >>> > compiler front ends as part of my college's compiler design course
> >>> > combined with my experience in working with large libraries written in
> >>> > C++ (such as CTF) I believe that these two projects are something that
> >>> > I can do.
> >>>
> >>> You seem to be quite ready!
> >>>
> >>> >
> >>> > Proposed Timeline:
> >>> > I can start working as soon as my end-semester exams finish, i.e. 9th
> >>> > May 2024, and continue to work for however long it requires me to
> >>> > complete the project.
> >>> > Week 1-2 -> Knowing the existing code and understanding how it works.
> >>>
> >>> Right, but please try to do a bit of this, at least on the high level,
> >>> also now when preparing the proposal.  There will be lots to learn in
> >>> the first weeks even so.  Mainly because...
> >>>
> >>> > Week 3-8 -> Working on the implementation of whichever project we
> >>> > decide to move forward with
> >>> > Week 9-12 -> Testing and creating understandable documentation for the same.
> >>> >
> >>> > This is a very rough timeline,
> >>>
> >>> ...eventually the milestones in the application will have to be more
> >>> specific, mainly to demonstrate you have basic understanding of the
> >>> proposed project.
> >>>
> >>> > and I will refine it further as we
> >>> > discuss the project idea. This email is more of a call for guidance
> >>> > than an application, and I would appreciate any feedback you give me.
> >>>
> >>> This is a very good start.
> >>>
> >>> Good luck!
> >>>
> >>> Martin

Re: GSoC

[Martin Jambor]

Hello Abhinav,

sorry for a very brief answer, I'm not much online during Easter (the
timing of the application deadline is a bit unfortunate in this regard).

On Sat, Mar 30 2024, Abhinav Gupta wrote:
> Hello GCC community,
>      Since resuming work for the GSoC proposal recently, I have made
> significant progress in understanding the code snippets provided by
> Mr. Jambor and delved into the libgomp folder, compiling random
> snippets in the examples folder and reading the IR. Building GCC from
> source has also provided me with a deeper understanding of its inner
> workings.
> I've also looked at the GOMP_target_ext function and its
> implementations for both the existing targets as pointed out by Mr.
> Schwinge, these config folders have also given me an understanding of
> what is actually expected for this project, however I need some
> clarity regarding a few things-
>
> The GOMP_* functions calls for different targets: I need further
> assistance to understand how they should be implemented for the host
> machine, I am aware that this is exactly what the project is, I am
> just looking for a start point.

Look at functions with names starting with GOMP_OFFLOAD (such as
GOMP_OFFLOAD_run) in files in src/libgomp/plugin/*.c - those are the
things you'll need to implement but instead of setting up a GPU and
talking to AMD HSA or Nvidia CUDA you'll be setting up a new process and
communicating with it.

The entire list of functions that a plugin can have (probably not all
have to be implemented) is in struct gomp_device_descr in libgomp.h.

>
> Host Fallback Mode: I came across mentions of a host fallback mode in
> the offloading Wiki page and as you mentioned that there is already a
> oacc-host.c api. Could you please provide more clarity on whether this
> existing feature can be repurposed for offloading to a separate
> process on the same host? If not, could you elaborate on what exactly
> the host fallback mode is and how it differs from the proposed
> offloading mechanism?

Partially.  The host fallback means that, for one reason or another
(most commonly because the machine does not have any accelerators to
which it can offload), the OpenMP run time simply runs the target
regions on the CPU, almost as if it was not there (but there are
caveats, of course).

Which means that the outlined regions can be compiled basically as they
are for host fallback (though doing it with the same compiler will be
funny), on the GOMP side, things will need to look very differently.

I hope this helps,

Martin


>
> Thanking you
> Abhinav
>
> On Fri, 15 Mar 2024 at 03:54, Thomas Schwinge <tschwinge@baylibre.com> wrote:
>>
>> Hi Abhinav!
>>
>> Thanks for your interest in contributing to GCC, and thanks to Martin for
>> all the information you already provided.  Just a bit more, to get you
>> started on developing a proper project proposal:
>>
>> On 2024-03-13T14:54:52+0100, Martin Jambor <mjambor@suse.cz> wrote:
>> > On Tue, Mar 12 2024, Abhinav Gupta wrote:
>> >>     I looked at all the links you provided in the reply and read the
>> >> paper cited on the GCC page for GSoC. I also looked into the rust
>> >> frontend project during this time, and the Offloading project
>> >> interests me more, so I will focus solely on it in the remaining seven
>> >> days before the deadline for GSoC application submission.
>> >
>> > AFAIU, in five days (from now) the application period *opens*, the
>> > deadline is 2nd April.  Still it is good idea not to lose any time.
>>
>> Indeed, no rush yet.  :-)
>>
>> >> Are there other resources I can look at to better understand the whole
>> >> process?
>>
>> At some point, no too late, you should spend some time on learning how to
>> build GCC, and run the test suite.
>> <https://gcc.gnu.org/wiki/SummerOfCode> and
>> <https://gcc.gnu.org/wiki/#Getting_Started_with_GCC_Development> have
>> some pointers to get started.  If you have specific questions, we're
>> happy to help, of course.
>>
>> >> Reading the git commit on the website is proving to be very
>> >> slow.
>>
>> Yes, that's not going to be necessary.
>>
>> >> I think the git commit about Intel MIC would be like a
>> >> "template" in loose terms
>>
>> Right, that's in fact a very good description.  The idea here is not to
>> bring back the whole Intel MIC emulator, but something very much simpler.
>>
>> >> to implement the host-ISA mode at least, but
>> >> for the libgomp plugin, I need a better understanding.
>>
>> Find existing libgomp plugins as 'libgomp/plugin/plugin-*.c'.  The
>> 'GOMP_OFFLOAD_[...]' functions implement the entry points, the offloading
>> plugin API.  Actually also the very simple 'libgomp/oacc-host.c' should
>> be helpful.  That's essentially the API you need to care about (for
>> OpenACC; but OpenMP 'target' also won't require much more, for a start).
>>
>> Make some thoughts (or actual experiments) about how we could
>> use/implement a separate host process for code offloading.
>>
>> And otherwise, as Martin said:
>>
>> > You need to understand how OpenMP programs are internally represented.
>> >
>> > Look at the following (hopefully correct, at least as long as you try it
>> > on a system without any Offloading enabled) simple testcase for OpenMP
>> > target construct:
>> >
>> > ----------------------------------------------------------------------
>> > #include <stdio.h>
>> >
>> > volatile int v = 1;
>> >
>> > int main (int argc, char **argv)
>> > {
>> >   int i = v;
>> >
>> > #pragma omp target map(to:i)
>> >   {
>> >     printf ("OpenMP target hello world, i is: %i\n", i);
>> >   }
>> >
>> >   return 0;
>> > }
>> > ----------------------------------------------------------------------
>> >
>> > and compile it with:
>> >
>> >   gcc -fopenmp omptgt-hello.c -O2 -fdump-tree-optimized
>> >
>> > and then look at the generated optimized dump.  This should give you an
>> > idea how OpenMP regions are internally represented (as outlined
>> > functions) and how calls into libgomp, the run-time library doing a lot
>> > of the magic, look like.
>> >
>> > You can try to do more similar exercises with more OpenMP testcase which
>> > you can find in sub-directory libgomp/testsuite/libgomp.c of in the GCC
>> > repository.
>> >
>> > And then you should perhaps have a look into libgomp itself (you'll find
>> > it in libgomp sub-directory) how GOMP_target_ext is implemented - though
>> > don't worry if you don't understand many of the details, it is complex
>> > stuff.  At this point hopefully more of the contents of the Offloading
>> > wiki page will make sense.
>> >
>> > Again, ask on the mailing list if you have any specific questions.
>>
>>
>> Grüße
>>  Thomas
>>
>>
>> >> On Thu, 7 Mar 2024 at 20:37, Martin Jambor <mjambor@suse.cz> wrote:
>> >>>
>> >>> Hello,
>> >>>
>> >>> On Wed, Mar 06 2024, Abhinav Gupta via Gcc wrote:
>> >>> > Dear GCC Community,
>> >>> >     I hope this email finds you well. My name is Abhinav Gupta. I am a
>> >>> > 3rd-year student at IIT Tirupati pursuing a bachelor's degree in
>> >>> > computer science and engineering. I am writing to express my interest
>> >>> > in contributing to the GCC project under GSoC.
>> >>>
>> >>> We are very happy that you find contributing to GCC interesting.
>> >>>
>> >>> > I am interested in two projects - Offloading to a separate process on
>> >>> > the same host, I am already working in parallel computing,
>> >>> > specifically parallelising tensor algorithms using various techniques
>> >>> > as part of my research project at IIT Tirupati. Although this is not
>> >>> > directly related to compilers, I will be able to get going with the
>> >>> > project quickly.
>> >>>
>> >>> I'd personally very much like to see this project implemented.  There is
>> >>> a lot of information on offloading at
>> >>> https://gcc.gnu.org/wiki/Offloading
>> >>>
>> >>> To give you a bit more context where the idea comes from, it was first
>> >>> thought of in email thread starting with
>> >>> https://gcc.gnu.org/pipermail/gcc-patches/2022-October/603984.html and
>> >>> the patch that was "scrubbed" from the email archive eventually became
>> >>> commit
>> >>> https://gcc.gnu.org/cgit/gcc/commit/?id=e4cba49413ca429dc82f6aa2e88129ecb3fdd943
>> >>>
>> >>> I hope these pointers will further help help you find out where to look
>> >>> when planning the project.  If you need more help, please feel free to
>> >>> ask (I'm CCing Thomas who then is perhaps best placed to answer).
>> >>>
>> >>> > The second project is Rust Front-End - both the BIR location support
>> >>> > and rustc testsuite adapter are of interest to me,
>> >>>
>> >>> Please note that Rust-GCC projects are a bit special in the sense that
>> >>> they are often discussed primarily on Zulip of the gcc-rust team:
>> >>>
>> >>> https://gcc-rust.zulipchat.com/
>> >>>
>> >>> So you may want to reach out to them there as well.
>> >>>
>> >>> > having worked on
>> >>> > compiler front ends as part of my college's compiler design course
>> >>> > combined with my experience in working with large libraries written in
>> >>> > C++ (such as CTF) I believe that these two projects are something that
>> >>> > I can do.
>> >>>
>> >>> You seem to be quite ready!
>> >>>
>> >>> >
>> >>> > Proposed Timeline:
>> >>> > I can start working as soon as my end-semester exams finish, i.e. 9th
>> >>> > May 2024, and continue to work for however long it requires me to
>> >>> > complete the project.
>> >>> > Week 1-2 -> Knowing the existing code and understanding how it works.
>> >>>
>> >>> Right, but please try to do a bit of this, at least on the high level,
>> >>> also now when preparing the proposal.  There will be lots to learn in
>> >>> the first weeks even so.  Mainly because...
>> >>>
>> >>> > Week 3-8 -> Working on the implementation of whichever project we
>> >>> > decide to move forward with
>> >>> > Week 9-12 -> Testing and creating understandable documentation for the same.
>> >>> >
>> >>> > This is a very rough timeline,
>> >>>
>> >>> ...eventually the milestones in the application will have to be more
>> >>> specific, mainly to demonstrate you have basic understanding of the
>> >>> proposed project.
>> >>>
>> >>> > and I will refine it further as we
>> >>> > discuss the project idea. This email is more of a call for guidance
>> >>> > than an application, and I would appreciate any feedback you give me.
>> >>>
>> >>> This is a very good start.
>> >>>
>> >>> Good luck!
>> >>>
>> >>> Martin

GSOC

[Pratush Rai]

[-- Attachment #1: Type: text/plain, Size: 196 bytes --]

I want to work on the Rust Front End project as per the idea list. I am a
rust compiler contributor and want to start contributing to gcc.
I need some guidance on how to start.
Thank you,
Pratush

Re: GSOC

[Martin Jambor]

Hello,

On Tue, Feb 27 2024, Pratush Rai via Gcc wrote:
> I want to work on the Rust Front End project as per the idea list. I am a
> rust compiler contributor and want to start contributing to gcc.
> I need some guidance on how to start.

Please note that Rust-GCC projects are a bit special in the sense that
they are often discussed primarily on Zulip of the gcc-rust team:

https://gcc-rust.zulipchat.com/

So you may want to reach out to gcc-rust developers there there as well.

Apart from this, I can only point you to
https://gcc.gnu.org/wiki/SummerOfCode and specifically to
https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply which explains
some of the steps you can take to familiarize yourself with our code
base.

Good luck!

Martin Jambor

GSoC

[Tim Lange]

Hi everyone,

my name is Tim and I'm also working on the static analyzer this summer. 
Some of you might already noticed my nooby questions in the IRC ;).
Specifically, I'll be working on extending the analyzer with several 
smaller warnings that the clang analyzer already has. David created a 
meta-bug[0] with the results of the discussion between him and me about 
the gap and what seems to be useful.

I won't do all of those but rather look how many of them I'm able to 
get done until September. I will begin with a Cast Size warning. This 
emits a warning when the tracked allocation size is not a multiple of 
the pointee's size, e.g., when casting malloc(10) to int*.

Furthermore, in preparation for the official coding phase, I played 
around a bit with a state machine that tracks whether an int is zero or 
not. So this is probably my next candidate after cast size.

- Tim

[0] 
https://gcc.gnu.org/bugzilla/showdependencytree.cgi?id=105887&hide_resolved=1

Re: GSoC

[David Malcolm]

On Thu, 2022-06-09 at 11:04 +0200, Tim Lange wrote:
> Hi everyone,
> 
> my name is Tim and I'm also working on the static analyzer this summer.

Hi Tim - and welcome to GCC development.

> Some of you might already noticed my nooby questions in the IRC ;).
> Specifically, I'll be working on extending the analyzer with several 
> smaller warnings that the clang analyzer already has. David created a
> meta-bug[0] with the results of the discussion between him and me about
> the gap and what seems to be useful.
> 
> I won't do all of those but rather look how many of them I'm able to 
> get done until September. I will begin with a Cast Size warning. This
> emits a warning when the tracked allocation size is not a multiple of
> the pointee's size, e.g., when casting malloc(10) to int*.

I think this is:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105900

You ought to mark that bug as ASSIGNED to you.  Do you have an account
for GCC's git and bugzilla yet?  See:
  https://gcc.gnu.org/gitwrite.html#authenticated
for info on how to get an account on sourceware.org/gcc.gnu.org

I can be your sponsor.

Once you have that, I think you automatically get a
USERNAME@gcc.gnu.org account that you can use to login to the GCC
bugzilla.

> 
> Furthermore, in preparation for the official coding phase, I played 
> around a bit with a state machine that tracks whether an int is zero or
> not. So this is probably my next candidate after cast size.

This would be for:
  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=99669
(generalizing the division-by-zero check)


Dave

> 
> - Tim
> 
> [0] 
>   
> https://gcc.gnu.org/bugzilla/showdependencytree.cgi?id=105887&hide_resolved=1
> 
> 

Gsoc

[20-cs Kunal Rajnish]

Dear Sir/Ma'am I beginners in coding but has lot's of interest in working
with people like you and learning new skills as much as possible. I had a
idea that why not "make a simple interface for user to create there own
library and import them". This will attract more new coders so that they
can make there own library.
Thanks for giving your precious time.

Re: Gsoc

[Martin Jambor]

Hello,

On Mon, Apr 11 2022, 20-cs Kunal Rajnish via Gcc wrote:
> Dear Sir/Ma'am I beginners in coding but has lot's of interest in working
> with people like you and learning new skills as much as possible. I had a
> idea that why not "make a simple interface for user to create there own
> library and import them". This will attract more new coders so that they
> can make there own library.
> Thanks for giving your precious time.

Thanks for your interest in contributing to the GCC.  Unfortunately, I
do not quite understand what your idea is about.  GCC of course already
allows users to create libraries and I do not quite know what you mean
by "importing."

Moreover, I am worried that you also need at least some rudimentary
theoretical background in the area of compilers in order to apply.
Nevertheless, if you still want to try applying, have a look at
https://gcc.gnu.org/wiki/SummerOfCode and specifically at
https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply which explains
some of the steps you can take to familiarize yourself with our code
base and start thinking about a project.

Good luck,

Martin Jambor

GSoC

[Γιωργος Μελλιος]

Greetings,

I am sending this email in order to show my interest in your GSoC program
for this summer.
To begin with, I would like to learn some general information about your
project in order to judge if I am capable of participating in it. I am
currently studying Compilers and Programming Languages in my university
(Computer Science Department, University of Crete) and I am very interested
in it, so finding you as a GSoC organization made me extremely happy.

Thanks in advance,
Georgios Panagiotis Mellios

Re: GSoC

[David Edelsohn]

On Sat, Mar 12, 2022 at 10:40 AM Γιωργος Μελλιος via Gcc
<gcc@gcc.gnu.org> wrote:
>
> Greetings,
>
> I am sending this email in order to show my interest in your GSoC program
> for this summer.
> To begin with, I would like to learn some general information about your
> project in order to judge if I am capable of participating in it. I am
> currently studying Compilers and Programming Languages in my university
> (Computer Science Department, University of Crete) and I am very interested
> in it, so finding you as a GSoC organization made me extremely happy.
>
> Thanks in advance,
> Georgios Panagiotis Mellios

Hi, Georgios

Thanks for your interest in GCC.  Welcome!

A good place to start is the GCC Wiki Getting Started page:
https://gcc.gnu.org/wiki/#Getting_Started_with_GCC_Development

The GSoC application process is not open yet, but you can look at
previous GSoC projects and project ideas on the GCC Wiki:

https://gcc.gnu.org/wiki/SummerOfCode

Thanks, David

GSOC

[Manish Sahani]

Hey,

I'm a  student at Delhi Technological University, and interested in
contributing to the *Make cp-demangle non-recursive* and *Fortran improved
argument compile-time checking*. Before reaching out here In the past week
I have gone through most of the introduction wiki on building and testing,
some wikis on the structure of gcc, have played around with configure
script (explored various options), build the code multiple times, and ran
test cases.

My previous intern at adobe included some work to analyze the MSVC manglers
and so I'm very interested in the cp-demangle project and I also want to
work on the improvement of compile-time argument. So, If somebody can
redirect me to the starting point that would be very helpful.

Also, while compiling without bootstrapping. I ran into an error (screenshot
of the error
<https://s3.us-west-2.amazonaws.com/secure.notion-static.com/1323c46e-a170-4c85-a83a-4ea302e7c689/Screenshot_2021-03-18_at_9.14.34_PM.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAT73L2G45O3KS52Y5%2F20210320%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210320T121908Z&X-Amz-Expires=86400&X-Amz-Signature=fc8925835e531aa06ef8bfbc8b427fe5f58b282200d898e2b25f82e596094d28&X-Amz-SignedHeaders=host&response-content-disposition=filename%20%3D%22Screenshot_2021-03-18_at_9.14.34_PM.png%22>)
couldn't find why this happened, any comments on that would be appreciated.

-- 
Regards,
Manish Sahani
rec.manish.sahani@gmail.com | linkedin@manishsahani <https://bit.ly/to-lnkd>
| github@kalkayan <https://bit.ly/to-gh>

Re: GSOC

[Jonathan Wakely]

On Sat, 20 Mar 2021 at 12:24, Manish Sahani via Gcc <gcc@gcc.gnu.org> wrote:
> Also, while compiling without bootstrapping. I ran into an error (screenshot
> of the error
> <https://s3.us-west-2.amazonaws.com/secure.notion-static.com/1323c46e-a170-4c85-a83a-4ea302e7c689/Screenshot_2021-03-18_at_9.14.34_PM.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAT73L2G45O3KS52Y5%2F20210320%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20210320T121908Z&X-Amz-Expires=86400&X-Amz-Signature=fc8925835e531aa06ef8bfbc8b427fe5f58b282200d898e2b25f82e596094d28&X-Amz-SignedHeaders=host&response-content-disposition=filename%20%3D%22Screenshot_2021-03-18_at_9.14.34_PM.png%22>)
> couldn't find why this happened, any comments on that would be appreciated.

The error message was plain text, so there's no need to use a
screenshot to show it (and certainly not one that disappears after a
few days).

When I first looked at the screenshot I don't think it showed the
error, only the end of the compilation output, some time after the
actual error was shown. You need to look at the earlier output for the
problem. Please use the gcc-help list for help with problems like
that.

Re: GSOC

[Martin Jambor]

Hello,

On Sat, Mar 20 2021, Manish Sahani via Gcc wrote:
> Hey,
>
> I'm a  student at Delhi Technological University, and interested in
> contributing to the *Make cp-demangle non-recursive* and *Fortran improved
> argument compile-time checking*.

we are very happy to hear you are interested in two topics, especially
since they are quite different.  It is going to be difficult, but unless
you want to submit two GCC GSoC proposals, you will have to choose one.

> Before reaching out here In the past week
> I have gone through most of the introduction wiki on building and testing,
> some wikis on the structure of gcc, have played around with configure
> script (explored various options), build the code multiple times, and ran
> test cases.

Great, thanks for taking the effort before reaching out to us.  What
next now depends on your particular choice of project.  The C++
demangler is part of libiberty (subdirectory), is fairly self-contained
and you can just go and read the source and (assuming the stated goal is
fairly clear to you), start planning on how you would go about carrying
out the task.  Ask here if you have any questions.

I cannot really help you with the Fortran project idea too, much, I
would suggest you request help and guidance on the fortran@gcc.gnu.org
mailing list (but thanks for announcing your intention here anyway, I am
not subscribed to the fortran list myself).

>
> My previous intern at adobe included some work to analyze the MSVC manglers
> and so I'm very interested in the cp-demangle project and I also want to
> work on the improvement of compile-time argument. So, If somebody can
> redirect me to the starting point that would be very helpful.
>
> Also, while compiling without bootstrapping. I ran into an error (screenshot
> of the error

I cannot access that URL any more and as Jonathan pointed out, there is
really no need to send a screenshot when cut and pasted error would do.
If the error is not an internal compiler error, requesting help on
avoiding it indeed is more appropriate on the gcc-help@gcc.gnu.org
mailing list.

Apologies for referring you to two different mailing lists, but you see,
the world of GCC big!

Good luck, please do not hesitate to ask for any help if you struggle
with any particular problem.

Martin

GSoC

[Isitha Subasinghe]

To whom it may concern,

I am a student interested in participating in GSoC this year. After having
a look at some of the available PL projects, gccrs caught my attention. I
love Rust and have an interest in exploring more about type theory and
automatic garbage collection.

My background is that I am a Masters's student at the University of
Melbourne in Australia, I have undertaken a graduate-level compiler class
where we implemented a stack-based compiler in Haskell.

I am quite interested in working on the static analysis project but wanted
feedback to iron out and address my proposal before I submit it.

I am quite confident in my C/C++ skills but somewhat unsure about the level
of knowledge of static analysis that I would need. Unfortunately, I am yet
to take any classes in this particular subfield but I am absolutely happy
to learn on my own time and have purchased the book Principles of Program
Analysis to assist with this matter.

Also, I did want to notify you that I would be available for less than the
entire coding duration of GSoC due to university commitments.
Unfortunately, my exams overlap with GSoC, and it is hard to compromise on
University studies since I am hoping to do a PhD in PL after the completion
of my master's. I would be absolutely happy to make up this time at the end
of the year where I have a 3-month break.

Best Regards,
Isitha

Re: GSoC

[Philip Herron]

Hi Isitha,

Thanks for your interest in GCC Rust, it's an exciting project that is
early on in development, so there is plenty of scoping for making your mark
on the compiler. In regards to your proposal feel free to join our Zulip
server https://gcc-rust.zulipchat.com/ and it can be discussed with the
community.

As for the Google Summer of Code timeline, I would have to defer to their
rules. Maybe others here know better in this mailing list but as far as I
know, to complete the google summer of code there are dated milestones of
review so this might break the rules if you have exams and are unable to
allocate the time towards it.

Hope this helps, I hope it works out for you.

Thanks

--Phil



On Fri, 19 Mar 2021 at 08:04, Isitha Subasinghe via Gcc <gcc@gcc.gnu.org>
wrote:

> To whom it may concern,
>
> I am a student interested in participating in GSoC this year. After having
> a look at some of the available PL projects, gccrs caught my attention. I
> love Rust and have an interest in exploring more about type theory and
> automatic garbage collection.
>
> My background is that I am a Masters's student at the University of
> Melbourne in Australia, I have undertaken a graduate-level compiler class
> where we implemented a stack-based compiler in Haskell.
>
> I am quite interested in working on the static analysis project but wanted
> feedback to iron out and address my proposal before I submit it.
>
> I am quite confident in my C/C++ skills but somewhat unsure about the level
> of knowledge of static analysis that I would need. Unfortunately, I am yet
> to take any classes in this particular subfield but I am absolutely happy
> to learn on my own time and have purchased the book Principles of Program
> Analysis to assist with this matter.
>
> Also, I did want to notify you that I would be available for less than the
> entire coding duration of GSoC due to university commitments.
> Unfortunately, my exams overlap with GSoC, and it is hard to compromise on
> University studies since I am hoping to do a PhD in PL after the completion
> of my master's. I would be absolutely happy to make up this time at the end
> of the year where I have a 3-month break.
>
> Best Regards,
> Isitha
>

Re: GSoC

[David Malcolm]

Hi Isitha (and Philip!)

If I'm reading Isitha's email correctly, it talks about static
analysis, whereas Philip's talks about GCC Rust, so some wires got
crossed somewhere.

I'm the author of the GCC static analysis pass.  I should confess that
I still feel like I'm learning static analysis myself - I too own a
copy of the Nielson, Nielson & Hankin book you mention, but have only
skimmed it.  FWIW, I find the very early papers by Patrick and Radhia
Cousot from the beginning of the field much easier to read, as they
take more time spelling out the meaning of the mathematics.  I should
also confess that the analysis pass takes some liberties compared to a
formal approach, grabbing ideas from here and there, plugging them into
30+-year-old codebase in a way that I hope is a reasonable trade-off
between speed, (lack of) soundness, (lack of) completeness, and
readability of output by end-user.

The static analysis pass is meant to be reasonably modular, so the
various suggested projects listed on the wiki page ought to be
implementable without knowing everything all at once.

However, as Philip says, GSoC imposes a particular timeline, and I
don't know to what extent might be a dealbreaker.

Hope this is helpful
Dave

On Fri, 2021-03-19 at 13:24 +0000, Philip Herron wrote:
> Hi Isitha,
> 
> Thanks for your interest in GCC Rust, it's an exciting project that
> is
> early on in development, so there is plenty of scoping for making
> your mark
> on the compiler. In regards to your proposal feel free to join our
> Zulip
> server https://gcc-rust.zulipchat.com/ and it can be discussed with
> the
> community.
> 
> As for the Google Summer of Code timeline, I would have to defer to
> their
> rules. Maybe others here know better in this mailing list but as far
> as I
> know, to complete the google summer of code there are dated
> milestones of
> review so this might break the rules if you have exams and are unable
> to
> allocate the time towards it.
> 
> Hope this helps, I hope it works out for you.
> 
> Thanks
> 
> --Phil
> 
> 
> 
> On Fri, 19 Mar 2021 at 08:04, Isitha Subasinghe via Gcc
> <gcc@gcc.gnu.org>
> wrote:
> 
> > To whom it may concern,
> > 
> > I am a student interested in participating in GSoC this year. After
> > having
> > a look at some of the available PL projects, gccrs caught my
> > attention. I
> > love Rust and have an interest in exploring more about type theory
> > and
> > automatic garbage collection.
> > 
> > My background is that I am a Masters's student at the University of
> > Melbourne in Australia, I have undertaken a graduate-level compiler
> > class
> > where we implemented a stack-based compiler in Haskell.
> > 
> > I am quite interested in working on the static analysis project but
> > wanted
> > feedback to iron out and address my proposal before I submit it.
> > 
> > I am quite confident in my C/C++ skills but somewhat unsure about
> > the level
> > of knowledge of static analysis that I would need. Unfortunately, I
> > am yet
> > to take any classes in this particular subfield but I am absolutely
> > happy
> > to learn on my own time and have purchased the book Principles of
> > Program
> > Analysis to assist with this matter.
> > 
> > Also, I did want to notify you that I would be available for less
> > than the
> > entire coding duration of GSoC due to university commitments.
> > Unfortunately, my exams overlap with GSoC, and it is hard to
> > compromise on
> > University studies since I am hoping to do a PhD in PL after the
> > completion
> > of my master's. I would be absolutely happy to make up this time at
> > the end
> > of the year where I have a 3-month break.
> > 
> > Best Regards,
> > Isitha
> > 
> 

Re: GSoC

[Martin Jambor]

Hello Isitha,

On Fri, Mar 19 2021, Isitha Subasinghe via Gcc wrote:
> To whom it may concern,
>
> I am a student interested in participating in GSoC this year. After having
> a look at some of the available PL projects, gccrs caught my attention. I
> love Rust and have an interest in exploring more about type theory and
> automatic garbage collection.
>
> My background is that I am a Masters's student at the University of
> Melbourne in Australia, I have undertaken a graduate-level compiler class
> where we implemented a stack-based compiler in Haskell.
>
> I am quite interested in working on the static analysis project but wanted
> feedback to iron out and address my proposal before I submit it.
>
> I am quite confident in my C/C++ skills but somewhat unsure about the level
> of knowledge of static analysis that I would need. Unfortunately, I am yet
> to take any classes in this particular subfield but I am absolutely happy
> to learn on my own time and have purchased the book Principles of Program
> Analysis to assist with this matter.
>
> Also, I did want to notify you that I would be available for less than the
> entire coding duration of GSoC due to university commitments.
> Unfortunately, my exams overlap with GSoC, and it is hard to compromise on
> University studies since I am hoping to do a PhD in PL after the completion
> of my master's. I would be absolutely happy to make up this time at the end
> of the year where I have a 3-month break.

Google Summer of code coding period is 10 week long and takes place
between June 7 and August 23.  Students are expected to work total 175
hours over these 10 weeks, so on average 18 hours a week, but it can be
more during some weeks and fewer (even zero) during others, as long as
the hours add up.  There is however one evaluation taking place between
July 12-16, so some part of work must be done and evaluated before then.

Unfortunately, the GSoC rules require that all evaluated work is
submitted before the deadline on August 16, no work done later can be
considered.

I hope the flexibility to distribute work will allow you to participate
but unfortunately we are not in a position to make an exception for
anybody, we have to adhere to GSoC rules too.

Martin

GSoC

[ΓΙΩΡΓΟΣ ΛΙΑΚΟΠΟΥΛΟΣ]

To whom it may concern ,

My name is George Liakopoulos and I am an undergraduate student in the
Department of Informatics at the University of Athens .
The past 4 years I code in C/C++ , as my department is focused very much in
these languages .

I would like to contribute to one of your GSoc 2021 projects and more precisely
in the "Rust Front-End" project . The last few days ( from the
organizations announcing ) I look into the git-repo code and I am quite
excited .

I would like to ask you if there is something that i could do from now , to
warm up and get more familiar with the project . Can you point me to a more
specific direction ?

I look forward to hearing from you ,
George Liakopoulos

Re: GSoC

[Philip Herron]

Hi George,

I am the maintainer for GCC Rust, please feel free to join our zulip server
https://gcc-rust.zulipchat.com/, there are other students on the server
also. In terms of getting involved, the first thing to do is make sure you
are able to download the code via git, compile and run the test suite.

After you are able to do that I would suggest writing some test cases and
running the compiler yourself to get familiar with it. As part of helping
students get started, I have been trying to create issues tagged with
good-first-pr
https://github.com/Rust-GCC/gccrs/issues?q=is%3Aissue+is%3Aopen+label%3Agood-first-pr
and feel free to ask questions and guidance on those specific issues as a
coding starting point.

Does this help?

Thanks

--Phil

On Fri, 12 Mar 2021 at 09:26, ΓΙΩΡΓΟΣ ΛΙΑΚΟΠΟΥΛΟΣ via Gcc <gcc@gcc.gnu.org>
wrote:

> To whom it may concern ,
>
> My name is George Liakopoulos and I am an undergraduate student in the
> Department of Informatics at the University of Athens .
> The past 4 years I code in C/C++ , as my department is focused very much in
> these languages .
>
> I would like to contribute to one of your GSoc 2021 projects and more
> precisely
> in the "Rust Front-End" project . The last few days ( from the
> organizations announcing ) I look into the git-repo code and I am quite
> excited .
>
> I would like to ask you if there is something that i could do from now , to
> warm up and get more familiar with the project . Can you point me to a more
> specific direction ?
>
> I look forward to hearing from you ,
> George Liakopoulos
>

GSoC

[Ravi Kumar]

Hello Sir,
I am Ravi Kumar. I am currently a 2nd year undergraduate(B.Tech) student. I want to participate in GSoC 2021 and want to work under the mentorship of GCC.    
WHY ME?
Because:
1.I have a proper knowledge and experience in C and C++ Language.
2. I have learnt to use git and GitHub.
3. I also have a theoretical knowledge of compilers and compiler organization.
4. I am ready to give 4-5 hours daily to the project.
5.I am familiar with GCC source code.

Since I am a new contributor and I have a little knowledge regarding this so I am facing lot of difficulties. But I am a good learner and I want to explore more in contribution and project completion under a mentor. Since I have the required skills for the project ideas that you have provided, I would love to work for it. 
Here is the project that I am opting:
Extend the static analysis pass



 


Sent from Mail for Windows 10

Re: GSoC

[Martin Jambor]

Hello,

On Sun, Feb 07 2021, Ravi Kumar via Gcc wrote:
> Hello Sir,
> I am Ravi Kumar. I am currently a 2nd year undergraduate(B.Tech) student. I want to participate in GSoC 2021 and want to work under the mentorship of GCC.    
> WHY ME?
> Because:
> 1.I have a proper knowledge and experience in C and C++ Language.
> 2. I have learnt to use git and GitHub.
> 3. I also have a theoretical knowledge of compilers and compiler organization.
> 4. I am ready to give 4-5 hours daily to the project.
> 5.I am familiar with GCC source code.
>
> Since I am a new contributor and I have a little knowledge regarding this so I am facing lot of difficulties. But I am a good learner and I want to explore more in contribution and project completion under a mentor. Since I have the required skills for the project ideas that you have provided, I would love to work for it. 
> Here is the project that I am opting:
> Extend the static analysis pass
>

we are delighted that you decided to apply for GSoC and that you have
chosen GCC as the organization.  If you think you need any help
selecting a particular static analyzer project or improving your
application proposal, feel free to email the mailing list again.

Thanks,

Martin

GSoC

[Yerassyl Sagynov]

Hi there! I would like to choose a project called “Parallelize compilation
using threads, part two”. My first programming language was C++ at school.
I used it for competitive programming. Later in university I have improved
this skill by taking courses related to C/C++. Precisely, Programming 1
course which includes basic knowledge about C like variables, types,
structures, pointers, arrays and different algorithms. In addition to that
I took courses called Programming Languages, Performance and Data
Structures, Operating Systems which taught deeper knowledge about C++,
various algorithms and data structures. Also by the end of the course
Operating
systems I made a project which is CLI based Quiz Server. In order to
complete this project threads, mutexes, semaphores, syscalls of Unix and
other stuff of currency were used. Since I am experienced with threads, I
would be very excited to work with parallelize compilation using threads.

Kind Regards!

GSOC

[shivam tiwari]

In the given link I shared My GSOC 2020 Project Proposal And Tell Me If Any
Other Extra Point I have to Add on it. Any Give Me feedback About My
Project Proposal .
https://docs.google.com/document/d/1J2lCtkDPoU0gwp5zk8Ygy0Vg82fNn-ZH0gsv5dBkKok/edit?usp=sharing

GSOC

[shivam tiwari]

In Which Format I have to Send Gsoc Proposal On this Mail PDF or DOC Format

Re: GSOC

[Segher Boessenkool]

Hi!

On Sun, Mar 15, 2020 at 06:49:32PM +0530, shivam tiwari via Gcc wrote:
> In Which Format I have to Send Gsoc Proposal On this Mail PDF or DOC Format

On this email list, use plain text.

Please read <https://gcc.gnu.org/wiki/SummerOfCode>, the "Application"
section there in particular, in this case (but read it all :-) )

Good luck,


Segher

Re: GSOC

[Martin Jambor]

Hello,

On Sun, Mar 15 2020, shivam tiwari via Gcc wrote:
> In Which Format I have to Send Gsoc Proposal On this Mail PDF or DOC Format

if you want to discuss your proposal on this mailing list, then plain
text in an email message is the best option.

Per GSoC rules (IIRC), the final application has to be in PDF.

Please make sure you read
https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply
and https://gcc.gnu.org/wiki/SummerOfCode#Application

Good luck,

Martin

Re: GSOC

[Giuliano Belinassi]

Hi, Shivan.

On 03/15, shivam tiwari via Gcc wrote:
> In Which Format I have to Send Gsoc Proposal On this Mail PDF or DOC Format

Perhaps the best way to write your proposal is to write LaTeX, and
then convert it to Markdown using pandoc, and attach it to the mailing
list. One example of this approach is:

https://gcc.gnu.org/pipermail/gcc/2020-March/231825.html

Thank you,
Giuliano.

GSoC

[utkarsh shrivastava]

Hello  There,
I am very eager to participate in GSoC 2019 and am interested in the idea
to add new functions to math.h and complex.h.
I wanted to know that can i apply i this project.
P.S. I've already uploaded my proposal.

GSoC

[utkarsh shrivastava]

Hello  There,
I very eager to participate in GSoC 2019 and am interested in the idea to
add new functions to math.h and complex.h.
I wanted to know that can i apply i this project.
P.S. I've already uploaded my proposal.

GSOC

[ashwina kumar]

Hi ,

While working I just figured out that -Wconversion is buggy. Please see the
below code- -

$ cat b.c
#include <stdint.h>

void main (void)
{
        //contains build errors
        uint16_t x = 1;
        uint16_t y = 2;
        y += x;  /* contains error */

}

$ gcc b.c -Wconversion
b.c: In function ‘main’:
b.c:22:7: warning: conversion to ‘uint16_t {aka short unsigned int}’
from ‘int’ may alter its value [-Wconversion]
  y += x;  /* contains error */

Please help me to know as an GSOC student can I work on this for this year
to make -Wconversion more robust.

Thanks & Regards,
Ashwina

-- 
Ashwina Kumar
BIT Mesra

Re: GSOC

[Richard Biener]

On Sun, Apr 7, 2019 at 11:10 AM ashwina kumar <kumarashwina31@gmail.com> wrote:
>
> Hi ,
>
> While working I just figured out that -Wconversion is buggy. Please see the
> below code- -
>
> $ cat b.c
> #include <stdint.h>
>
> void main (void)
> {
>         //contains build errors
>         uint16_t x = 1;
>         uint16_t y = 2;
>         y += x;  /* contains error */
>
> }
>
> $ gcc b.c -Wconversion
> b.c: In function ‘main’:
> b.c:22:7: warning: conversion to ‘uint16_t {aka short unsigned int}’
> from ‘int’ may alter its value [-Wconversion]
>   y += x;  /* contains error */

The warning is correct unless you factor in that x == 1 and y == 2.

> Please help me to know as an GSOC student can I work on this for this year
> to make -Wconversion more robust.
>
> Thanks & Regards,
> Ashwina
>
> --
> Ashwina Kumar
> BIT Mesra

Re: GSOC

[Martin Jambor]

Hello,

On Sun, Apr 07 2019, ashwina kumar wrote:
> Hi ,
>
> While working I just figured out that -Wconversion is buggy. Please see the
> below code- -
>
> $ cat b.c
> #include <stdint.h>
>
> void main (void)
> {
>         //contains build errors
>         uint16_t x = 1;
>         uint16_t y = 2;
>         y += x;  /* contains error */
>
> }
>
> $ gcc b.c -Wconversion
> b.c: In function ‘main’:
> b.c:22:7: warning: conversion to ‘uint16_t {aka short unsigned int}’
> from ‘int’ may alter its value [-Wconversion]
>   y += x;  /* contains error */
>
> Please help me to know as an GSOC student can I work on this for this year
> to make -Wconversion more robust.

Unfortunately I am not quite sure what you think is a problem.  The
option -Werror warns "for implicit conversions that may alter a value."
In this case, the C language mandates that the addition is performed in
full integer type, which is then stored to a shorter type, which is a
conversion which may alter the value.  Thus the warning.

In this particular example the values are known constants and so one
could argue that the result of addition is known at compile not to
exceed the uint16_t range.  However, the infrastructure that would have
to be added to the C/C++ front-ends to perform such value tracking would
not be justified with this simple use-case - especially given that the
warning should be perfectly silence-able with an explicit conversion.

Therefore, we are very unlikely to accept such GSoC proposal, sorry.

Martin

GSoC

[Muhammad Shehzad]

Hello GCC Community,

For GSoC I Would love to work on "*Add new math.h and complex.h functions
as built-ins*" project from given project ideas. I wanted to know which
mailing list should I subscribe to?

I also wanted to know If I have my own idea, I want to present, what steps
should I follow?

Regards,
Muhammad Shehzad

Re: GSoC

[Martin Jambor]

Hello Muhammad,

On Thu, Apr 04 2019, Muhammad Shehzad wrote:
> Hello GCC Community,
>
> For GSoC I Would love to work on "*Add new math.h and complex.h functions
> as built-ins*" project from given project ideas.

we are already in fairly advanced stage of putting together a proposal
for the same project with Tejas Joshi.  You can of course try to submit
a better one but it will be difficult.  Or if you act really quicly, you
just have about time to switch and attempt a different project idea.

> I wanted to know which
> mailing list should I subscribe to?

This one.  And look at https://gcc.gnu.org/lists.html to find if perhaps
to some more.

>
> I also wanted to know If I have my own idea, I want to present, what steps
> should I follow?

Send it to this (gcc@gcc.gnu.org) mailing list (and remember to put GSoC
somewhere into the subject).

In any case, remember to read "Before you apply" section of our GSoC
wiki page at https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply and
make sure you are able to build, install and test GCC and then have it
generate dumps and step through some function during compilation.

Good luck,

Martin Jambor

Gsoc

[FuN traveller]

Hello ,
I am Darshan jadhao, computer engineering student at Lovely Professional
University Jalandhar,
I would like to work with GNU on the project *Make C/C++ not automatically
promote memory_order_consume to memory_order_acquire*.
Email: darshan18jadhao99@gmail.com

Re: Gsoc

[Martin Jambor]

Hello Darshan,

On Tue, Mar 26 2019, FuN traveller wrote:
> Hello ,
> I am Darshan jadhao, computer engineering student at Lovely Professional
> University Jalandhar,
> I would like to work with GNU on the project *Make C/C++ not automatically
> promote memory_order_consume to memory_order_acquire*.
> Email: darshan18jadhao99@gmail.com

You posted the email twice, so I assume you are expecting an answer.
Your intention has been noted, I assume that you have also completed all
steps described in the "Before you apply" section of our GSoC wiki page
at https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply

If you have any specific question, please do not hesitate to ask on this
list.

Good luck,

Martin

Re: Gsoc

[FuN traveller]

Thank you, I will do that.

On Wed, 27 Mar 2019, 00:04 Martin Jambor <mjambor@suse.cz wrote:

> Hello Darshan,
>
> On Tue, Mar 26 2019, FuN traveller wrote:
> > Hello ,
> > I am Darshan jadhao, computer engineering student at Lovely Professional
> > University Jalandhar,
> > I would like to work with GNU on the project *Make C/C++ not
> automatically
> > promote memory_order_consume to memory_order_acquire*.
> > Email: darshan18jadhao99@gmail.com
>
> You posted the email twice, so I assume you are expecting an answer.
> Your intention has been noted, I assume that you have also completed all
> steps described in the "Before you apply" section of our GSoC wiki page
> at https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply
>
> If you have any specific question, please do not hesitate to ask on this
> list.
>
> Good luck,
>
> Martin
>

GSOC

[nick]

Greetings All,

I would like to take up parallelize compilation using threads or make c++/c 
memory issues not automatically promote. I did ask about this before but
not get a reply. When someone replies I'm just a little concerned as 
my writing for proposals has never been great so if someone just reviews
and doubt checks that's fine.

As for the other things building gcc and running the testsuite is fine. Plus
I already working on gcc so I've pretty aware of most things and this would
be a great steeping stone into more serious gcc development work.

If sample code is required that's in mainline gcc I sent out a trial patch
for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395

Cheers,

Nick

Re: GSOC

[David Malcolm]

On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> Greetings All,
> 
> I would like to take up parallelize compilation using threads or make
> c++/c 
> memory issues not automatically promote. I did ask about this before
> but
> not get a reply. When someone replies I'm just a little concerned as 
> my writing for proposals has never been great so if someone just
> reviews
> and doubt checks that's fine.
> 
> As for the other things building gcc and running the testsuite is
> fine. Plus
> I already working on gcc so I've pretty aware of most things and this
> would
> be a great steeping stone into more serious gcc development work.
> 
> If sample code is required that's in mainline gcc I sent out a trial
> patch
> for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> 
> Cheers,
> 
> Nick

It's good to see that you've gotten as far as attaching a patch to BZ
[1]

I think someone was going to attempt the "parallelize compilation using
threads" idea last year, but then pulled out before the summer; you may
want to check the archives (or was that you?)

IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
I don't know if he's still interested/able to spare the cycles.

That said, the parallel compilation one strikes me as very ambitious;
it's not clear to me what could realistically be done as a GSoC
project.  I think a good proposal on that would come up with some
subset of the problem that's doable over a summer, whilst also being
useful to the project.  The RTL infrastructure has a lot of global
state, so maybe either focus on the gimple passes, or on fixing global
state on the RTL side?  (I'm not sure)   Or maybe a project to be more
explicit about regions of the code that assume that the garbage-
collector can't run within them?[3] (since the GC is state that would
be shared by the threads).

Hope this is constructive/helpful
Dave

[1] though typically our workflow involved sending patches to the gcc-
patches mailing list
[2] as libgccjit maintainer I have an interest in global state within
the compiler
[3] I posted some ideas about this back in 2013 IIRC; probably
massively bit-rotted since then.  I also gave a talk at Cauldron 2013
about global state in the compiler (with a view to gcc-as-a-shared-
library); likewise I expect much of the ideas there to be out-of-date); 
for libgccjit I went with a different approach

Re: GSOC

[Richard Biener]

On Tue, 26 Mar 2019, David Malcolm wrote:

> On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > Greetings All,
> > 
> > I would like to take up parallelize compilation using threads or make
> > c++/c 
> > memory issues not automatically promote. I did ask about this before
> > but
> > not get a reply. When someone replies I'm just a little concerned as 
> > my writing for proposals has never been great so if someone just
> > reviews
> > and doubt checks that's fine.
> > 
> > As for the other things building gcc and running the testsuite is
> > fine. Plus
> > I already working on gcc so I've pretty aware of most things and this
> > would
> > be a great steeping stone into more serious gcc development work.
> > 
> > If sample code is required that's in mainline gcc I sent out a trial
> > patch
> > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > 
> > Cheers,
> > 
> > Nick
> 
> It's good to see that you've gotten as far as attaching a patch to BZ
> [1]
> 
> I think someone was going to attempt the "parallelize compilation using
> threads" idea last year, but then pulled out before the summer; you may
> want to check the archives (or was that you?)

There's also Giuliano Belinassi who is interested in the same project
(CCed).

> IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> I don't know if he's still interested/able to spare the cycles.

I've offered mentoring to Giuliano, so yes.

> That said, the parallel compilation one strikes me as very ambitious;
> it's not clear to me what could realistically be done as a GSoC
> project.  I think a good proposal on that would come up with some
> subset of the problem that's doable over a summer, whilst also being
> useful to the project.  The RTL infrastructure has a lot of global
> state, so maybe either focus on the gimple passes, or on fixing global
> state on the RTL side?  (I'm not sure)

That was the original intent for the experiment.  There's also
the already somewhat parallel WPA stage in LTO compilation mode
(but it simply forks for the sake of simplicity...).

> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).

The GC will be one obstackle.  The original idea was to drive
parallelization on the pass level by the pass manager for the
GIMPLE passes, so serialization points would be in it.

Richard.

> Hope this is constructive/helpful
> Dave
> 
> [1] though typically our workflow involved sending patches to the gcc-
> patches mailing list
> [2] as libgccjit maintainer I have an interest in global state within
> the compiler
> [3] I posted some ideas about this back in 2013 IIRC; probably
> massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> about global state in the compiler (with a view to gcc-as-a-shared-
> library); likewise I expect much of the ideas there to be out-of-date); 
> for libgccjit I went with a different approach

Re: GSOC

[nick]

On 2019-03-26 9:41 a.m., Richard Biener wrote:
> On Tue, 26 Mar 2019, David Malcolm wrote:
> 
>> On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
>>> Greetings All,
>>>
>>> I would like to take up parallelize compilation using threads or make
>>> c++/c 
>>> memory issues not automatically promote. I did ask about this before
>>> but
>>> not get a reply. When someone replies I'm just a little concerned as 
>>> my writing for proposals has never been great so if someone just
>>> reviews
>>> and doubt checks that's fine.
>>>
>>> As for the other things building gcc and running the testsuite is
>>> fine. Plus
>>> I already working on gcc so I've pretty aware of most things and this
>>> would
>>> be a great steeping stone into more serious gcc development work.
>>>
>>> If sample code is required that's in mainline gcc I sent out a trial
>>> patch
>>> for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
>>>
>>> Cheers,
>>>
>>> Nick
>>
>> It's good to see that you've gotten as far as attaching a patch to BZ
>> [1]
>>
>> I think someone was going to attempt the "parallelize compilation using
>> threads" idea last year, but then pulled out before the summer; you may
>> want to check the archives (or was that you?)
> 
> There's also Giuliano Belinassi who is interested in the same project
> (CCed).
> 
>> IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
>> I don't know if he's still interested/able to spare the cycles.
> 
> I've offered mentoring to Giuliano, so yes.
> 
That was just there because it was assumed wrong by me. I sent a proposed
probably correct patch to the gcc patches list without a commit message
as I wanted to make sure it was correct first. This is the said patch:
From a5fcad0cd1d1b79606ad9cec9a37d6a77ee50fc8 Mon Sep 17 00:00:00 2001
From: Nicholas Krause <xerofoify@gmail.com>
Date: Sun, 24 Mar 2019 15:07:18 -0400
Subject: [PATCH] Proposed patch to fix bug id, 89796 on bugzilla

Not sure if this is a correct fix to this bug found here:
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=89796 but
comments are welcome. If a backtrace is required please
let me know. I am just sending it to the development list
for review to make sure it's OK in terms of my understanding
the code.

Signed-off-by: Nicholas Krause <xerofoify@gmail.com>
---
 gcc/cp/constraint.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gcc/cp/constraint.cc b/gcc/cp/constraint.cc
index 9884eb0db50..a78d0a9a49b 100644
--- a/gcc/cp/constraint.cc
+++ b/gcc/cp/constraint.cc
@@ -1882,7 +1882,7 @@ tsubst_requires_expr (tree t, tree args,
   tree parms = TREE_OPERAND (t, 0);
   if (parms)
     {
-      parms = tsubst_constraint_variables (parms, args, complain, in_decl);
+      parms = tsubst_constraint_variables (PARM_CONSTR_PARMS (parms), args, complain, in_decl);
       if (parms == error_mark_node)
         return error_mark_node;
     }
-- 
2.17.1

>> That said, the parallel compilation one strikes me as very ambitious;
>> it's not clear to me what could realistically be done as a GSoC
>> project.  I think a good proposal on that would come up with some
>> subset of the problem that's doable over a summer, whilst also being
>> useful to the project.  The RTL infrastructure has a lot of global
>> state, so maybe either focus on the gimple passes, or on fixing global
>> state on the RTL side?  (I'm not sure)
> 
> That was the original intent for the experiment.  There's also
> the already somewhat parallel WPA stage in LTO compilation mode
> (but it simply forks for the sake of simplicity...).
> 
>> Or maybe a project to be more
>> explicit about regions of the code that assume that the garbage-
>> collector can't run within them?[3] (since the GC is state that would
>> be shared by the threads).
> 
The GC would make the most sense I think in terms of making this better
as shared state would normally slow this down.

Nick

> The GC will be one obstackle.  The original idea was to drive
> parallelization on the pass level by the pass manager for the
> GIMPLE passes, so serialization points would be in it.
> 
> Richard.
> 
>> Hope this is constructive/helpful
>> Dave
>>
>> [1] though typically our workflow involved sending patches to the gcc-
>> patches mailing list
>> [2] as libgccjit maintainer I have an interest in global state within
>> the compiler
>> [3] I posted some ideas about this back in 2013 IIRC; probably
>> massively bit-rotted since then.  I also gave a talk at Cauldron 2013
>> about global state in the compiler (with a view to gcc-as-a-shared-
>> library); likewise I expect much of the ideas there to be out-of-date); 
>> for libgccjit I went with a different approach

Re: GSOC

[Giuliano Belinassi]

Hi,

On 03/26, Richard Biener wrote:
> On Tue, 26 Mar 2019, David Malcolm wrote:
> 
> > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > Greetings All,
> > > 
> > > I would like to take up parallelize compilation using threads or make
> > > c++/c 
> > > memory issues not automatically promote. I did ask about this before
> > > but
> > > not get a reply. When someone replies I'm just a little concerned as 
> > > my writing for proposals has never been great so if someone just
> > > reviews
> > > and doubt checks that's fine.
> > > 
> > > As for the other things building gcc and running the testsuite is
> > > fine. Plus
> > > I already working on gcc so I've pretty aware of most things and this
> > > would
> > > be a great steeping stone into more serious gcc development work.
> > > 
> > > If sample code is required that's in mainline gcc I sent out a trial
> > > patch
> > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > 
> > > Cheers,
> > > 
> > > Nick
> > 
> > It's good to see that you've gotten as far as attaching a patch to BZ
> > [1]
> > 
> > I think someone was going to attempt the "parallelize compilation using
> > threads" idea last year, but then pulled out before the summer; you may
> > want to check the archives (or was that you?)
> 
> There's also Giuliano Belinassi who is interested in the same project
> (CCed).

Yes, I will apply for this project, and I will submit the final version
of my proposal by the end of the week.

Currently, my target is the `expand_all_functions` routine, as most of
the time is spent on it according to the experiments that I performed as
part of my Master's research on compiler parallelization.
(-O2, --disable-checking)

Thank you,
Giuliano.


> 
> > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > I don't know if he's still interested/able to spare the cycles.
> 
> I've offered mentoring to Giuliano, so yes.
> 
> > That said, the parallel compilation one strikes me as very ambitious;
> > it's not clear to me what could realistically be done as a GSoC
> > project.  I think a good proposal on that would come up with some
> > subset of the problem that's doable over a summer, whilst also being
> > useful to the project.  The RTL infrastructure has a lot of global
> > state, so maybe either focus on the gimple passes, or on fixing global
> > state on the RTL side?  (I'm not sure)
> 
> That was the original intent for the experiment.  There's also
> the already somewhat parallel WPA stage in LTO compilation mode
> (but it simply forks for the sake of simplicity...).
> 
> > Or maybe a project to be more
> > explicit about regions of the code that assume that the garbage-
> > collector can't run within them?[3] (since the GC is state that would
> > be shared by the threads).
> 
> The GC will be one obstackle.  The original idea was to drive
> parallelization on the pass level by the pass manager for the
> GIMPLE passes, so serialization points would be in it.
> 
> Richard.
> 
> > Hope this is constructive/helpful
> > Dave
> > 
> > [1] though typically our workflow involved sending patches to the gcc-
> > patches mailing list
> > [2] as libgccjit maintainer I have an interest in global state within
> > the compiler
> > [3] I posted some ideas about this back in 2013 IIRC; probably
> > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > about global state in the compiler (with a view to gcc-as-a-shared-
> > library); likewise I expect much of the ideas there to be out-of-date); 
> > for libgccjit I went with a different approach

Re: GSOC

[nick]

On 2019-03-27 9:55 a.m., Giuliano Belinassi wrote:
> Hi,
> 
> On 03/26, Richard Biener wrote:
>> On Tue, 26 Mar 2019, David Malcolm wrote:
>>
>>> On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
>>>> Greetings All,
>>>>
>>>> I would like to take up parallelize compilation using threads or make
>>>> c++/c 
>>>> memory issues not automatically promote. I did ask about this before
>>>> but
>>>> not get a reply. When someone replies I'm just a little concerned as 
>>>> my writing for proposals has never been great so if someone just
>>>> reviews
>>>> and doubt checks that's fine.
>>>>
>>>> As for the other things building gcc and running the testsuite is
>>>> fine. Plus
>>>> I already working on gcc so I've pretty aware of most things and this
>>>> would
>>>> be a great steeping stone into more serious gcc development work.
>>>>
>>>> If sample code is required that's in mainline gcc I sent out a trial
>>>> patch
>>>> for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
>>>>
>>>> Cheers,
>>>>
>>>> Nick
>>>
>>> It's good to see that you've gotten as far as attaching a patch to BZ
>>> [1]
>>>
>>> I think someone was going to attempt the "parallelize compilation using
>>> threads" idea last year, but then pulled out before the summer; you may
>>> want to check the archives (or was that you?)
>>
>> There's also Giuliano Belinassi who is interested in the same project
>> (CCed).
> 
> Yes, I will apply for this project, and I will submit the final version
> of my proposal by the end of the week.
> 
> Currently, my target is the `expand_all_functions` routine, as most of
> the time is spent on it according to the experiments that I performed as
> part of my Master's research on compiler parallelization.
> (-O2, --disable-checking)
> 
> Thank you,
> Giuliano.
> 
> 
My goal was this:
Or maybe a project to be more
explicit about regions of the code that assume that the garbage-
collector can't run within them?[3] (since the GC is state that would
be shared by the threads).

So there is no conflict between me and Giuliano. Richard and me have
already been going back and forth. The remaining tasks for me are
just write the proposal as the big one and I asked Richard to sent
me a example you guys liked. I've signed up to contribute for the
last year so that's fine.

Just letting the list known as well as Richard where I am,

Nick
>>
>>> IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
>>> I don't know if he's still interested/able to spare the cycles.
>>
>> I've offered mentoring to Giuliano, so yes.
>>
>>> That said, the parallel compilation one strikes me as very ambitious;
>>> it's not clear to me what could realistically be done as a GSoC
>>> project.  I think a good proposal on that would come up with some
>>> subset of the problem that's doable over a summer, whilst also being
>>> useful to the project.  The RTL infrastructure has a lot of global
>>> state, so maybe either focus on the gimple passes, or on fixing global
>>> state on the RTL side?  (I'm not sure)
>>
>> That was the original intent for the experiment.  There's also
>> the already somewhat parallel WPA stage in LTO compilation mode
>> (but it simply forks for the sake of simplicity...).
>>
>>> Or maybe a project to be more
>>> explicit about regions of the code that assume that the garbage-
>>> collector can't run within them?[3] (since the GC is state that would
>>> be shared by the threads).
>>
>> The GC will be one obstackle.  The original idea was to drive
>> parallelization on the pass level by the pass manager for the
>> GIMPLE passes, so serialization points would be in it.
>>
>> Richard.
>>
>>> Hope this is constructive/helpful
>>> Dave
>>>
>>> [1] though typically our workflow involved sending patches to the gcc-
>>> patches mailing list
>>> [2] as libgccjit maintainer I have an interest in global state within
>>> the compiler
>>> [3] I posted some ideas about this back in 2013 IIRC; probably
>>> massively bit-rotted since then.  I also gave a talk at Cauldron 2013
>>> about global state in the compiler (with a view to gcc-as-a-shared-
>>> library); likewise I expect much of the ideas there to be out-of-date); 
>>> for libgccjit I went with a different approach

Re: GSOC

[Richard Biener]

On Wed, Mar 27, 2019 at 3:43 PM nick <xerofoify@gmail.com> wrote:
>
>
>
> On 2019-03-27 9:55 a.m., Giuliano Belinassi wrote:
> > Hi,
> >
> > On 03/26, Richard Biener wrote:
> >> On Tue, 26 Mar 2019, David Malcolm wrote:
> >>
> >>> On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> >>>> Greetings All,
> >>>>
> >>>> I would like to take up parallelize compilation using threads or make
> >>>> c++/c
> >>>> memory issues not automatically promote. I did ask about this before
> >>>> but
> >>>> not get a reply. When someone replies I'm just a little concerned as
> >>>> my writing for proposals has never been great so if someone just
> >>>> reviews
> >>>> and doubt checks that's fine.
> >>>>
> >>>> As for the other things building gcc and running the testsuite is
> >>>> fine. Plus
> >>>> I already working on gcc so I've pretty aware of most things and this
> >>>> would
> >>>> be a great steeping stone into more serious gcc development work.
> >>>>
> >>>> If sample code is required that's in mainline gcc I sent out a trial
> >>>> patch
> >>>> for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Nick
> >>>
> >>> It's good to see that you've gotten as far as attaching a patch to BZ
> >>> [1]
> >>>
> >>> I think someone was going to attempt the "parallelize compilation using
> >>> threads" idea last year, but then pulled out before the summer; you may
> >>> want to check the archives (or was that you?)
> >>
> >> There's also Giuliano Belinassi who is interested in the same project
> >> (CCed).
> >
> > Yes, I will apply for this project, and I will submit the final version
> > of my proposal by the end of the week.
> >
> > Currently, my target is the `expand_all_functions` routine, as most of
> > the time is spent on it according to the experiments that I performed as
> > part of my Master's research on compiler parallelization.
> > (-O2, --disable-checking)
> >
> > Thank you,
> > Giuliano.
> >
> >
> My goal was this:
> Or maybe a project to be more
> explicit about regions of the code that assume that the garbage-
> collector can't run within them?[3] (since the GC is state that would
> be shared by the threads).

That's already pretty clear so a non-project.  Honestly you are somewhat
late to the project but if you can come up with a solid proposal we will
definitely have a look.  The project itself is of course large enough to
cover 10s of GSoC students ;)

Richard.

> So there is no conflict between me and Giuliano. Richard and me have
> already been going back and forth. The remaining tasks for me are
> just write the proposal as the big one and I asked Richard to sent
> me a example you guys liked. I've signed up to contribute for the
> last year so that's fine.
>
> Just letting the list known as well as Richard where I am,
>
> Nick
> >>
> >>> IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> >>> I don't know if he's still interested/able to spare the cycles.
> >>
> >> I've offered mentoring to Giuliano, so yes.
> >>
> >>> That said, the parallel compilation one strikes me as very ambitious;
> >>> it's not clear to me what could realistically be done as a GSoC
> >>> project.  I think a good proposal on that would come up with some
> >>> subset of the problem that's doable over a summer, whilst also being
> >>> useful to the project.  The RTL infrastructure has a lot of global
> >>> state, so maybe either focus on the gimple passes, or on fixing global
> >>> state on the RTL side?  (I'm not sure)
> >>
> >> That was the original intent for the experiment.  There's also
> >> the already somewhat parallel WPA stage in LTO compilation mode
> >> (but it simply forks for the sake of simplicity...).
> >>
> >>> Or maybe a project to be more
> >>> explicit about regions of the code that assume that the garbage-
> >>> collector can't run within them?[3] (since the GC is state that would
> >>> be shared by the threads).
> >>
> >> The GC will be one obstackle.  The original idea was to drive
> >> parallelization on the pass level by the pass manager for the
> >> GIMPLE passes, so serialization points would be in it.
> >>
> >> Richard.
> >>
> >>> Hope this is constructive/helpful
> >>> Dave
> >>>
> >>> [1] though typically our workflow involved sending patches to the gcc-
> >>> patches mailing list
> >>> [2] as libgccjit maintainer I have an interest in global state within
> >>> the compiler
> >>> [3] I posted some ideas about this back in 2013 IIRC; probably
> >>> massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> >>> about global state in the compiler (with a view to gcc-as-a-shared-
> >>> library); likewise I expect much of the ideas there to be out-of-date);
> >>> for libgccjit I went with a different approach

Re: GSOC

[Richard Biener]

On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
<giuliano.belinassi@usp.br> wrote:
>
> Hi,
>
> On 03/26, Richard Biener wrote:
> > On Tue, 26 Mar 2019, David Malcolm wrote:
> >
> > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > Greetings All,
> > > >
> > > > I would like to take up parallelize compilation using threads or make
> > > > c++/c
> > > > memory issues not automatically promote. I did ask about this before
> > > > but
> > > > not get a reply. When someone replies I'm just a little concerned as
> > > > my writing for proposals has never been great so if someone just
> > > > reviews
> > > > and doubt checks that's fine.
> > > >
> > > > As for the other things building gcc and running the testsuite is
> > > > fine. Plus
> > > > I already working on gcc so I've pretty aware of most things and this
> > > > would
> > > > be a great steeping stone into more serious gcc development work.
> > > >
> > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > patch
> > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > >
> > > > Cheers,
> > > >
> > > > Nick
> > >
> > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > [1]
> > >
> > > I think someone was going to attempt the "parallelize compilation using
> > > threads" idea last year, but then pulled out before the summer; you may
> > > want to check the archives (or was that you?)
> >
> > There's also Giuliano Belinassi who is interested in the same project
> > (CCed).
>
> Yes, I will apply for this project, and I will submit the final version
> of my proposal by the end of the week.
>
> Currently, my target is the `expand_all_functions` routine, as most of
> the time is spent on it according to the experiments that I performed as
> part of my Master's research on compiler parallelization.
> (-O2, --disable-checking)

Yes, more specifically I think the realistic target is the GIMPLE part
of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
cgraph_node::expand.  If you look at passes.def you'll see all_passes
also contains RTL expansion (pass_expand) and the RTL optimization
queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
Without changing the pass hierarchy the obvious part that can be
handled would be the pass_all_optimizations pass sub-queue of
all_passes since those are all passes that perform transforms on the
GIMPLE IL where we have all functions in this state at the same time
and where no interactions between the functions happen anymore
and thus functions can be processed in parallel (as much as make
processes individual translation units in parallel).

To simplify the taks further a useful constraint is to not have
a single optimization pass executed multiple times at the same time
(otherwise you have to look at pass specific global states as well),
thus the parallel part could be coded in a way keeping per function
the state of what pass to execute next and have a scheduler pick
a function its next pass is "free", scheduling that to a fixed set of
worker threads.  There's no dependences between functions
for the scheduling but each pass has only one execution resource
in the pipeline.  You can start processing an arbitrarily large number
of functions but slow functions will keep others from advancing across
the pass it executes on.

Passes could of course be individually marked as thread-safe
(multiple instances execute concurrently).

Garbage collection is already in control of the pass manager which
would also be the thread scheduler.  For GC the remaining issue
is allocation which passes occasionally do.  Locking is the short
term solution for GSoC I guess, long-term per-thread GC pools
might be better (to not slow down non-threaded parts of the compiler).

Richard.

>
> Thank you,
> Giuliano.
>
>
> >
> > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > I don't know if he's still interested/able to spare the cycles.
> >
> > I've offered mentoring to Giuliano, so yes.
> >
> > > That said, the parallel compilation one strikes me as very ambitious;
> > > it's not clear to me what could realistically be done as a GSoC
> > > project.  I think a good proposal on that would come up with some
> > > subset of the problem that's doable over a summer, whilst also being
> > > useful to the project.  The RTL infrastructure has a lot of global
> > > state, so maybe either focus on the gimple passes, or on fixing global
> > > state on the RTL side?  (I'm not sure)
> >
> > That was the original intent for the experiment.  There's also
> > the already somewhat parallel WPA stage in LTO compilation mode
> > (but it simply forks for the sake of simplicity...).
> >
> > > Or maybe a project to be more
> > > explicit about regions of the code that assume that the garbage-
> > > collector can't run within them?[3] (since the GC is state that would
> > > be shared by the threads).
> >
> > The GC will be one obstackle.  The original idea was to drive
> > parallelization on the pass level by the pass manager for the
> > GIMPLE passes, so serialization points would be in it.
> >
> > Richard.
> >
> > > Hope this is constructive/helpful
> > > Dave
> > >
> > > [1] though typically our workflow involved sending patches to the gcc-
> > > patches mailing list
> > > [2] as libgccjit maintainer I have an interest in global state within
> > > the compiler
> > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > library); likewise I expect much of the ideas there to be out-of-date);
> > > for libgccjit I went with a different approach

Re: GSOC

[Giuliano Belinassi]

Hi, Richard

On 03/28, Richard Biener wrote:
> On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> <giuliano.belinassi@usp.br> wrote:
> >
> > Hi,
> >
> > On 03/26, Richard Biener wrote:
> > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > >
> > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > Greetings All,
> > > > >
> > > > > I would like to take up parallelize compilation using threads or make
> > > > > c++/c
> > > > > memory issues not automatically promote. I did ask about this before
> > > > > but
> > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > my writing for proposals has never been great so if someone just
> > > > > reviews
> > > > > and doubt checks that's fine.
> > > > >
> > > > > As for the other things building gcc and running the testsuite is
> > > > > fine. Plus
> > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > would
> > > > > be a great steeping stone into more serious gcc development work.
> > > > >
> > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > patch
> > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Nick
> > > >
> > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > [1]
> > > >
> > > > I think someone was going to attempt the "parallelize compilation using
> > > > threads" idea last year, but then pulled out before the summer; you may
> > > > want to check the archives (or was that you?)
> > >
> > > There's also Giuliano Belinassi who is interested in the same project
> > > (CCed).
> >
> > Yes, I will apply for this project, and I will submit the final version
> > of my proposal by the end of the week.
> >
> > Currently, my target is the `expand_all_functions` routine, as most of
> > the time is spent on it according to the experiments that I performed as
> > part of my Master's research on compiler parallelization.
> > (-O2, --disable-checking)
> 
> Yes, more specifically I think the realistic target is the GIMPLE part
> of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> cgraph_node::expand.  If you look at passes.def you'll see all_passes
> also contains RTL expansion (pass_expand) and the RTL optimization
> queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> Without changing the pass hierarchy the obvious part that can be
> handled would be the pass_all_optimizations pass sub-queue of
> all_passes since those are all passes that perform transforms on the
> GIMPLE IL where we have all functions in this state at the same time
> and where no interactions between the functions happen anymore
> and thus functions can be processed in parallel (as much as make
> processes individual translation units in parallel).
> 

Great. So if I understood correctly, I will need to split
cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
refactor `expand_all_functions` so that the loop

     for (i = new_order_pos - 1; i >= 0; i--)

 use these three functions, then partition

     g->get_passes()->all_passes

into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
can run RTL after GIMPLE is finished, to finally start the
paralellization of per function GIMPLE passes.

> To simplify the taks further a useful constraint is to not have
> a single optimization pass executed multiple times at the same time
> (otherwise you have to look at pass specific global states as well),
> thus the parallel part could be coded in a way keeping per function
> the state of what pass to execute next and have a scheduler pick
> a function its next pass is "free", scheduling that to a fixed set of
> worker threads.  There's no dependences between functions
> for the scheduling but each pass has only one execution resource
> in the pipeline.  You can start processing an arbitrarily large number
> of functions but slow functions will keep others from advancing across
> the pass it executes on.
>

Something like a pipeline? That is certainly a start, but if one pass is
very slow wouldn't it bottleneck everything?

> Passes could of course be individually marked as thread-safe
> (multiple instances execute concurrently).
> 
> Garbage collection is already in control of the pass manager which
> would also be the thread scheduler.  For GC the remaining issue
> is allocation which passes occasionally do.  Locking is the short
> term solution for GSoC I guess, long-term per-thread GC pools
> might be better (to not slow down non-threaded parts of the compiler).
> 
> Richard.
> 
> >
> > Thank you,
> > Giuliano.
> >
> >
> > >
> > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > I don't know if he's still interested/able to spare the cycles.
> > >
> > > I've offered mentoring to Giuliano, so yes.
> > >
> > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > it's not clear to me what could realistically be done as a GSoC
> > > > project.  I think a good proposal on that would come up with some
> > > > subset of the problem that's doable over a summer, whilst also being
> > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > state on the RTL side?  (I'm not sure)
> > >
> > > That was the original intent for the experiment.  There's also
> > > the already somewhat parallel WPA stage in LTO compilation mode
> > > (but it simply forks for the sake of simplicity...).
> > >
> > > > Or maybe a project to be more
> > > > explicit about regions of the code that assume that the garbage-
> > > > collector can't run within them?[3] (since the GC is state that would
> > > > be shared by the threads).
> > >
> > > The GC will be one obstackle.  The original idea was to drive
> > > parallelization on the pass level by the pass manager for the
> > > GIMPLE passes, so serialization points would be in it.
> > >
> > > Richard.
> > >
> > > > Hope this is constructive/helpful
> > > > Dave
> > > >
> > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > patches mailing list
> > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > the compiler
> > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > for libgccjit I went with a different approach

Thank you,
Giuliano.

Re: GSOC

[Richard Biener]

[-- Attachment #1: Type: text/plain, Size: 9672 bytes --]

On Thu, 28 Mar 2019, Giuliano Belinassi wrote:

> Hi, Richard
> 
> On 03/28, Richard Biener wrote:
> > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > <giuliano.belinassi@usp.br> wrote:
> > >
> > > Hi,
> > >
> > > On 03/26, Richard Biener wrote:
> > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > >
> > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > Greetings All,
> > > > > >
> > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > c++/c
> > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > but
> > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > my writing for proposals has never been great so if someone just
> > > > > > reviews
> > > > > > and doubt checks that's fine.
> > > > > >
> > > > > > As for the other things building gcc and running the testsuite is
> > > > > > fine. Plus
> > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > would
> > > > > > be a great steeping stone into more serious gcc development work.
> > > > > >
> > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > patch
> > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Nick
> > > > >
> > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > [1]
> > > > >
> > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > want to check the archives (or was that you?)
> > > >
> > > > There's also Giuliano Belinassi who is interested in the same project
> > > > (CCed).
> > >
> > > Yes, I will apply for this project, and I will submit the final version
> > > of my proposal by the end of the week.
> > >
> > > Currently, my target is the `expand_all_functions` routine, as most of
> > > the time is spent on it according to the experiments that I performed as
> > > part of my Master's research on compiler parallelization.
> > > (-O2, --disable-checking)
> > 
> > Yes, more specifically I think the realistic target is the GIMPLE part
> > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > also contains RTL expansion (pass_expand) and the RTL optimization
> > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > Without changing the pass hierarchy the obvious part that can be
> > handled would be the pass_all_optimizations pass sub-queue of
> > all_passes since those are all passes that perform transforms on the
> > GIMPLE IL where we have all functions in this state at the same time
> > and where no interactions between the functions happen anymore
> > and thus functions can be processed in parallel (as much as make
> > processes individual translation units in parallel).
> > 
> 
> Great. So if I understood correctly, I will need to split
> cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> refactor `expand_all_functions` so that the loop
> 
>      for (i = new_order_pos - 1; i >= 0; i--)
> 
>  use these three functions, then partition
> 
>      g->get_passes()->all_passes
> 
> into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> can run RTL after GIMPLE is finished, to finally start the
> paralellization of per function GIMPLE passes.

Yes, it involves refactoring of the loop - you may notice that
parts of the compilation pipeline are under control of the
pass manager (passes.c) but some is still manually driven
by symbol_table::compile.  Whether it's more convenient to
get more control stuffed to the pass manager and perform the
threading under its control (I'd say that would be the cleaner
design) or to try do this in the current ad-hoc parts remains
to be seen.  You can see symbol_table::compile hands over
control to the pass manager multiple times, first ipa_passes ()
then all_late_ipa_passes and finally the expand_all_functions code.

I guess it would simplify things if you'd split pass_all_passes
in passes.def at pass_expand like so:

diff --git a/gcc/passes.def b/gcc/passes.def
index 2fcd80e53a3..bb0453b36a7 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
   NEXT_PASS (pass_spectrev1);
   NEXT_PASS (pass_warn_function_noreturn);
   NEXT_PASS (pass_gen_hsail);
+  TERMINATE_PASS_LIST (all_passes)
 
-  NEXT_PASS (pass_expand);
-
-  NEXT_PASS (pass_rest_of_compilation);
-  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
+  INSERT_PASSES_AFTER (pass_rest_of_compilation)
+      NEXT_PASS (pass_expand);
       NEXT_PASS (pass_instantiate_virtual_regs);
       NEXT_PASS (pass_into_cfg_layout_mode);
       NEXT_PASS (pass_jump);
@@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
          NEXT_PASS (pass_final);
       POP_INSERT_PASSES ()
       NEXT_PASS (pass_df_finish);
-  POP_INSERT_PASSES ()
   NEXT_PASS (pass_clean_state);
-  TERMINATE_PASS_LIST (all_passes)
+  TERMINATE_PASS_LIST (pass_rest_of_compilation)

where to make things "work" again w/o threading you'd invoke
execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
right after the all_passes invocation in cgraph_node::expand.

You then can refactor things so the loop over the 'order' array
is done twice, once over all_passes (the set you then parallelize)
and once over pass_rest_of_compilation (which you can't parallelize
because of being in RTL).

The above patch needs more changes in pass manager code - a chance
to dive into it a little since that's where you'd change code.

> > To simplify the taks further a useful constraint is to not have
> > a single optimization pass executed multiple times at the same time
> > (otherwise you have to look at pass specific global states as well),
> > thus the parallel part could be coded in a way keeping per function
> > the state of what pass to execute next and have a scheduler pick
> > a function its next pass is "free", scheduling that to a fixed set of
> > worker threads.  There's no dependences between functions
> > for the scheduling but each pass has only one execution resource
> > in the pipeline.  You can start processing an arbitrarily large number
> > of functions but slow functions will keep others from advancing across
> > the pass it executes on.
> >
> 
> Something like a pipeline? That is certainly a start, but if one pass is
> very slow wouldn't it bottleneck everything?

Yes, something like a pipeline.  It's true a slow pass would
bottleneck things - as said, we can selectively make passes
thread safe in such cases.

> > Passes could of course be individually marked as thread-safe
> > (multiple instances execute concurrently).
> > 
> > Garbage collection is already in control of the pass manager which
> > would also be the thread scheduler.  For GC the remaining issue
> > is allocation which passes occasionally do.  Locking is the short
> > term solution for GSoC I guess, long-term per-thread GC pools
> > might be better (to not slow down non-threaded parts of the compiler).
> > 
> > Richard.
> > 
> > >
> > > Thank you,
> > > Giuliano.
> > >
> > >
> > > >
> > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > I don't know if he's still interested/able to spare the cycles.
> > > >
> > > > I've offered mentoring to Giuliano, so yes.
> > > >
> > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > project.  I think a good proposal on that would come up with some
> > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > state on the RTL side?  (I'm not sure)
> > > >
> > > > That was the original intent for the experiment.  There's also
> > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > (but it simply forks for the sake of simplicity...).
> > > >
> > > > > Or maybe a project to be more
> > > > > explicit about regions of the code that assume that the garbage-
> > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > be shared by the threads).
> > > >
> > > > The GC will be one obstackle.  The original idea was to drive
> > > > parallelization on the pass level by the pass manager for the
> > > > GIMPLE passes, so serialization points would be in it.
> > > >
> > > > Richard.
> > > >
> > > > > Hope this is constructive/helpful
> > > > > Dave
> > > > >
> > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > patches mailing list
> > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > the compiler
> > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > for libgccjit I went with a different approach
> 
> Thank you,
> Giuliano.
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG NÌrnberg)

Re: GSOC

[Giuliano Belinassi]

[-- Attachment #1: Type: text/plain, Size: 10505 bytes --]

Hi,

On 03/29, Richard Biener wrote:
> On Thu, 28 Mar 2019, Giuliano Belinassi wrote:
> 
> > Hi, Richard
> > 
> > On 03/28, Richard Biener wrote:
> > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > > <giuliano.belinassi@usp.br> wrote:
> > > >
> > > > Hi,
> > > >
> > > > On 03/26, Richard Biener wrote:
> > > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > > >
> > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > > Greetings All,
> > > > > > >
> > > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > > c++/c
> > > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > > but
> > > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > > my writing for proposals has never been great so if someone just
> > > > > > > reviews
> > > > > > > and doubt checks that's fine.
> > > > > > >
> > > > > > > As for the other things building gcc and running the testsuite is
> > > > > > > fine. Plus
> > > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > > would
> > > > > > > be a great steeping stone into more serious gcc development work.
> > > > > > >
> > > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > > patch
> > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Nick
> > > > > >
> > > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > > [1]
> > > > > >
> > > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > > want to check the archives (or was that you?)
> > > > >
> > > > > There's also Giuliano Belinassi who is interested in the same project
> > > > > (CCed).
> > > >
> > > > Yes, I will apply for this project, and I will submit the final version
> > > > of my proposal by the end of the week.
> > > >
> > > > Currently, my target is the `expand_all_functions` routine, as most of
> > > > the time is spent on it according to the experiments that I performed as
> > > > part of my Master's research on compiler parallelization.
> > > > (-O2, --disable-checking)
> > > 
> > > Yes, more specifically I think the realistic target is the GIMPLE part
> > > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > > also contains RTL expansion (pass_expand) and the RTL optimization
> > > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > > Without changing the pass hierarchy the obvious part that can be
> > > handled would be the pass_all_optimizations pass sub-queue of
> > > all_passes since those are all passes that perform transforms on the
> > > GIMPLE IL where we have all functions in this state at the same time
> > > and where no interactions between the functions happen anymore
> > > and thus functions can be processed in parallel (as much as make
> > > processes individual translation units in parallel).
> > > 
> > 
> > Great. So if I understood correctly, I will need to split
> > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> > refactor `expand_all_functions` so that the loop
> > 
> >      for (i = new_order_pos - 1; i >= 0; i--)
> > 
> >  use these three functions, then partition
> > 
> >      g->get_passes()->all_passes
> > 
> > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> > can run RTL after GIMPLE is finished, to finally start the
> > paralellization of per function GIMPLE passes.
> 
> Yes, it involves refactoring of the loop - you may notice that
> parts of the compilation pipeline are under control of the
> pass manager (passes.c) but some is still manually driven
> by symbol_table::compile.  Whether it's more convenient to
> get more control stuffed to the pass manager and perform the
> threading under its control (I'd say that would be the cleaner
> design) or to try do this in the current ad-hoc parts remains
> to be seen.  You can see symbol_table::compile hands over
> control to the pass manager multiple times, first ipa_passes ()
> then all_late_ipa_passes and finally the expand_all_functions code.
> 
> I guess it would simplify things if you'd split pass_all_passes
> in passes.def at pass_expand like so:
> 
> diff --git a/gcc/passes.def b/gcc/passes.def
> index 2fcd80e53a3..bb0453b36a7 100644
> --- a/gcc/passes.def
> +++ b/gcc/passes.def
> @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
>    NEXT_PASS (pass_spectrev1);
>    NEXT_PASS (pass_warn_function_noreturn);
>    NEXT_PASS (pass_gen_hsail);
> +  TERMINATE_PASS_LIST (all_passes)
>  
> -  NEXT_PASS (pass_expand);
> -
> -  NEXT_PASS (pass_rest_of_compilation);
> -  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
> +  INSERT_PASSES_AFTER (pass_rest_of_compilation)
> +      NEXT_PASS (pass_expand);
>        NEXT_PASS (pass_instantiate_virtual_regs);
>        NEXT_PASS (pass_into_cfg_layout_mode);
>        NEXT_PASS (pass_jump);
> @@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
>           NEXT_PASS (pass_final);
>        POP_INSERT_PASSES ()
>        NEXT_PASS (pass_df_finish);
> -  POP_INSERT_PASSES ()
>    NEXT_PASS (pass_clean_state);
> -  TERMINATE_PASS_LIST (all_passes)
> +  TERMINATE_PASS_LIST (pass_rest_of_compilation)
> 
> where to make things "work" again w/o threading you'd invoke
> execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
> right after the all_passes invocation in cgraph_node::expand.
> 
> You then can refactor things so the loop over the 'order' array
> is done twice, once over all_passes (the set you then parallelize)
> and once over pass_rest_of_compilation (which you can't parallelize
> because of being in RTL).
>

I managed to get it working today. However, I found an issue with the
statistics_fini_pass() and pass_init_dump_file(), which I had to
comment, and force a `return false` for every case, respectively. Then I
managed to compile some programs correctly with -O2. I have no idea why
yet, but I will keep searching. I've attached my patch here.



> The above patch needs more changes in pass manager code - a chance
> to dive into it a little since that's where you'd change code.
> 
> > > To simplify the taks further a useful constraint is to not have
> > > a single optimization pass executed multiple times at the same time
> > > (otherwise you have to look at pass specific global states as well),
> > > thus the parallel part could be coded in a way keeping per function
> > > the state of what pass to execute next and have a scheduler pick
> > > a function its next pass is "free", scheduling that to a fixed set of
> > > worker threads.  There's no dependences between functions
> > > for the scheduling but each pass has only one execution resource
> > > in the pipeline.  You can start processing an arbitrarily large number
> > > of functions but slow functions will keep others from advancing across
> > > the pass it executes on.
> > >
> > 
> > Something like a pipeline? That is certainly a start, but if one pass is
> > very slow wouldn't it bottleneck everything?
> 
> Yes, something like a pipeline.  It's true a slow pass would
> bottleneck things - as said, we can selectively make passes
> thread safe in such cases.
> 
> > > Passes could of course be individually marked as thread-safe
> > > (multiple instances execute concurrently).
> > > 
> > > Garbage collection is already in control of the pass manager which
> > > would also be the thread scheduler.  For GC the remaining issue
> > > is allocation which passes occasionally do.  Locking is the short
> > > term solution for GSoC I guess, long-term per-thread GC pools
> > > might be better (to not slow down non-threaded parts of the compiler).
> > > 
> > > Richard.
> > > 
> > > >
> > > > Thank you,
> > > > Giuliano.
> > > >
> > > >
> > > > >
> > > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > > I don't know if he's still interested/able to spare the cycles.
> > > > >
> > > > > I've offered mentoring to Giuliano, so yes.
> > > > >
> > > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > > project.  I think a good proposal on that would come up with some
> > > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > > state on the RTL side?  (I'm not sure)
> > > > >
> > > > > That was the original intent for the experiment.  There's also
> > > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > > (but it simply forks for the sake of simplicity...).
> > > > >
> > > > > > Or maybe a project to be more
> > > > > > explicit about regions of the code that assume that the garbage-
> > > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > > be shared by the threads).
> > > > >
> > > > > The GC will be one obstackle.  The original idea was to drive
> > > > > parallelization on the pass level by the pass manager for the
> > > > > GIMPLE passes, so serialization points would be in it.
> > > > >
> > > > > Richard.
> > > > >
> > > > > > Hope this is constructive/helpful
> > > > > > Dave
> > > > > >
> > > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > > patches mailing list
> > > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > > the compiler
> > > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > > for libgccjit I went with a different approach
> > 
> > Thank you,
> > Giuliano.
> > 
> 
> -- 
> Richard Biener <rguenther@suse.de>
> SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)


[-- Attachment #2: restof_compilation.patch --]
[-- Type: text/x-diff, Size: 3997 bytes --]

Index: gcc/cgraphunit.c
===================================================================
--- gcc/cgraphunit.c	(revision 270905)
+++ gcc/cgraphunit.c	(working copy)
@@ -2196,6 +2196,7 @@
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_START, NULL);
 
   execute_pass_list (cfun, g->get_passes ()->all_passes);
+  execute_pass_list (cfun, g->get_passes ()->get_rest_of_compilation ());
 
   /* Signal the end of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_END, NULL);
Index: gcc/pass_manager.h
===================================================================
--- gcc/pass_manager.h	(revision 270905)
+++ gcc/pass_manager.h	(working copy)
@@ -81,8 +81,9 @@
 
   opt_pass *get_rest_of_compilation () const
   {
-    return pass_rest_of_compilation_1;
+    return pass_rest_of_compilation;
   }
+
   opt_pass *get_clean_slate () const { return pass_clean_state_1; }
 
 public:
@@ -92,6 +93,7 @@
   opt_pass *all_lowering_passes;
   opt_pass *all_regular_ipa_passes;
   opt_pass *all_late_ipa_passes;
+  opt_pass *pass_rest_of_compilation;
 
   /* A map from static pass id to optimization pass.  */
   opt_pass **passes_by_id;
Index: gcc/passes.c
===================================================================
--- gcc/passes.c	(revision 270905)
+++ gcc/passes.c	(working copy)
@@ -958,6 +958,7 @@
   dump_pass_list (all_regular_ipa_passes, 1);
   dump_pass_list (all_late_ipa_passes, 1);
   dump_pass_list (all_passes, 1);
+  //dump_pass_list (pass_rest_of_compilation, 1);
 
   pop_dummy_function ();
 }
@@ -1468,6 +1469,8 @@
     success |= position_pass (pass_info, &all_late_ipa_passes);
   if (!success || all_instances)
     success |= position_pass (pass_info, &all_passes);
+  if (!success || all_instances)
+    success |= position_pass (pass_info, &pass_rest_of_compilation);
   if (!success)
     fatal_error
       (input_location,
@@ -1516,8 +1519,8 @@
 
 pass_manager::pass_manager (context *ctxt)
 : all_passes (NULL), all_small_ipa_passes (NULL), all_lowering_passes (NULL),
-  all_regular_ipa_passes (NULL),
-  all_late_ipa_passes (NULL), passes_by_id (NULL), passes_by_id_size (0),
+  all_regular_ipa_passes (NULL),  all_late_ipa_passes (NULL), 
+  pass_rest_of_compilation(NULL), passes_by_id (NULL), passes_by_id_size (0),
   m_ctxt (ctxt), m_name_to_pass_map (NULL)
 {
   opt_pass **p;
@@ -2011,7 +2014,7 @@
       && need_ssa_update_p (cfun))
     gcc_assert (flags & TODO_update_ssa_any);
 
-  statistics_fini_pass ();
+  //statistics_fini_pass ();
 
   if (flags)
     do_per_function (execute_function_todo, (void *)(size_t) flags);
@@ -2091,9 +2094,15 @@
 bool
 pass_init_dump_file (opt_pass *pass)
 {
+  return false;
   /* If a dump file name is present, open it if enabled.  */
   if (pass->static_pass_number != -1)
     {
+      if (pass->static_pass_number < 0)
+      {
+	fprintf(stderr, "pass_name: %s, pass->static_pass_number: %d\n", pass->name, 
+	       pass->static_pass_number);
+      }
       timevar_push (TV_DUMP);
       gcc::dump_manager *dumps = g->get_dumps ();
       bool initializing_dump =
Index: gcc/passes.def
===================================================================
--- gcc/passes.def	(revision 270905)
+++ gcc/passes.def	(working copy)
@@ -395,11 +395,13 @@
   NEXT_PASS (pass_cleanup_cfg_post_optimizing);
   NEXT_PASS (pass_warn_function_noreturn);
   NEXT_PASS (pass_gen_hsail);
+  TERMINATE_PASS_LIST(all_passes)
 
-  NEXT_PASS (pass_expand);
 
-  NEXT_PASS (pass_rest_of_compilation);
-  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
+
+
+  INSERT_PASSES_AFTER (pass_rest_of_compilation)
+      NEXT_PASS (pass_expand);
       NEXT_PASS (pass_instantiate_virtual_regs);
       NEXT_PASS (pass_into_cfg_layout_mode);
       NEXT_PASS (pass_jump);
@@ -497,6 +499,5 @@
 	  NEXT_PASS (pass_final);
       POP_INSERT_PASSES ()
       NEXT_PASS (pass_df_finish);
-  POP_INSERT_PASSES ()
   NEXT_PASS (pass_clean_state);
-  TERMINATE_PASS_LIST (all_passes)
+  TERMINATE_PASS_LIST (pass_rest_of_compilation)

Re: GSOC

[Richard Biener]

[-- Attachment #1: Type: text/plain, Size: 11726 bytes --]

On Mon, 6 May 2019, Giuliano Belinassi wrote:

> Hi,
> 
> On 03/29, Richard Biener wrote:
> > On Thu, 28 Mar 2019, Giuliano Belinassi wrote:
> > 
> > > Hi, Richard
> > > 
> > > On 03/28, Richard Biener wrote:
> > > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > > > <giuliano.belinassi@usp.br> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > On 03/26, Richard Biener wrote:
> > > > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > > > >
> > > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > > > Greetings All,
> > > > > > > >
> > > > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > > > c++/c
> > > > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > > > but
> > > > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > > > my writing for proposals has never been great so if someone just
> > > > > > > > reviews
> > > > > > > > and doubt checks that's fine.
> > > > > > > >
> > > > > > > > As for the other things building gcc and running the testsuite is
> > > > > > > > fine. Plus
> > > > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > > > would
> > > > > > > > be a great steeping stone into more serious gcc development work.
> > > > > > > >
> > > > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > > > patch
> > > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Nick
> > > > > > >
> > > > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > > > [1]
> > > > > > >
> > > > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > > > want to check the archives (or was that you?)
> > > > > >
> > > > > > There's also Giuliano Belinassi who is interested in the same project
> > > > > > (CCed).
> > > > >
> > > > > Yes, I will apply for this project, and I will submit the final version
> > > > > of my proposal by the end of the week.
> > > > >
> > > > > Currently, my target is the `expand_all_functions` routine, as most of
> > > > > the time is spent on it according to the experiments that I performed as
> > > > > part of my Master's research on compiler parallelization.
> > > > > (-O2, --disable-checking)
> > > > 
> > > > Yes, more specifically I think the realistic target is the GIMPLE part
> > > > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > > > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > > > also contains RTL expansion (pass_expand) and the RTL optimization
> > > > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > > > Without changing the pass hierarchy the obvious part that can be
> > > > handled would be the pass_all_optimizations pass sub-queue of
> > > > all_passes since those are all passes that perform transforms on the
> > > > GIMPLE IL where we have all functions in this state at the same time
> > > > and where no interactions between the functions happen anymore
> > > > and thus functions can be processed in parallel (as much as make
> > > > processes individual translation units in parallel).
> > > > 
> > > 
> > > Great. So if I understood correctly, I will need to split
> > > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> > > refactor `expand_all_functions` so that the loop
> > > 
> > >      for (i = new_order_pos - 1; i >= 0; i--)
> > > 
> > >  use these three functions, then partition
> > > 
> > >      g->get_passes()->all_passes
> > > 
> > > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> > > can run RTL after GIMPLE is finished, to finally start the
> > > paralellization of per function GIMPLE passes.
> > 
> > Yes, it involves refactoring of the loop - you may notice that
> > parts of the compilation pipeline are under control of the
> > pass manager (passes.c) but some is still manually driven
> > by symbol_table::compile.  Whether it's more convenient to
> > get more control stuffed to the pass manager and perform the
> > threading under its control (I'd say that would be the cleaner
> > design) or to try do this in the current ad-hoc parts remains
> > to be seen.  You can see symbol_table::compile hands over
> > control to the pass manager multiple times, first ipa_passes ()
> > then all_late_ipa_passes and finally the expand_all_functions code.
> > 
> > I guess it would simplify things if you'd split pass_all_passes
> > in passes.def at pass_expand like so:
> > 
> > diff --git a/gcc/passes.def b/gcc/passes.def
> > index 2fcd80e53a3..bb0453b36a7 100644
> > --- a/gcc/passes.def
> > +++ b/gcc/passes.def
> > @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
> >    NEXT_PASS (pass_spectrev1);
> >    NEXT_PASS (pass_warn_function_noreturn);
> >    NEXT_PASS (pass_gen_hsail);
> > +  TERMINATE_PASS_LIST (all_passes)
> >  
> > -  NEXT_PASS (pass_expand);
> > -
> > -  NEXT_PASS (pass_rest_of_compilation);
> > -  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
> > +  INSERT_PASSES_AFTER (pass_rest_of_compilation)
> > +      NEXT_PASS (pass_expand);
> >        NEXT_PASS (pass_instantiate_virtual_regs);
> >        NEXT_PASS (pass_into_cfg_layout_mode);
> >        NEXT_PASS (pass_jump);
> > @@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
> >           NEXT_PASS (pass_final);
> >        POP_INSERT_PASSES ()
> >        NEXT_PASS (pass_df_finish);
> > -  POP_INSERT_PASSES ()
> >    NEXT_PASS (pass_clean_state);
> > -  TERMINATE_PASS_LIST (all_passes)
> > +  TERMINATE_PASS_LIST (pass_rest_of_compilation)
> > 
> > where to make things "work" again w/o threading you'd invoke
> > execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
> > right after the all_passes invocation in cgraph_node::expand.
> > 
> > You then can refactor things so the loop over the 'order' array
> > is done twice, once over all_passes (the set you then parallelize)
> > and once over pass_rest_of_compilation (which you can't parallelize
> > because of being in RTL).
> >
> 
> I managed to get it working today. However, I found an issue with the
> statistics_fini_pass() and pass_init_dump_file(), which I had to
> comment, and force a `return false` for every case, respectively. Then I
> managed to compile some programs correctly with -O2. I have no idea why
> yet, but I will keep searching. I've attached my patch here.

It may be that you need to adjust the GCC_PASS_LISTS define in
pass_manager.h, changing pass_rest_of_compilation to a pass list
and also remove its "old" definition in passes.c.  Or it might
be simpler to not re-use pass_rest_of_compilation but wrap
the tail in a new all_passes2 or so.  You'll also see
a call to register_dump_files (all_passes) in passes.c where
you probably need to do the same for the new tail.

As usual grep is your best friend when figuring out what to do
(doing that right now myself).

Richard.

> 
> 
> > The above patch needs more changes in pass manager code - a chance
> > to dive into it a little since that's where you'd change code.
> > 
> > > > To simplify the taks further a useful constraint is to not have
> > > > a single optimization pass executed multiple times at the same time
> > > > (otherwise you have to look at pass specific global states as well),
> > > > thus the parallel part could be coded in a way keeping per function
> > > > the state of what pass to execute next and have a scheduler pick
> > > > a function its next pass is "free", scheduling that to a fixed set of
> > > > worker threads.  There's no dependences between functions
> > > > for the scheduling but each pass has only one execution resource
> > > > in the pipeline.  You can start processing an arbitrarily large number
> > > > of functions but slow functions will keep others from advancing across
> > > > the pass it executes on.
> > > >
> > > 
> > > Something like a pipeline? That is certainly a start, but if one pass is
> > > very slow wouldn't it bottleneck everything?
> > 
> > Yes, something like a pipeline.  It's true a slow pass would
> > bottleneck things - as said, we can selectively make passes
> > thread safe in such cases.
> > 
> > > > Passes could of course be individually marked as thread-safe
> > > > (multiple instances execute concurrently).
> > > > 
> > > > Garbage collection is already in control of the pass manager which
> > > > would also be the thread scheduler.  For GC the remaining issue
> > > > is allocation which passes occasionally do.  Locking is the short
> > > > term solution for GSoC I guess, long-term per-thread GC pools
> > > > might be better (to not slow down non-threaded parts of the compiler).
> > > > 
> > > > Richard.
> > > > 
> > > > >
> > > > > Thank you,
> > > > > Giuliano.
> > > > >
> > > > >
> > > > > >
> > > > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > > > I don't know if he's still interested/able to spare the cycles.
> > > > > >
> > > > > > I've offered mentoring to Giuliano, so yes.
> > > > > >
> > > > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > > > project.  I think a good proposal on that would come up with some
> > > > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > > > state on the RTL side?  (I'm not sure)
> > > > > >
> > > > > > That was the original intent for the experiment.  There's also
> > > > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > > > (but it simply forks for the sake of simplicity...).
> > > > > >
> > > > > > > Or maybe a project to be more
> > > > > > > explicit about regions of the code that assume that the garbage-
> > > > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > > > be shared by the threads).
> > > > > >
> > > > > > The GC will be one obstackle.  The original idea was to drive
> > > > > > parallelization on the pass level by the pass manager for the
> > > > > > GIMPLE passes, so serialization points would be in it.
> > > > > >
> > > > > > Richard.
> > > > > >
> > > > > > > Hope this is constructive/helpful
> > > > > > > Dave
> > > > > > >
> > > > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > > > patches mailing list
> > > > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > > > the compiler
> > > > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > > > for libgccjit I went with a different approach
> > > 
> > > Thank you,
> > > Giuliano.
> > > 
> > 
> > -- 
> > Richard Biener <rguenther@suse.de>
> > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)

Re: GSOC

[Giuliano Belinassi]

[-- Attachment #1: Type: text/plain, Size: 14204 bytes --]

Hi, Richard

On 05/07, Richard Biener wrote:
> On Mon, 6 May 2019, Giuliano Belinassi wrote:
> 
> > Hi,
> > 
> > On 03/29, Richard Biener wrote:
> > > On Thu, 28 Mar 2019, Giuliano Belinassi wrote:
> > > 
> > > > Hi, Richard
> > > > 
> > > > On 03/28, Richard Biener wrote:
> > > > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > > > > <giuliano.belinassi@usp.br> wrote:
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > On 03/26, Richard Biener wrote:
> > > > > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > > > > >
> > > > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > > > > Greetings All,
> > > > > > > > >
> > > > > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > > > > c++/c
> > > > > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > > > > but
> > > > > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > > > > my writing for proposals has never been great so if someone just
> > > > > > > > > reviews
> > > > > > > > > and doubt checks that's fine.
> > > > > > > > >
> > > > > > > > > As for the other things building gcc and running the testsuite is
> > > > > > > > > fine. Plus
> > > > > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > > > > would
> > > > > > > > > be a great steeping stone into more serious gcc development work.
> > > > > > > > >
> > > > > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > > > > patch
> > > > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Nick
> > > > > > > >
> > > > > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > > > > [1]
> > > > > > > >
> > > > > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > > > > want to check the archives (or was that you?)
> > > > > > >
> > > > > > > There's also Giuliano Belinassi who is interested in the same project
> > > > > > > (CCed).
> > > > > >
> > > > > > Yes, I will apply for this project, and I will submit the final version
> > > > > > of my proposal by the end of the week.
> > > > > >
> > > > > > Currently, my target is the `expand_all_functions` routine, as most of
> > > > > > the time is spent on it according to the experiments that I performed as
> > > > > > part of my Master's research on compiler parallelization.
> > > > > > (-O2, --disable-checking)
> > > > > 
> > > > > Yes, more specifically I think the realistic target is the GIMPLE part
> > > > > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > > > > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > > > > also contains RTL expansion (pass_expand) and the RTL optimization
> > > > > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > > > > Without changing the pass hierarchy the obvious part that can be
> > > > > handled would be the pass_all_optimizations pass sub-queue of
> > > > > all_passes since those are all passes that perform transforms on the
> > > > > GIMPLE IL where we have all functions in this state at the same time
> > > > > and where no interactions between the functions happen anymore
> > > > > and thus functions can be processed in parallel (as much as make
> > > > > processes individual translation units in parallel).
> > > > > 
> > > > 
> > > > Great. So if I understood correctly, I will need to split
> > > > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> > > > refactor `expand_all_functions` so that the loop
> > > > 
> > > >      for (i = new_order_pos - 1; i >= 0; i--)
> > > > 
> > > >  use these three functions, then partition
> > > > 
> > > >      g->get_passes()->all_passes
> > > > 
> > > > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> > > > can run RTL after GIMPLE is finished, to finally start the
> > > > paralellization of per function GIMPLE passes.
> > > 
> > > Yes, it involves refactoring of the loop - you may notice that
> > > parts of the compilation pipeline are under control of the
> > > pass manager (passes.c) but some is still manually driven
> > > by symbol_table::compile.  Whether it's more convenient to
> > > get more control stuffed to the pass manager and perform the
> > > threading under its control (I'd say that would be the cleaner
> > > design) or to try do this in the current ad-hoc parts remains
> > > to be seen.  You can see symbol_table::compile hands over
> > > control to the pass manager multiple times, first ipa_passes ()
> > > then all_late_ipa_passes and finally the expand_all_functions code.
> > > 
> > > I guess it would simplify things if you'd split pass_all_passes
> > > in passes.def at pass_expand like so:
> > > 
> > > diff --git a/gcc/passes.def b/gcc/passes.def
> > > index 2fcd80e53a3..bb0453b36a7 100644
> > > --- a/gcc/passes.def
> > > +++ b/gcc/passes.def
> > > @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
> > >    NEXT_PASS (pass_spectrev1);
> > >    NEXT_PASS (pass_warn_function_noreturn);
> > >    NEXT_PASS (pass_gen_hsail);
> > > +  TERMINATE_PASS_LIST (all_passes)
> > >  
> > > -  NEXT_PASS (pass_expand);
> > > -
> > > -  NEXT_PASS (pass_rest_of_compilation);
> > > -  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
> > > +  INSERT_PASSES_AFTER (pass_rest_of_compilation)
> > > +      NEXT_PASS (pass_expand);
> > >        NEXT_PASS (pass_instantiate_virtual_regs);
> > >        NEXT_PASS (pass_into_cfg_layout_mode);
> > >        NEXT_PASS (pass_jump);
> > > @@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
> > >           NEXT_PASS (pass_final);
> > >        POP_INSERT_PASSES ()
> > >        NEXT_PASS (pass_df_finish);
> > > -  POP_INSERT_PASSES ()
> > >    NEXT_PASS (pass_clean_state);
> > > -  TERMINATE_PASS_LIST (all_passes)
> > > +  TERMINATE_PASS_LIST (pass_rest_of_compilation)
> > > 
> > > where to make things "work" again w/o threading you'd invoke
> > > execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
> > > right after the all_passes invocation in cgraph_node::expand.
> > > 
> > > You then can refactor things so the loop over the 'order' array
> > > is done twice, once over all_passes (the set you then parallelize)
> > > and once over pass_rest_of_compilation (which you can't parallelize
> > > because of being in RTL).
> > >
> > 
> > I managed to get it working today. However, I found an issue with the
> > statistics_fini_pass() and pass_init_dump_file(), which I had to
> > comment, and force a `return false` for every case, respectively. Then I
> > managed to compile some programs correctly with -O2. I have no idea why
> > yet, but I will keep searching. I've attached my patch here.
> 
> It may be that you need to adjust the GCC_PASS_LISTS define in
> pass_manager.h, changing pass_rest_of_compilation to a pass list
> and also remove its "old" definition in passes.c.  Or it might
> be simpler to not re-use pass_rest_of_compilation but wrap
> the tail in a new all_passes2 or so.  You'll also see
> a call to register_dump_files (all_passes) in passes.c where
> you probably need to do the same for the new tail.
> 
> As usual grep is your best friend when figuring out what to do
> (doing that right now myself).
> 
> Richard.

I am just posting this as an update.

Just to contextualize, I've managed to fix this issue by following Biener's
advices. However what happened next was that the pr87600-2.c was
crashing the compiler, as discussed in IRC. The reason of the crash was
that after the discussed modifications, the `pass->next` after `pass_expand`
was not `pass_cleanup`, since the NEXT_PASS after `pass_expand` was
`pass_instantiate_virtual_regs`. I fixed that by not reusing the
`pass_rest_of_compilation`, but by creating a new pass called `all_rtl_passes`
and keeping `pass_expand` before the `PUSH_INSERT_PASSES_WITHIN`.

After that, I've run the full `dg.exp` testsuite and found that the
following tests are also failing:

FAIL: gcc.dg/20020919-1.c (internal compiler error)
FAIL: gcc.dg/always_inline3.c (internal compiler error)
FAIL: gcc.dg/dx-test.c (internal compiler error)
FAIL: gcc.dg/pr32370.c (internal compiler error)
FAIL: gcc.dg/pr61692.c (internal compiler error)
FAIL: gcc.dg/pr87600-2.c (internal compiler error)
FAIL: gcc.dg/va-arg-pack-1.c (internal compiler error)
FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++14  (test for warnings, line 125)
FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++17  (test for warnings, line 125)
FAIL: g++.dg/opt/pr69213.C  -std=gnu++98 (internal compiler error)
FAIL: g++.dg/opt/pr69213.C  -std=gnu++14 (internal compiler error)
FAIL: g++.dg/opt/pr69213.C  -std=gnu++17 (internal compiler error)

Notice that pr87600-2.c is failing again, but now when trying to compile the
`long test3 (void)` function. Interestingly, if I comment every function but
test3(), the test passes. All tests seems to be failed in the assert:

 gcc_assert (TREE_ASM_WRITTEN (decl))

of cgraphunit.c:2255, and the rtl dumps seems always to be different from
trunk.

I've attached a patch to my modifications, if anyone is interested in it.
I hope to provide a better feedback in the next days.

> 
> > 
> > 
> > > The above patch needs more changes in pass manager code - a chance
> > > to dive into it a little since that's where you'd change code.
> > > 
> > > > > To simplify the taks further a useful constraint is to not have
> > > > > a single optimization pass executed multiple times at the same time
> > > > > (otherwise you have to look at pass specific global states as well),
> > > > > thus the parallel part could be coded in a way keeping per function
> > > > > the state of what pass to execute next and have a scheduler pick
> > > > > a function its next pass is "free", scheduling that to a fixed set of
> > > > > worker threads.  There's no dependences between functions
> > > > > for the scheduling but each pass has only one execution resource
> > > > > in the pipeline.  You can start processing an arbitrarily large number
> > > > > of functions but slow functions will keep others from advancing across
> > > > > the pass it executes on.
> > > > >
> > > > 
> > > > Something like a pipeline? That is certainly a start, but if one pass is
> > > > very slow wouldn't it bottleneck everything?
> > > 
> > > Yes, something like a pipeline.  It's true a slow pass would
> > > bottleneck things - as said, we can selectively make passes
> > > thread safe in such cases.
> > > 
> > > > > Passes could of course be individually marked as thread-safe
> > > > > (multiple instances execute concurrently).
> > > > > 
> > > > > Garbage collection is already in control of the pass manager which
> > > > > would also be the thread scheduler.  For GC the remaining issue
> > > > > is allocation which passes occasionally do.  Locking is the short
> > > > > term solution for GSoC I guess, long-term per-thread GC pools
> > > > > might be better (to not slow down non-threaded parts of the compiler).
> > > > > 
> > > > > Richard.
> > > > > 
> > > > > >
> > > > > > Thank you,
> > > > > > Giuliano.
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > > > > I don't know if he's still interested/able to spare the cycles.
> > > > > > >
> > > > > > > I've offered mentoring to Giuliano, so yes.
> > > > > > >
> > > > > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > > > > project.  I think a good proposal on that would come up with some
> > > > > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > > > > state on the RTL side?  (I'm not sure)
> > > > > > >
> > > > > > > That was the original intent for the experiment.  There's also
> > > > > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > > > > (but it simply forks for the sake of simplicity...).
> > > > > > >
> > > > > > > > Or maybe a project to be more
> > > > > > > > explicit about regions of the code that assume that the garbage-
> > > > > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > > > > be shared by the threads).
> > > > > > >
> > > > > > > The GC will be one obstackle.  The original idea was to drive
> > > > > > > parallelization on the pass level by the pass manager for the
> > > > > > > GIMPLE passes, so serialization points would be in it.
> > > > > > >
> > > > > > > Richard.
> > > > > > >
> > > > > > > > Hope this is constructive/helpful
> > > > > > > > Dave
> > > > > > > >
> > > > > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > > > > patches mailing list
> > > > > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > > > > the compiler
> > > > > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > > > > for libgccjit I went with a different approach
> > > > 
> > > > Thank you,
> > > > Giuliano.
> > > > 
> > > 
> > > -- 
> > > Richard Biener <rguenther@suse.de>
> > > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> > 
> > 
> 
> -- 
> Richard Biener <rguenther@suse.de>
> SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
Giuliano.


[-- Attachment #2: 0001-Split-rtl_passes-from-all_passes.patch --]
[-- Type: text/x-diff, Size: 4107 bytes --]

From d3fcfa095d209550c814e878b744f7d9b2394729 Mon Sep 17 00:00:00 2001
From: Giuliano Belinassi <giuliano.belinassi@usp.br>
Date: Sun, 12 May 2019 13:07:57 -0300
Subject: Split rtl_passes from all_passes

Signed-off-by: Giuliano Belinassi <giuliano.belinassi@usp.br>
---
 gcc/cgraphunit.c   |  2 ++
 gcc/pass_manager.h |  2 ++
 gcc/passes.c       | 10 +++++++---
 gcc/passes.def     |  5 +++--
 4 files changed, 14 insertions(+), 5 deletions(-)

diff --git a/gcc/cgraphunit.c b/gcc/cgraphunit.c
index 8bfbd0bb12f..2c730dfba12 100644
--- a/gcc/cgraphunit.c
+++ b/gcc/cgraphunit.c
@@ -2196,6 +2196,8 @@ cgraph_node::expand (void)
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_START, NULL);
 
   execute_pass_list (cfun, g->get_passes ()->all_passes);
+  if (!(rtl_dump_and_exit || flag_syntax_only || seen_error ()))
+      execute_pass_list (cfun, g->get_passes ()->all_rtl_passes);
 
   /* Signal the end of passes.  */
   invoke_plugin_callbacks (PLUGIN_ALL_PASSES_END, NULL);
diff --git a/gcc/pass_manager.h b/gcc/pass_manager.h
index 8c7d272f20f..473fc6a9df5 100644
--- a/gcc/pass_manager.h
+++ b/gcc/pass_manager.h
@@ -30,6 +30,7 @@ struct register_pass_info;
   DEF_PASS_LIST (all_small_ipa_passes) \
   DEF_PASS_LIST (all_regular_ipa_passes) \
   DEF_PASS_LIST (all_late_ipa_passes) \
+  DEF_PASS_LIST (all_rtl_passes) \
   DEF_PASS_LIST (all_passes)
 
 #define DEF_PASS_LIST(LIST) PASS_LIST_NO_##LIST,
@@ -92,6 +93,7 @@ public:
   opt_pass *all_lowering_passes;
   opt_pass *all_regular_ipa_passes;
   opt_pass *all_late_ipa_passes;
+  opt_pass *all_rtl_passes;
 
   /* A map from static pass id to optimization pass.  */
   opt_pass **passes_by_id;
diff --git a/gcc/passes.c b/gcc/passes.c
index a9c22889d71..b76065470a8 100644
--- a/gcc/passes.c
+++ b/gcc/passes.c
@@ -958,6 +958,7 @@ pass_manager::dump_passes () const
   dump_pass_list (all_regular_ipa_passes, 1);
   dump_pass_list (all_late_ipa_passes, 1);
   dump_pass_list (all_passes, 1);
+  dump_pass_list (all_rtl_passes, 1);
 
   pop_dummy_function ();
 }
@@ -1468,6 +1469,8 @@ pass_manager::register_pass (struct register_pass_info *pass_info)
     success |= position_pass (pass_info, &all_late_ipa_passes);
   if (!success || all_instances)
     success |= position_pass (pass_info, &all_passes);
+  if (!success || all_instances)
+    success |= position_pass (pass_info, &all_rtl_passes);
   if (!success)
     fatal_error
       (input_location,
@@ -1515,9 +1518,9 @@ pass_manager::register_pass (struct register_pass_info *pass_info)
 */
 
 pass_manager::pass_manager (context *ctxt)
-: all_passes (NULL), all_small_ipa_passes (NULL), all_lowering_passes (NULL),
-  all_regular_ipa_passes (NULL),
-  all_late_ipa_passes (NULL), passes_by_id (NULL), passes_by_id_size (0),
+: all_passes (NULL), all_rtl_passes (NULL), all_small_ipa_passes (NULL), all_lowering_passes (NULL),
+  all_regular_ipa_passes (NULL),  all_late_ipa_passes (NULL),
+  passes_by_id (NULL), passes_by_id_size (0),
   m_ctxt (ctxt), m_name_to_pass_map (NULL)
 {
   opt_pass **p;
@@ -1595,6 +1598,7 @@ pass_manager::pass_manager (context *ctxt)
   register_dump_files (all_regular_ipa_passes);
   register_dump_files (all_late_ipa_passes);
   register_dump_files (all_passes);
+  register_dump_files (all_rtl_passes);
 }
 
 static void
diff --git a/gcc/passes.def b/gcc/passes.def
index ad2efabd385..246775c1bed 100644
--- a/gcc/passes.def
+++ b/gcc/passes.def
@@ -395,9 +395,10 @@ along with GCC; see the file COPYING3.  If not see
   NEXT_PASS (pass_cleanup_cfg_post_optimizing);
   NEXT_PASS (pass_warn_function_noreturn);
   NEXT_PASS (pass_gen_hsail);
+  TERMINATE_PASS_LIST (all_passes)
 
+  INSERT_PASSES_AFTER (all_rtl_passes)
   NEXT_PASS (pass_expand);
-
   NEXT_PASS (pass_rest_of_compilation);
   PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
       NEXT_PASS (pass_instantiate_virtual_regs);
@@ -499,4 +500,4 @@ along with GCC; see the file COPYING3.  If not see
       NEXT_PASS (pass_df_finish);
   POP_INSERT_PASSES ()
   NEXT_PASS (pass_clean_state);
-  TERMINATE_PASS_LIST (all_passes)
+  TERMINATE_PASS_LIST (all_rtl_passes)
-- 
2.20.1

Re: GSOC

[Richard Biener]

[-- Attachment #1: Type: text/plain, Size: 15385 bytes --]

On Sun, 12 May 2019, Giuliano Belinassi wrote:

> Hi, Richard
> 
> On 05/07, Richard Biener wrote:
> > On Mon, 6 May 2019, Giuliano Belinassi wrote:
> > 
> > > Hi,
> > > 
> > > On 03/29, Richard Biener wrote:
> > > > On Thu, 28 Mar 2019, Giuliano Belinassi wrote:
> > > > 
> > > > > Hi, Richard
> > > > > 
> > > > > On 03/28, Richard Biener wrote:
> > > > > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > > > > > <giuliano.belinassi@usp.br> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > On 03/26, Richard Biener wrote:
> > > > > > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > > > > > >
> > > > > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > > > > > Greetings All,
> > > > > > > > > >
> > > > > > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > > > > > c++/c
> > > > > > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > > > > > but
> > > > > > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > > > > > my writing for proposals has never been great so if someone just
> > > > > > > > > > reviews
> > > > > > > > > > and doubt checks that's fine.
> > > > > > > > > >
> > > > > > > > > > As for the other things building gcc and running the testsuite is
> > > > > > > > > > fine. Plus
> > > > > > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > > > > > would
> > > > > > > > > > be a great steeping stone into more serious gcc development work.
> > > > > > > > > >
> > > > > > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > > > > > patch
> > > > > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > >
> > > > > > > > > > Nick
> > > > > > > > >
> > > > > > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > > > > > [1]
> > > > > > > > >
> > > > > > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > > > > > want to check the archives (or was that you?)
> > > > > > > >
> > > > > > > > There's also Giuliano Belinassi who is interested in the same project
> > > > > > > > (CCed).
> > > > > > >
> > > > > > > Yes, I will apply for this project, and I will submit the final version
> > > > > > > of my proposal by the end of the week.
> > > > > > >
> > > > > > > Currently, my target is the `expand_all_functions` routine, as most of
> > > > > > > the time is spent on it according to the experiments that I performed as
> > > > > > > part of my Master's research on compiler parallelization.
> > > > > > > (-O2, --disable-checking)
> > > > > > 
> > > > > > Yes, more specifically I think the realistic target is the GIMPLE part
> > > > > > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > > > > > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > > > > > also contains RTL expansion (pass_expand) and the RTL optimization
> > > > > > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > > > > > Without changing the pass hierarchy the obvious part that can be
> > > > > > handled would be the pass_all_optimizations pass sub-queue of
> > > > > > all_passes since those are all passes that perform transforms on the
> > > > > > GIMPLE IL where we have all functions in this state at the same time
> > > > > > and where no interactions between the functions happen anymore
> > > > > > and thus functions can be processed in parallel (as much as make
> > > > > > processes individual translation units in parallel).
> > > > > > 
> > > > > 
> > > > > Great. So if I understood correctly, I will need to split
> > > > > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> > > > > refactor `expand_all_functions` so that the loop
> > > > > 
> > > > >      for (i = new_order_pos - 1; i >= 0; i--)
> > > > > 
> > > > >  use these three functions, then partition
> > > > > 
> > > > >      g->get_passes()->all_passes
> > > > > 
> > > > > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> > > > > can run RTL after GIMPLE is finished, to finally start the
> > > > > paralellization of per function GIMPLE passes.
> > > > 
> > > > Yes, it involves refactoring of the loop - you may notice that
> > > > parts of the compilation pipeline are under control of the
> > > > pass manager (passes.c) but some is still manually driven
> > > > by symbol_table::compile.  Whether it's more convenient to
> > > > get more control stuffed to the pass manager and perform the
> > > > threading under its control (I'd say that would be the cleaner
> > > > design) or to try do this in the current ad-hoc parts remains
> > > > to be seen.  You can see symbol_table::compile hands over
> > > > control to the pass manager multiple times, first ipa_passes ()
> > > > then all_late_ipa_passes and finally the expand_all_functions code.
> > > > 
> > > > I guess it would simplify things if you'd split pass_all_passes
> > > > in passes.def at pass_expand like so:
> > > > 
> > > > diff --git a/gcc/passes.def b/gcc/passes.def
> > > > index 2fcd80e53a3..bb0453b36a7 100644
> > > > --- a/gcc/passes.def
> > > > +++ b/gcc/passes.def
> > > > @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
> > > >    NEXT_PASS (pass_spectrev1);
> > > >    NEXT_PASS (pass_warn_function_noreturn);
> > > >    NEXT_PASS (pass_gen_hsail);
> > > > +  TERMINATE_PASS_LIST (all_passes)
> > > >  
> > > > -  NEXT_PASS (pass_expand);
> > > > -
> > > > -  NEXT_PASS (pass_rest_of_compilation);
> > > > -  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
> > > > +  INSERT_PASSES_AFTER (pass_rest_of_compilation)
> > > > +      NEXT_PASS (pass_expand);
> > > >        NEXT_PASS (pass_instantiate_virtual_regs);
> > > >        NEXT_PASS (pass_into_cfg_layout_mode);
> > > >        NEXT_PASS (pass_jump);
> > > > @@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
> > > >           NEXT_PASS (pass_final);
> > > >        POP_INSERT_PASSES ()
> > > >        NEXT_PASS (pass_df_finish);
> > > > -  POP_INSERT_PASSES ()
> > > >    NEXT_PASS (pass_clean_state);
> > > > -  TERMINATE_PASS_LIST (all_passes)
> > > > +  TERMINATE_PASS_LIST (pass_rest_of_compilation)
> > > > 
> > > > where to make things "work" again w/o threading you'd invoke
> > > > execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
> > > > right after the all_passes invocation in cgraph_node::expand.
> > > > 
> > > > You then can refactor things so the loop over the 'order' array
> > > > is done twice, once over all_passes (the set you then parallelize)
> > > > and once over pass_rest_of_compilation (which you can't parallelize
> > > > because of being in RTL).
> > > >
> > > 
> > > I managed to get it working today. However, I found an issue with the
> > > statistics_fini_pass() and pass_init_dump_file(), which I had to
> > > comment, and force a `return false` for every case, respectively. Then I
> > > managed to compile some programs correctly with -O2. I have no idea why
> > > yet, but I will keep searching. I've attached my patch here.
> > 
> > It may be that you need to adjust the GCC_PASS_LISTS define in
> > pass_manager.h, changing pass_rest_of_compilation to a pass list
> > and also remove its "old" definition in passes.c.  Or it might
> > be simpler to not re-use pass_rest_of_compilation but wrap
> > the tail in a new all_passes2 or so.  You'll also see
> > a call to register_dump_files (all_passes) in passes.c where
> > you probably need to do the same for the new tail.
> > 
> > As usual grep is your best friend when figuring out what to do
> > (doing that right now myself).
> > 
> > Richard.
> 
> I am just posting this as an update.
> 
> Just to contextualize, I've managed to fix this issue by following Biener's
> advices. However what happened next was that the pr87600-2.c was
> crashing the compiler, as discussed in IRC. The reason of the crash was
> that after the discussed modifications, the `pass->next` after `pass_expand`
> was not `pass_cleanup`, since the NEXT_PASS after `pass_expand` was
> `pass_instantiate_virtual_regs`. I fixed that by not reusing the
> `pass_rest_of_compilation`, but by creating a new pass called `all_rtl_passes`
> and keeping `pass_expand` before the `PUSH_INSERT_PASSES_WITHIN`.
> 
> After that, I've run the full `dg.exp` testsuite and found that the
> following tests are also failing:
> 
> FAIL: gcc.dg/20020919-1.c (internal compiler error)
> FAIL: gcc.dg/always_inline3.c (internal compiler error)
> FAIL: gcc.dg/dx-test.c (internal compiler error)
> FAIL: gcc.dg/pr32370.c (internal compiler error)
> FAIL: gcc.dg/pr61692.c (internal compiler error)
> FAIL: gcc.dg/pr87600-2.c (internal compiler error)
> FAIL: gcc.dg/va-arg-pack-1.c (internal compiler error)
> FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++14  (test for warnings, line 125)
> FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++17  (test for warnings, line 125)
> FAIL: g++.dg/opt/pr69213.C  -std=gnu++98 (internal compiler error)
> FAIL: g++.dg/opt/pr69213.C  -std=gnu++14 (internal compiler error)
> FAIL: g++.dg/opt/pr69213.C  -std=gnu++17 (internal compiler error)
> 
> Notice that pr87600-2.c is failing again, but now when trying to compile the
> `long test3 (void)` function. Interestingly, if I comment every function but
> test3(), the test passes. All tests seems to be failed in the assert:
> 
>  gcc_assert (TREE_ASM_WRITTEN (decl))
> 
> of cgraphunit.c:2255, and the rtl dumps seems always to be different from
> trunk.
> 
> I've attached a patch to my modifications, if anyone is interested in it.
> I hope to provide a better feedback in the next days.

Btw, if you are familiar with git I encourage you to publish your
changes on a branch.  Conveniently there's a github mirror of the
GCC repository you can branch off.  It might be easier to work
on a not-so-fast moving target which would mean branching off the
gcc-9-branch for example (or just not rebase your branch too often).

Thanks,
Richard.

> > 
> > > 
> > > 
> > > > The above patch needs more changes in pass manager code - a chance
> > > > to dive into it a little since that's where you'd change code.
> > > > 
> > > > > > To simplify the taks further a useful constraint is to not have
> > > > > > a single optimization pass executed multiple times at the same time
> > > > > > (otherwise you have to look at pass specific global states as well),
> > > > > > thus the parallel part could be coded in a way keeping per function
> > > > > > the state of what pass to execute next and have a scheduler pick
> > > > > > a function its next pass is "free", scheduling that to a fixed set of
> > > > > > worker threads.  There's no dependences between functions
> > > > > > for the scheduling but each pass has only one execution resource
> > > > > > in the pipeline.  You can start processing an arbitrarily large number
> > > > > > of functions but slow functions will keep others from advancing across
> > > > > > the pass it executes on.
> > > > > >
> > > > > 
> > > > > Something like a pipeline? That is certainly a start, but if one pass is
> > > > > very slow wouldn't it bottleneck everything?
> > > > 
> > > > Yes, something like a pipeline.  It's true a slow pass would
> > > > bottleneck things - as said, we can selectively make passes
> > > > thread safe in such cases.
> > > > 
> > > > > > Passes could of course be individually marked as thread-safe
> > > > > > (multiple instances execute concurrently).
> > > > > > 
> > > > > > Garbage collection is already in control of the pass manager which
> > > > > > would also be the thread scheduler.  For GC the remaining issue
> > > > > > is allocation which passes occasionally do.  Locking is the short
> > > > > > term solution for GSoC I guess, long-term per-thread GC pools
> > > > > > might be better (to not slow down non-threaded parts of the compiler).
> > > > > > 
> > > > > > Richard.
> > > > > > 
> > > > > > >
> > > > > > > Thank you,
> > > > > > > Giuliano.
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > > > > > I don't know if he's still interested/able to spare the cycles.
> > > > > > > >
> > > > > > > > I've offered mentoring to Giuliano, so yes.
> > > > > > > >
> > > > > > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > > > > > project.  I think a good proposal on that would come up with some
> > > > > > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > > > > > state on the RTL side?  (I'm not sure)
> > > > > > > >
> > > > > > > > That was the original intent for the experiment.  There's also
> > > > > > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > > > > > (but it simply forks for the sake of simplicity...).
> > > > > > > >
> > > > > > > > > Or maybe a project to be more
> > > > > > > > > explicit about regions of the code that assume that the garbage-
> > > > > > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > > > > > be shared by the threads).
> > > > > > > >
> > > > > > > > The GC will be one obstackle.  The original idea was to drive
> > > > > > > > parallelization on the pass level by the pass manager for the
> > > > > > > > GIMPLE passes, so serialization points would be in it.
> > > > > > > >
> > > > > > > > Richard.
> > > > > > > >
> > > > > > > > > Hope this is constructive/helpful
> > > > > > > > > Dave
> > > > > > > > >
> > > > > > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > > > > > patches mailing list
> > > > > > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > > > > > the compiler
> > > > > > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > > > > > for libgccjit I went with a different approach
> > > > > 
> > > > > Thank you,
> > > > > Giuliano.
> > > > > 
> > > > 
> > > > -- 
> > > > Richard Biener <rguenther@suse.de>
> > > > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> > > 
> > > 
> > 
> > -- 
> > Richard Biener <rguenther@suse.de>
> > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> Giuliano.
> 
> 

-- 
Richard Biener <rguenther@suse.de>
SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)

Re: GSOC

[Giuliano Belinassi]

On 05/13, Richard Biener wrote:
> On Sun, 12 May 2019, Giuliano Belinassi wrote:
> 
> > Hi, Richard
> > 
> > On 05/07, Richard Biener wrote:
> > > On Mon, 6 May 2019, Giuliano Belinassi wrote:
> > > 
> > > > Hi,
> > > > 
> > > > On 03/29, Richard Biener wrote:
> > > > > On Thu, 28 Mar 2019, Giuliano Belinassi wrote:
> > > > > 
> > > > > > Hi, Richard
> > > > > > 
> > > > > > On 03/28, Richard Biener wrote:
> > > > > > > On Wed, Mar 27, 2019 at 2:55 PM Giuliano Belinassi
> > > > > > > <giuliano.belinassi@usp.br> wrote:
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > On 03/26, Richard Biener wrote:
> > > > > > > > > On Tue, 26 Mar 2019, David Malcolm wrote:
> > > > > > > > >
> > > > > > > > > > On Mon, 2019-03-25 at 19:51 -0400, nick wrote:
> > > > > > > > > > > Greetings All,
> > > > > > > > > > >
> > > > > > > > > > > I would like to take up parallelize compilation using threads or make
> > > > > > > > > > > c++/c
> > > > > > > > > > > memory issues not automatically promote. I did ask about this before
> > > > > > > > > > > but
> > > > > > > > > > > not get a reply. When someone replies I'm just a little concerned as
> > > > > > > > > > > my writing for proposals has never been great so if someone just
> > > > > > > > > > > reviews
> > > > > > > > > > > and doubt checks that's fine.
> > > > > > > > > > >
> > > > > > > > > > > As for the other things building gcc and running the testsuite is
> > > > > > > > > > > fine. Plus
> > > > > > > > > > > I already working on gcc so I've pretty aware of most things and this
> > > > > > > > > > > would
> > > > > > > > > > > be a great steeping stone into more serious gcc development work.
> > > > > > > > > > >
> > > > > > > > > > > If sample code is required that's in mainline gcc I sent out a trial
> > > > > > > > > > > patch
> > > > > > > > > > > for this issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88395
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > >
> > > > > > > > > > > Nick
> > > > > > > > > >
> > > > > > > > > > It's good to see that you've gotten as far as attaching a patch to BZ
> > > > > > > > > > [1]
> > > > > > > > > >
> > > > > > > > > > I think someone was going to attempt the "parallelize compilation using
> > > > > > > > > > threads" idea last year, but then pulled out before the summer; you may
> > > > > > > > > > want to check the archives (or was that you?)
> > > > > > > > >
> > > > > > > > > There's also Giuliano Belinassi who is interested in the same project
> > > > > > > > > (CCed).
> > > > > > > >
> > > > > > > > Yes, I will apply for this project, and I will submit the final version
> > > > > > > > of my proposal by the end of the week.
> > > > > > > >
> > > > > > > > Currently, my target is the `expand_all_functions` routine, as most of
> > > > > > > > the time is spent on it according to the experiments that I performed as
> > > > > > > > part of my Master's research on compiler parallelization.
> > > > > > > > (-O2, --disable-checking)
> > > > > > > 
> > > > > > > Yes, more specifically I think the realistic target is the GIMPLE part
> > > > > > > of   execute_pass_list (cfun, g->get_passes ()->all_passes);  done in
> > > > > > > cgraph_node::expand.  If you look at passes.def you'll see all_passes
> > > > > > > also contains RTL expansion (pass_expand) and the RTL optimization
> > > > > > > queue (pass_rest_of_compilation).  The RTL part isn't a realistic target.
> > > > > > > Without changing the pass hierarchy the obvious part that can be
> > > > > > > handled would be the pass_all_optimizations pass sub-queue of
> > > > > > > all_passes since those are all passes that perform transforms on the
> > > > > > > GIMPLE IL where we have all functions in this state at the same time
> > > > > > > and where no interactions between the functions happen anymore
> > > > > > > and thus functions can be processed in parallel (as much as make
> > > > > > > processes individual translation units in parallel).
> > > > > > > 
> > > > > > 
> > > > > > Great. So if I understood correctly, I will need to split
> > > > > > cgraph_node::expand() into three parts: IPA, GIMPLE and RTL, and then
> > > > > > refactor `expand_all_functions` so that the loop
> > > > > > 
> > > > > >      for (i = new_order_pos - 1; i >= 0; i--)
> > > > > > 
> > > > > >  use these three functions, then partition
> > > > > > 
> > > > > >      g->get_passes()->all_passes
> > > > > > 
> > > > > > into get_passes()->gimple_passes and get_passes()->rtl_passes, so I
> > > > > > can run RTL after GIMPLE is finished, to finally start the
> > > > > > paralellization of per function GIMPLE passes.
> > > > > 
> > > > > Yes, it involves refactoring of the loop - you may notice that
> > > > > parts of the compilation pipeline are under control of the
> > > > > pass manager (passes.c) but some is still manually driven
> > > > > by symbol_table::compile.  Whether it's more convenient to
> > > > > get more control stuffed to the pass manager and perform the
> > > > > threading under its control (I'd say that would be the cleaner
> > > > > design) or to try do this in the current ad-hoc parts remains
> > > > > to be seen.  You can see symbol_table::compile hands over
> > > > > control to the pass manager multiple times, first ipa_passes ()
> > > > > then all_late_ipa_passes and finally the expand_all_functions code.
> > > > > 
> > > > > I guess it would simplify things if you'd split pass_all_passes
> > > > > in passes.def at pass_expand like so:
> > > > > 
> > > > > diff --git a/gcc/passes.def b/gcc/passes.def
> > > > > index 2fcd80e53a3..bb0453b36a7 100644
> > > > > --- a/gcc/passes.def
> > > > > +++ b/gcc/passes.def
> > > > > @@ -403,11 +403,10 @@ along with GCC; see the file COPYING3.  If not see
> > > > >    NEXT_PASS (pass_spectrev1);
> > > > >    NEXT_PASS (pass_warn_function_noreturn);
> > > > >    NEXT_PASS (pass_gen_hsail);
> > > > > +  TERMINATE_PASS_LIST (all_passes)
> > > > >  
> > > > > -  NEXT_PASS (pass_expand);
> > > > > -
> > > > > -  NEXT_PASS (pass_rest_of_compilation);
> > > > > -  PUSH_INSERT_PASSES_WITHIN (pass_rest_of_compilation)
> > > > > +  INSERT_PASSES_AFTER (pass_rest_of_compilation)
> > > > > +      NEXT_PASS (pass_expand);
> > > > >        NEXT_PASS (pass_instantiate_virtual_regs);
> > > > >        NEXT_PASS (pass_into_cfg_layout_mode);
> > > > >        NEXT_PASS (pass_jump);
> > > > > @@ -505,6 +504,5 @@ along with GCC; see the file COPYING3.  If not see
> > > > >           NEXT_PASS (pass_final);
> > > > >        POP_INSERT_PASSES ()
> > > > >        NEXT_PASS (pass_df_finish);
> > > > > -  POP_INSERT_PASSES ()
> > > > >    NEXT_PASS (pass_clean_state);
> > > > > -  TERMINATE_PASS_LIST (all_passes)
> > > > > +  TERMINATE_PASS_LIST (pass_rest_of_compilation)
> > > > > 
> > > > > where to make things "work" again w/o threading you'd invoke
> > > > > execute_pass_list (cfun, g->get_passes ()->pass_rest_of_compilation)
> > > > > right after the all_passes invocation in cgraph_node::expand.
> > > > > 
> > > > > You then can refactor things so the loop over the 'order' array
> > > > > is done twice, once over all_passes (the set you then parallelize)
> > > > > and once over pass_rest_of_compilation (which you can't parallelize
> > > > > because of being in RTL).
> > > > >
> > > > 
> > > > I managed to get it working today. However, I found an issue with the
> > > > statistics_fini_pass() and pass_init_dump_file(), which I had to
> > > > comment, and force a `return false` for every case, respectively. Then I
> > > > managed to compile some programs correctly with -O2. I have no idea why
> > > > yet, but I will keep searching. I've attached my patch here.
> > > 
> > > It may be that you need to adjust the GCC_PASS_LISTS define in
> > > pass_manager.h, changing pass_rest_of_compilation to a pass list
> > > and also remove its "old" definition in passes.c.  Or it might
> > > be simpler to not re-use pass_rest_of_compilation but wrap
> > > the tail in a new all_passes2 or so.  You'll also see
> > > a call to register_dump_files (all_passes) in passes.c where
> > > you probably need to do the same for the new tail.
> > > 
> > > As usual grep is your best friend when figuring out what to do
> > > (doing that right now myself).
> > > 
> > > Richard.
> > 
> > I am just posting this as an update.
> > 
> > Just to contextualize, I've managed to fix this issue by following Biener's
> > advices. However what happened next was that the pr87600-2.c was
> > crashing the compiler, as discussed in IRC. The reason of the crash was
> > that after the discussed modifications, the `pass->next` after `pass_expand`
> > was not `pass_cleanup`, since the NEXT_PASS after `pass_expand` was
> > `pass_instantiate_virtual_regs`. I fixed that by not reusing the
> > `pass_rest_of_compilation`, but by creating a new pass called `all_rtl_passes`
> > and keeping `pass_expand` before the `PUSH_INSERT_PASSES_WITHIN`.
> > 
> > After that, I've run the full `dg.exp` testsuite and found that the
> > following tests are also failing:
> > 
> > FAIL: gcc.dg/20020919-1.c (internal compiler error)
> > FAIL: gcc.dg/always_inline3.c (internal compiler error)
> > FAIL: gcc.dg/dx-test.c (internal compiler error)
> > FAIL: gcc.dg/pr32370.c (internal compiler error)
> > FAIL: gcc.dg/pr61692.c (internal compiler error)
> > FAIL: gcc.dg/pr87600-2.c (internal compiler error)
> > FAIL: gcc.dg/va-arg-pack-1.c (internal compiler error)
> > FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++14  (test for warnings, line 125)
> > FAIL: g++.dg/cpp0x/Wattributes1.C  -std=c++17  (test for warnings, line 125)
> > FAIL: g++.dg/opt/pr69213.C  -std=gnu++98 (internal compiler error)
> > FAIL: g++.dg/opt/pr69213.C  -std=gnu++14 (internal compiler error)
> > FAIL: g++.dg/opt/pr69213.C  -std=gnu++17 (internal compiler error)
> > 
> > Notice that pr87600-2.c is failing again, but now when trying to compile the
> > `long test3 (void)` function. Interestingly, if I comment every function but
> > test3(), the test passes. All tests seems to be failed in the assert:
> > 
> >  gcc_assert (TREE_ASM_WRITTEN (decl))
> > 
> > of cgraphunit.c:2255, and the rtl dumps seems always to be different from
> > trunk.
> > 
> > I've attached a patch to my modifications, if anyone is interested in it.
> > I hope to provide a better feedback in the next days.
> 
> Btw, if you are familiar with git I encourage you to publish your
> changes on a branch.  Conveniently there's a github mirror of the
> GCC repository you can branch off.  It might be easier to work
> on a not-so-fast moving target which would mean branching off the
> gcc-9-branch for example (or just not rebase your branch too often).
> 

I am working on a mirror that I created:
https://gitlab.com/flusp/gcc/tree/giulianob_parallel

I also managed to fix all problems that I mentioned earlier. Basically
the check before calling the get_all_rtl_passes () was incorrect.

> Thanks,
> Richard.
> 
> > > 
> > > > 
> > > > 
> > > > > The above patch needs more changes in pass manager code - a chance
> > > > > to dive into it a little since that's where you'd change code.
> > > > > 
> > > > > > > To simplify the taks further a useful constraint is to not have
> > > > > > > a single optimization pass executed multiple times at the same time
> > > > > > > (otherwise you have to look at pass specific global states as well),
> > > > > > > thus the parallel part could be coded in a way keeping per function
> > > > > > > the state of what pass to execute next and have a scheduler pick
> > > > > > > a function its next pass is "free", scheduling that to a fixed set of
> > > > > > > worker threads.  There's no dependences between functions
> > > > > > > for the scheduling but each pass has only one execution resource
> > > > > > > in the pipeline.  You can start processing an arbitrarily large number
> > > > > > > of functions but slow functions will keep others from advancing across
> > > > > > > the pass it executes on.
> > > > > > >
> > > > > > 
> > > > > > Something like a pipeline? That is certainly a start, but if one pass is
> > > > > > very slow wouldn't it bottleneck everything?
> > > > > 
> > > > > Yes, something like a pipeline.  It's true a slow pass would
> > > > > bottleneck things - as said, we can selectively make passes
> > > > > thread safe in such cases.
> > > > > 
> > > > > > > Passes could of course be individually marked as thread-safe
> > > > > > > (multiple instances execute concurrently).
> > > > > > > 
> > > > > > > Garbage collection is already in control of the pass manager which
> > > > > > > would also be the thread scheduler.  For GC the remaining issue
> > > > > > > is allocation which passes occasionally do.  Locking is the short
> > > > > > > term solution for GSoC I guess, long-term per-thread GC pools
> > > > > > > might be better (to not slow down non-threaded parts of the compiler).
> > > > > > > 
> > > > > > > Richard.
> > > > > > > 
> > > > > > > >
> > > > > > > > Thank you,
> > > > > > > > Giuliano.
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > IIRC Richard [CCed] was going to mentor, with me co-mentoring [2] - but
> > > > > > > > > > I don't know if he's still interested/able to spare the cycles.
> > > > > > > > >
> > > > > > > > > I've offered mentoring to Giuliano, so yes.
> > > > > > > > >
> > > > > > > > > > That said, the parallel compilation one strikes me as very ambitious;
> > > > > > > > > > it's not clear to me what could realistically be done as a GSoC
> > > > > > > > > > project.  I think a good proposal on that would come up with some
> > > > > > > > > > subset of the problem that's doable over a summer, whilst also being
> > > > > > > > > > useful to the project.  The RTL infrastructure has a lot of global
> > > > > > > > > > state, so maybe either focus on the gimple passes, or on fixing global
> > > > > > > > > > state on the RTL side?  (I'm not sure)
> > > > > > > > >
> > > > > > > > > That was the original intent for the experiment.  There's also
> > > > > > > > > the already somewhat parallel WPA stage in LTO compilation mode
> > > > > > > > > (but it simply forks for the sake of simplicity...).
> > > > > > > > >
> > > > > > > > > > Or maybe a project to be more
> > > > > > > > > > explicit about regions of the code that assume that the garbage-
> > > > > > > > > > collector can't run within them?[3] (since the GC is state that would
> > > > > > > > > > be shared by the threads).
> > > > > > > > >
> > > > > > > > > The GC will be one obstackle.  The original idea was to drive
> > > > > > > > > parallelization on the pass level by the pass manager for the
> > > > > > > > > GIMPLE passes, so serialization points would be in it.
> > > > > > > > >
> > > > > > > > > Richard.
> > > > > > > > >
> > > > > > > > > > Hope this is constructive/helpful
> > > > > > > > > > Dave
> > > > > > > > > >
> > > > > > > > > > [1] though typically our workflow involved sending patches to the gcc-
> > > > > > > > > > patches mailing list
> > > > > > > > > > [2] as libgccjit maintainer I have an interest in global state within
> > > > > > > > > > the compiler
> > > > > > > > > > [3] I posted some ideas about this back in 2013 IIRC; probably
> > > > > > > > > > massively bit-rotted since then.  I also gave a talk at Cauldron 2013
> > > > > > > > > > about global state in the compiler (with a view to gcc-as-a-shared-
> > > > > > > > > > library); likewise I expect much of the ideas there to be out-of-date);
> > > > > > > > > > for libgccjit I went with a different approach
> > > > > > 
> > > > > > Thank you,
> > > > > > Giuliano.
> > > > > > 
> > > > > 
> > > > > -- 
> > > > > Richard Biener <rguenther@suse.de>
> > > > > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > > > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> > > > 
> > > > 
> > > 
> > > -- 
> > > Richard Biener <rguenther@suse.de>
> > > SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> > > GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)
> > Giuliano.
> > 
> > 
> 
> -- 
> Richard Biener <rguenther@suse.de>
> SUSE Linux GmbH, Maxfeldstrasse 5, 90409 Nuernberg, Germany;
> GF: Felix Imendörffer, Mary Higgins, Sri Rasiah; HRB 21284 (AG Nürnberg)

Thank you,
Giuliano.

Gsoc

[FuN traveller]

Hello,
I am Darshan jadhao ,I am a computer engineering student at Lovely
Professional University, Jalandhar.
And I would like to work with GNU on the project  "c/c++
Not automatically promote memory".

Email:- darshan18jadhao99@gmail.com

GSoC

[youssef Elmasry]

What kind of experience i should have in c/c++ before applying? Is it possible to learn what’s missing before the deadline? 
Thank you

Re: GSoC

[Martin Jambor]

Hello Youssuf,

On Sat, Mar 23 2019, youssef Elmasry wrote:
> What kind of experience i should have in c/c++ before applying?

I am afraid that not only you need a very solid command of C and have to
be comfortable navigating C++ mazes but that you also need at least some
rudimentary theoretical background in the area of compilers in order to
apply.

>  Is it possible to learn what’s missing before the deadline?

No, unfortunately I don't think that is realistic.

I do not mean to be discouraging though, we are delighted you found
contributing to GCC interesting.  If your interest in C, C++ and
compilers in particular persists, I'm sure we will be only very happy to
consider you another year.

Good luck,

Martin

GSOC

[nick]

Greetings All,

I was interested in the following two projects from the wiki for this summer if possible,
Parallelize compilation using threads and Make C/C++ not automatically promote
 memory_order_consume to memory_order_acquire.

Thanks,
Nick

GSoC

[Matias Barrientos]

Hello,

I am Matías Barrientos, I am in my third year studying computing
engineering at the Pontifical University of Valparaiso.

I am very interested in contributing at GCC this year in GSoC. I have
followed the steps of the link
(https://gcc.gnu.org/wiki/SummerOfCode), so I was able to do bootstrap
and installation of GCC in my computer.

On the other hand, I have been studying GCC’s codes, optimization
flags to be more specific. In that time I could realize that the
warnings have their dependence among them, but that is not the case of
optimization. This is how I have had the idea of implementing
dependence among optimization flags by modifying the .opt files.

It would be as follow:

1. Embed the attribute EnabledBy o LangEnabledBy to create the dependence.
2. Implement the dependency of the optimization flag (“-O<number>”) to
detect the different levels.

Regards,
Matías Barrientos.

Re: GSoC

[Martin Jambor]

Hello Matias,

On Thu, Mar 14 2019, Matias Barrientos wrote:
> Hello,
>
> I am Matías Barrientos, I am in my third year studying computing
> engineering at the Pontifical University of Valparaiso.
>
> I am very interested in contributing at GCC this year in GSoC. I have
> followed the steps of the link
> (https://gcc.gnu.org/wiki/SummerOfCode), so I was able to do bootstrap
> and installation of GCC in my computer.

Wonderful, we are delighted you found contributing to GCC interesting
and happy to hear that you have got past the first steps.

>
> On the other hand, I have been studying GCC’s codes, optimization
> flags to be more specific. In that time I could realize that the
> warnings have their dependence among them, but that is not the case of
> optimization. This is how I have had the idea of implementing
> dependence among optimization flags by modifying the .opt files.
>
> It would be as follow:
>
> 1. Embed the attribute EnabledBy o LangEnabledBy to create the dependence.
> 2. Implement the dependency of the optimization flag (“-O<number>”) to
> detect the different levels.

Unfortunately, this project idea apparently has not quite caught much
attention.  I myself struggle with understanding what is supposed to be
dependent on what and don't think that simply mapping dependencies from
optimization levels to individual warning options is a project that
should span three months.  You'd need to substantially extend the scope
somehow or perhaps look for another idea, our wiki still has a few that
nobody took up.

If I misunderstood something, please feel free to correct me and/or ask
anything specific that you'd like to know about GCC.

Thanks,

Martin

GSoC

[Martin Emil]

Hello ,
I am Martin Emil last year computer engineering student from Egypt .
I came through your project in GSoC  and i am very interested about it and
want to work on it.
I have strong knowledge in C,C++,Java and python programming languages .
I am interested in *Implementation of OMPD
<http://www.openmp.org/wp-content/uploads/SC17-Protze-2017-11-15_OpenMP-Tools.pdf>
in
GCC idea  , and i want to know more about it .*
*I am seeking a mentor to work with on this idea*
Here is the link to my github page :
https://github.com/MartinEmilSaad

I am looking forward to hearing from you

Re: GSoC

[Martin Jambor]

Hello Martin,

On Sun, Mar 10 2019, Martin Emil wrote:
> Hello ,
> I am Martin Emil last year computer engineering student from Egypt .
> I came through your project in GSoC  and i am very interested about it and
> want to work on it.
> I have strong knowledge in C,C++,Java and python programming
> languages.

Given the project you are interested in, do you have background in
parallel programming?  OpenMP in particular?  I am also worried that you
need at least some rudimentary theoretical background in the area of
compilers in order to apply too.

> I am interested in *Implementation of OMPD
> <http://www.openmp.org/wp-content/uploads/SC17-Protze-2017-11-15_OpenMP-Tools.pdf>
> in  GCC idea  , and i want to know more about it .*

We are happy to answer any specific question but this one is quite a bit
too broad.  Apart from what is covered by the spec you have linked, what
would you like to know?

Thanks,

Martin

Re: GSoC

[Jakub Jelinek]

On Tue, Mar 26, 2019 at 01:05:37AM +0100, Martin Jambor wrote:
> On Sun, Mar 10 2019, Martin Emil wrote:
> > Hello ,
> > I am Martin Emil last year computer engineering student from Egypt .
> > I came through your project in GSoC  and i am very interested about it and
> > want to work on it.
> > I have strong knowledge in C,C++,Java and python programming
> > languages.
> 
> Given the project you are interested in, do you have background in
> parallel programming?  OpenMP in particular?  I am also worried that you
> need at least some rudimentary theoretical background in the area of
> compilers in order to apply too.
> 
> > I am interested in *Implementation of OMPD
> > <http://www.openmp.org/wp-content/uploads/SC17-Protze-2017-11-15_OpenMP-Tools.pdf>
> > in  GCC idea  , and i want to know more about it .*

Note, the current OMPD spec is in
https://www.openmp.org/wp-content/uploads/OpenMP-API-Specification-5.0.pdf

	Jakub

GSoC

[Ahmed Ashraf]

Hello,
I am Ahmed Ashraf, a first-year student at the Computer Engineering
Department, Faculty of Engineering - Cairo University in Egypt.

In the first semester, I was asked to create a CAD as the Circuits course
project. The requirements were to simulate simple AC circuits that contain
only (independent voltage sources, independent current sources, dependent
voltage source, dependent current source, resistors, capacitors, and
inductors).

The main problem which faced me was the math behind the code. I needed to
solve a system of equations in more than 3 variables and the equations
contain complex numbers and the results also were complex but the "math.h"
and "complex.h" were very poor and useless to my project.

So, it's a great chance for me to participate in a project will improve
that libraries to help other people who need it to have a better experiment
in using them.

I am very excited about participating with you in the project "  Add new
math.h and complex.h functions as built-ins". I studied 2 courses in
programming the first was about the basics of C++ and the second was about
Object-Oriented Programming using C++. Currently, I'm studying the third
course about "Data Structures and Algorithms" also using C++ but recently I
started to learn Python from an MIT course and "Data Structures and
Algorithms" specialization in Coursera. Participating in that will be a
very good step in my career closer to my goal.

I also studied some courses in mathematics such that three courses in
Calculus, in different areas like "Differential Calculus", "Integral
Calculus", "Differential Equations" and " Partial Derivatives and Laplace".
Currently, I am studying a course on "Discrete Mathematics" and " Discrete
Mathematics" specialization in Coursera.

I am also training on problem-solving skills constantly from online judges
such that Codeforces.

So, is this enough to have a chance in your project? and how I can improve
my self to be qualified to participate with you before the application
deadline?

GitHub Account <https://github.com/aashrafh>

Re: GSoC

[Dmitry Mikushin]

Linear equation solvers is not the scope of <math.h>. There are many
packages serving this particular purpose, try looking into e.g. LAPACK.

Kind regards,
- Dmitry.


пт, 1 мар. 2019 г. в 23:09, Ahmed Ashraf <aashrafh31@gmail.com>:

> Hello,
> I am Ahmed Ashraf, a first-year student at the Computer Engineering
> Department, Faculty of Engineering - Cairo University in Egypt.
>
> In the first semester, I was asked to create a CAD as the Circuits course
> project. The requirements were to simulate simple AC circuits that contain
> only (independent voltage sources, independent current sources, dependent
> voltage source, dependent current source, resistors, capacitors, and
> inductors).
>
> The main problem which faced me was the math behind the code. I needed to
> solve a system of equations in more than 3 variables and the equations
> contain complex numbers and the results also were complex but the "math.h"
> and "complex.h" were very poor and useless to my project.
>
> So, it's a great chance for me to participate in a project will improve
> that libraries to help other people who need it to have a better experiment
> in using them.
>
> I am very excited about participating with you in the project "  Add new
> math.h and complex.h functions as built-ins". I studied 2 courses in
> programming the first was about the basics of C++ and the second was about
> Object-Oriented Programming using C++. Currently, I'm studying the third
> course about "Data Structures and Algorithms" also using C++ but recently I
> started to learn Python from an MIT course and "Data Structures and
> Algorithms" specialization in Coursera. Participating in that will be a
> very good step in my career closer to my goal.
>
> I also studied some courses in mathematics such that three courses in
> Calculus, in different areas like "Differential Calculus", "Integral
> Calculus", "Differential Equations" and " Partial Derivatives and Laplace".
> Currently, I am studying a course on "Discrete Mathematics" and " Discrete
> Mathematics" specialization in Coursera.
>
> I am also training on problem-solving skills constantly from online judges
> such that Codeforces.
>
> So, is this enough to have a chance in your project? and how I can improve
> my self to be qualified to participate with you before the application
> deadline?
>
> GitHub Account <https://github.com/aashrafh>
>

GSOC

[Siddhartha Sen]

I am Siddhartha Sen,currently pursuing my B.Tech degree in Information
Science and Engineering,2nd year. I have taken a keen interest in your
projects and have some ideas of my own as well. I am really interested in
working with you in G-SOC 2019. I am proficient in C and C++ and am eager
to work on brushing my skills in whichever field required. I have already
checked out the GCC trunk source-code and am being able to build GCC from
it.I have run the testsuite and saved the results. However, upon building
it and saving it again, the results don't match. I apologise for my lack of
knowledge but any help on your part would be great.
Thank You.

GSoC

[Basil George]

Stating my intention to apply for GSoS as instructed here
<https://gcc.gnu.org/wiki/SummerOfCode#Application> .

Name - Basil George Poulose
Email - basilgeo99@gmail.com

Proposal submitted for back-end cleanup project.

GSoC

[Gaurav Ahuja]

I had a query, while choosing project for GSoC. Actually I'm
contributing to GCC for the first time, and I am not sure about what
is expected of me.

So, can you please explain me about , what GCC organisation expects me
to do, like detect bugs, or add a new feature ,etc.?

Re: GSoC

[Martin Jambor]

Hello Gurav,

On Thu, Mar 15 2018, Gaurav Ahuja wrote:
> I had a query, while choosing project for GSoC. Actually I'm
> contributing to GCC for the first time, and I am not sure about what
> is expected of me.
>
> So, can you please explain me about , what GCC organisation expects me
> to do, like detect bugs, or add a new feature ,etc.?

I am not sure I understand your question.  GCC as an GSoC organization
does not give prospective students any tasks they need to complete
before applying.  However, it is a good idea to at least check out,
build and test the compiler suite before you apply for a project with
us.  See the information at

   https://gcc.gnu.org/wiki/SummerOfCode#Before_you_apply

and generally speaking the entire wiki page on GSoC for what to do.

Martin

Re: GSOC

[Martin Jambor]

Hello Prashant,

On Thu, Mar 08 2018, prashant kumar wrote:
> Hello sir my self *Prashant kumar *second year B.TECH student and i am
> interesting in *"GCC plugin Api" *project .i have a good hand in c,c++
> languages .Can u tell me what the further process i have to do and what
> kind of thing i have to do

first and foremost, please discuss any GSoC idea or topic, use the
gcc@gcc.gnu.org mailing list (CCed), this way other people may also
participate in the discussion which will eventually help you as well.
Moreover, in your particular case, you sent me your message while I was
on vacation and so it took quite some time before I got to replying.
Other members of the GCC community might have done that meanwhile.

The aim of the gcc API project would be to start building a real GCC API
for plugins, so that simple but already useful plugins (operating on
GIMPLE level) could be written that use only this API (as opposed to the
current state where they can - actually they have to - use any normal
public symbol there is in GCC.  I do not think it is reasonable to
attempt make the API really entirely compiler agnostic but whenever
practical, the interface should be built on top of generic compiler
constructs (e.g. statements and operands as opposed to gimple and
trees), for many reasons.

In the first half of the project one would aim to build various dumping
plugins... that would simply dump CFG, statements and perhaps also some
information about types queried through the new interface.  In the
second half, you should try to rewrite some existing simple plugins
using this API (such as https://pagure.io/funcp-encrypt that is
described at
https://developers.redhat.com/blog/2017/03/17/diagnosing-function-pointer-security-flaws-with-a-gcc-plugin/
or perhaps the structleak_plugin.c kernel GCC plugin or
https://rwmj.wordpress.com/2016/02/24/playing-with-gcc-plugins/ though I
have not had a very close look any of those).

If you are interested, make sure you can check out the GCC trunk source
code, can build GCC from it.  The following links should help you:

 - How to check out our sources using svn and git is described at
   https://gcc.gnu.org/svn.html and https://gcc.gnu.org/wiki/GitMirror
   respectively.

 - Steps linked from https://gcc.gnu.org/install/ show you how to
   configure, build and test GCC is described in (look for
   --disable-bootstrap, among other things).

 - Also make sure you also look at https://gcc.gnu.org/wiki/InstallingGCC and
   https://gcc.gnu.org/wiki/GettingStarted wiki pages.

And while you are at it, try also to look at the source.  But you have
gone through all of the above and still find a little bit intimidating
or difficult to see where to start looking, do not despair.  That is
something the mentors and the community at large are willing to help
you with.

Regarding plugins, I'd suggest reading one ore more tutorials on their
current implementation
(e.g. http://thinkingeek.com/2015/08/16/a-simple-plugin-for-gcc-part-1/)
and then look at some real plugins to get the sense of what would need
to be APIzied.

Good luck,

Martin

Re: GSOC

[Richard Biener]

On Wed, Mar 14, 2018 at 5:34 PM, Martin Jambor <mjambor@suse.cz> wrote:
> Hello Prashant,
>
> On Thu, Mar 08 2018, prashant kumar wrote:
>> Hello sir my self *Prashant kumar *second year B.TECH student and i am
>> interesting in *"GCC plugin Api" *project .i have a good hand in c,c++
>> languages .Can u tell me what the further process i have to do and what
>> kind of thing i have to do
>
> first and foremost, please discuss any GSoC idea or topic, use the
> gcc@gcc.gnu.org mailing list (CCed), this way other people may also
> participate in the discussion which will eventually help you as well.
> Moreover, in your particular case, you sent me your message while I was
> on vacation and so it took quite some time before I got to replying.
> Other members of the GCC community might have done that meanwhile.
>
> The aim of the gcc API project would be to start building a real GCC API
> for plugins, so that simple but already useful plugins (operating on
> GIMPLE level) could be written that use only this API (as opposed to the
> current state where they can - actually they have to - use any normal
> public symbol there is in GCC.  I do not think it is reasonable to
> attempt make the API really entirely compiler agnostic but whenever
> practical, the interface should be built on top of generic compiler
> constructs (e.g. statements and operands as opposed to gimple and
> trees), for many reasons.
>
> In the first half of the project one would aim to build various dumping
> plugins... that would simply dump CFG, statements and perhaps also some
> information about types queried through the new interface.  In the
> second half, you should try to rewrite some existing simple plugins
> using this API (such as https://pagure.io/funcp-encrypt that is
> described at
> https://developers.redhat.com/blog/2017/03/17/diagnosing-function-pointer-security-flaws-with-a-gcc-plugin/
> or perhaps the structleak_plugin.c kernel GCC plugin or
> https://rwmj.wordpress.com/2016/02/24/playing-with-gcc-plugins/ though I
> have not had a very close look any of those).
>
> If you are interested, make sure you can check out the GCC trunk source
> code, can build GCC from it.  The following links should help you:
>
>  - How to check out our sources using svn and git is described at
>    https://gcc.gnu.org/svn.html and https://gcc.gnu.org/wiki/GitMirror
>    respectively.
>
>  - Steps linked from https://gcc.gnu.org/install/ show you how to
>    configure, build and test GCC is described in (look for
>    --disable-bootstrap, among other things).
>
>  - Also make sure you also look at https://gcc.gnu.org/wiki/InstallingGCC and
>    https://gcc.gnu.org/wiki/GettingStarted wiki pages.
>
> And while you are at it, try also to look at the source.  But you have
> gone through all of the above and still find a little bit intimidating
> or difficult to see where to start looking, do not despair.  That is
> something the mentors and the community at large are willing to help
> you with.
>
> Regarding plugins, I'd suggest reading one ore more tutorials on their
> current implementation
> (e.g. http://thinkingeek.com/2015/08/16/a-simple-plugin-for-gcc-part-1/)
> and then look at some real plugins to get the sense of what would need
> to be APIzied.

Note that there are also (half-way?) finished implementations, several
that allow writing plugins in other languages (python, etc.) that usually
come with "shim" layers to support multiple GCC versions underneath.
Those "shim" layers are a good hint at what needs abstracting.

There's also an incomplete attempt at providing a mostly compiler-agnostic
plugin API for introspection (but techincally not limited to that).  Not sure
if Micha ever posted what he had implemented.

If I had to choose then I would suggest to embed a "common" shim layer
into GCC itself and call that the C plugin API.  The python plugin API
implementation should be able to bolt on that as well as the introspection API.

This means the GSoC project would be about implementing such a layer
(and eventually simply start by picking one of the available out-of-tree ones,
massaging it so it can replace the others).  And of course documenting it,
doing some "real" example plugins using that layer directly.

Richard.

> Good luck,
>
> Martin

GSoC

[Maria Kalikas]

Hello anyone this concerns,

My name is Maria Kalikas and I am a senior majoring in Computer Science at
The University of Akron.

My familiarity and continuous reading of the C++ Standard has encouraged me
to extend the previously developed support for concepts in GCC. I want to
modify the implementation to conform with the current specification in the
C++ Standard. I am curious to know if this would be a good project to
propose for Google Summer of Code and am interested in finding a mentor.

 I took a Compiler Design course in Spring 2017 where I designed a compiler
for a C-like programming language. My implementation is written in C++.
Since completing the course, I became familiar with Clang and LLVM and am
currently completing a capstone project where I am creating a virtual
machine to implement the C++ abstract machine using LLVM instructions and
rewriting the conversions, expressions and declarations specified in the
C++ Standard as three address code instructions.

I look forward to any responses and feedback.

Thank you,

Maria Kalikas

GSoC

[Ko Phyo]

Hello

        I want to apply the project "Improvements to GCC on Windows" of GCC
foundation. As a student, I have some theorectical knowledges about C/C++
and compiler technologies. And I am sure that the Windows platform has
steps that are little behind than other open source platforms in pure C/C++
technology. So I decided to work on GSoC 2018 with your "Windows
Improvement program". Please let me know if I am selected as a mentored
student in yours great project. I will be waiting for your kind reply.

GSoC

[Tejas Joshi]

"GCC supports built-in functions for math.h and complex.h functions in
the C99/C11 standards (both folding calls for constant arguments, and
expanding inline when the processor supports appropriate
functionality). More such functions have been added in ISO/IEC TS
18661, supporting features of IEEE 754-2008. It would be useful to
have built-in functions for those, both folding for constant
arguments, and expanding inline where appropriate (e.g. for roundeven
and the functions rounding result to narrower type, on some
processors; roundeven can be inlined on x86 for SSE4.1 and later, and
the narrowing functions on IA64 and POWER8, for example)".

The above Project Idea is made available on 'Summer of Code' wiki of
GNU GCC Website. I wanted to have some more details about above idea
regarding to what is expected for implementation and expected output
for the same.

-Tejas Joshi

Re: GSoC

[Joseph Myers]

On Thu, 1 Mar 2018, Tejas Joshi wrote:

> "GCC supports built-in functions for math.h and complex.h functions in
> the C99/C11 standards (both folding calls for constant arguments, and
> expanding inline when the processor supports appropriate
> functionality). More such functions have been added in ISO/IEC TS
> 18661, supporting features of IEEE 754-2008. It would be useful to
> have built-in functions for those, both folding for constant
> arguments, and expanding inline where appropriate (e.g. for roundeven
> and the functions rounding result to narrower type, on some
> processors; roundeven can be inlined on x86 for SSE4.1 and later, and
> the narrowing functions on IA64 and POWER8, for example)".
> 
> The above Project Idea is made available on 'Summer of Code' wiki of
> GNU GCC Website. I wanted to have some more details about above idea
> regarding to what is expected for implementation and expected output
> for the same.

See <http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1778.pdf> for a 
near-final draft of TS 18661-1 that says what various of the functions 
mentioned should do.  In most cases you'd be folding for constant 
arguments and generating appropriate instructions for non-constant 
arguments if the target processor has such instructions - for non-constant 
arguments where the processor doesn't have relevant instructions, the 
implementation would be left to libm (e.g. glibc 2.25 has most of these 
functions) as it would be too big to expand inline in that case.

Here are some examples:

* roundeven is similar to existing functions round / ceil / floor / trunc.  
So you'd define built-in functions (roundeven / roundevenf / roundevenl 
and _FloatN and _FloatNx variants) similar to those for the older rounding 
functions, in builtins.def.  You'd look at all the places through the 
compiler that are needed to handle those functions - where the existing 
ones get folded for constant arguments, where they end up getting 
converted to appropriate RTL using corresponding .md patterns for 
processors supporting expanding them inline for non-constant arguments, 
and update all those places to handle these functions.  Documentation of 
built-in functions (extend.texi) and instruction patterns (md.texi) would 
need updating.  Testcases would need adding to verify appropriate folding 
for constant arguments.

You'd also implement the new instruction patterns for at least one target 
- for example, for x86 (roundss and roundsd SSE4.1 instructions, immediate 
operand 8).  There would then be architecture-specific tests needed for 
the testsuite to verify the appropriate code generation (both scanning 
assembler output and execution tests).  Some other architectures also have 
appropriate instructions, but once things have been illustrated by doing 
it for one architecture, the target maintainers for other architectures 
might well do those updates.

Doing roundeven would be a good starting point for becoming familiar with 
the relevant parts of the compiler, because there are existing built-in 
functions that are very similar and so serve as a guide to what you need 
to change.  There are lots of other functions for which built-in functions 
are also useful, some of which bring their own complications - this 
project is open-ended, and the number of functions implemented can depend 
on how long it takes to do each one.

* The narrowing functions, e.g. fadd, faddl, daddl, are a bit different 
from most other built-in math.h functions because the return type is 
different from the argument types.  You could start by adding them to 
builtins.def similarly to roundeven (with new macros to handle adding such 
functions for relevant pairs of _FloatN, _FloatNx types).  These functions 
could be folded for constant arguments only if the result is exact, or if 
-fno-rounding-math -fno-trapping-math (and -fno-math-errno if the result 
involves overflow / underflow).

Probably more interesting than folding them for constant arguments is 
expanding them inline for non-constant arguments (provided 
-fno-math-errno).  Given -fno-math-errno, those functions can always be 
expanded inline if the argument and result types have the same 
floating-point format - for example, f32xaddf64 on all GCC configurations 
that support _Float32x and _Float64.  So you want that to expand to a 
normal ADD operation (on GIMPLE and then RTL) just as if the user had 
written normal floating-point addition.

But when the functions are genuinely narrowing - fadd taking double 
arguments and returning float, for example - you can only expand inline 
for non-constant arguments when the target architecture has appropriate 
support.  There are various existing standard instruction patterns with 
arguments and results of different modes - for example, mulsidi3, a 
widening multiplication of SImode (32-bit) operands producing a DImode 
(64-bit) result.  So you might add support for e.g. adddfsf3, a narrowing 
addition of DFmode operands producing an SFmode result.  Then that pattern 
would need implementing for some suitable target, say POWER8 (rs6000 
port), where the instructions for float arithmetic work with double 
arguments and have the required narrowing semantics.

* The fromfp / fromfpx / ufromfp / ufromfpx functions (round to integers 
of a specified number of bits, in a specified rounding mode, with 
specified handling of inexact results) are a case with some other 
complications.  Typically I'd expect them to be expanded inline only (for 
constant arguments or) for constant values of the number of bits and 
rounding mode, if the target machine has an appropriate instruction.  A 
target hook would need adding for a target to specify the FP_INT_* values 
used in libm, since that's an ABI that's defined by libm, not by GCC.  
Then you'd need instruction patterns that might only be supported in 
certain cases.

For example, AArch64 has FCVT instructions to convert floating-point 
values to 32-bit or 64-bit integers, signed and unsigned, with a rounding 
mode specified in the instruction.  Those instructions set "inexact" for 
non-integer arguments, so are appropriate for fromfpx / ufromfpx, but for 
the functions that don't set "inexact" (fromfp / ufromfp) you might use an 
appropriate FRINT instruction to round to integer in the floating-point 
type, without setting "inexact", followed by an FCVT instruction to 
convert that to an integer type.

* Classification and comparison macros also naturally have corresponding 
type-generic built-in functions which are a bit different from the 
examples discussed above.  For example, there ought to be 
__builtin_iseqsig, see 
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=77928> for details.  
__builtin_issignaling, which would need to work by examining the bits of 
the representation of the argument passed (except on architectures with 
RISC-V that have instructions that help), would also be useful.  (Some 
such functions, see e.g. bug 77925, were implemented by Tamar Christina 
but that patch had to be reverted because it caused problems which haven't 
yet been resolved.  However, I don't think anyone has implemented 
__builtin_iseqsig or __builtin_issignaling for GCC.)

-- 
Joseph S. Myers
joseph@codesourcery.com

GSoC

[Dushyant Pratap Singh]

Hello GCC community,
My name is Dushyant and I am interested in Gsoc at GCC .I am pursuing
Integrated B.Tech (Computer Science) and M.Tech (software engineering) from
Gautam Buddha University, India. Currently I am in 4th year of the
programme.

I have experience in Competitive Programming (primary language C++) and
have won and participated in many of the Algorithmic & Data Structures
competitions.

Here is my Linkedin:
https://www.linkedin.com/in/dushyant-pratap-singh-044658128/

I have taken a course on Compiler Design in my last semester in which i
scored A+ grade.

Project that i am most interested in is Textual Representation of LTO
Object Files.

In my compiler lab I learned about the Compilation process how the
preproccessed files (.i file) ,assembly code (.s file) ,object code (.o
file) and finally executable file(a.out) were generated (as they were
temporary files they needed an extra GCC flag -save-temps to store
"temporary files" permanently) using gcc –Wall –save-temps prog1.c –o prog1
.

I have started to investigate on converting binary files to human readable
forms.It would be great if someone could explain how to get started with
the process.

Regards

Dushyant Pratap Singh

Re: GSoC

[Richard Biener]

On Mon, Feb 26, 2018 at 5:35 PM, Dushyant Pratap Singh
<iamdushyant865@gmail.com> wrote:
> Hello GCC community,
> My name is Dushyant and I am interested in Gsoc at GCC .I am pursuing
> Integrated B.Tech (Computer Science) and M.Tech (software engineering) from
> Gautam Buddha University, India. Currently I am in 4th year of the
> programme.
>
> I have experience in Competitive Programming (primary language C++) and
> have won and participated in many of the Algorithmic & Data Structures
> competitions.
>
> Here is my Linkedin:
> https://www.linkedin.com/in/dushyant-pratap-singh-044658128/
>
> I have taken a course on Compiler Design in my last semester in which i
> scored A+ grade.
>
> Project that i am most interested in is Textual Representation of LTO
> Object Files.
>
> In my compiler lab I learned about the Compilation process how the
> preproccessed files (.i file) ,assembly code (.s file) ,object code (.o
> file) and finally executable file(a.out) were generated (as they were
> temporary files they needed an extra GCC flag -save-temps to store
> "temporary files" permanently) using gcc –Wall –save-temps prog1.c –o prog1
> .
>
> I have started to investigate on converting binary files to human readable
> forms.It would be great if someone could explain how to get started with
> the process.

The suggested way was to handle things similar to the gcov tools -- piggy-back
on the existing {lto,tree}-streamer-in.c files, #ifdef'ing relevant parts so you
can link them into a new lto-dump "driver" binary.

To get started you might want to look at lto/lto-object.c and/or follow the
lto1 binary when it processes its input files.

Eventually integrating the dumping of a textual representation into
the lto1 binary itself would also be feasible.  Still the actual dumping
should probably be intermangled with the binary reading code for
ease of maintainance.

Note the project isn't about providing a way of editing the textual
representation and reading that back in but only for for inspection / debugging.

There are several interesting pieces of information in the LTO IL so
having a way to dump only parts with a specifyable level of detail
is desirable (thus the suggestion to do this in a new binary rather
than lto1).

Thanks for the interest in this project!

Richard.


> Regards
>
> Dushyant Pratap Singh

GSoC

[Thejazeto Lhousa]

Hello,

I went through the GSoC 2018 idealist and found 'Improving GCC
Developer Documentation' to be interesting and feasible for me.
But I am confused because I remember reading somewhere that
documentation will not be accepted as a valid project for GSoC.
Can anyone shed some light?

Regards,
Theja

Re: GSoC

[Thomas Schwinge]

Hi!

On Wed, 21 Feb 2018 21:35:35 +0530, Thejazeto Lhousa <thejazeto@gmail.com> wrote:
> I went through the GSoC 2018 idealist

Thanks for your interest!

> found 'Improving GCC
> Developer Documentation' to be interesting and feasible for me.
> But I am confused because I remember reading somewhere that
> documentation will not be accepted as a valid project for GSoC.
> Can anyone shed some light?

You're correct: per <https://summerofcode.withgoogle.com/rules/>, 1.26
"Project", "Projects do not include projects for documentation only".
Martin has now removed that item from the Project Ideas list,
<https://gcc.gnu.org/wiki/SummerOfCode?action=diff&rev1=137&rev2=138>.


Grüße
 Thomas

Re: GSoC

[Thejazeto Lhousa]

Thank you for the clarification.

On Feb 24, 2018 4:25 PM, "Thomas Schwinge" <thomas@codesourcery.com> wrote:

> Hi!
>
> On Wed, 21 Feb 2018 21:35:35 +0530, Thejazeto Lhousa <thejazeto@gmail.com>
> wrote:
> > I went through the GSoC 2018 idealist
>
> Thanks for your interest!
>
> > found 'Improving GCC
> > Developer Documentation' to be interesting and feasible for me.
> > But I am confused because I remember reading somewhere that
> > documentation will not be accepted as a valid project for GSoC.
> > Can anyone shed some light?
>
> You're correct: per <https://summerofcode.withgoogle.com/rules/>, 1.26
> "Project", "Projects do not include projects for documentation only".
> Martin has now removed that item from the Project Ideas list,
> <https://gcc.gnu.org/wiki/SummerOfCode?action=diff&rev1=137&rev2=138>.
>
>
> Grüße
>  Thomas
>

Re: GSoC

[Martin Jambor]

Hi Thejazeto,

I apologize for replying this late, I've been traveling and did not
manage to write to you earlier as a consequence.

On Wed, Feb 21 2018, Thejazeto Lhousa wrote:
> Hello,
>
> I went through the GSoC 2018 idealist and found 'Improving GCC
> Developer Documentation' to be interesting and feasible for me.
> But I am confused because I remember reading somewhere that
> documentation will not be accepted as a valid project for GSoC.
> Can anyone shed some light?

You are right, the rules do not allow documentation-only projects, thank
you very much for pointing this out. As Thomas pointed out, I have
corrected the wiki.

Moreover, I think that writing good internal documentation is actually a
fairly difficult task that is not really suitable for newcomers, because
it often takes quite some knowledge to understand all important caveats.

Nevertheless, if you are really interested in compilers in general and
GCC in particular, please not feel put off by this.  Check out and build
GCC (as described at https://gcc.gnu.org/install/) and experiment with
it a little, you may find it is not actually as hard as it seems.  Feel
free to ask for help here on the mailing list or on our IRC when you get
stuck, we are a friendly bunch that want to help beginners, even though
we are often busy.

If the scope of GCC still intimidates you (but we all struggle with it
sometimes, trust me), consider reaching out to Andi Kleen and discuss
his fuzzer project idea with him (he may tell you what to check out and
experiment with to get the feeling about the task at hand).

Thanks again for notifying us about the documentation and I hope that
you will eventually find a project you will enjoy,

Martin

Re: GSoC

[Andi Kleen]

> If the scope of GCC still intimidates you (but we all struggle with it
> sometimes, trust me), consider reaching out to Andi Kleen and discuss
> his fuzzer project idea with him (he may tell you what to check out and
> experiment with to get the feeling about the task at hand).

The fuzzer project is to extend an existing C fuzzer (such as csmith
or yarpgen) with gcc language extensions and use that to test the compiler

Tasks:
- evaluate existing fuzzer sources for suitable base
- study gcc documentation and other references for suitable language
extensions to implement (e.g. openmp or transactions or vector extensions)
- implement a suitable subset of extensions in one of the fuzzers
- validate fuzzer output is valid code (that's the hard part)
- run fuzzer against compiler and report bugs

-Andi